Added link to tag

Update 08_routing_htlcs - wrong var name in RIPEMD160 description

.gitattributes

@@ -0,0 +1 @@
+*.asciidoc linguist-detectable

.gitignore

@@ -0,0 +1,9 @@
+*.pdf
+*.html
+!author_bio.html
+!colo.html
+!copyright.html
+!cover.html
+!ix.html
+!titlepage.html
+!toc.html

.gitignore/.gitignore

@@ -1,2 +0,0 @@
-*.pdf
-.debris

01_introduction.asciidoc

@@ -2,106 +2,232 @@
 [[intro_what_is_the_lightning_network]]
 == Introduction
 
-Welcome to Mastering the Lightning Network!
+Welcome to _Mastering the Lightning Network_!
 
-The Lightning Network, abbreviated with LN, is a protocol for using Bitcoin in a smart and non-obvious way.
-Thus it is a second layer technology on top of Bitcoin.
-It is changing the way people exchange value online and it's one of the most exciting advancements to happen to the Bitcoin network over the past few years. Right now, the Lightning Network is in its infancy. In concept it's about 5 years old, in implementation about 3 years old. We're only beginning to see the opportunities the Lightning Network provides including improved privacy, speed, and scale. With core knowledge of the Lightning Network, you can help shape the future of the network while building opportunities for yourself as well. Some basic knowledge about Bitcoin is assumed but can be readily acquired by reading the first two chapters of _Mastering Bitcoin_ which are available for free online.
+((("Lightning Network (generally)", seealso="innovations in Lightning", id="ix_01_introduction-asciidoc0", range="startofrange")))The Lightning Network (often abbreviated as LN), is changing the way people exchange value online, and it's one of the most exciting advancements to happen in Bitcoin's history.
+Today, in 2021, the Lightning Network is still in its infancy. The Lightning Network is a protocol for using Bitcoin in a smart and nonobvious way. It is a second layer technology on top of Bitcoin.
 
-While the bulk of this book is written for programmers, the first two chapters are written to be approachable by anyone regardless of technical experience. In order to better understand how the technology actually works, and why people use it, we'll be following a number of users and their stories. But first, we'll introduce some of the key concepts in the Lightning Network. Let's get started with why the Lightning Network was proposed in the first place.
+The concept of the Lightning Network was proposed in 2015, and the first implementation was launched in 2018. As of 2021, we're only beginning to see the opportunities the Lightning Network provides to Bitcoin, including improved privacy, speed, and scale.
+With core knowledge of the Lightning Network, you can help shape the future of the network while also building opportunities for yourself.
 
-=== Motivation for the Lightning Network
+We assume you already have some basic knowledge about Bitcoin, but if not, don't worry—we will explain the most important Bitcoin concepts, those you must know to understand the Lightning Network, in <<bitcoin_fundamentals_review>>. If you want to learn more about Bitcoin, you can read _Mastering Bitcoin_, 2nd edition, by Andreas M. Antonopoulos (O'Reilly), available https://github.com/bitcoinbook/bitcoinbook[for free online].
 
-As Bitcoin and the demand for transactions grows, the number of transactions in each block will increase until it eventually hits the block size limit. When blocks are full, excess transactions are left to wait in a queue. Many users increase the fees they're willing to pay in order to buy space for their transaction in the next block. At the same time, an increasing number of users are left behind. Their transactions, e.g. microtransactions such as common small spendings, are not economically qualified to be on the network. However, increasing block size simply shifts the problem to node operators, where each increase in blocksize results in a resource increase multiplied by an order of magnitude.
+While the bulk of this book is written for programmers, the first few chapters are written to be approachable by anyone regardless of technical experience. In this chapter, we'll start with some terminology, then move to look at trust and its application in these systems, and finally we'll discuss the history and future of the Lightning Network. Let's get started.
 
-Because blockchains are gossip protocols, each node is required to know and validate every single transaction that occurs on the network. Furthermore, once validated, each transaction and block must be propagated to the node's "neighbors", multiplying the bandwidth requirements. As such, the greater the block size, the greater the bandwidth, processing, and storage requirements for each individual node, effectively limiting the amount of scaling that can be done this way. Furthermore, scaling in this fashion has an undesirable side effect of centralizing the network by reducing the number of nodes and node operators. Since node operators are not compensated for running nodes, if nodes are very expensive to run, only a few well funded node operators will continue to run nodes.
-
-[NOTE]
-====
-The side effects of increasing the block size or decreasing the block time with respect to centralization of the network are severe as a few calculations with the numbers show. 
-Let us assume the usage of Bitcoin grows so that the network has to process 40,000 transactions per second.
-Assuming 250 Bytes on average per transaction this would result in a data stream of 10 Megabyte per second or 80 Mbit/s just to be able to receive all the transactions.
-This does not include the traffic overhead of forwarding the transaction information to other peers.
-While single hosts on the internet could handle such a load of traffic our current internet would not be able to support this traffic for a large fraction of hosts.
-Also storing this information locally would result in 864,000 Megabyte per day. This is roughly 1 Terabyte of data or the size of a hard drive.
-While verifying 40,000 ECDSA signatures per second seems barely feasible (c.f.: https://bitcoin.stackexchange.com/questions/95339/how-many-bitcoin-transactions-can-be-verified-per-second) nodes could hardly catch up initial sync of the blockchain. 
-====
-
-But what if each node wasn't required to know and validate every single transaction? What if there was a way to have scalable off-chain transactions, without losing the security of the Bitcoin network?
-
-In February 2015, Joseph Poon and Thaddeus Dryja proposed a possible solution to the Bitcoin Scalability Problem, with the publication of _"The Bitcoin Lightning Network: Scalable Off-Chain Instant Payments"_ footnote:[Joseph Poon, Thaddeus Dryja - "The Bitcoin Lightning Network:
-Scalable Off-Chain Instant Payments" (https://lightning.network/lightning-network-paper.pdf).] In the meanwhile outdated whitepaper, Poon and Dryja estimate that in order for Bitcoin to reach the 47,000 transactions per second processed at peak by Visa, it would require 8 GB blocks. This would make running a node completely untenable for anyone but large scale enterprises and industrial grade operations. The result would be a network in which only a few users can actually validate the state of the ledger, which ultimately breaks the trust model of Bitcoin.
-
-The Lightning Network proposes a new network, a "second layer", where users can make payments to each other peer-to-peer, without the necessity to publish a transaction to the Bitcoin blockchain for each payment.
-Users may pay each other on the Lightning Network as many times as they want, making use of the Bitcoin blockchain only in order to load bitcoin onto the Lightning network initially and to "settle", that is: remove bitcoin from the Lightning Network.
-The result is that many more Bitcoin payments can take place "off-chain", with only the initial loading and final settlement transactions needing to be validated and stored by Bitcoin nodes.
-Aside from reducing the burden on nodes, payments on the Lightning Network will be cheaper for users as they do not need to pay blockchain fees, and more private for users as they are not published to all participants of the network and furthermore not stored permanently. 
-
-While the Lightning Network was initially conceived for Bitcoin, it can be implemented on any blockchain that meets its technical requirements.
 
 === Lightning Network Basic Concepts
 
-As we start exploring the Lightning Network, we will encounter some technical terminology that might, at first, be confusing and a bit difficult to understand. While all of these concepts and terms will be explained in detail as we progress through the book, and are defined in the glossary, we need some basic clarifications to get started. Here are some of the concepts you will encounter in the first two chapters of this book:
+((("Lightning Network (generally)","basic concepts", id="ix_01_introduction-asciidoc1", range="startofrange")))As we explore how the Lightning Network actually works, we will encounter some technical terminology that might, at first, be a bit confusing. While all of these concepts and terms will be explained in detail as we progress through the book and are defined in the glossary, some basic definitions now will make it easier to understand the concepts in the next two chapters. If you don't understand all of the words in these definitions yet, that's OK. You'll understand more as you move through the text.
 
-Node:: A computer that participates in a network. A Lightning node is a computer that participates in the Lightning Network. A Bitcoin node is a computer that participates in the Bitcoin Network. Typically a Lightning Network user will run a Lightning node and a Bitcoin node.
+Blockchain:: ((("blockchain","defined")))A distributed transaction ledger, produced by a network of computers. Bitcoin, for example, is a system that produces a blockchain. The Lightning Network is not itself a blockchain, nor does it produce a blockchain. It is a network that relies on an existing external blockchain for its security.
 
-Blockchain:: A distributed transaction ledger, produced by a network of computers. Bitcoin, for example, is a system that produces a blockchain. The Lightning Network is not itself a blockchain, nor does it produce a blockchain. It is a network that relies on an existing external blockchain for its security.
+Digital signature:: ((("digital signatures")))A digital signature is a mathematical scheme for verifying the authenticity of digital messages or documents. A valid digital signature gives a recipient reason to believe that the message was created by a known sender, that the sender cannot deny having sent the message, and that the message was not altered in transit.
 
-Transaction:: A data structure that records the transfer of control over some funds (e.g. some bitcoin). The Lightning Network relies moreover on Bitcoin transactions (or those of another blockchain) to track control of funds.
+Hash function:: ((("hash function, defined")))A cryptographic hash function is a mathematical algorithm that maps data of arbitrary size to a bit string of a fixed size (a hash) and is designed to be a one-way function, that is, a function which is infeasible to invert.
 
-Payment:: When value is exchanged on the Lightning Network we call this a payment as the term Transaction is the technical term for entries on the Bitcoin Blockchain.
+Node:: ((("node, defined")))A computer that participates in a network. A Lightning node is a computer that participates in the Lightning Network. A Bitcoin node is a computer that participates in the Bitcoin network. Typically an LN user will run a Lightning node _and_ a Bitcoin node.
 
-Payment Channel:: A _financial relationship_ between two nodes on the Lightning Network, typically implemented by multi-signature Bitcoin transactions that share control over bitcoin between the two Lightning nodes.
+On-chain versus off-chain:: ((("on-chain payment","defined")))A payment is _on-chain_ if it is recorded as a transaction on the Bitcoin (or other underlying) blockchain. ((("off-chain payment")))Payments sent via payment channels between Lightning nodes, and which are not visible in the underlying blockchain, are called _off-chain_ payments. Usually in the Lightning Network, the only on-chain transactions are those used to open and close a Lightning payment channel. A third type of channel modifying transaction exists, called splicing, which can be used to add/remove the amount of funds committed in a channel.
 
-On-Chain vs. Off-Chain:: A payment is "on-chain" if it is recorded as a transaction on the Bitcoin (or other underlying) blockchain. Payments sent via payment channels between Lightning nodes, and which are not visible in the underlying blockchain, are called "off-chain" payments.
+Payment:: ((("payment","defined")))When value is exchanged on the Lightning Network, we call this a "payment" as compared to a "transaction" on the Bitcoin blockchain.
 
-More detailed definitions of these and many other terms can be found in the <<glossary>>. Throughout this book we will explain what these concepts mean and how these technologies actually work.
+Payment channel:: ((("payment channel", seealso="channel entries")))A _financial relationship_ between two nodes on the Lightning Network, typically implemented by multisignature Bitcoin transactions that share control over bitcoin between the two Lightning nodes.
+
+Routing versus sending:: ((("routing","sending versus")))((("sending, routing versus")))Unlike Bitcoin where transactions are "sent" by broadcasting them to everyone, Lightning is a routed network where payments are "routed" across one or more payment channels following a _path_ from sender to recipient.
+
+Transaction:: ((("transaction, defined")))A data structure that records the transfer of control over some funds (e.g., some bitcoin). The Lightning Network relies on Bitcoin transactions (or those of another blockchain) to track control of funds.
+
+More detailed definitions of these and many other terms can be found in the <<glossary>>. Throughout this book, we will explain what these concepts mean and how these technologies actually work.
 
 [TIP]
 ====
-Throughout this book you will see "Bitcoin" with the first letter capitalized, which refers to the _Bitcoin System_ and is a proper noun. You will also see "bitcoin", with a lower-case _b_, which refers to the currency unit.
+Throughout this book, you will see "Bitcoin" with the first letter capitalized, which refers to the _Bitcoin system_ and is a proper noun. You will also see "bitcoin," with a lowercase _b_, which refers to the currency unit. Each bitcoin is further subdivided into 100 million units each called a "satoshi" (singular) or "satoshis" (plural).(((range="endofrange", startref="ix_01_introduction-asciidoc1")))
 ====
 
-=== What is the Lightning Network?
+Now that you're familiar with these basic terms, let's move to a concept you are already comfortable with: trust.
 
-The Lightning Network is a network that operates as a "second layer" protocol on top of Bitcoin and other blockchains. The Lightning Network enables fast, secure, private, trustless, and permissionless payments. Here are some of the features of the Lightning Network:
+
+=== Trust in Decentralized Networks
+
+((("Lightning Network (generally)","trust in decentralized networks")))((("trustless systems","trust in decentralized networks")))You will often hear people calling Bitcoin and the Lightning Network "trustless." At first glance this is confusing. After all, isn't trust a good thing? Banks even use it in their names! Isn't a "trustless" system, a system devoid of trust, a bad thing?
+
+The use of the word "trustless" is intended to convey the ability to operate without _needing_ trust in the other participants in the system. In a decentralized system like Bitcoin, you can always choose to transact with someone you trust. However, the system ensures you can't be cheated even if you can't trust the other party in a transaction. Trust is a nice-to-have instead of a must-have property of the system.
+
+Contrast that to traditional systems like banking where you must place your trust in a third party, since it controls your money. If the bank violates your trust, you may be able to find some recourse from a regulator or court, but at an enormous cost of time, money, and effort.
+
+Trustless does not mean devoid of trust. It means that trust is not a necessary prerequisite to all transactions and that you can transact even with people you don't trust because the system prevents cheating.
+
+Before we get into how the Lightning Network works, it's important to understand one basic concept that underlies Bitcoin, the Lightning Network, and many other such systems: something we call a _fairness protocol_. A fairness protocol is a way to achieve fair outcomes between participants, who do not need to trust each other, without the need for a central authority, and it is the backbone of decentralized systems like Bitcoin.
+
+
+=== Fairness Without Central Authority
+
+((("fairness, ensuring")))((("Lightning Network (generally)","fairness without central authority")))When people have competing interests, how can they establish enough trust to engage in some cooperative or transactional behavior? The answer to this question lies at the core of several scientific and humanistic disciplines, such as economics, sociology, behavioral psychology, and mathematics. Some of those disciplines give us "soft" answers that depend on concepts such as reputation, fairness, morality, and even religion. Other disciplines give us concrete answers that depend only on the assumption that the participants in these interactions will act rationally, with their self-interest as the main objective.
+
+In broad terms, there are a handful of ways to ensure fair outcomes in interactions between individuals who may have competing interests:
+
+Require trust:: You only interact with people whom you already trust, due to prior interactions, reputation, or familial relationships. This works well enough at small scale, especially within families and small groups, that it is the most common basis for cooperative behavior. Unfortunately, it doesn't scale and it suffers from tribalist (in-group) bias.
+
+Rule of law:: Establish rules for interactions that are enforced by an institution. This scales better, but it cannot scale globally due to differences in customs and traditions, as well as the inability to scale the institutions of enforcement. One nasty side effect of this solution is that the institutions become more and more powerful as they get bigger and that may lead to corruption.
+
+Trusted third parties:: Put an intermediary in every interaction to enforce fairness. Combined with the "rule of law" to provide oversight of intermediaries, this scales better, but suffers from the same imbalance of power: the intermediaries get very powerful and may attract corruption. Concentration of power leads to systemic risk and systemic failure ("too big to fail").
+
+Game theoretical fairness protocols:: This last category emerges from the combination of the internet and cryptography and is the subject of this section. Let's see how it works and what its advantages and disadvantages are.
+
+==== Trusted Protocols Without Intermediaries
+
+((("fairness protocol","trusted protocols without intermediaries")))Cryptographic systems like Bitcoin and the Lightning Network are systems that allow you to transact with people (and computers) that you don't trust. This is often referred to as "trustless" operation, even though it is not actually trustless. You have to trust in the software that you run, and you have to trust that the protocol implemented by that software will result in fair outcomes.
+
+The big distinction between a cryptographic system like this and a traditional financial system is that in traditional finance you have a _trusted third party_, for example a bank, to ensure that outcomes are fair. A significant problem with such systems is that they give too much power to the third party, and they are also vulnerable to a _single point of failure_. If the trusted third party itself violates trust or attempts to cheat, the basis of trust breaks.
+
+As you study cryptographic systems, you will notice a certain pattern: instead of relying on a trusted third party, these systems attempt to prevent unfair outcomes by using a system of incentives and disincentives. In cryptographic systems you place trust in the ((("protocol, defined")))_protocol_, which is effectively a system with a set of rules that, if properly designed, will correctly apply the desired incentives and disincentives. The advantage of this approach is twofold: not only do you avoid trusting a third party, you also reduce the need to enforce fair outcomes. So long as the participants follow the agreed protocol and stay within the system, the incentive mechanism in that protocol achieves fair outcomes without enforcement.
+
+((("game theory")))The use of incentives and disincentives to achieve fair outcomes is one aspect of a branch of mathematics called _game theory_, which studies "models of strategic interaction among rational decision makers."footnote:[The Wikipedia https://en.wikipedia.org/wiki/Game_theory[entry on game theory] provides more information.] Cryptographic systems that control financial interactions between participants, such as Bitcoin and the Lightning Network, rely heavily on game theory to prevent participants from cheating and allow participants who don't trust each other to achieve fair outcomes.
+
+While game theory and its use in cryptographic systems may appear confounding and unfamiliar at first, chances are you're already familiar with these systems in your everyday life; you just don't recognize them yet. In the following section we'll use a simple example from childhood to help us identify the basic pattern. Once you understand the basic pattern, you will see it everywhere in the blockchain space and you will come to recognize it quickly and intuitively.
+
+((("fairness protocol", id="ix_01_introduction-asciidoc2", range="startofrange")))In this book, we call this pattern a ((("fairness protocol","defined")))_fairness protocol_, defined as a process that uses a system of incentives and/or disincentives to ensure fair outcomes for participants who don't trust each other. Enforcement of a fairness protocol is only necessary to ensure that the participants can't escape the incentives or disincentives.
+
+==== A Fairness Protocol in Action
+
+((("fairness protocol","real-world example")))Let's look at an example of a fairness protocol that you may already be familiar with.
+
+Imagine a family lunch, with a parent and two children. The children are fussy eaters and the only thing they will agree to eat is fried potatoes. The parent has prepared a bowl of fried potatoes ("french fries" or "chips" depending on which English dialect you use). The two siblings must share the plate of chips. The parent must ensure a fair distribution of chips to each child; otherwise, the parent will have to hear constant complaining (maybe all day), and there's always a possibility of an unfair situation escalating to violence. What is a parent to do?
+
+There are a few different ways that fairness can be achieved in this strategic interaction between two siblings that do not trust each other and have competing interests. The naive but commonly used method is for the parent to use their authority as a trusted third party: they split the bowl of chips into two servings. This is similar to traditional finance, where a bank, accountant, or lawyer acts as a trusted third party to prevent any cheating between two parties who want to transact.
+
+The problem with this scenario is that it vests a lot of power and responsibility in the hands of the trusted third party. In this example, the parent is fully responsible for the equal allocation of chips, and the parties merely wait, watch, and complain.  The children accuse the parent of playing favorites and not allocating the chips fairly. The siblings fight over the chips, yelling "that chip is bigger!" and dragging the parent into their fight. It sounds awful, doesn't it? Should the parent yell louder? Take all of the chips away? Threaten to never make chips again and let those ungrateful children go hungry?
+
+A much better solution exists: the siblings are taught to play a game called "split and choose." At each lunch one sibling splits the bowl of chips into two servings and the _other_ sibling gets to choose which serving they want. Almost immediately, the siblings figure out the dynamic of this game. If the one splitting makes a mistake or tries to cheat, the other sibling can "punish" them by choosing the bigger bowl. It is in the best interest of both siblings, but especially the one splitting the bowl, to play fair. Only the cheater loses in this scenario. The parent doesn't even have to use their authority or enforce fairness. All the parent has to do is _enforce the protocol_; as long as the siblings cannot escape their assigned roles of "splitter" and "chooser," the protocol itself ensures a fair outcome without the need for any intervention. The parent can't play favorites or distort the outcome.
+
+[WARNING]
+====
+While the infamous chip battles of the 1980s neatly illustrate the point, any similarity between the preceding scenario and any of the authors' actual childhood experiences with their cousins is entirely coincidental...or is it?
+====
+
+==== Security Primitives as Building Blocks
+
+((("fairness protocol","security primitives as building blocks")))((("security primitives")))In order for a fairness protocol like this to work, there need to be certain guarantees, or _security primitives_, that can be combined to ensure enforcement. The first security primitive is _strict time ordering/sequencing_: the "splitting" action must happen before the "choosing" action. It's not immediately obvious, but unless you can guarantee that action A happens before action B, then the protocol falls apart. The second security primitive is _commitment with nonrepudiation_. Each sibling must commit to their choice of role: either splitter or chooser. Also, once the splitting has been completed, the splitter is committed to the split they created—they cannot repudiate that choice and go try again.
+
+Cryptographic systems offer a number of security primitives that can be combined in different ways to construct a fairness protocol. In addition to sequencing and commitment, we can also use many other tools:
+
+- Hash functions to fingerprint data, as a form of commitment, or as the basis for a digital signature
+- Digital signatures for authentication, nonrepudiation, and proof of ownership of a secret
+- Encryption/decryption to restrict access to information to authorized participants only
+
+This is only a small list of a whole "menagerie" of security and cryptographic primitives that are in use. More basic primitives and combinations are invented all the time.
+
+In our real-life example, we saw one form of fairness protocol called "split and choose." This is just one of a myriad different fairness protocols that can be built by combining the building blocks of security primitives in different ways. But the basic pattern is always the same: two or more participants interact without trusting each other by engaging in a series of steps that are part of an agreed protocol. The protocol's steps arrange incentives and disincentives to ensure that if the participants are rational, cheating is counterproductive and fairness is the automatic outcome. Enforcement is not necessary to get fair outcomes—it is only necessary to keep the participants from breaking out of the agreed protocol.
+
+Now that you understand this basic pattern, you will start seeing it everywhere in Bitcoin, the Lightning Network, and many other systems. Let's look at some specific examples next.
+
+==== Example of the Fairness Protocol
+
+((("fairness protocol","Proof of Work example")))((("PoW (Proof of Work) algorithm")))((("Proof of Work (PoW) algorithm")))The most prominent example of a fairness protocol is Bitcoin's consensus algorithm, Proof of Work (PoW). In Bitcoin, miners compete to verify transactions and aggregate them in blocks. To ensure that the miners do not cheat, without entrusting them with authority, Bitcoin uses a system of incentives and disincentives. Miners have to use electricity and dedicate hardware doing "work" that is embedded as a "proof" inside every block. This is achieved because of a property of hash functions where the output value is randomly distributed across the entire range of possible outputs. If miners succeed in producing a valid block fast enough, they are rewarded by earning the block reward for that block. Forcing miners to use a lot of electricity before the network considers their block means that they have an incentive to correctly validate the transactions in the block. If they cheat or make any kind of mistake, their block is rejected and the electricity they used to "prove" it is wasted. No one needs to force miners to produce valid blocks; the reward and punishment incentivize them to do so. All the protocol needs to do is ensure that only valid blocks with Proof of Work are accepted.
+
+The fairness protocol pattern can also be found in many different aspects of the Lightning Network:
+
+* Those who fund channels make sure that they have a refund transaction signed before they publish the funding transaction.
+
+* Whenever a channel is moved to a new state, the old state is "revoked" by ensuring that if anyone tries to broadcast it, they lose the entire balance and get punished.
+
+* Those who forward payments know that if they commit funds forward, they can either get a refund or get paid by the node preceding them.
+
+Again and again, we see this pattern. Fair outcomes are not enforced by any authority. They emerge as the natural consequence of a protocol that rewards fairness and punishes cheating, a fairness protocol that harnesses self-interest by directing it toward fair outcomes.
+
+Bitcoin and the Lightning Network are both implementations of fairness protocols. So why do we need the Lightning Network? Isn't Bitcoin enough?(((range="endofrange", startref="ix_01_introduction-asciidoc2")))
+
+
+=== Motivation for the Lightning Network
+
+((("Lightning Network (generally)","motivation for", id="ix_01_introduction-asciidoc3", range="startofrange")))Bitcoin is a system that records transactions on a globally replicated public ledger. Every transaction is seen, validated, and stored by every participating computer. As you can imagine, this generates a lot of data and is difficult to scale.
+
+As Bitcoin and the demand for transactions grew, the number of transactions in each block increased until it eventually reached the block size limit.
+Once blocks are "full," excess transactions are left to wait in a queue. Many users will increase the fees they're willing to pay to buy space for their transactions in the next block.
+
+If demand continues to outpace the capacity of the network, an increasing number of users' transactions are left waiting unconfirmed. Competition for fees also increases the cost of each transaction, making many smaller-value transactions (e.g., microtransactions) completely uneconomical during periods of particularly high demand.
+
+To solve this problem, we could increase the block size limit to create space for more transactions. An increase in the "supply" of block space will lead to a lower price equilibrium for transaction fees.
+
+However, increasing block size shifts the cost to node operators and requires them to expend more resources to validate and store the blockchain. Because blockchains are gossip protocols, each node is required to know and validate every single transaction that occurs on the network. Furthermore, once validated, each transaction and block must be propagated to the node's "neighbors," multiplying the bandwidth requirements. As such, the greater the block size, the greater the bandwidth, processing, and storage requirements for each individual node. Increasing transaction capacity in this way has the undesirable effect of centralizing the system by reducing the number of nodes and node operators. Since node operators are not compensated for running nodes, if nodes are very expensive to run, only a few well-funded node operators will continue to run nodes.
+
+==== Scaling Blockchains
+
+((("blockchain","scaling", id="ix_01_introduction-asciidoc4", range="startofrange")))((("Lightning Network (generally)","scaling blockchains", id="ix_01_introduction-asciidoc5", range="startofrange")))The side effects of increasing the block size or decreasing the block time with respect to centralization of the network are severe, as a few calculations with the numbers show.
+
+Let us assume the usage of Bitcoin grows so that the network has to process 40,000 transactions per second, which is the approximate transaction processing level of the Visa network during peak usage.
+
+Assuming 250 bytes on average per transaction, this would result in a data stream of 10 megabytes per second (MBps) or 80 megabits per second (Mbps) just to be able to receive all the transactions.
+This does not include the traffic overhead of forwarding the transaction information to other peers.
+While 10 MBps does not seem extreme in the context of high-speed fiber optic and 5G mobile speeds, it would effectively exclude anyone who cannot meet this requirement from running a node, especially in countries where high-performance internet is not affordable or widely available.
+
+Users also have many other demands on their bandwidth and cannot be expected to expend this much only to receive transactions.
+
+Furthermore, storing this information locally would result in 864 gigabytes per day. This is roughly one terabyte of data, or the size of a hard drive.
+
+
+Verifying 40,000 Elliptic Curve Digital Signature Algorithm (ECDSA) signatures per second is also barely feasible (see https://bitcoin.stackexchange.com/questions/95339/how-many-bitcoin-transactions-can-be-verified-per-second[this article on StackExchange]), making the _initial block download (IBD)_ of the Bitcoin blockchain  (synchronizing and verifying everything starting from the genesis block) almost impossible without very expensive hardware.
+
+While 40,000 transactions per second seems like a lot, it only achieves parity with traditional financial payment networks at peak times. Innovations in machine-to-machine payments, microtransactions, and other applications are likely to push demand to many orders higher than that.
+
+Simply put: you can't scale a blockchain to validate the entire world's transactions in a decentralized way.
+
+_But what if each node wasn't required to know and validate every single transaction? What if there was a way to have scalable off-chain transactions, without losing the security of the Bitcoin network?_
+
+In February 2015, Joseph Poon and Thaddeus Dryja proposed a possible solution to the Bitcoin scalability problem, with the publication of "The Bitcoin Lightning Network: Scalable Off-Chain Instant Payments."footnote:[Joseph Poon and Thaddeus Dryja. "The Bitcoin Lightning Network: Scalable Off-Chain Instant Payments." DRAFT Version 0.5.9.2. January 14, 2016. https://lightning.network/lightning-network-paper.pdf[].]
+
+In the (now outdated) whitepaper, Poon and Dryja estimate that in order for Bitcoin to reach the 47,000 transactions per second processed at peak by Visa, it would require 8 GB blocks.
+This would make running a node completely untenable for anyone but large-scale enterprises and industrial-grade operations.
+The result would be a network in which only a few users could actually validate the state of the ledger.
+Bitcoin relies on users validating the ledger for themselves, without explicitly trusting third parties, in order to stay decentralized.
+Pricing users out of running nodes would force the average user to trust third parties to discover the state of the ledger, ultimately breaking the trust model of Bitcoin.
+
+The Lightning Network proposes a new network, a second layer, where users can make payments to each other peer-to-peer, without the necessity of publishing a transaction to the Bitcoin blockchain for each payment.
+Users may pay each other on the Lightning Network as many times as they want, without creating additional Bitcoin transactions or incurring on-chain fees.
+They only make use of the Bitcoin blockchain to load bitcoin onto the Lightning Network initially and to _settle_, that is, to remove bitcoin from the Lightning Network.
+The result is that many more Bitcoin payments can take place off-chain, with only the initial loading and final settlement transactions needing to be validated and stored by Bitcoin nodes.
+Aside from reducing the burden on nodes, payments on the Lightning Network are cheaper for users because they do not need to pay blockchain fees, and more private for users because they are not published to all participants of the network and furthermore are not stored permanently.
+
+While the Lightning Network was initially conceived for Bitcoin, it can be implemented on any blockchain that meets some basic technical requirements. Other blockchains, such as Litecoin, already support the Lightning Network. Additionally, several other blockchains are developing similar second layer or "layer 2" solutions to help them scale(((range="endofrange", startref="ix_01_introduction-asciidoc5")))(((range="endofrange", startref="ix_01_introduction-asciidoc4"))).(((range="endofrange", startref="ix_01_introduction-asciidoc3")))
+
+=== The Lightning Network's Defining Features
+
+((("Lightning Network (generally)","defining features")))The Lightning Network is a network that operates as a second layer protocol on top of Bitcoin and other blockchains. The Lightning Network enables fast, secure, private, trustless, and permissionless payments. Here are some of the features of the Lightning Network:
 
  * Users of the Lightning Network can route payments to each other for low cost and in real time.
  * Users who exchange value over the Lightning Network do not need to wait for block confirmations for payments.
  * Once a payment on the Lightning Network has completed, usually within a few seconds, it is final and cannot be reversed. Like a Bitcoin transaction, a payment on the Lightning Network can only be refunded by the recipient.
- * While "on-chain" Bitcoin transactions are broadcast and verified by all nodes in the network, payments routed on the Lightning Network are transmitted between pairs of nodes and are not visible to everyone, resulting in much greater privacy. 
- * Unlike transactions on the Bitcoin Network, payments routed on the Lightning Network do not need to be stored permanently. Lightning thus uses fewer resources, hence it is cheaper. This property also has benefits for privacy. 
- * The Lightning Network uses onion routing, similar to the protocol used by The Onion Router (TOR) privacy network, so that even the nodes involved in routing a payment are only directly aware of their predecessor and successor in the payment route.
+ * Whereas on-chain Bitcoin transactions are broadcast and verified by all nodes in the network, payments routed on the Lightning Network are transmitted between pairs of nodes and are not visible to everyone, resulting in much greater privacy.
+ * Unlike transactions on the Bitcoin network, payments routed on the Lightning Network do not need to be stored permanently. Lightning thus uses fewer resources and hence is cheaper. This property also has benefits for privacy.
+ * The Lightning Network uses onion routing, similar to the protocol used by The Onion Router (Tor) privacy network, so that even the nodes involved in routing a payment are only directly aware of their predecessor and successor in the payment route.
+ * When used on top of Bitcoin, the Lightning Network uses real bitcoin, which is always in the possession (custody) and full control of the user. Lightning is not a separate token or coin, it _is_ Bitcoin.
+
 
 [[user-stories]]
 === Lightning Network Use Cases, Users, and Their Stories
 
-As an electronic cash system, Bitcoin preserves the three most important properties of money (medium of exchange, store of value, and unit of account). Other relevant properties of digital payment systems include the ability of third parties to use them as a method of control and/or a tool of surveillance.
-The invention of money (and in particular Bitcoin) was primarily made to facilitate trade and enable the exchange of value between people. However, without the Lightning Network (or another second layer or scaling solution), it would be infeasible for millions of people to concurrently use Bitcoin as a medium of exchange because the network itself would become overloaded, slow, and costly.
+((("Lightning Network (generally)","use cases and users")))To better understand how the Lightning Network actually works, and why people use it, we'll be following a number of users and their stories.
 
-To date, Bitcoin is the longest running, most secure cryptocurrency or electronic cash system and many people believe it represents the most stable store of value of all of the current cryptocurrencies. The Lightning Network allows people to send and receive bitcoin, without the overhead associated with on-chain transactions. This might seem confusing at first. You might be wondering how can the Lightning Network actually achieve this? While we could explain the network in computer science terms, it will be much easier to understand if we examine it from the perspective of people using it. In our examples, some of the people have already used Bitcoin and others are completely new to the Bitcoin network. Each of the people and their stories, as listed here, illustrates one or more specific use cases. We'll be revisiting them throughout this book:
+In our examples, some of the people have already used Bitcoin and others are completely new to the Bitcoin network. Each person and their story, as listed here, illustrate one or more specific use cases. We'll be revisiting them throughout this book:
 
-consumer::
+Consumer::
 Alice is a Bitcoin user who wants to make fast, secure, cheap, and private payments for small retail purchases. She buys coffee with bitcoin, using the Lightning Network.
 
-merchant::
-Bob owns a coffee shop, "Bob's Cafe". "On-chain" bitcoin payments don't scale for small amounts like a cup of coffee, so he uses the Lightning Network to accept bitcoin payments almost instantaneously and for low fees.
+Merchant::
+Bob owns a coffee shop, "Bob's Cafe." On-chain Bitcoin payments don't scale for small amounts like a cup of coffee, so he uses the Lightning Network to accept Bitcoin payments almost instantaneously and for low fees.
 
-web designer::
-Saanvi is a web designer and developer in Bangalore, India. She accepts bitcoin for her work, but would prefer to get paid more frequently and so uses the Lightning Network to get paid incrementally for each small milestone she completes. With the Lightning Network, she can do more small jobs for more clients without worrying about fees or delays.
+Software service business::
+Chan is a Chinese entrepreneur who sells information services related to the Lightning Network, as well as Bitcoin and other cryptocurrencies. Chan is selling these information services over the internet by implementing micropayments over the Lightning Network. Additionally, Chan has implemented a liquidity provider service that rents inbound channel capacity on the Lightning Network, charging a small bitcoin fee for each rental period.
 
-content creator / curator::
-John is a 9-year-old boy from New Zealand, who wanted a games console just like his friends. However, his dad told him that in order to buy it, he had to earn the money himself. Now John is an aspiring artist, so he knows that while he is still improving, he can't charge much for his artwork. After learning about Bitcoin, he managed to set up a website to sell his drawings across the internet. By using the Lightning Network, John was able to charge as little as $1 for one of his drawings, which would normally be considered a micro-payment and, as such, not possible with other payment methods. Furthermore, by using a global currency such as bitcoin, John was able to sell his artwork to customers all over the world and, in the end, buy the games console he so desperately wanted.
+Gamer::
+Dina is a teenage gamer from Russia. She plays many different computer games, but her favorite ones are those that have an "in-game economy" based on real money. As she plays games, she also earns money by acquiring and selling virtual in-game items. The Lightning Network allows her to transact in small amounts for in-game items as well as earn small amounts for completing quests.
 
-gamer::
-Gloria is a teenage gamer from the Philippines. She plays many different computer games, but her favorite ones are those that have an "in-game economy" based on real money. As she plays games, she also earns money by acquiring and selling virtual in-game items. The Lightning Network allows her to transact in small amounts for in-game items as well as earn small amounts for completing quests.
+=== Conclusion
 
-migrant::
-Farel is an immigrant who works in the Middle East and sends money home to his family in Indonesia. Remittance companies and banks charge high fees, and Farel prefers to send smaller amounts more often. Using the Lightning Network, Farel can send bitcoin as often as he wants, with negligible fees.
+In this chapter, we talked about the fundamental concept that underlies both Bitcoin and the Lightning Network: the fairness protocol.
 
-software service business::
-Wei is an entrepreneur who sells information services related to the Lightning Network, as well as Bitcoin and other cryptocurrencies. Wei is monetizing his API endpoints by implementing micro-payments over the Lightning Network. Additionally, Wei has implemented a liquidity provider service that rents inbound channel capacity on the Lightning Network, charging a small bitcoin fee for each rental period.
+We looked at the history of the Lightning Network and the motivations behind second layer scaling solutions for Bitcoin and other blockchain-based networks.
 
-=== Chapter Summary
+We learned basic terminology including node, payment channel, on-chain transactions, and off-chain payments.
 
-In this chapter we looked at the history of the Lightning Network and the motivations behind second layer scaling solutions for Bitcoin and other blockchain based networks. We learned basic terminology including node, payment channel, on-chain transactions, and off-chain payments. Finally, we met Alice, Bob, Saanvi, John, Gloria, Farel, and Wei who we'll be following throughout the rest of the book. In the next chapter we'll meet Alice and walk through her thought process as she selects a Lightning wallet and prepares to make her first Lightning payment to buy a cup of coffee from Bob's Cafe.
+Finally, we met Alice, Bob, Chan, and Dina, whom we'll be following throughout the rest of the book.(((range="endofrange", startref="ix_01_introduction-asciidoc0"))) In the next chapter, we'll meet Alice and walk through her thought process as she selects a Lightning wallet and prepares to make her first Lightning payment to buy a cup of coffee from Bob's Cafe.((("Bitcoin (system)","Lightning Network compared to", see="Bitcoin–Lightning Network comparisons")))((("channel", see="payment channel")))((("containers", see="Docker containers")))((("delivering payment", see="payment delivery")))((("DoS attacks", see="denial-of-service attacks")))((("encrypted message transport", see="Lightning encrypted transport protocol")))((("future issues", see="innovations in Lightning")))((("HTLCs", see="hash time-locked contracts")))((("invoices", see="Lightning invoices")))((("Lightning Network (generally)","Bitcoin compared to", see="Bitcoin–Lightning Network comparisons")))((("Lightning Network (generally)","invoices", see="Lightning invoices")))((("Lightning Network (generally)","network architecture", see="architecture, Lightning Network")))((("Lightning Network node", see="Lightning node entries")))((("Lightning payment requests", see="Lightning invoices")))((("LN node", see="Lightning node entries")))((("LND node project", see="Lightning Network Daemon node project")))((("message transport", see="Lightning encrypted transport protocol")))((("node", see="Lightning node entries")))((("payment channel","routing on network of", see="routing")))((("payment requests", see="Lightning invoices")))((("payment routing", see="routing")))((("privacy", see="breaches of privacy")))((("privacy", see="security and privacy")))((("private channels", see="unannounced channels")))((("TLV", see="Type-Length-Value")))((("wallet", see="Lightning wallet")))

02_getting_started.asciidoc

@@ -1,145 +1,192 @@
 [[getting-started]]
 == Getting Started
 
-In this chapter, we will begin where most people start when encountering the Lightning Network for the first time - choosing software to participate in the Lightning Network economy. We will examine the choices of two users who represent a common use-case for the Lightning Network and learn by example. Alice, a coffee shop customer, will be using a Lightning wallet on her mobile device to buy coffee from Bob's Cafe. Bob, a merchant, will be using a Lightning node and wallet to run a point-of-sale system at his cafe, so he can accept payments over the Lightning Network.
+
+((("Lightning Network (generally)","example", id="ix_02_getting_started-asciidoc0", range="startofrange")))In this chapter, we will begin where most people start when encountering the Lightning Network for the first time—choosing software to participate in the LN economy. We will examine the choices of two users who represent a common use case for the Lightning Network and learn by example. Alice, a coffee shop customer, will be using a Lightning wallet on her mobile device to buy coffee from Bob's Cafe. Bob, a merchant, will be using a Lightning node and wallet to run a point-of-sale system at his cafe, so he can accept payments over the Lightning Network.
+
+=== Alice's First Lightning Wallet
+
+((("Lightning Network (generally)","Lightning wallet")))((("Lightning wallet")))Alice is a longtime Bitcoin user. We first met Alice in Chapter 1 of _Mastering Bitcoin_,footnote:[Andreas M. Antonopoulos, _Mastering Bitcoin_, 2nd Edition, https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch01.asciidoc[Chapter 1] (O'Reilly)] when she bought a cup of coffee from Bob's cafe using a Bitcoin transaction. If you are not yet familiar with how Bitcoin transactions work or need a refresher, please read _Mastering Bitcoin_ or the summary in <<bitcoin_fundamentals_review>>.
+
+Alice recently learned that Bob's Cafe just started accepting LN payments! Alice is eager to learn about and experiment with the Lightning Network; she wants to be one of Bob's first LN customers. To do this, first, Alice has to select a Lightning wallet that meets her needs.
+
+Alice does not want to entrust custody of her bitcoin to third parties. She has learned enough about cryptocurrency to know how to use a wallet. She also wants a mobile wallet so that she can use it for small payments on the go, so she chooses the _Eclair_ wallet, a popular noncustodial mobile Lightning wallet. Let's learn more about how and why she's made these choices.
 
 === Lightning Nodes
 
-The Lightning Network is accessed via software applications that can speak the Lightning Network protocol. A _Lightning Network Node_ (or simply "node") is a software application that has three important characteristics. First, nodes must communicate on a peer-to-peer basis with other Lightning nodes creating the network. Nodes also include "wallet" functionality, so they can send and receive payments over the Lightning Network as well as on the Bitcoin network. Finally, Lightning nodes also need access to the Bitcoin blockchain (or other blockchains for other cryptocurrencies) to secure the funds used for payments.
+((("Lightning node operation")))The Lightning Network is accessed via software applications that can speak the LN protocol. A _Lightning Network node_ (LN node or simply node) is a software application that has three important characteristics. First, Lightning nodes are wallets, so they send and receive payments over the Lightning Network as well as on the Bitcoin network. Second, nodes must communicate on a peer-to-peer basis with other Lightning nodes creating the network. Finally, Lightning nodes also need access to the Bitcoin blockchain (or other blockchains for other cryptocurrencies) to secure the funds used for payments.
 
-Users have the highest degree of control by running their own Bitcoin node and Lightning node. However, Lightning nodes can also use a lightweight Bitcoin client (commonly referred to as Simplified Payment Verification (SPV)) to interact with the Bitcoin blockchain.
+Users have the highest degree of control by running their own Bitcoin node and Lightning node. However, ((("simplified payment verification (SPV)")))((("SPV (simplified payment verification)")))Lightning nodes can also use a lightweight Bitcoin client, commonly referred to as simplified payment verification (SPV), to interact with the Bitcoin blockchain.
 
-=== Lightning Explorer
+[[ln_explorer]]
+=== Lightning Explorers
 
-A Lightning network explorer is an useful tool to show the statistics of nodes, channels and network capacity.
+((("Lightning explorers")))LN explorers are useful tools to show the statistics of nodes, channels, and network capacity.
 
-The below is an inexhaustive list (in alphanumerical order):
+Following is an inexhaustive list:
 
-* https://1ml.com/, 1ML Lightning explorer
-* https://explorer.acinq.co/, ACINQ's Lightning explorer, with fancy visualization
-* https://lightblock.me/, Lightblock Lightning explorer
-* https://hashxp.org/lightning/node/, hashXP Lightning explorer
+* https://1ml.com[1ML Lightning explorer] 
+* https://explorer.acinq.co[ACINQ's Lightning explorer], with fancy visualization 
+* https://amboss.space[Amboss Space Lightning explorer], with community metrics and intuitive pass:[<span class="keep-together">visualizations</span>]
+* https://ln.bigsun.xyz[Fiatjaf's Lightning explorer] with many diagrams
+*  https://hashxp.org/lightning/node[hashXP Lightning explorer]
 
 [TIP]
 ====
-Note that if using Lightning explorers, just like in existing block explorers,
-the privacy can be a concern because, if users are careless, the website may track their IP addresses and collect their behavior records (for example, the nodes users are interested in).
+Note that when using Lightning explorers, just like with other block explorers, privacy can be a concern.
+If users are careless, the website may track their IP addresses and collect their behavior records (for example, the nodes users are interested in).
 
-Also what should be noted is that, as there is no global consensus of the current Lightning graph, nor the current state of any existing channel policy, users should never rely on Lightning explorers to retrieve the most updated information. That is, Lightning explorers should only be used to gather loosely statistics of Lightning Network.
+Also, it should be noted that because there is no global consensus of the current Lightning graph or the current state of any existing channel policy, users should never rely on Lightning explorers to retrieve the most current information.
+Furthermore, as users open, close, and update channels, the graph will change and individual Lightning explorers may not be up to date.
+Use Lightning explorers to visualize the network or gather information, but not as an authoritative source of what is happening on the Lightning Network.
+To have an authoritative view of the Lightning Network, run your own Lightning node that will build a channel graph and collect various statistics, which you can view with a web-based interface.
 ====
 
 === Lightning Wallets
 
-The term "Lightning Wallet" is somewhat ambiguous, as it can describe a broad variety of components combined with some user interface. The most common components of lightning wallet software include:
+((("Lightning wallet","basics", id="ix_02_getting_started-asciidoc1", range="startofrange")))The term _Lightning wallet_ is somewhat ambiguous because it can describe a broad variety of components combined with some user interface. The most common components of Lightning wallet software include:
 
-* A keystore that holds secrets, such as private keys.
-* A Lightning Network node (Lightning node) that communicates on the Peer-to-Peer network, as described previously.
-* A Bitcoin node that stores blockchain data and communicates with other Bitcoin nodes.
-* A database "map" of nodes and channels that are announced on the Lightning network.
-* A "gossip" participant that learns about nodes and channels on the network to construct the "map" of the network.
-* A channel data store containing state about channels on the Lightning Network as well as signatures to settle them.
-* A channel manager that can open and close Lightning Network channels.
-* A path-finding system that can identify a path of connected channels from payment source to payment destination.
+* A keystore that holds secrets, such as private keys
+* An LN node (Lightning node) that communicates on the peer-to-peer network, as described previously
+* A Bitcoin node that stores blockchain data and communicates with other Bitcoin nodes
+* A database "map" of nodes and channels that are announced on the Lightning Network
+* A channel manager that can open and close LN channels
+* A close-up system that can find a path of connected channels from payment source to payment destination
 
-A Lightning wallet may contain all of these functions, acting as a "full" wallet, with no reliance on any third-party services. Or, one or more of these components may rely (partially or entirely) on third-party services that mediate those functions.
+A Lightning wallet may contain all of these functions, acting as a "full" wallet, with no reliance on any third-party services. Or one or more of these components may rely (partially or entirely) on third-party services that mediate those functions.
 
-A key distinction (pun intended) is whether the keystore function is internal or outsourced. In blockchains, control of keys determines custody of funds, as memorialized by the phrase "your keys, your coins; not your keys, not your coins". Any wallet that outsources management of keys is called a "custodial" wallet because a third party acting as custodian has control of the user's funds, not the user himself. A "non-custodial" or "self-custody" wallet, by comparison, is one where the keystore is part of the wallet, and keys are controlled directly by the user.  The term "non-custodial" wallet just implies that the keystore is local and under the user's control. However, one or more of the other wallet components might or might not be outsourced and rely on some trusted third parties even if some of the components (other than the keystore) rely on some trusted third parties.
+A _key_ distinction (pun intended) is whether the keystore function is internal or outsourced. In blockchains, control of keys determines custody of funds, as memorialized by the phrase "your keys, your coins; not your keys, not your coins." ((("custodial wallet")))Any wallet that outsources management of keys is called a _custodial_ wallet because a third party acting as custodian has control of the user's funds, not the user. ((("noncustodial wallet")))A _noncustodial_ or ((("self-custody wallet")))_self-custody_ wallet, by comparison, is one where the keystore is part of the wallet, and keys are controlled directly by the user.  The term noncustodial wallet just implies that the keystore is local and under the user's control. However, one or more of the other wallet components may or may not be outsourced and rely on trusted third parties.
 
-Blockchains, especially open blockchains like Bitcoin, attempt to minimize or eliminate trust in third parties and empower users. This is often called a "trustless" model, though "trust-minimized" is a better term. In such systems, the user trusts the software rules, not third parties. Therefore, the issue of control over keys is a principal consideration when choosing a Lightning wallet.
+Blockchains, especially open blockchains like Bitcoin, attempt to minimize or eliminate trust in third parties and empower users. ((("trustless systems","blockchains as")))This is often called a "trustless" model, though "trust minimized" is a better term. In such systems, the user trusts the software rules, not third parties. Therefore, the issue of control over keys is a principal consideration when choosing a Lightning wallet.
 
-Every other component of a Lightning wallet brings similar considerations of trust. If all the components are under the control of the user, then the amount of trust in third parties is minimized, bringing maximum power to the user. Of course, this brings a direct trade-off, as with that power comes the corresponding responsibility to manage complex software.
+Every other component of a Lightning wallet brings similar considerations of trust. If all the components are under the control of the user, then the amount of trust in third parties is minimized, bringing maximum power to the user. Of course, this brings a direct trade-off because with that power comes the corresponding responsibility to manage complex software.
 
-Every user must consider their own technical skills before deciding what type of Lightning wallet to use. Those with strong technical skills should use a Lightning wallet that puts all of the components under the direct control of the user. Those with fewer technical skills but a desire to control their funds, should choose a _non-custodial_ Lightning wallet.
-Often the trust in those cases relates to privacy.
-If users decide to outsource some functionality to a third party they usually give up some privacy as the third party will learn some information about them.
+Every user must consider their own technical skills before deciding what type of Lightning wallet to use. Those with strong technical skills should use a Lightning wallet that puts all of the components under the direct control of the user. Those with fewer technical skills, but with a desire to control their funds, should choose a noncustodial Lightning wallet.
+Often the trust in these cases relates to privacy.
+If users decide to outsource some functionality to a third party, they usually give up some privacy as the third party will learn some information about them.
 
-Finally, those seeking simplicity and convenience, even at the expense of control and security, may choose a custodial Lightning wallet. This is the least technically challenging option, but it _undermines the trust model of cryptocurrency_ and should therefore be considered only as a stepping stone towards more control and self-reliance.
+Finally, those seeking simplicity and convenience, even at the expense of control and security, may choose a custodial Lightning wallet. This is the least technically challenging option, but it _undermines the trust model of cryptocurrency_ and should therefore be considered only as a stepping stone toward more control and self-reliance.
 
 There are many ways wallets can be characterized or categorized.
 The most important questions to ask about a specific wallet are:
 
-- Does this Lightning wallet have a full Lightning Node or does it use a third-party Lightning Node?
-- Does this Lightning wallet have a full Bitcoin Node or does it use a third-party Bitcoin Node? footnote:[If a Lightning wallet uses a third-party Lightning node, it is this third-party Lightning node who decides how to communicate with Bitcoin. Hence, using a third-party Lightning node implies that you as a wallet user also use a third-party Bitcoin node. Only in the other case, when the Lightning wallet uses its own Lightning node, does the choice "full Bitcoin-node" vs. "third-party Bitcoin node" exist. ]
-- Does this Lightning wallet store its own keys under user control (self-custody) or are the keys held by a third-party custodian?
+. Does this Lightning wallet have a full Lightning node or does it use a third-party Lightning node?
+. Does this Lightning wallet have a full Bitcoin node or does it use a third-party Bitcoin node?
+. Does this Lightning wallet store its own keys under user control (self-custody) or are the keys held by a third-party custodian?
 
-At the highest level of abstraction, questions 1 and 3 are the most elementary ones.
+[TIP]
+====
+If a Lightning wallet uses a third-party Lightning node, it is this third-party Lightning node that decides how to communicate with Bitcoin. Hence, using a third-party Lightning node implies that you are also using a third-party Bitcoin node. Only when the Lightning wallet uses its own Lightning node does the choice between full Bitcoin node and third-party Bitcoin node exist.
+====
+
+At the highest level of abstraction, Questions 1 and 3 are the most elementary ones.
 From these two questions, we can derive four possible categories.
-We can place these four categories into a quadrant as seen in Table <<lnwallet-categories>>.
+We can place these four categories into a quadrant, as seen in <<lnwallet-categories>>.
 But remember that this is just one way of categorizing Lightning wallets.
 
 [[lnwallet-categories]]
-.Lightning Wallets Quadrant
+.Lightning wallets quadrant
 [options="header"]
 |===
-|                        | *Full Lightning Node*      | *3rd-party Lightning Node*
-| *Self-Custody*         | Q1: High technical skill, least trust in 3rd parties, most permissionless | Q2: Below medium technical skills, below medium trust in 3rd parties, requires some permissions
-| *Custodial*            | Q3: Above medium technical skills, above medium trust in 3rd parties, requires some permissions | Q4: Low technical skills, high trust in 3rd parties, least permissionless
+|                        | *Full Lightning node*      | *Third-party Lightning node*
+| *Self-custody*         | Q1: High technical skill, least trust in third parties, most permissionless | Q2: Below medium technical skills, below medium trust in third parties, requires some permissions
+| *Custodial*            | Q3: Above medium technical skills, above medium trust in third parties, requires some permissions | Q4: Low technical skills, high trust in third parties, least permissionless
 |===
 
-Q3, quadrant 3, where a full Lightning node is used but the keys are held by a custodian is currently not common.
-Future wallets from that quadrant would let a user worry about the operational aspects of their node, but then delegate the access to the keys to a third party which may use primarily cold storage.
+Quadrant 3 (Q3), where a full Lightning node is used, but the keys are held by a custodian, is currently not common.
+Future wallets from that quadrant may let a user worry about the operational aspects of their node, but then delegate access to the keys to a third party which primarily uses cold storage.
 
-Lightning wallets can be installed on a variety of devices, including laptops, servers, and mobile devices. To run a full Lightning node (one that participates in "gossip" and creates its own map of the network for path finding and routing) you will need to use a server or desktop computer, as mobile devices and laptops are usually not powerful enough in terms of capacity, processing, battery life, and connectivity.
+Lightning wallets can be installed on a variety of devices, including laptops, servers, and mobile devices. To run a full Lightning node, you will need to use a server or desktop computer, because mobile devices and laptops are usually not powerful enough in terms of capacity, processing, battery life, and connectivity.
 
-The category "Third-party Lightning Nodes" can again be subdivided into:
+The category third-party Lightning nodes can again be subdivided:
 
-- Lightweight: means that the Lightning Node is operated by a third party and that the wallet obtains the required information through an API that connects wallet and third-party Lightning node.
-- None: means that not only is the Lightning Node operated by a third party but most of the wallet is operated by a third party in the cloud such that the wallet does not even need to make calls on a Lightning node API.
+Lightweight:: 
+ This means that the wallet does not operate a Lightning node and thus needs to obtain information about the Lightning Network over the internet from someone else's Lightning node.
+None:: 
+ This means that not only is the Lightning node operated by a third party, but most of the wallet is operated by a third party in the cloud. This is a custodial wallet where someone else controls custody of the funds.
 
-These subcategories are used in Table <<lnwallet-examples>>.
+These subcategories are used in <<lnwallet-examples>>.
 
-Other terms that need explanation in Table <<lnwallet-examples>> in column "Bitcoin Node" are:
+[role="pagebreak-before"]
+Other terms that need explanation in <<lnwallet-examples>> in the column "Bitcoin node" are:
 
-- Neutrino: Neutrino is a specific implementation of a protocol to get data from Bitcoin as outlined in BIP 157 and BIP 158. The Bitcoin Node is run by a third-party and accessed via "neutrino".
-- Electrum: This means the wallet connects to an Electrum Server. The Bitcoin Node is run by a third-party and accessed via the  "Electrum" protocol.
-- Bitcoin Core: implementation of Bitcoin full node
-- btcd: another implementation of Bitcoin full node
+Neutrino::
+ This wallet does not operate a Bitcoin node. Instead, a Bitcoin node operated by someone else (a third party) is accessed via the Neutrino Protocol.
+Electrum::
+ This wallet does not operate a Bitcoin node. Instead, a Bitcoin node operated by someone else (a third party) is accessed via the Electrum Protocol.
+Bitcoin Core::
+ This is an implementation of a Bitcoin node.
+btcd::
+ This is another implementation of a Bitcoin node.
 
-In <<lnwallet-examples>> we see some examples of currently popular Lightning node and wallet applications for different types of devices.
+In <<lnwallet-examples>>, we see some examples of currently popular Lightning node and wallet applications for different types of devices. The list is sorted first by device type and then alphabetically.
 
-// TODO: Add a lot more wallet/node examples, confirm the details for correctness
 [[lnwallet-examples]]
-.Examples of Popular Lightning Wallets
+.Examples of popular Lightning wallets
 [options="header"]
 |===
-| Application    | Device  | Lightning Node | Bitcoin Node          | Keystore
-| lnd            | Server  | Full Node   | Bitcoin Core/btcd     | Self-Custody
-| c-lightning    | Server  | Full Node   | Bitcoin Core          | Self-Custody
-| Eclair Server  | Server  | Full Node   | Bitcoin Core/Electrum | Self-Custody
-| Zap Desktop    | Desktop | Full Node   | Neutrino              | Self-Custody
-| Eclair Mobile  | Mobile  | Lightweight | Electrum              | Self-Custody
-| Breez Wallet   | Mobile  | Full Node   | Neutrino              | Self-Custody
-| Phoenix Wallet | Mobile  | Lightweight | Electrum              | Self-Custody
+| Application    | Device  | Lightning node | Bitcoin node          | Keystore
 | Blue Wallet    | Mobile  | None        | None                  | Custodial
+| Breez Wallet   | Mobile  | Full node   | Neutrino              | Self-custody
+| Eclair Mobile  | Mobile  | Lightweight | Electrum              | Self-custody
+| lntxbot        | Mobile  | None        | None                  | Custodial
+| Muun | Mobile  | Lightweight | Neutrino              | Self-custody
+| Phoenix Wallet | Mobile  | Lightweight | Electrum              | Self-custody
+| Zeus           | Mobile  | Full node   | Bitcoin Core/btcd     | Self-custody
+| Electrum       | Desktop | Full node   | Bitcoin Core/Electrum | Self-custody
+| Zap Desktop    | Desktop | Full node   | Neutrino              | Self-custody
+| c-lightning    | Server  | Full node   | Bitcoin Core          | Self-custody
+| Eclair Server  | Server  | Full node   | Bitcoin Core/Electrum | Self-custody
+| lnd            | Server  | Full node   | Bitcoin Core/btcd     | Self-custody
 |===
 
-=== Balancing complexity and control
+[[testnet-bitcoin]]
+==== Testnet Bitcoin
 
-Lightning wallets have to strike a careful balance between complexity and user control. Those that give the user the most control over their funds, the highest degree of privacy, and the greatest independence from third party services are necessarily more complex and difficult to operate. As the technology advances, some of these trade-offs will become less stark, and users may be able to get more control without more complexity. However, for now, different companies and projects are exploring different positions along this control-complexity spectrum and hoping to find the "sweet spot" for the users they are targeting.
+((("Lightning wallet","testnet bitcoin and")))((("testnet bitcoin (tBTC)")))The Bitcoin system offers an alternative chain for testing purposes called _testnet_, in contrast with the "normal" Bitcoin chain which is referred to as _mainnet_. On testnet, the currency is _testnet bitcoin_ (_tBTC_), which is a worthless copy of bitcoin used exclusively for testing. Every function of Bitcoin is replicated exactly, but the money is worth nothing, so you literally have nothing to lose!
 
-When selecting a wallet, keep in mind that even if you don't see these trade-offs, they still exist. For example, many wallets will attempt to remove the burden of channel management from its users. To do so, they introduce central "hub" nodes that all their wallets connect to automatically. While this trade-off simplifies the user interface and user experience, it introduces a Single Point of Failure (SPoF) as these "hub nodes" become indispensable for the wallet operation. Furthermore, relying on a "hub" like this can reduce user privacy since the hub knows the sender and potentially (if constructing the payment route on behalf of the user) also the recipient of each payment made by the user's wallet.
+Some Lightning wallets can also operate on testnet, allowing you to make Lightning payments with testnet bitcoin, without risking real funds. This is a great way to experiment with Lightning safely. Eclair Mobile, which Alice uses in this chapter, is one example of a Lightning wallet that supports testnet operation.
 
-In the next section, we will return to our first user and walk through her first Lightning wallet setup. She has chosen a wallet that is more sophisticated than the easier custodial wallets. This allows us to show some of the underlying complexity and introduce some of the inner workings of an advanced wallet during our example. You may find that your first ideal wallet is further towards "ease of use", by accepting some of the control and privacy trade-offs. Or perhaps you are more of a "power user" and want to run your own Lightning and Bitcoin nodes as part of your wallet solution.
+You can get some tBTC to play with from a _testnet bitcoin faucet_, which gives out free tBTC on demand. Here are a few testnet faucets:
 
-=== Alice's First Lightning Wallet
+++++
+<ul class="simplelist">
+<li><a href="https://coinfaucet.eu/en/btc-testnet/"><em>https://coinfaucet.eu/en/btc-testnet</em></a></li>
+<li><a href="https://testnet-faucet.mempool.co/"><em>https://testnet-faucet.mempool.co</em></a></li>
+<li><a href="https://bitcoinfaucet.uo1.net/"><em>https://bitcoinfaucet.uo1.net</em></a></li>
+<li><a href="https://testnet.help/en/btcfaucet/testnet"><em>https://testnet.help/en/btcfaucet/testnet</em></a></li>
+</ul>
+++++
 
-Alice is a long time Bitcoin user. We first met Alice in Chapter 1 of _"Mastering Bitcoin"_ footnote:["Mastering Bitcoin 2nd Edition, Chapter 1" Andreas M. Antonopoulos (https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch01.asciidoc).], when she bought a cup of coffee from Bob's cafe using a bitcoin transaction. Now, Alice is eager to learn about and experiment with the Lightning Network. First, she has to select a Lightning wallet that meets her needs.
+All of the examples in this book can be replicated exactly on testnet with tBTC, so you can follow along if you want without risking real money.(((range="endofrange", startref="ix_02_getting_started-asciidoc1")))
 
-Alice does not want to entrust custody of her bitcoin to third parties. She has learned enough about cryptocurrency to know how to use a wallet. She also wants a mobile wallet so that she can use it for small payments on-the-go, so she chooses the _Eclair_ wallet, a popular non-custodial mobile Lightning wallet.
+=== Balancing Complexity and Control
 
-==== Downloading and Installing a Lightning Wallet
+((("Lightning wallet","balancing complexity and control")))Lightning wallets have to strike a careful balance between complexity and user control. Those that give the user the most control over their funds, the highest degree of privacy, and the greatest independence from third-party services are necessarily more complex and difficult to operate. As the technology advances, some of these trade-offs will become less stark, and users may be able to get more control without more complexity. However, for now, different companies and projects are exploring different positions along this control-complexity spectrum, hoping to find the "sweet spot" for the users they are targeting.
 
-When looking for a new cryptocurrency wallet, you must be very careful to select a secure source for the software.
+When selecting a wallet, keep in mind that even if you don't see these trade-offs, they still exist. For example, many wallets will attempt to remove the burden of channel management from their users. To do so, they introduce central _hub nodes_ that all their wallets connect to automatically. While this trade-off simplifies the user interface and user experience, it introduces a single point of failure (SPoF) as these hub nodes become indispensable for the wallet's operation. Furthermore, relying on a "hub" like this can reduce user privacy since the hub knows the sender and potentially (if constructing the payment route on behalf of the user) also the recipient of each payment made by the user's wallet.
 
-Unfortunately, many fake wallet applications will steal your money, and some of these even find their way onto reputable and supposedly vetted software sites like the Apple and Google application stores. Whether you are installing your first or your tenth wallet, always exercise extreme caution. A rogue app cannot only steal any money you entrust it with, but it might also be able to steal keys and passwords from other applications by compromising your mobile device operating system.
+In the next section, we will return to our first user and walk through her first Lightning wallet setup. She has chosen a wallet that is more sophisticated than the easier custodial wallets. This allows us to show some of the underlying complexity and introduce some of the inner workings of an advanced wallet. You may find that your first ideal wallet is oriented toward ease of use, accepting some of the control and privacy trade-offs. Or perhaps you are more of a power user and want to run your own Lightning and Bitcoin nodes as part of your wallet solution.
 
-Alice uses an Android device and will use the Google Play Store to download and install the Eclair wallet. Searching on Google Play, she finds an entry for "Eclair Mobile", as shown in <<eclair-playstore>>.
+=== Downloading and Installing a Lightning Wallet
+
+((("Lightning wallet","downloading/installing")))When looking for a new cryptocurrency wallet, you must be very careful to select a secure source for the software.
+
+Unfortunately, many fake wallet applications will steal your money, and some of these even find their way onto reputable and supposedly vetted software sites like the Apple and Google application stores. Whether you are installing your first or your tenth wallet, always exercise extreme caution. A rogue app may not just steal any money you entrust it with, but it might also be able to steal keys and passwords from other applications by compromising your mobile device operating system.
+
+((("Eclair wallet, downloading/installing")))Alice uses an Android device and will use the Google Play Store to download and install the Eclair wallet. Searching on Google Play, she finds an entry for "Eclair Mobile," as shown in <<eclair-playstore>>.
 
 [[eclair-playstore]]
 .Eclair Mobile in the Google Play Store
-image:images/eclair-playstore.png["Eclair wallet in the Google Play Store"]
+image::images/mtln_0201.png["Eclair wallet in the Google Play Store"]
 
-Alice notices a few different elements on this page, that help her ascertain that this is, most likely, the correct "Eclair Mobile" wallet she is looking for. Firstly, the organization "ACINQ" footnote:[ACINQ: Developers of the Eclair Mobile Lightning wallet (https://acinq.co/).] is listed as the developer of this mobile wallet, which Alice knows from her research is the correct developer. Secondly, the wallet has been installed "10,000+" times and has more than 320 positive reviews. It is unlikely this is a rogue app that has snuck into the Play Store. As third step, she goes to the ACINQ website (https://acinq.co/). She verifies that the webpage is secure (https, not http) which is shown as a green lock by some browsers. On the website she goes to the Download section or looks for the link to the Google App store. She finds the link and clicks it. She compares that this link brings her to the very same app in the Google App Store. Satisfied by these findings, Alice installs the Eclair app on her mobile device.
+
+[TIP]
+====
+It is possible to experiment and test all Bitcoin-type software with zero risk (except for your own time) by using testnet bitcoins. You can also download the Eclair testnet wallet to try Lightning (on testnet) by going to the Google Play Store.
+====
+
+Alice notices a few different elements on this page that help her ascertain that this is, most likely, the correct "Eclair Mobile" wallet she is looking for. Firstly, the organization ACINQfootnote:[ACINQ: Developers of the Eclair Mobile Lightning wallet.] is listed as the developer of this mobile wallet, which Alice knows from her research is the correct developer. Secondly, the wallet has been installed "10,000+" times and has more than 320 positive reviews. It is unlikely that this is a rogue app that has snuck into the Google Play Store. As a third step, she goes to the https://acinq.co[ACINQ website]. She verifies that the web page is secure by checking that the address begins with https, or prefixed by a padlock in some browsers. On the website, she goes to the Download section or looks for the link to the Google App Store. She finds the link and clicks it. She compares that this link brings her to the very same app in the Google App Store. Satisfied by these findings, Alice installs the Eclair app on her mobile device.
 
 [WARNING]
 ====
@@ -148,11 +195,11 @@ Always exercise great care when installing software on any device. There are man
 
 === Creating a New Wallet
 
-When Alice opens the Eclair Mobile app for the first time, she is presented with a choice to "Create a New Wallet" or to "Import an Existing Wallet". Alice will create a new wallet, but let's first discuss why these options are presented here and what it means to "import an existing wallet".
+((("Lightning wallet","creating a new wallet", id="ix_02_getting_started-asciidoc2", range="startofrange")))When Alice opens the Eclair Mobile app for the first time, she is presented with a choice to "Create a New Wallet" or to "Import an Existing Wallet." Alice will create a new wallet, but let's first discuss why these options are presented here and what it means to import an existing wallet.
 
 ==== Responsibility with Key Custody
 
-As we mentioned in the beginning of this section, Eclair is a _non-custodial_ wallet, meaning that Alice has sole custody of the keys used to control her bitcoin. This also means that Alice is responsible for protecting and backing up those keys. If Alice loses the keys, no one can help her recover the bitcoin, and they will be lost forever.
+((("keys","Lightning wallet and")))((("Lightning wallet","responsibility with key custody")))As we mentioned at the beginning of this section, Eclair is a _noncustodial_ wallet, meaning that Alice has sole custody of the keys used to control her bitcoin. This also means that Alice is responsible for protecting and backing up those keys. If Alice loses the keys, no one can help her recover the bitcoin, and they will be lost forever.
 
 [WARNING]
 ====
@@ -161,123 +208,143 @@ With the Eclair Mobile wallet, Alice has custody and control of the keys and, th
 
 ==== Mnemonic Words
 
-Similar to most Bitcoin wallets, Eclair Mobile provides a _mnemonic phrase_ for Alice to back up. The mnemonic phrase consists of 24 English words, selected randomly by the software, and used as the basis for the keys that are generated by the wallet. The mnemonic phrase can be used by Alice to restore all the transactions and funds in the Eclair Mobile wallet in the case of an event such as a lost mobile device, a software bug, or memory corruption.
+((("Lightning wallet","mnemonic phrase")))((("mnemonic phrase")))((("seed (mnemonic) phrase")))Similar to most Bitcoin wallets, Eclair Mobile provides a _mnemonic phrase_ (also sometimes called a "seed" or "seed phrase") for Alice to back up. The mnemonic phrase consists of 24 English words, selected randomly by the software and used as the basis for the keys that are generated by the wallet. Alice can use the mnemonic phrase to restore all the transactions and funds in the Eclair Mobile wallet in the case of a lost mobile device, a software bug, or memory corruption.
 
 [TIP]
 ====
-The _mnemonic phrase_ is often mistakenly called a "seed". In fact, a seed is constructed _from the mnemonic_ and is something different.
+The correct term for these backup words is "mnemonic phrase." We avoid the use of the term "seed" to refer to a mnemonic phrase because even though its use is common, it is incorrect.
 ====
 
-When Alice chooses to "Create a New Wallet", she will be shown a screen with her mnemonic phrase, which looks like the screenshot in <<eclair-mnemonic>>.
+When Alice chooses to create a new wallet, she will see a screen with her mnemonic phrase, which looks like the screenshot in <<eclair-mnemonic>>.
 
 [[eclair-mnemonic]]
-.New Wallet Mnemonic Phrase
-image:images/eclair-mnemonic.png["New Wallet Mnemonic Phrase"]
+.New wallet mnemonic phrase
+image::images/mtln_0202.png["New Wallet Mnemonic Phrase"]
 
 In <<eclair-mnemonic>>, we have purposely obscured part of the mnemonic phrase to prevent readers of this book from reusing the mnemonic.
 
 [[mnemonic-storage]]
 ==== Storing the Mnemonic Safely
 
-Alice needs to be careful to store the mnemonic phrase in a way that balances the need to prevent theft and accidental loss. The recommended way to properly balance these risks is to write two copies of the mnemonic phrase on paper, with each of the words numbered as the order matters.
+((("Lightning wallet","mnemonic phrase storage")))Alice needs to be careful to store the mnemonic phrase in a way that prevents theft but also avoids accidental loss. The recommended way to properly balance these risks is to write two copies of the mnemonic phrase on paper, with each of the words numbered&#x2014;the order matters.
 
-Once Alice has recorded the mnemonic phrase, after touching "OK GOT IT" on her screen, she will be presented with a _quiz_ to make sure that she correctly recorded the mnemonic. The quiz will ask for three or four of the words at random. Alice wasn't expecting a quiz, but since she recorded the mnemonic correctly, she passes without any difficulty.
+Once Alice has recorded the mnemonic phrase, after touching "OK GOT IT" on her screen, she will be presented with a quiz to make sure that she correctly recorded the mnemonic. The quiz will ask for three or four of the words at random. Alice isn't expecting a quiz, but since she recorded the mnemonic correctly, she passes without any difficulty.
 
-Once Alice has recorded the mnemonic phrase and passed the quiz, she should store each copy in a separate secure location such as a locked desk drawer or a fireproof safe.
+Once Alice has recorded the mnemonic phrase and passed the quiz, she should store each copy in a separate secure location, such as a locked desk drawer or a fireproof safe.
 
 [WARNING]
 ====
-Never attempt a "DIY" security scheme that deviates in any way from the best practice recommendation in <<mnemonic-storage>>. Do not cut your mnemonic in half, make screenshots, store on USB drives or cloud drives, encrypt it, or try any other non-standard method. You will tip the balance in such a way as to risk permanent loss or theft. Many people have lost funds, not from theft but because they tried a non-standard solution without having the expertise to balance the risks involved. The best practice recommendation is carefully balanced by experts and suitable for the vast majority of users.
+Never attempt a "DIY" security scheme that deviates in any way from the best practice recommendation in <<mnemonic-storage>>. Do not cut your mnemonic in half, make screenshots, store it on USB drives or cloud drives, encrypt it, or try any other nonstandard method. You will tip the balance in such a way as to risk permanent loss. Many people have lost funds, not from theft, but because they tried a nonstandard solution without having the expertise to balance the risks involved. The best practice recommendation is carefully considered by experts and suitable for the vast majority of users.
 ====
 
-After Alice initializes her Eclair Mobile wallet, she will see a brief tutorial that highlights the various elements of the user interface. We won't replicate the tutorial here, but we will explore all of those elements as we follow Alice's attempt to buy a cup of coffee!
+After Alice initializes her Eclair Mobile wallet, she will see a brief tutorial that highlights the various elements of the user interface. We won't replicate the tutorial here, but we will explore all of those elements as we follow Alice's attempt to buy a cup of coffee!(((range="endofrange", startref="ix_02_getting_started-asciidoc2")))
 
-=== Loading Bitcoin Into the Wallet
+=== Loading Bitcoin onto the Wallet
 
-Alice now has a Lightning wallet. But, it's empty! She now faces one of the more challenging aspects of this experiment: she has to find a way to acquire some bitcoin and load it onto her Eclair wallet.
+((("bitcoin (currency)","loading onto Lightning wallet", id="ix_02_getting_started-asciidoc3", range="startofrange")))((("Lightning wallet","loading bitcoin onto", id="ix_02_getting_started-asciidoc4", range="startofrange")))Alice now has a Lightning wallet. But it's empty! She now faces one of the more challenging aspects of this experiment: she has to find a way to acquire some bitcoin and load it onto her Eclair wallet.
+
+[TIP]
+====
+If Alice already has bitcoin in another wallet, she could choose to send that bitcoin to her Eclair wallet instead of acquiring new bitcoin to load onto her new wallet.
+====
 
 [[acquiring-bitcoin]]
 ==== Acquiring Bitcoin
 
-There are several ways Alice can acquire bitcoin:
+((("bitcoin (currency)","acquiring for Lightning wallet")))((("Lightning wallet","acquiring bitcoin for")))There are several ways Alice can acquire bitcoin:
 
-* She can exchange some of her national currency (e.g. USD) at a crypto-currency exchange
-* She can buy some from a friend, or an acquaintance from a Bitcoin Meetup, in exchange for cash
-* She can find a _Bitcoin ATM_ in her area, which acts as a vending machine, selling bitcoin for cash
-* She can offer her skills or a product she sells and accepts payment in bitcoin
-* She can ask her employer or clients to pay her in bitcoin
+* She can exchange some of her national currency (e.g., USD) on a cryptocurrency exchange.
+* She can buy some from a friend, or an acquaintance from a Bitcoin meetup, in exchange for cash.
+* She can find a _Bitcoin ATM_ in her area, which acts as a vending machine, selling bitcoin for cash.
+[role="pagebreak-before"]
+* She can offer her skills or a product she sells and accept payment in bitcoin.
+* She can ask her employer or clients to pay her in bitcoin.
 
 All of these methods have varying degrees of difficulty, and many will involve paying a fee. Some will also require Alice to provide identification documents to comply with local banking regulations. However, with all these methods, Alice will be able to receive bitcoin.
 
 ==== Receiving Bitcoin
 
-Let's assume Alice has found a local Bitcoin ATM and has decided to buy some bitcoin in exchange for cash. An example of a Bitcoin ATM, one built by the Lamassu company, is shown in <<bitcoin-atm>>. Such Bitcoin ATMs accept national currency (cash) through a cash slot and send bitcoin to a Bitcoin Address scanned from a user's wallet using a built-in camera.
+((("bitcoin (currency)","receiving for Lightning wallet", id="ix_02_getting_started-asciidoc5", range="startofrange")))((("Bitcoin ATM", id="ix_02_getting_started-asciidoc6", range="startofrange")))((("Lightning wallet","receiving bitcoin", id="ix_02_getting_started-asciidoc7", range="startofrange")))Let's assume Alice has found a local Bitcoin ATM and has decided to buy some bitcoin in exchange for cash. An example of a Bitcoin ATM, one built by the Lamassu Company, is shown in <<bitcoin-atm>>. Such Bitcoin ATMs accept national currency (cash) through a cash slot and send bitcoin to a Bitcoin address scanned from a user's wallet using a built-in camera.
 
 [[bitcoin-atm]]
 .A Lamassu Bitcoin ATM
-image:images/bitcoin-atm.png[]
+image::images/mtln_0203.png["Lamassu Bitcoin ATM"]
 
-To receive the bitcoin in her Eclair Lightning wallet, Alice will need to present a _Bitcoin Address_ from the Eclair Lightning wallet to the ATM. The ATM can then send Alice's newly acquired bitcoin to this bitcoin address.
+To receive the bitcoin in her Eclair Lightning wallet, Alice will need to present a Bitcoin address from the Eclair Lightning wallet to the ATM. The ATM can then send Alice's newly acquired bitcoin to this Bitcoin address.
 
-To see a Bitcoin Address on the Eclair wallet, Alice must swipe to the left column titled "YOUR BITCOIN ADDRESS" (see <<eclair-receive>>), where she will see a square barcode (called a _QR code_) and a string of letters and numbers below.
+To see a Bitcoin address on the Eclair wallet, Alice must swipe to the left column titled YOUR BITCOIN ADDRESS (see <<eclair-receive>>), where she will see a square barcode (called a _QR code_) and a string of letters and numbers below that.
+
+The QR code contains the same string of letters and numbers shown below it, in an easy to scan format. This way, Alice doesn't have to type the Bitcoin address. In the screenshot (<<eclair-receive>>), we have purposely blurred both, to prevent readers from inadvertently sending bitcoin to this address.
 
 [[eclair-receive]]
 .Alice's bitcoin address, shown in Eclair
-image:images/eclair-receive.png[]
-
-The QR code contains the same string of letters and numbers as shown below it, in an easy to scan format. This way, Alice doesn't have to type the Bitcoin Address. In the screenshot <<eclair-receive>>, we have purposely blurred both, to prevent readers from inadvertently sending bitcoin to this address.
+image::images/mtln_0204.png["Eclair bitcoin address QR code"]
 
 [NOTE]
 ====
-Both Bitcoin addresses and QR codes contain error detection information that prevents any typing or scanning errors from producing a "wrong" Bitcoin address. If there is a mistake in the address, any Bitcoin wallet will notice the error and refuse to accept the Bitcoin Address as valid.
+Both Bitcoin addresses and QR codes contain error detection information that prevents any typing or scanning errors from producing a "wrong" Bitcoin address. If there is a mistake in the address, any Bitcoin wallet will notice the error and refuse to accept the Bitcoin address as valid.
 ====
 
+[role="pagebreak-before"]
 Alice can take her mobile device to the ATM and show it to the built-in camera, as shown in <<bitcoin-atm-receive>>. After inserting some cash into the slot, she will receive bitcoin in Eclair!
 
 [[bitcoin-atm-receive]]
-.Bitcoin ATM scans the QR code.
-image:images/bitcoin-atm-receive.png[]
+.Bitcoin ATM scans the QR code
+image::images/mtln_0205.png["Bitcoin ATM scans the QR code"]
 
-Alice will see the transaction from the ATM in the "TRANSACTION HISTORY" tab of the Eclair wallet. While Eclair will detect the bitcoin transaction in just a few seconds, it will take approximately one hour for the bitcoin transaction to be "confirmed" on the Bitcoin blockchain. As you can see in <<eclair-tx1>>, Alice's Eclair wallet shows "6+ conf" below the transaction, indicating that the transaction has received the required minimum of six confirmations, and her funds are now ready to use.
+Alice will see the transaction from the ATM in the TRANSACTION HISTORY tab of the Eclair wallet. Although Eclair will detect the bitcoin transaction in just a few seconds, it will take approximately one hour for the bitcoin transaction to be "confirmed" on the Bitcoin blockchain. As you can see in <<eclair-tx1>>, Alice's Eclair wallet shows "6+ conf" below the transaction, indicating that the transaction has received the required minimum of six confirmations, and her funds are now ready to use.
+
+[TIP]
+====
+The number of confirmations on a transaction is the number of blocks mined since (and inclusive of) the block that contained that transaction. Six confirmations is best practice, but different Lightning wallets can consider a channel open after any number of confirmations. Some wallets even scale up the number of expected confirmations by the monetary value of the channel.
+====
+
+Although in this example Alice used an ATM to acquire her first bitcoin, the same basic concepts would apply even if she used one of the other methods in <<acquiring-bitcoin>>. For example, if Alice wanted to sell a product or provide a professional service in exchange for bitcoin, her customers could scan the Bitcoin address with their wallets and pay her in bitcoin.
 
 [[eclair-tx1]]
 .Alice receives bitcoin
-image:images/eclair-tx1-btc.png[]
+image::images/mtln_0206.png["Bitcoin transaction received"]
 
-While in this example Alice used an ATM to acquire her first bitcoin, the same basic concepts would apply even if she used one of the other methods in <<acquiring-bitcoin>>. For example, if Alice wanted to sell a product or provide a professional service in exchange for bitcoin, her customers could scan the Bitcoin Address with their wallets and pay her in bitcoin.
+Similarly, if she billed a client for a service offered over the internet, Alice could send an email or instant message with the Bitcoin address or the QR code to her client, and they could paste or scan the information into a Bitcoin wallet to pay her.
 
-Similarly, if she billed a client for a service offered over the Internet, Alice could send an email or instant message with the Bitcoin Address or the QR code to her client, and they could paste or scan the information into a Bitcoin wallet to pay her.
+Alice could even print the QR code and affix it to a sign and display it publicly to receive tips. For example, she could have a QR code affixed to her guitar and receive tips while performing on the street!footnote:[It is generally not advisable to reuse the same Bitcoin address for multiple payments because all Bitcoin transactions are public.
+A nosy person passing by could scan Alice's QR code and see how many tips Alice has already received to this address on the Bitcoin blockchain.
+Fortunately, the Lightning Network offers more private solutions to this, discussed later in the book!]
 
-Alice could even print the QR code and affix it to a sign and display it publicly to receive tips. For example, she could have a QR code affixed to her guitar and receive tips while performing on the street!
-
-Finally, if Alice bought bitcoin from a crypto-currency exchange, she could (and should) "withdraw" the bitcoin by pasting her Bitcoin Address into the exchange website. The exchange will then send the bitcoin to her address directly.
+Finally, if Alice bought bitcoin from a cryptocurrency exchange, she could (and should) "withdraw" the bitcoin by pasting her Bitcoin address into the exchange website. The exchange will then send the bitcoin to her address directly(((range="endofrange", startref="ix_02_getting_started-asciidoc7")))(((range="endofrange", startref="ix_02_getting_started-asciidoc6")))(((range="endofrange", startref="ix_02_getting_started-asciidoc5"))).(((range="endofrange", startref="ix_02_getting_started-asciidoc4")))(((range="endofrange", startref="ix_02_getting_started-asciidoc3")))
 
 === From Bitcoin to Lightning Network
 
-Alice's bitcoin is now controlled by her Eclair wallet and has been recorded on the Bitcoin blockchain. At this point, Alice's bitcoin is "on-chain," meaning that the transaction has been broadcast to the entire Bitcoin network, verified by all Bitcoin nodes, and "mined" (recorded) onto the Bitcoin blockchain.
+((("Lightning wallet","bridging of Bitcoin and Lightning networks", id="ix_02_getting_started-asciidoc8", range="startofrange")))Alice's bitcoin is now controlled by her Eclair wallet and has been recorded on the Bitcoin blockchain. At this point, Alice's bitcoin is _on-chain_, meaning that the transaction has been broadcast to the entire Bitcoin network, verified by all Bitcoin nodes, and _mined_ (recorded) onto the Bitcoin blockchain.
 
 So far, the Eclair Mobile wallet has behaved only as a Bitcoin wallet, and Alice hasn't used the Lightning Network features of Eclair. As is the case with many Lightning wallets, Eclair bridges Bitcoin and the Lightning Network by acting as both a Bitcoin wallet and a Lightning wallet.
 
-Now, Alice is ready to start using the Lightning Network by taking her bitcoin "off-chain" in order to take advantage of the fast, cheap, and private payments that the Lightning Network offers.
+Now, Alice is ready to start using the Lightning Network by taking her bitcoin off-chain to take advantage of the fast, cheap, and private payments that the Lightning Network offers.
 
 ==== Lightning Network Channels
 
-Swiping right, Alice accesses the "LIGHTNING CHANNELS" section of Eclair. Here she can manage the channels that will connect her wallet to the Lightning Network.
+((("Lightning Network channels","basics", id="ix_02_getting_started-asciidoc9", range="startofrange")))((("Lightning Network channels","opening a channel", id="ix_02_getting_started-asciidoc10", range="startofrange")))((("Lightning wallet","LN channels and", id="ix_02_getting_started-asciidoc11", range="startofrange")))Swiping right, Alice accesses the LIGHTNING CHANNELS section of Eclair. Here she can manage the channels that will connect her wallet to the Lightning Network.
 
-Let's review the definition of a "Lightning Network Channel" at this point, to make things a bit clearer. Firstly, the word "channel" is a metaphor for a _financial relationship_ between Alice's Lightning wallet and another Lightning wallet. We call it a channel because it is a means for Alice's wallet and this other wallet to exchange many payments with each other on the Lightning Network (off-chain) without committing transactions to the Bitcoin blockchain (on-chain).
+Let's review the definition of an LN channel at this point, to make things a bit clearer. Firstly, the word "channel" is a metaphor for a _financial relationship_ between Alice's Lightning wallet and another Lightning wallet. We call it a channel because it is a means for Alice's wallet and this other wallet to exchange many payments with each other on the Lightning Network (off-chain) without committing transactions to the Bitcoin blockchain (on-chain).
 
-The wallet or _node_ that Alice opens a channel to is called her _channel peer_. Once "opened", a channel can be used to send many payments back and forth between Alice's wallet and her channel peer.
+((("channel peer")))The wallet or _node_ that Alice opens a channel to is called her _channel peer_. Once "opened," a channel can be used to send many payments back and forth between Alice's wallet and her channel peer.
 
-Furthermore, Alice's channel peer can _forward_ payments via other channels further into the Lightning Network. This way, Alice can _route_ a payment to any wallet (e.g. Bob's Lightning wallet) as long as Alice's wallet can find a _path_ made by hopping from channel to channel, all the way to Bob's wallet.
+Furthermore, Alice's channel peer can _forward_ payments via other channels further into the Lightning Network. This way, Alice can _route_ a payment to any wallet (e.g., Bob's Lightning wallet) as long as Alice's wallet can find a viable _path_ made by hopping from channel to channel, all the way to Bob's wallet.
 
-In other words: Alice needs one or more channels that connects her to one or more other nodes on the Lightning Network. She doesn't need a channel to connect her wallet directly to Bob's Cafe in order to send Bob a payment, though she can choose to open a direct channel too. Any node in the Lightning Network can be used for Alice's first channel. The more well-connected a node is the more people Alice can reach. In this example, since we want to also demonstrate payment routing, we won't have Alice open a channel directly to Bob's wallet. Instead, we will have Alice open a channel to a well-connected node and then later use that node to forward her payment, routing it through any other nodes as necessary to reach Bob.
+[TIP]
+====
+Not all channel peers are _good_ peers for routing payments. Well-connected peers will be able to route payments over shorter paths to the destination, increasing the chance of success. Channel peers with ample funds will be able to route larger payments. 
+====
 
-At first, there are no open channels, so as we see in <<eclair-tutorial2.png>>, the "LIGHTNING CHANNELS" tab displays an empty list. If you notice, on the bottom right corner, there is a plus symbol (+), which is a button to open a new channel.
+In other words, Alice needs one or more channels that connect her to one or more other nodes on the Lightning Network. She doesn't need a channel to connect her wallet directly to Bob's Cafe in order to send Bob a payment, though she can choose to open a direct channel, too. Any node in the Lightning Network can be used for Alice's first channel. The more well-connected a node is, the more people Alice can reach. In this example, since we want to also demonstrate payment routing, we won't have Alice open a channel directly to Bob's wallet. Instead, we will have Alice open a channel to a well-connected node and then later use that node to forward her payment, routing it through any other nodes as necessary to reach Bob.
+
+At first, there are no open channels, so as we see in <<eclair-channels>>, the LIGHTNING CHANNELS tab displays an empty list. If you notice, in the bottom-right corner there is a plus symbol (+), which is a button to open a new channel.
 
 [[eclair-channels]]
-.Lightning Channels Tab
-image:images/eclair-tutorial2.png["Lightning Channels Tab"]
+.LIGHTNING CHANNELS tab
+image::images/mtln_0207.png["LIGHTNING CHANNELS tab"]
 
+[role="pagebreak-before"]
 Alice presses the plus symbol and is presented with four possible ways to open a channel:
 
 * Paste a node URI
@@ -285,98 +352,105 @@ Alice presses the plus symbol and is presented with four possible ways to open a
 * Random node
 * ACINQ node
 
-A "node URI" is a Universal Resource Identifier (URI) that identifies a specific Lightning node. Alice can either paste such a URI from her clipboard or scan a QR code containing that same information. An example of a node URI is shown as a QR code in <<node-URI-QR>> and below it as a text string:
+A "node URI" is a Universal Resource Identifier (URI) that identifies a specific Lightning node. Alice can either paste such a URI from her clipboard or scan a QR code containing that same information. An example of a node URI is shown as a QR code in <<node-URI-QR>> and then as a text string.
 
 [[node-URI-QR]]
-.node URI as a QR code
-image:images/node-URI-QR.png[width=120]
+.Node URI as a QR code
+image::images/mtln_0208.png["Lightning node URI QR code",width=120]
 
 [[node-URI-example]]
 .node URI
-++++
+----
 0237fefbe8626bf888de0cad8c73630e32746a22a2c4faa91c1d9877a3826e1174@1.ln.aantonop.com:9735
-++++
+----
 
-While Alice could select a specific Lightning node, or use the "Random node" option to have the Eclair wallet select a node at random, she will select the "ACINQ Node" option to connect to one of ACINQ's well-connected Lightning nodes.
+While Alice could select a specific Lightning node, or use the "Random node" option to have the Eclair wallet select a node at random, she will select the ACINQ Node option to connect to one of ACINQ's well-connected Lightning nodes.
 
-Choosing the ACINQ node will slightly reduce Alice's privacy, as it will give ACINQ the ability to see all of Alice's transactions. It will also create a Single Point of Failure, since Alice will only have one channel, and if the ACINQ node is not available, Alice will not be able to make payments. To keep things simple at first, we will accept these trade-offs. In subsequent chapters we will gradually learn how to gain more independence and make fewer trade-offs!
+Choosing the ACINQ node will slightly reduce Alice's privacy, because it will give ACINQ the ability to see all of Alice's transactions. It will also create a single point of failure, since Alice will only have one channel, and if the ACINQ node is not available, Alice will not be able to make payments. To keep things simple at first, we will accept these trade-offs. In subsequent chapters, we will gradually learn how to gain more independence and make fewer trade-offs!
 
-Alice selects "ACINQ Node" and is ready to open her first channel on the Lightning network.
+Alice selects ACINQ Node and is ready to open her first channel on the Lightning Network.(((range="endofrange", startref="ix_02_getting_started-asciidoc11")))(((range="endofrange", startref="ix_02_getting_started-asciidoc10")))(((range="endofrange", startref="ix_02_getting_started-asciidoc9")))
 
 ==== Opening a Lightning Channel
 
-When Alice selects a node to open a new channel, she is asked to select how much bitcoin she wants to allocate to this channel. In subsequent chapters, we will discuss the implications of these choices, but for now, Alice will allocate almost all her funds to the channel. Since she will have to pay transaction fees to open the channel, she will select an amount a few dollars (or a few thousandths of a bitcoin) less than her total balance.
+((("Lightning wallet","opening a Lightning channel", id="ix_02_getting_started-asciidoc12", range="startofrange")))When Alice selects a node to open a new channel, she is asked to select how much bitcoin she wants to allocate to this channel. In subsequent chapters, we will discuss the implications of these choices, but for now, Alice will allocate almost all her funds to the channel. Since she will have to pay transaction fees to open the channel, she will select an amount slightly less than her total balance.footnote:[The Eclair wallet doesn't offer an option to automatically calculate the necessary fees and allocate the maximum amount of funds to a channel, so Alice has to calculate this herself.]
 
-Alice allocates 0.018BTC of her 0.020 total to her channel and accepts the default fee rate, as shown in <<eclair-open-channel>>.
+Alice allocates 0.018 BTC of her 0.020 BTC total to her channel and accepts the default fee rate, as shown in <<eclair-open-channel>>.
 
 [[eclair-open-channel]]
-.Opening a Lightning Channel
-image:images/eclair-open-channel-detail.png[]
+.Opening a Lightning channel
+image::images/mtln_0209.png["Opening a Lightning Channel"]
 
-Once she clicks "OPEN", her wallet constructs the special Bitcoin transaction that opens a Lightning channel, known as the _funding transaction_. The "on-chain" funding transaction is sent to the Bitcoin Network for confirmation.
+Once she clicks OPEN, her wallet constructs the special Bitcoin transaction that ((("funding transaction")))opens a Lightning channel, known as the _funding transaction_. The on-chain funding transaction is sent to the Bitcoin network for confirmation.
 
 Alice now has to wait again (see <<eclair-channel-waiting>>) for the transaction to be recorded on the Bitcoin blockchain. As with the initial Bitcoin transaction that she used to acquire her bitcoin, she has to wait for six or more confirmations (approximately one hour).
 
 [[eclair-channel-waiting]]
-.Waiting for the Funding Transaction to Open the Channel
-image:images/eclair-channel-waiting.png["Waiting for the Funding Transaction to Open the Channel"]
+.Waiting for the funding transaction to open the channel
+image::images/mtln_0210.png["Waiting for the Funding Transaction to Open the Channel"]
 
-Once the funding transaction is confirmed, Alice's channel to the ACINQ node is open, funded and ready, as shown in <<eclair-channel-open>>:
+Once the funding transaction is confirmed, Alice's channel to the ACINQ node is open, funded, and ready, as shown in <<eclair-channel-open>>.
 
 [[eclair-channel-open]]
-.Channel is Open
-image:images/eclair-channel-open.png["Channel is Open"]
+.Channel is open
+image::images/mtln_0211.png["Channel is Open"]
 
 [TIP]
 ====
-Did you notice that the channel amount seems to have changed? It hasn't: the channel contains 0.018 BTC, but in the time between screenshots the BTC exchange rate changed, so the USD value is different. You can choose to show balances in BTC or USD, but keep in mind that USD values are calculated in real time and will change!
+Did you notice that the channel amount seems to have changed? It hasn't: the channel contains 0.018 BTC, but in the time between screenshots, the BTC exchange rate changed, so the USD value is different. You can choose to show balances in BTC or USD, but keep in mind that USD values are calculated in real time and will change(((range="endofrange", startref="ix_02_getting_started-asciidoc12")))!(((range="endofrange", startref="ix_02_getting_started-asciidoc8")))
 ====
 
-=== Buying a Cup of Coffee
+=== Buying a Cup of Coffee Using the Lightning Network
 
-Alice now has everything ready to start using the Lightning Network. As you can see, it took a bit of work and a bit of time waiting for confirmations. However, now subsequent actions are fast and easy. The Lightning Network enables payments without having to wait for confirmations, as funds get settled in seconds.
+((("Lightning Network (generally)","example: buying a cup of coffee", id="ix_02_getting_started-asciidoc13", range="startofrange")))((("Lightning wallet","example: buying a cup of coffee", id="ix_02_getting_started-asciidoc14", range="startofrange")))Alice now has everything ready to start using the Lightning Network. As you can see, it took a bit of work and a bit of time waiting for confirmations. However, now subsequent actions are fast and easy. The Lightning Network enables payments without having to wait for confirmations, as funds get settled in seconds.
 
 Alice grabs her mobile device and runs to Bob's Cafe in her neighborhood. She is excited to try her new Lightning wallet and use it to buy something!
 
 ==== Bob's Cafe
 
-Bob has a simple Point-of-Sale (PoS) application for the use of any customer who wants to pay with bitcoin over the Lightning Network. As we will see in the next chapter, Bob uses the popular open source platform _BTCPay Server_ which contains all the necessary components for an e-commerce or retail solution, such as:
+Bob has a simple point-of-sale (PoS) application for the use of any customer who wants to pay with bitcoin over the Lightning Network. As we will see in the next chapter, Bob uses the popular open source platform _BTCPay Server_ which contains all the necessary components for an ecommerce or retail solution, such as:
 
-* A Bitcoin Node using the Bitcoin Core software
-* A Lightning Node using the c-lightning software
+* A Bitcoin node using the Bitcoin Core software
+* A Lightning node using the c-lightning software
 * A simple PoS application for a tablet
 
 BTCPay Server makes it simple to install all the necessary software, upload pictures and product prices, and launch a store quickly.
 
-On the counter at Bob's Cafe, there is a tablet device showing <<bob-cafe-posapp>>:
+On the counter at Bob's Cafe, there is a tablet device showing what you see in <<bob-cafe-posapp>>.
 
 [[bob-cafe-posapp]]
-.Bob's Point-of-Sale Application
-image:images/bob-cafe-posapp.png[]
+.Bob's point-of-sale application
+image::images/mtln_0212.png["Bob's Point-of-Sale Application"]
 
 ==== A Lightning Invoice
 
-Alice selects the "Cafe Latte" option from the screen and is presented with a _Lightning Invoice_ as shown in <<bob-cafe-invoice>>
+((("Lightning invoices", id="ix_02_getting_started-asciidoc15", range="startofrange")))((("Lightning wallet","invoices", id="ix_02_getting_started-asciidoc16", range="startofrange")))Alice selects the Cafe Latte option from the screen and is presented with a _Lightning invoice_ (also known as a "payment request"), as shown in <<bob-cafe-invoice>>.
 
 [[bob-cafe-invoice]]
-.Lightning Invoice for Alice's latte
-image:images/bob-cafe-invoice.png[]
+.Lightning invoice for Alice's latte
+image::images/mtln_0213.png["BTCPay Server Lightning invoice"]
 
-To pay the invoice, Alice opens her Eclair wallet and selects the "Send" button (which looks like a right-facing arrow) under the "TRANSACTION HISTORY" tab, as shown in <<alice-send-start>>.
+[role="pagebreak-before"]
+To pay the invoice, Alice opens her Eclair wallet and selects the Send button (which looks like an up-facing arrow) under the TRANSACTION HISTORY tab, as shown in <<alice-send-start>>.
 
 [[alice-send-start]]
-.Alice Send
-image:images/alice-send-start.png[width=300]
+.Alice selecting Send
+image::images/mtln_0214.png["Lightning transaction send",width=300]
 
-Alice selects the option to "scan a payment request" and scans the QR code displayed on the screen of the tablet (see <<bob-cafe-invoice>>), and is prompted to confirm her payment, as shown in <<alice-send-detail>>:
+[TIP]
+====
+The term "payment request" can refer to a Bitcoin payment request or a Lightning invoice, and the terms "invoice" and "payment request" are often used interchangeably. The correct technical term is "Lightning invoice," regardless of how it is named in the wallet.
+====
+
+Alice selects the option to "scan a payment request," scans the QR code displayed on the screen of the tablet (see <<bob-cafe-invoice>>), and is prompted to confirm her payment, as shown in <<alice-send-detail>>.
+
+Alice presses PAY, and a second later, Bob's tablet shows a successful payment. Alice has completed her first LN payment! It was fast, inexpensive, and easy. Now she can enjoy her latte which was purchased using bitcoin through a payment system that is fast, cheap, and decentralized. From now on, Alice can simply select an item on Bob's tablet screen, scan the QR code with her cell phone, click PAY, and be served a coffee, all within seconds and all without an on-chain transaction.
 
 [[alice-send-detail]]
-.Alice's Send Confirmation
-image:images/alice-send-detail.png[width=300]
-
-Alice presses "PAY," and a second later, Bob's tablet shows a successful payment. Alice has completed her first Lightning Network payment! It was fast, inexpensive, and easy. Now she can enjoy her latte which was purchased using the most advanced payment technology in the world. And from now on, whenever Alice feels like drinking a coffee at Bob's Cafe she selects an item on Bob's tablet screen, scans the QR code with her cell phone, clicks pay and is served a coffee, all within seconds and all without "on-chain" transaction.
+.Alice's send confirmation
+image::images/mtln_0215.png["Lightning transaction send confirmation",width=300]
 
+Lightning payments are better for Bob, too. He's confident that he will be paid for Alice's latte without waiting for an on-chain confirmation. In the future, whenever Alice feels like drinking a coffee at Bob's Cafe, she can choose to pay with bitcoin on the Bitcoin network or the Lightning Network. Which one do you think she will choose(((range="endofrange", startref="ix_02_getting_started-asciidoc16")))(((range="endofrange", startref="ix_02_getting_started-asciidoc15")))?(((range="endofrange", startref="ix_02_getting_started-asciidoc14")))(((range="endofrange", startref="ix_02_getting_started-asciidoc13")))
 
 === Conclusion
 
-In this chapter, we followed Alice as she downloaded and installed her first Lightning wallet, acquired and transferred some bitcoin, opened her first Lightning channel, and bought a cup of coffee by making her first payment on the Lightning Network. In the following chapters, we will look "under the covers" at how each component in the Lightning Network works, and how Alice's payment reached Bob's Cafe.
+In this chapter, we followed Alice as she downloaded and installed her first Lightning wallet, acquired and transferred some bitcoin, opened her first Lightning channel, and bought a cup of coffee by making her first payment on the Lightning Network.(((range="endofrange", startref="ix_02_getting_started-asciidoc0"))) In the following chapters, we will look "under the covers" at how each component in the Lightning Network works and how Alice's payment reached Bob's Cafe.

05_node_operations.asciidoc

@@ -0,0 +1,970 @@
+[[operating_ln_node]]
+[[node_operations]]
+== Operating a Lightning Network Node
+
+((("Lightning node operation", id="ix_05_node_operations-asciidoc0", range="startofrange")))After having read this far, you have probably set up a Lightning wallet. In this chapter, we will take things one step further and set up a full Lightning node. In addition to setting one up, we will learn how to operate it and maintain it over time.
+
+There are many reasons why you might want to set up your own Lightning node. They include:
+
+* To be a full, active participant in the Lightning Network, not just an end user
+* To run an ecommerce store or receive income via Lightning payments
+* To earn income from Lightning routing fees or by renting channel liquidity
+* To develop new services, applications, or plug-ins for the Lightning Network
+* To increase your financial privacy while using Lightning
+* To use some apps built on top of Lightning, like Lightning-powered instant messaging apps
+* For financial freedom, independence, and sovereignty
+
+There are costs associated with running an LN node. You need a computer, a permanent internet connection, lots of disk space, and lots of time!
+Operational costs will include electricity expenses.
+
+But the skills you will learn from this experience are valuable and can be applied to a variety of other tasks too.
+
+Let's get started!
+
+[NOTE]
+====
+It is important that you set your own expectations correctly on accurate facts.
+If you plan to operate a Lightning node _solely_ to gain income by earning routing fees,
+please do your homework diligently first. Running a profitable business by operating a Lightning node is
+definitely _not_ easy. Calculate all your initial and ongoing costs in a spreadsheet. Study LN statistics carefully.
+What is the current payment volume? What is the volume per node? What are the current average routing fees? Consult forums and ask
+for advice or feedback from other community members who have already gained real-world experience. Form your own educated opinion only
+_after_ you have done this due diligence exercise. Most people will find their motivation for running a node not in financial gain,
+but somewhere else.
+====
+
+=== Choosing Your Platform
+
+((("Lightning node operation","choosing a platform", id="ix_05_node_operations-asciidoc1", range="startofrange")))There are many ways you can run a Lightning node, ranging from a small mini PC hosted in your home or a dedicated server, to a hosted server in the cloud. The method you choose will depend on the resources you have and how much money you want to spend.
+
+[[continuous_operation]]
+==== Why Is Reliability Important for Running a Lightning Node?
+
+((("Lightning node operation","reliability issues")))((("reliability, Lightning node and")))In Bitcoin, hardware is not particularly important unless one is specifically running a mining node.
+The Bitcoin Core node software can be run on any machine that meets its minimum requirements and does not need to be online to receive payments—only to send them.
+If a Bitcoin node goes down for an extended period of time, the user can simply reboot the node, and once it connects to the rest of the network, it will resync the blockchain.
+
+In Lightning, however, the user needs to be online both to send _and_ to receive payments. If the Lightning node is offline, it cannot receive any payments from anyone, and thus its open invoices cannot be fulfilled.
+Furthermore, the open channels of an offline node cannot be used to route payments. Your channel partners will notice that you are offline and cannot contact you to route a payment. If you are offline too often, they may consider the bitcoin locked up in their channels with you to be underutilized capacity, and may close those channels. We already discussed the case of a protocol attack in which your channel partner tries to cheat you by submitting an earlier commitment transaction. If you are offline and your channels aren't being monitored, then the attempted theft could succeed, and you will have no recourse once the timelock expires.
+Hence, node reliability is extremely important for a Lightning node.
+
+((("hardware failure")))There are also the issues of hardware failure and loss of data. In Bitcoin, a hardware failure can be a trivial problem if the user has a backup of their mnemonic phrase or private keys. The Bitcoin wallet and the bitcoin inside the wallet can be easily restored from the private keys on a new computer. Most information can be redownloaded from the blockchain.
+
+In contrast, in Lightning the information about the user's channels, including the commitment transactions and revocation secrets, are not publicly known and are only stored on the individual user's hardware.
+Thus, software and hardware failures in the Lightning Network can easily result in loss of funds.
+
+==== Types of Hardware Lightning Nodes
+
+((("Lightning node operation","types of hardware Lightning nodes")))There are three main types of hardware Lightning nodes:
+
+General-purpose computers:: An LN node can be run on a home computer or laptop running Windows, macOS, or Linux. Typically this is run alongside a Bitcoin node.
+Dedicated hardware:: A Lightning node can also be run on dedicated hardware like a Raspberry Pi, Rock64, or mini PC. This setup would usually run a software stack, including a Bitcoin node and other applications. This setup is popular because the hardware is dedicated to running and maintaining the Lightning node only and is usually set up with an installation "helper."
+Preconfigured hardware:: An LN node can also be run on purpose-built hardware specifically selected and configured for it. This would include "out-of-the-box" Lightning node solutions that can be purchased as a kit or a turnkey system.
+
+==== Running in the "Cloud"
+
+((("cloud, Lightning node operation in")))((("Lightning node operation","running in the cloud")))((("virtual private server (VPS)")))_Virtual private server_ (VPS) and cloud computing services such as Microsoft Azure, Google Cloud, Amazon Web Services (AWS), or DigitalOcean are quite affordable and can be set up very quickly. A Lightning node can be hosted for between $20 and $40 per month on such a service.
+
+However, as the saying goes, "‘Cloud’ is just other people's computers." Using these services means running your node on other people's computers. This brings along the corresponding advantages and disadvantages. The key advantages are convenience, efficiency, uptime, and possibly even cost. The cloud operator manages and runs the node to a high degree, automatically providing you with convenience and efficiency. They provide excellent uptime and availability, often much better than what an individual can achieve at home. If you consider that just the electricity cost of running a server in many Western countries is around $10 per month, then add to that the cost of network bandwidth and the hardware itself, the VPS offering becomes financially competitive. Lastly, with a VPS you need no space for a PC at home and don't have any issues with PC noise or heat.
+On the other hand, there are several notable disadvantages. A Lightning node running in the "cloud" will always be less secure and less private than one running on your own computer. Additionally, these cloud computing services are very centralized. The vast majority of Bitcoin and Lightning nodes running on such services are located in a handful of data centers in Virginia, Sunnyvale, Seattle, London, and Frankfurt. When the networks or data centers of these providers have service problems, it affects thousands of nodes on so-called "decentralized" networks.
+
+If you have the possibility and capacity of running a node on your own computer at home or in your office, then this might be preferable to running it
+in the cloud. Nonetheless, if running your own server is not an option, by all means consider running one on a VPS.
+
+==== Running a Node at Home
+
+((("Lightning node operation","running a node at home")))If you have a reasonable-capacity internet connection at home or in your office, you can certainly run a Lightning node there. Any "broadband" connection is sufficient for the purpose of running a lightweight node, and a fast connection will allow you to run a Bitcoin full node too.
+
+While you can run a Lightning node (and even a Bitcoin node) on your laptop, it will become annoying quite fast. These programs consume your computer's resources and need to run 24/7. Your user applications like your browser or your spreadsheet will be competing against the Lightning background services for your computer's resources. In other words, your browser and other desktop workloads will be slowed down.
+And when your word-processing app freezes up your laptop, your Lightning node will go down as well, leaving you unable to receive transactions and potentially vulnerable to attacks. Furthermore, you should never turn off your laptop.
+All this combined together results in a setup that is not ideal. The same will apply to your daily-use personal desktop PC.
+
+Instead, most users will choose to run a node on a dedicated computer.
+Fortunately, you don't need a "server" class computer to do this.
+You can run a Lightning node on a single-board computer, such as a Raspberry Pi or on a mini PC (usually marketed as home theater PCs).
+These are simple computers which are commonly used as a home automation hub or a media server.
+They are relatively inexpensive when compared to a PC or a laptop.
+The advantage of a dedicated device as a platform for Lightning and Bitcoin nodes is that it can run continuously, silently, and unobtrusively on your home network, tucked behind your router or TV.
+No one will even know that this little box is actually part of a global banking system!
+
+[WARNING]
+====
+Operating a node on a 32-bit operating system and/or 32-bit CPU is not recommended, because the node software may run into resource issues, causing a crash and possibly a loss of funds.
+====
+
+==== What Hardware Is Required to Run a Lightning Node?
+
+((("hardware, Lightning node")))((("Lightning node operation","hardware requirements")))At a minimum, the following are required to run a Lightning node:
+
+CPU:: Sufficient processing power is required to run a Bitcoin node, which will continuously download and validate new blocks. The user also needs to consider the initial block download (IBD) when setting up a new Bitcoin node, which can take anywhere from several hours to several days. A 2-core or 4-core CPU is recommended.
+
+RAM:: A system with 2 GB of RAM will _barely_ run both Bitcoin and Lightning nodes. It will perform much better with at least 4 GB of RAM. The IBD will be especially challenging with less than 4 GB of RAM. More than 8 GB of RAM is unnecessary because the CPU is the greater bottleneck for these types of services, due to cryptographic operations such as signature validation.
+
+Storage drive:: This can be a hard disk drive (HDD) or a solid state drive (SSD).
+An SSD will be significantly quicker (but more expensive) for running a node.
+Most of the storage is used for the Bitcoin blockchain, which is hundreds of gigabytes in size.
+A fair trade-off (cost for complexity) is to buy a small SSD to boot the OS from, and a larger HDD to store large data objects (mostly databases).
+
+[NOTE]
+====
+Raspberry Pis are a common choice for running node software, due to the cost and parts availability.
+The OS that runs on the device usually boots from a secure digital (SD) card.
+For most use cases, this is a nonissue, but Bitcoin Core is notorious for being I/O heavy.
+You should make sure to place the Bitcoin blockchain and Lightning data directory on a different drive because long-term intensive I/O can cause an SD card to fail.
+====
+
+Internet connection:: A reliable internet connection is required to download new Bitcoin blocks, as well as to communicate with other Lightning peers. During operation the estimated data use ranges from 10 to 100 GB per month, depending on configuration. At startup, a Bitcoin full node downloads the full blockchain.
+
+Power supply:: A reliable power supply is required because Lightning nodes need to be online at all times. A power failure will cause in-progress payments to fail. For heavy duty routing nodes, a backup or uninterruptible power supply (UPS) is useful in the event of power outages.
+Ideally, you should connect your internet router to this UPS as well.
+
+Backup:: Backup is crucial because a failure can result in loss of data and hence in loss of funds.
+You will want to consider some kind of data backup solution. This could be a cloud-based automated backup to a server or web service you control. Alternatively, it could be an automated local hardware backup, such as a second hard drive. For best results, both local and remote backup can be combined.
+
+==== Switching Server Configuration in the Cloud
+
+((("Lightning node operation","switching server configuration in the cloud")))When renting a cloud server, it is often cost effective to change the configuration between two phases of operation. A faster CPU and faster storage will be needed during the IBD (e.g., the first day). After the blockchain has synced, the CPU and storage speed requirements are much less, so the performance can be downgraded to a more cost-effective level.
+
+For example, on Amazon's cloud, we would use an 8–16 GB RAM, 8-core CPU (e.g., t3-large or m3.large) and faster 400 GB SSD (1000+ provisioned input/output operations per second [IOPS]) for the IBD, reducing its time to just 6-8 hours. Once that is complete, we would switch the server instance to a 2 GB RAM, 2-core CPU (e.g., t3.small) and storage to a general purpose 1 TB HDD. This will cost about the same as if you ran it on the slower server the entire time, but it will get you up and running in less than a day instead of having to wait almost a week for the IBD.
+
+===== Permanent data storage (drive)
+
+((("data storage")))((("Lightning node operation","permanent data storage")))If you use a mini PC or rent a server, the storage can be the most expensive part, costing as much as the computer and connectivity (data) added together.
+
+Let's have a look at the different options available. First, there are two main types of drives, HDDs and SSDs. HDDs are cheaper and SSDs are faster, but both do the job.
+
+((("Non-Volatile Memory Express (NVMe)")))((("solid state drives (SSDs)")))((("SSDs (solid state drives)")))The fastest SSDs available today use the Non-Volatile Memory Express (NVMe) interface. The NVMe SSDs are faster in high-end machines, but also more costly.
+Traditional SATA-based SSDs are cheaper, but not as fast. SATA SSDs perform sufficiently well for your node setup.
+Smaller computers might not be able to take advantage of NVMe SSDs.
+For example, the Raspberry Pi 4 cannot benefit from them because of the limited bandwidth of its USB port.
+
+To choose the size, let's look at the Bitcoin blockchain. As of August 2021, its size is 360 GB, including the transaction index, and grows by roughly 60 GB per year. If you want to have some margin available for future growth or to install other data on your node, purchase at least a 512 GB drive, or better yet, a 1 TB drive.(((range="endofrange", startref="ix_05_node_operations-asciidoc1")))
+
+[[helpers]]
+=== Using an Installer or Helper
+
+((("helpers (installation/configuration software)", id="ix_05_node_operations-asciidoc2", range="startofrange")))((("Lightning node operation","using an installer or helper", id="ix_05_node_operations-asciidoc3", range="startofrange")))Installing a Lightning node or a Bitcoin node may be daunting if you are not familiar with a command-line environment. Luckily, there are a number of projects that make "helpers," i.e., software that installs and configures the various components for you. You will still need to learn some command-line incantations to interact with your node, but most of the initial work is done for you.
+
+==== RaspiBlitz
+
+((("helpers (installation/configuration software)","RaspiBlitz")))((("RaspiBlitz")))One of the most popular and complete "helpers" is _RaspiBlitz_ (<<RaspiBlitz>>), a project built by Christian Rotzoll. It is intended to be installed on a Raspberry Pi 4. RaspiBlitz comes with a recommended hardware kit that you can build in a matter of hours or at most a weekend. If you attend a Lightning "hackathon" in your city, you are likely to see many people working on their RaspiBlitz setup, swapping tips, and helping each other. You can find the RaspiBlitz project on https://github.com/rootzoll/raspiblitz[GitHub].
+
+In addition to a Bitcoin and Lightning node, RaspiBlitz can install a number of additional services, such as:
+
+* Tor (run as hidden service)
+* ElectRS (Electrum server in Rust)
+* BTCPay Server (cryptocurrency payment processor)
+* BTC RPC Explorer (Bitcoin blockchain explorer)
+* Ride The Lightning (Lightning node management GUI)
+* LNbits (Lightning wallet/accounts system)
+* Specter Desktop (multisig Trezor, Ledger, Coldcard wallet, and Specter-DIY)
+* lndmanage (command-line interface for advanced channel management)
+* Loop (submarine swaps service)
+* JoinMarket (CoinJoin service)
+
+[[RaspiBlitz]]
+.A RaspiBlitz node
+image::images/mtln_0501.png[]
+
+==== Mynode
+
+((("helpers (installation/configuration software)","myNode")))((("myNode")))https://mynodebtc.com[_myNode_] is another popular open source "helper" project including a lot of Bitcoin related software. It is easy to install: you "flash" the installer onto an SD card and boot your mini PC from the SD card. You do not need any monitor to use myNode because the administrative tools are accessible remotely from a browser. If your mini PC has no monitor, mouse, or keyboard, you can manage it from another computer or even from your smartphone. Once installed, go to http://mynode.local and create a Lightning wallet and node in two clicks.
+
+In addition to a Bitcoin and Lightning node, myNode can optionally install a variety of additional services, such as:
+
+* Ride The Lightning (Lightning node management GUI)
+* OpenVPN (virtual private network [VPN] support for remote management or wallet)
+* lndmanage (command-line interface for advanced channel management)
+* BTC RPC Explorer (a Bitcoin blockchain explorer)
+
+==== Umbrel
+
+((("helpers (installation/configuration software)","Umbrel", id="ix_05_node_operations-asciidoc4", range="startofrange")))((("Umbrel", id="ix_05_node_operations-asciidoc5", range="startofrange")))Famous for their UX/UI (shown in <<umbrel>>), _Umbrel_ provides a very easy and accessible way to get your Bitcoin and Lightning node up and running in no time, especially for beginners. A very distinctive feature is that Umbrel utilizes Neutrino/SPV during the IBD so you can instantly start using your node. Once Bitcoin Core is fully synced in the background, it automatically switches over and disables SPV mode. Umbrel OS supports the Raspberry Pi 4 and can also be installed on any Linux-based OS or on a virtual machine on macOS or Windows. You can also connect any wallet that supports Bitcoin Core P2P, Bitcoin Core RPC, the Electrum protocol, or lndconnect.
+
+There's no need to wait for a rainy day&mdash;you can go right to https://getumbrel.com[Umbrel] to learn more.
+
+[[umbrel]]
+.The Umbrel web interface
+image::images/mtln_0502.png["The Umbrel web interface"]
+
+In addition to a Bitcoin and Lightning node, Umbrel introduced the Umbrel App Store, where you can easily install additional services, such as:
+
+* Lightning Terminal (interface for managing channel liquidity, Loop In, and Loop Out)
+* Ride The Lightning (Lightning node management GUI)
+* Specter Desktop (watch-only coordinator for multisignature and single-key Bitcoin wallets)
+* BTCPay Server (cryptocurrency payment processor)
+* BTC RPC Explorer (Bitcoin blockchain explorer)
+* ThunderHub (monitor and manage your node)
+* Sphinx Relay (handling connectivity and storage for Sphinx chat)
+* mempool.space (mempool visualizer and block explorer)
+* LNbits (Lightning wallet/accounts system)
+
+Umbrel is currently still in beta and is not considered secure.(((range="endofrange", startref="ix_05_node_operations-asciidoc5")))(((range="endofrange", startref="ix_05_node_operations-asciidoc4")))
+
+==== BTCPay Server
+
+((("BTCPay Server")))((("helpers (installation/configuration software)","BTCPay Server")))While not initially designed as an installation "helper," the ecommerce and payment platform _BTCPay Server_ has an incredibly easy installation system that uses Docker containers and +docker-compose+ to install a Bitcoin node, Lightning node, and payment gateway, among many other services. It can be installed on a variety of hardware platforms, from a simple Raspberry Pi 4 (4 GB recommended) to a mini PC or old laptop, desktop, or server.
+
+https://btcpayserver.org[BTCPay Server] is a fully featured self-hosted, self-custody ecommerce platform that can be integrated with many ecommerce platforms, such as WordPress WooCommerce and others. The installation of the full node is only a step of the ecommerce platform installation.
+While originally developed as a feature-for-feature replacement of the _BitPay_ commercial payment service and API, it has evolved past that to be a complete platform for BTC and Lightning services related to ecommerce. For many sellers or shops it is a one-shop turnkey solution to ecommerce.
+
+In addition to a Bitcoin and Lightning node, BTCPay Server can also install a variety of services, including:
+
+* `c-lightning` or LND Lightning node
+* Litecoin support
+* Monero support
+* Spark server (`c-lightning` web wallet)
+* Charge server (`c-lightning` ecommerce API)
+* Ride The Lightning (Lightning node management web GUI)
+* Many BTC forks
+* BTCTransmuter (event-action automation service supporting currency exchange)
+
+The number of additional services and features is growing rapidly, so the preceding list is only a small subset of what is available on the BTCPay Server platform.
+
+==== Bitcoin Node or Lightweight Lightning
+
+((("Bitcoin node")))((("helpers (installation/configuration software)","Bitcoin node versus lightweight Lightning node")))One critical choice for your setup will be the choice of the Bitcoin node and its configuration. _Bitcoin Core_, the reference implementation, is the most common choice but not the only choice available. One alternative choice is _btcd_, which is a Go-language implementation of a Bitcoin node. btcd supports some features that are useful for running an LND Lightning node and are not available in Bitcoin Core.
+
+A second consideration is whether you will run an _archival_ Bitcoin node with a full copy of the blockchain (some 350 GB in mid-2021) or a _pruned_ blockchain that only keeps the most recent blocks. A pruned blockchain can save you some disk space, but you will still need to download the full blockchain at least once (during the IBD). Hence it won't save you any network traffic. Using a pruned node to run a Lightning node is still an experimental capability and might not support all the functionality. However, many people are running a node like that successfully.
+
+((("lightweight Lightning node")))Finally, you also have the option of not running a Bitcoin node at all. Instead you can operate the LND Lightning node in "lightweight" mode, using the Neutrino Protocol to retrieve blockchain information from public Bitcoin nodes operated by others. Running like this means that you are taking resources from the Bitcoin network without offering any in return. Instead, you are offering your resources and contributing to the LN community. For smaller Lightning nodes this will generally reduce network traffic in comparison to running a full Bitcoin node.
+
+Keep in mind that operating a Bitcoin node allows you to support other services, besides and on top of a Lightning node. These other services may require an archival (not pruned) Bitcoin node and often can't run without a Bitcoin node. Consider up front what other services you may want to run now or in the future to make an informed decision on the type of Bitcoin node you select.
+
+The bottom line for this decision is: if you can afford a disk larger than 500 GB, run a full archival Bitcoin node. You will be contributing resources to the Bitcoin system and helping others who cannot afford to do so. If you can't afford such a big disk, run a pruned node. If you can't afford the disk or the bandwidth for even a pruned node, run a lightweight LND node over Neutrino.
+
+==== Operating System Choice
+
+((("Lightning node operation","operating system choice")))((("operating system","for Lightning node")))The next step is to select an operating system for your node. The vast majority of internet servers run on some variant of Linux. Linux is the platform of choice for the internet because it is a powerful open source operating system. Linux, however, has a steep learning curve and requires familiarity with a command-line environment. It is often intimidating for new users.
+
+Ultimately, most of the services can be run on any modern POSIX operating system, which includes macOS, Windows, and of course Linux. Your choice should be driven more by your familiarity and comfort with an operating system and your learning objectives. If you want to expand your knowledge and learn how to operate a Linux system, this is a great opportunity to do so with a specific project and a clear goal. If you just want to get a node up and running, go with what you know.
+
+Nowadays, many services are also delivered in the form of containers, usually based on the Docker system. These containers can be deployed on a variety of operating systems, abstracting the underlying OS. You may need to learn some Linux CLI commands nonetheless, as most of the containers run some variant of Linux inside.(((range="endofrange", startref="ix_05_node_operations-asciidoc3")))(((range="endofrange", startref="ix_05_node_operations-asciidoc2")))
+
+=== Choose Your Lightning Node Implementation
+
+((("Lightning node operation","implementation choice")))As with the choice of operating system, your choice of Lightning node implementation should depend primarily on your familiarity with the programming language and development tools used by the projects. While there are some small differences in features between the various node implementations, those are relatively minor, and most implementations converge on the common standards defined by the BOLTs.
+
+Familiarity with the programming language and build system, on the other hand, is a good basis for choosing a node. That's because installation, configuration, ongoing maintenance, and troubleshooting will all involve interacting with the various tools used by the build system. This includes:
+
+* Make, Autotools, and GNU utilities for `c-lightning`
+* Go utilities for LND
+* Java/Maven for Eclair
+
+The programming language influences not only the choice of build system but also many other aspects of the program. Each programming language comes with a whole design philosophy and affects many other aspects, such as:
+
+* Format and syntax of configuration files
+* File locations (in the filesystem)
+* Command-line arguments and their syntax
+* Error message formatting
+* Prerequisite libraries
+* Remote procedure call interfaces
+
+When you choose your Lightning node, you are also choosing all the aforementioned characteristics. So your familiarity with these tools and design philosophies will make it easier to run a node. Or harder, if you land in an unfamiliar domain.
+
+On the other hand, if this is your first foray into the command-line and server/service environment, you will find yourself unfamiliar with any implementation and have the opportunity to learn something completely new. In that case you might want to decide based on a number of other factors, such as:
+
+* Quality of support forums and chat rooms
+* Quality of documentation
+* Degree of integration with other tools you want to run
+
+As a final consideration, you may want to examine the performance and reliability of different node implementations. This is especially important if you will be using this node in a production environment and expect heavy traffic and high reliability requirements. This might be the case if you plan to run the payment system of a shop on it.
+
+=== Installing a Bitcoin or Lightning Node
+
+((("Bitcoin node","installation/configuration", id="ix_05_node_operations-asciidoc6", range="startofrange")))((("Lightning node operation","installing Bitcoin node or Lightning node", id="ix_05_node_operations-asciidoc7", range="startofrange")))((("Linux, installing Bitcoin node or Lightning node", id="ix_05_node_operations-asciidoc8", range="startofrange")))You decided not to use an installation "helper" and instead to dive into the command line of a Linux operating system? That is a brave decision, and we'll try to help you make it work. If you'd rather not try to do this manually, consider using an application that helps you install the node software or a container-based solution, as described in <<helpers>>.
+
+[WARNING]
+====
+This section will delve into the advanced topic of system administration from the command line. Linux administration is its own skill set that is outside the scope of this book. It is a complicated topic and there are many pitfalls. Proceed with caution!
+====
+
+In the next few sections we will briefly describe how to install and configure a Bitcoin and Lightning node on a Linux operating system. You will need to review the installation instructions for the specific Bitcoin and Lightning node applications you decided to use. You can usually find these in a file called _INSTALL_ or in the _docs_ subdirectory of each project. We will only describe some of the common steps that apply to all such services, and the instructions we offer will be necessarily incomplete.
+
+==== Background Services
+
+((("background services")))((("Lightning node operation","background services")))For those accustomed to running applications on their desktop or smartphone, an application always has a graphical user interface even if it may sometimes run in the background. The Bitcoin and Lightning node applications, however, are very different. These applications do not have a graphical user interface built in. Instead, they run as _headless_ background services, meaning they are always operating in the background and do not interact with the user directly.
+
+This can create some confusion for users who are not used to running background services. How do you know if such a service is currently running? How do you start and stop it? How do you interact with it? The answers to these questions depend on the operating system you are using. For now we will assume you are using some Linux variant and answer them in that context.
+
+==== Process Isolation
+
+((("Lightning node operation","process isolation")))Background services usually run under a specific user account to isolate them from the operating system and each other. For example, Bitcoin Core is configured to run as user +bitcoin+. You will need to use the command line to create a user for each of the services you run.
+
+In addition, if you have connected an external drive, you will need to tell the operating system to relocate the user's home directory to that drive. That's because a service like Bitcoin Core will create files under the user's home directory. If you are setting it up to download the full Bitcoin blockchain, these files will take up several hundred gigabytes. Here, we assume you have connected the external drive and it is located on the _/external_drive/_ path of the operating system.
+
+On most Linux systems you can create a new user with the +useradd+ command, like this:
+
+----
+$ sudo useradd -m -d /external_drive/bitcoin -s /dev/null bitcoin
+----
+
+The +m+ and +d+ flags create the user's home directory as specified by _/external_drive/bitcoin_ in this case. The +s+ flag assigns the user's interactive shell. In this case, we set it to _/dev/null_ to disable interactive shell use. The last argument is the new user's username +bitcoin+.
+
+==== Node Startup
+
+((("Lightning node operation","node startup")))((("startup script")))For both Bitcoin and Lightning node services, "installation" also involves creating a so-called _startup script_ to make sure that the node starts when the computer boots. Startup and shutdown of background services is handled by an operating system process, which in Linux is called +init+ or +systemd+. You can usually find a system startup script in the +contrib+ subdirectory of each project. For example, if you are on a modern Linux OS that uses +systemd+, you would find a script called _bitcoind.service_ that can start and stop the Bitcoin Core node service.
+
+Here's an example of what a Bitcoin node's startup script looks like, taken from the Bitcoin Core code repository:
+
+.From bitcoin/contrib/init/bitcoind.service
+----
+[Unit]
+Description=Bitcoin daemon
+After=network.target
+
+[Service]
+ExecStart=/usr/bin/bitcoind -daemon \
+                            -pid=/run/bitcoind/bitcoind.pid \
+                            -conf=/etc/bitcoin/bitcoin.conf \
+                            -datadir=/var/lib/bitcoind
+
+# Make sure the config directory is readable by the service user
+PermissionsStartOnly=true
+ExecStartPre=/bin/chgrp bitcoin /etc/bitcoin
+
+# Process management
+####################
+
+Type=forking
+PIDFile=/run/bitcoind/bitcoind.pid
+Restart=on-failure
+TimeoutStopSec=600
+
+# Directory creation and permissions
+####################################
+
+# Run as bitcoin:bitcoin
+User=bitcoin
+Group=bitcoin
+
+# /run/bitcoind
+RuntimeDirectory=bitcoind
+RuntimeDirectoryMode=0710
+
+# /etc/bitcoin
+ConfigurationDirectory=bitcoin
+ConfigurationDirectoryMode=0710
+
+# /var/lib/bitcoind
+StateDirectory=bitcoind
+StateDirectoryMode=0710
+
+[...]
+
+[Install]
+WantedBy=multi-user.target
+----
+
+As the root user, install the script by copying it into the +systemd+ service folder _/lib/systemd/system/_ and then reload +systemd+:
+
+----
+$ sudo systemctl daemon-reload
+----
+
+[role="pagebreak-before"]
+Next, enable the service:
+
+----
+$ sudo systemctl enable bitcoind
+----
+
+You can now start and stop the service. Don't start it yet, as we haven't configured the Bitcoin node.
+
+----
+$ sudo systemctl start bitcoind
+$ sudo systemctl stop bitcoind
+----
+
+==== Node Configuration
+
+((("Lightning node operation","node configuration")))To configure your node, you need to create and reference a configuration file. By convention, this file is usually created in _/etc_, under a directory with the name of the program. For example, Bitcoin Core and LND configurations would usually be stored in _/etc/bitcoin/bitcoin.conf_ and
+_/etc/lnd/lnd.conf_, respectively.
+
+These configuration files are text files with each line expressing one configuration option and its value. Default values are assumed for anything not defined in the configuration file. You can see what options can be set in the configuration in two ways. First, running the node application with a +help+ argument will show the options that can be defined on the command line. These same options can be defined in the configuration file. Second, you can usually find an example configuration file, with all the default options, in the code repository of the software.
+
+You can find one example of a configuration file in each of the Docker images we used in <<set_up_a_lightning_node>>. For example, the file _code/docker/bitcoind/bitcoind/bitcoin.conf_:
+
+.Configuration file for docker bitcoind (code/docker/bitcoind/bitcoind/bitcoin.conf)
+----
+include::code/docker/bitcoind/bitcoind/bitcoin.conf[]
+----
+
+That particular configuration file configures Bitcoin Core for operation as a +regtest+ node and provides a weak username and password for remote access, so you shouldn't use it for your node configuration. However, it serves to illustrate the syntax of a configuration file and you can make adjustments to it in the Docker container to experiment with different options. See if you can use the +bitcoind -help+ command to understand what each of the options does in the context of the Docker network we built in <<set_up_a_lightning_node>>.
+
+Often, the defaults suffice, and with a few modifications your node software can be configured quickly. To get a Bitcoin Core node running with minimal customization, you only need four lines of configuration:
+
+[source, subs="quotes"]
+----
+server=1
+daemon=1
+txindex=1
+rpcuser=_USERNAME_
+rpcpassword=_PASSWORD_
+----
+
+Even the +txindex+ option is not strictly necessary, though it will ensure your Bitcoin node creates an index of all transactions, which is required for some applications. The +txindex+ option is not required to run a Lightning node.
+
+A `c-lightning` Lightning node running on the same server also requires only a few lines in the configuration:
+
+[source, subs="quotes"]
+----
+network=mainnet
+bitcoin-rpcuser=_USERNAME_
+bitcoin-rpcpassword=_PASSWORD_
+----
+
+In general, it is a good idea to minimize the amount of customization of these systems. The default configuration is carefully designed to support the most common deployments. If you modify a default value, it may cause problems later on or reduce the performance of your node. In short, modify only when necessary!
+
+==== Network Configuration
+
+((("Lightning node operation","network configuration", id="ix_05_node_operations-asciidoc9", range="startofrange")))((("network configuration","Lightning node", id="ix_05_node_operations-asciidoc10", range="startofrange")))Network configuration is normally not an issue when configuring a new application. However, peer-to-peer networks like Bitcoin and the Lightning Network present some unique challenges for network configuration.
+
+In a centralized service, your computer connects to the "big servers" of some corporation, and not vice versa. Your home internet connection is actually configured on the assumption that you are simply a consumer of services provided by others. But in a peer-to-peer system, every peer both consumes from and provides services to other nodes. If you're running a Bitcoin or Lightning node at your home, you're providing a service to other computers on the internet. Your internet service by default is not configured to allow you to run servers and may need some additional configuration to enable others to reach your node.
+
+If you want to run a Bitcoin or Lightning node, you need to make it possible for other nodes on the internet to connect to you. That means enabling incoming TCP connections to the Bitcoin port (port 8333 by default) or Lightning port (port 9735 by default). While you can run a Bitcoin node without incoming connectivity, you can't do that with a Lightning node. A Lightning node must be accessible to others from outside your network.
+
+By default, your home internet router does not expect incoming connections from the outside, and in fact incoming connections are blocked. Your internet router IP address is the only externally accessible IP address, and all the computers you run inside your home network share that single IP address. This is achieved by a mechanism called _Network Address Translation_ (_NAT_), which allows your internet router to act as an intermediary for all outbound connections. ((("port forwarding","defined")))If you want to allow an inbound connection, you have to set up _port forwarding_, which tells your internet router that incoming connections on specific ports should be forwarded to specific computers inside the network. You can do this manually by changing your internet router configuration or,  if your router supports it, through an automatic port forwarding mechanism called _Universal Plug and Play_ (_UPnP_).
+
+An alternative mechanism to port forwarding is to enable The Onion Router (Tor), which provides a kind of virtual private network overlay that allows incoming connections to an _onion address_. If you run Tor, you don't need to do port forwarding or enable incoming connections to Bitcoin or Lightning ports. If you run your nodes using Tor, all traffic goes through Tor and no other ports are used.
+
+Let's look at different ways you can make it possible for others to connect to your node. We'll look at these alternatives in order, from easiest to most difficult.
+
+===== It just works!
+
+There is a possibility that your internet service provider or router is configured to support UPnP by default and everything just works automatically. Let's try this approach first, just in case we are lucky.
+
+Assuming you already have a Bitcoin or Lightning node running, we will try and see if they are accessible from the outside.
+
+[NOTE]
+====
+For this test to work, you have to have either a Bitcoin or Lightning node (or both) up and running on your home network. If your router supports UPnP, the incoming traffic will automatically be forwarded to the corresponding ports on the computer running the node.
+====
+
+You can use some very popular and useful websites to find out what is your external IP address and whether it allows and forwards incoming connections to a known port. Here are two that are reliable:
+
+* https://canyouseeme.org[]
+* https://www.whatismyip.com/port-scanner[]
+
+By default, these services only allow you to check incoming connections to the IP address from which you are connecting. This is done to prevent you from using the service to scan other people's networks and computers. You will see your router's external IP address and a field for entering a port number. If you haven't changed the default ports in your node configuration, try port 8333 (Bitcoin) and/or 9735 (Lightning).
+
+In <<ln_port_check>> you can see the result of checking port 9735 on a server running Lightning, using the _whatismyip.com_ port scanner tool. It shows that the server is accepting incoming connections to the Lightning port. If you see a result like this, you are all set!
+
+[[ln_port_check]]
+.Checking for incoming port 9735
+image::images/mtln_0503.png[]
+
+===== Automatic port forwarding using UPnP
+
+((("network configuration","automatic port forwarding using UPnP")))((("port forwarding","automatic")))((("Universal Plug and Play (UPnP)")))Sometimes, even if your internet router supports UPnP, it may be turned off by default. In that case you need to change your internet router configuration from its web administration interface:
+
+. Connect to your internet router's configuration website. Usually this can be done by connecting to the _gateway address_ of your home network using a web browser. You can find the gateway address by looking at the IP configuration of any computer on your home network. It is often the first address in one of the nonroutable networks, like 192.168.0.1 or 10.0.0.1. Check all stickers on your router as well for the _gateway address_. Once found, open a browser and enter the IP address into the browser URL/Search box, e.g., "192.168.0.1" or "http://192.168.0.1."
+
+. Find the administrator username and password for the web configuration panel of the router. This is often written on a sticker on the router itself and may be as simple as "admin" and "password." A quick web search for your ISP and router model can also help you find this information.
+
+. Find a setting for UPnP and turn it on.
+
+Restart your Bitcoin and/or Lightning node and repeat the open port test with one of the websites we used in the previous section.
+
+===== Using Tor for incoming connections
+
+((("network configuration","Tor for incoming connections")))((("The Onion Router (Tor)")))((("Tor (The Onion Router)")))_The Onion Router_ (_Tor_) is a VPN with the special property that it encrypts communications between hops, such that any intermediary node cannot determine the origin or destination of a packet. Both Bitcoin and Lightning nodes support operation over Tor, which enables you to operate a node without revealing your IP address or location. Hence, it provides a high level of privacy to your network traffic. An added benefit of running Tor is that, because it operates as a VPN, it resolves the problem of port forwarding from your internet router. Incoming connections are received over the Tor tunnel, and your node can be found through an ad hoc generated _onion address_ instead of an IP address.
+
+Enabling Tor requires two steps. First, you must install the Tor router and proxy on your computer. Second, you must enable the use of the Tor proxy in your Bitcoin or Lightning configuration.
+
+To install Tor on an Ubuntu Linux system that uses the +apt+ package manager, run:
+
+----
+sudo apt install tor
+----
+
+Next, we configure our Lightning node to use Tor for its external connectivity. Here is an example configuration for LND:
+
+----
+[Tor]
+tor.active=true
+tor.v3=true
+tor.streamisolation=true
+listen=localhost
+----
+
+This will enable Tor (+tor.active+), establish a v3 onion service (+tor.v3=true+), use a different onion stream for each connection (+tor.streamisolation+), and restrict listening for connections to the local host only, to avoid leaking your IP address (pass:[<code>l&#x2060;i&#x2060;s&#x2060;t&#x2060;e&#x2060;n&#x200b;=&#x2060;l&#x2060;o&#x2060;c&#x2060;a&#x2060;l&#x2060;h&#x2060;o&#x2060;s&#x2060;t</code>]).
+
+You can check if Tor is correctly installed and working by running a simple one-line command. This command should work on most flavors of Linux:
+
+----
+curl --socks5 localhost:9050 --socks5-hostname localhost:9050 -s https://check.torproject.org/ | cat | grep -m 1 Congratulations | xargs
+----
+
+If everything is working properly, the response of this command should be +"Congratulations. This browser is configured to use Tor."+
+
+Due to the nature of Tor, you can't easily use an external service to check if your node is reachable via an onion address. Nonetheless, you should see your Tor onion address in the logs of your Lightning node. It is a long string of letters and numbers followed by the suffix +.onion+. Your node should now be reachable from the internet, with the added bonus of privacy!
+
+===== Manual port forwarding
+
+((("network configuration","manual port forwarding")))((("port forwarding","manual")))This is the most complex process and requires quite a bit of technical skill. The details depend on the type of internet router you have, your service provider settings and policies, and a lot of other context. Try UPnP or Tor first, before you try this much more difficult mechanism.
+
+The basic steps are as follows:
+
+. Find the IP address of the computer your node is on. This is usually dynamically allocated by the Dynamic Host Configuration Protocol (DHCP) and is often somewhere in the 192.168.x.x or 10.x.x.x range.
+
+. Find the media access control (MAC) address of your node's network interface. This can be found in the internet settings of that computer.
+
+. Assign a static IP address for your node so that it is always the same one. You can use the IP address it currently has. On your internet router, look for "Static Leases" under the DHCP configuration. Map the MAC address to the IP address you selected. Now your node will always have that IP address allocated to it. Alternatively, you can look at your router's DHCP configuration and find out what its DHCP address range is. Select an unused address _outside_ of the DHCP address range. Then, on the server, configure the network to stop using DHCP and hardcode the selected non-DHCP IP address into the operating system network configuration.
+
+. Finally, set up "Port Forwarding" on your internet router to route incoming traffic on specific ports to the selected IP address of your server.
+
+Once done reconfiguring, repeat the port check using one of the websites from the previous sections(((range="endofrange", startref="ix_05_node_operations-asciidoc10")))(((range="endofrange", startref="ix_05_node_operations-asciidoc9"))).(((range="endofrange", startref="ix_05_node_operations-asciidoc8")))(((range="endofrange", startref="ix_05_node_operations-asciidoc7")))(((range="endofrange", startref="ix_05_node_operations-asciidoc6")))
+
+=== Security of Your Node
+
+((("Lightning node operation","security", id="ix_05_node_operations-asciidoc11", range="startofrange")))((("security and privacy","Lightning node", id="ix_05_node_operations-asciidoc12", range="startofrange")))A Lightning node is, by definition, a _hot wallet_. That means that the funds (both on-chain and off-chain) controlled by a Lightning node are directly controlled by keys that are loaded in the node's memory or stored on the node's hard disk. If a Lightning node is compromised, it is trivial to create on-chain or off-chain transactions to drain its funds. It is therefore critically important that you protect it from unauthorized access.
+
+Security is a holistic effort, meaning that you have to secure every layer of a system. As the saying goes: the chain is only as strong as the weakest link. This is an important concept in information security, and we will apply it to our node.
+
+Despite all the security measures you will take, remember that the Lightning Network is an early-stage experimental technology and there are likely to be exploitable bugs in the code of any project you use. _Do not put more money than you are willing to risk losing on the Lightning Network._
+
+
+==== Operating System Security
+
+((("operating system","security")))((("security and privacy","operating system security")))Securing an operating system is a vast topic that is beyond the scope of this book. However, we can establish some basic principles.
+
+To secure your operating system, here are some of the top items to consider:
+
+Provenance:: Start by ensuring that you are downloading the correct operating system image, and verify any signatures or checksums before installing it. Extend this to any software that you install. Double-check any source or URL from where you download. Verify the integrity and correctness of the downloaded software via signature and checksum verification.
+Maintenance:: Make sure that you keep your operating system up to date. Enable automated daily or weekly installation of security updates.
+Least privilege: set up users for specific processes and give them the least access needed to run a service. Do not run processes with admin privileges (e.g., +root+).
+Process isolation:: Use the operating system features to isolate processes from each other.
+Filesystem permissions:: Configure the filesystem carefully, on the least-privilege principle. Do not make files readable or writable by everyone.
+Strong authentication:: Use strong randomly generated passwords or, whenever possible, public-key authentication. For example, it is safer to use Secure Shell (SSH) with a cryptographic key pair instead of a password.
+Two-factor authentication (2FA):: Use two-factor authentication wherever possible, including Universal 2nd Factor (U2F) with hardware security keys. This applies to all external services you might be using, such as your cloud service provider. You can apply this also to your own setup, such as your own SSH configuration. Use 2FA also for indirect services. For example, say you are using a cloud service. You gave your cloud service provider an email address, so you should also protect your email address with 2FA.
+Backup:: Make backups of your system, and make sure you protect the backups with encryption too. Perform these backups periodically. At least once, test if you can restore your backup and that your backup is complete and accessible. If possible, keep one copy of your backups on a different disk to avoid a single hard disk failure destroying _both_ your active node as well as your backup copies.
+Vulnerability and exposure management:: Use remote scanning to ensure you have minimized the attack surface of your system. Close any unnecessary services or ports. Install only software and packages that you really need and use. Uninstall packages that you no longer use. It is recommended that you do _not_ use your node computer for non-node activities that you can perform on another of your computers. Especially, if you can, do _not_ use your node computer for browsing, surfing the internet, or reading your email.
+
+This is a list of the most basic security measures. It is by no means exhaustive.
+
+==== Node Access
+
+((("Lightning node operation","node access")))((("remote procedure call (RPC) API")))((("RPC (remote procedure call) API")))Your Lightning node will expose a remote procedure call (RPC) API. This means that your node can be controlled remotely by commands sent to a specific TCP port. Access control to that RPC API is achieved by some form of user authentication. Depending on the type of Lightning node you set up, this will either be done by pass:[<span class="keep-together">username/password</span>] authentication or by a mechanism called an authentication _macaroon_. As the name implies, a macaroon is a more sophisticated type of cookie. Unlike a cookie, it is cryptographically signed and can express a set of access pass:[<span class="keep-together">capabilities</span>].
+
+For example, LND uses macaroons to grant access to the RPC API. By default, the LND software creates three macaroons with different levels of access, called +admin+, +invoice+, and +readonly+. Depending on which macaroon you copy and use in your RPC client, you either have _read-only_ access, _invoice_ access (which includes the read-only capabilities), or _admin_ access, which gives you full control. There is also a macaroon +bakery+ function in LND that can construct macaroons with any combination of capabilities with very fine-grained control.
+
+If you use a username/password authentication model, make sure you select a long and random password. You will not have to type this password often, because it will be stored in the configuration files. You should therefore pick one that cannot be guessed. Many of the examples you will see include poorly chosen passwords, and often people copy these into their own systems, providing easy access to anyone. Don't do that! Use a password manager to generate a long random alphanumeric password. Since certain special characters such as +$?/!*\&%`"'+ can interfere with the command line, it is best to avoid these for passwords that will be used in a shell environment. To avoid problems, stick with long random alphanumeric passwords.
+
+A plain alphanumeric sequence that is longer than 12 characters and randomly generated is usually sufficient. If you plan to store large amounts of money on your Lightning node and are concerned about remote brute-force attacks, select a password length of more than 20 characters to make such attacks practically infeasible.(((range="endofrange", startref="ix_05_node_operations-asciidoc12")))(((range="endofrange", startref="ix_05_node_operations-asciidoc11")))
+
+=== Node and Channel Backups
+
+((("backups", id="ix_05_node_operations-asciidoc13", range="startofrange")))((("Lightning Network channels","backups", id="ix_05_node_operations-asciidoc14", range="startofrange")))((("Lightning node operation","node and channel backups", id="ix_05_node_operations-asciidoc15", range="startofrange")))A very important consideration when running a Lightning node is the issue of backups. Unlike a Bitcoin wallet, where a BIP-39 mnemonic phrase can recover all the state of the wallet, in Lightning this is _not_ the case.
+
+Lightning wallets do use a BIP-39 mnemonic phrase backup, but only for the on-chain wallet. However, due to the way channels are constructed, the mnemonic phrase is _not_ sufficient to restore a Lightning node. ((("SCB (static channel backup)")))((("static channel backup (SCB)")))An additional layer of backups is needed, which is called the _static channel backup_ (_SCB_). Without an SCB, a Lightning node operator may lose _all_ the funds that are in channels if they lose the Lightning node data store.
+
+[WARNING]
+====
+Do _not_ fund channels until you have put a system in place to continuously back up your channel state. Your backups should be moved "offsite" to a different system and location from your node, so that they can survive a variety of system failures (power loss, data corruption, etc.) or natural disasters (flood, fire, etc.).
+====
+
+SCBs are not a panacea. First, the state of each channel needs to be backed up every time there is a new commitment transaction. Second, restoring from a channel backup is dangerous. If you do not have the _last_ commitment transaction and you accidentally broadcast an old (revoked) commitment, your channel peer will assume you are trying to cheat and claim the entire channel balance with a penalty transaction. To make sure you are closing the channel, you need to do a _cooperative close_. But a malicious peer could mislead your node into broadcasting an old, revoked commitment during that cooperative close, thereby cheating you by making your node inadvertently try to cheat.
+
+Furthermore, the backups of your channels need to be encrypted to maintain your privacy and your channel security. Otherwise, anyone who finds the backups can not only see all your channels but also could use the backups to close all your channels in a way that hands over the balance to your channel peers. In other words, a malicious person that gets access to your backups can cause you to lose all your channel funds.
+
+You can see that SCBs are not a foolproof safeguard. They are a weak compromise because they swap one type of risk (data corruption or loss) for another type of risk (malicious peer). To restore from an SCB, you have to interact with your channel peers and hope they don't try to cheat you by either giving you an old commitment or by fooling your node into broadcasting a revoked commitment so they can penalize you. Despite the weaknesses of SCB, SCBs do make sense and you should perform them. If you do not perform SCBs and you lose your node data, you will lose your channel funds forever. Guaranteed! However, if you _do_ perform SCBs and you lose your node data, then you have a reasonable chance that some of your peers are honest and that you can recover some of your channel funds. If you are lucky, you might recover all your funds. In conclusion, it is best for you to perform continuous SCBs to a disk other than the primary node hard disk.
+
+Channel backup mechanisms are still a work in progress and a weakness in most Lightning implementations.
+
+((("Lightning Network Daemon (LND) node project","SCBs and")))At the time of writing this book, only LND offers a built-in mechanism for SCBs. Eclair has a similar mechanism deployed for server-side deployments, although Eclair Mobile does offer optional backup to a Google Drive. `c-lightning` recently merged the necessary interfaces for a plug-in to implement channel backups. Unfortunately, there is no consistent, agreed upon backup mechanism across different node pass:[<span class="keep-together">implementations</span>].
+
+File-based backups of the Lightning node databases are at best a partial solution because you run the risk of backing up an inconsistent database state. In addition, you may not reliably catch the latest state commitments. It is much better to have a backup mechanism that is triggered every time there is a state change in a channel, thereby ensuring data consistency.
+
+To set up SCBs in LND, set the +backupfilepath+ parameter either on the command line or in the configuration file. LND will then save an SCB file in that directory path. Of course, that's only the first step of the solution. Now, you have to set up a mechanism that monitors this file for changes. Each time the file changes, pass:[<span class="keep-together">the backup</span>] mechanism must copy this file to another, preferably off-site disk. Such backup mechanisms are beyond the scope of this book. Nonetheless, any sophisticated backup solution should be able to handle this scenario. Recall, the backup files should be encrypted too.
+
+==== Hot Wallet Risk
+
+((("Lightning node operation","hot wallet risk")))As ((("hot wallets","security issues", id="ix_05_node_operations-asciidoc16", range="startofrange")))((("security and privacy","hot wallet risk", id="ix_05_node_operations-asciidoc17", range="startofrange")))we've discussed previously, the Lightning Network consists of a network of _hot wallets_. The funds you store in a Lightning wallet are online all the time. This makes them vulnerable. Hence, you should not store large amounts in a Lightning wallet. Large amounts should be kept in a _cold_ wallet that is _not_ online and which can transact only on-chain.
+
+Even if you start small, as time passes you may still find you have a significant amount of money in a Lightning wallet. This is a typical scenario for store owners. If you use a Lightning node for an ecommerce operation, your wallet will likely receive funds often, but send funds rarely. You will therefore end up having two problems simultaneously. First, your channels will be imbalanced, with large local balances outweighing small remote balances. Secondly, you will have too much money in the wallet. Fortunately, you can also solve both of these problems simultaneously.
+
+Let's look at some of the solutions you can use to reduce the funds exposed in a hot wallet.
+
+==== Sweeping Funds
+
+((("hot wallets","sweeping funds")))((("sweeping funds","hot wallets")))If your Lightning wallet balance becomes too large for your risk tolerance, you will need to "sweep" funds out of the wallet. You can do so in three ways: on-chain, off-chain, and Loop Out. Let's look at each of these options in the next few sections. 
+
+===== On-chain sweep
+
+((("sweeping funds","on-chain sweep")))Sweeping funds on-chain is accomplished by moving the funds from the Lightning wallet to a Bitcoin wallet. You do that by closing channels. When you close a channel, all funds from your local balance are "swept" to a Bitcoin address. The Bitcoin address for on-chain funds is usually generated by your Lightning wallet, so it is still a hot wallet. You may need to do an additional on-chain transaction to move the funds to a more secure address, such as one generated on your hardware wallet.
+
+Closing channels will incur an on-chain fee and will reduce your Lightning node's capacity and connectivity. However, if you run a popular ecommerce node, you will not lack incoming capacity and can strategically close channels with large local balances, essentially "bundling" your funds for movement on-chain. You may need to use some channel rebalancing techniques (see <<channel_rebalancing>>) before you close channels to maximize the benefits of this strategy.
+
+===== Off-chain sweep
+
+((("sweeping funds","off-chain sweep")))Another technique you can use involves running a second Lightning node that is not advertised on the network. You can establish large capacity channels from your public node (e.g., the one running your shop) to your unadvertised (hidden) node. On a regular basis, "sweep" funds by making a Lightning payment to your hidden node.
+
+The advantage of this technique lies in the fact that the Lightning node that receives payments for your shop will be publicly known. This makes it a target for hackers, as any Lightning node associated with a shop would be assumed to have a large balance. A second node that is not associated with your shop will not easily be identified as a valuable target.
+
+As an additional measure of security, you can make your second node a hidden Tor service so that its IP address is not known. That further reduces the opportunity for attacks and increases your privacy.
+
+You will need to set up a script that runs at regular intervals. The purpose of this script is to create an invoice on your hidden node and to pay that invoice from your shop's node, thereby shifting funds over to your hidden node.
+
+Keep in mind that this technique does not move funds into cold storage. Both Lightning nodes are hot wallets. The objective of this sweep is to move funds from a very well-known hot wallet to an obscure hot wallet.
+
+===== Submarine swap sweep
+
+((("submarine swaps")))((("sweeping funds","submarine swap sweep")))Another way to reduce your Lightning hot-wallet balance is to use a technique called a _submarine swap_. Submarine swaps, conceptualized by coauthor Olaoluwa Osuntokun and Alex Bosworth, allow the exchange of on-chain bitcoin for Lightning payments and vice versa. Essentially, submarine swaps are atomic swaps between Lightning off-chain funds and Bitcoin on-chain funds.
+
+A node operator can initiate a submarine swap and send all available channel balances to the other party, who will send them on-chain bitcoin in exchange.
+
+In the future, this could be a paid service offered by nodes on the Lightning Network who advertise exchange rates or charge a flat fee for the conversion.
+
+The advantage of a submarine swap for sweeping funds is that no channel needs to be closed. That means that we preserve our channels, only rebalancing our channels through this operation. As we send a Lightning payment out, we shift some balance from local to remote on one or more of our channels. Not only does that reduce the balance exposed in our node's hot wallet, it also increases the balance available for future incoming payments.
+
+You could do this by trusting an intermediary to act as a gateway, but this risks your coins being stolen. However, in the case of a submarine swap, the operation does not require trust. Submarine swaps are noncustodial _atomic_ operations. That means that the counterparty in your submarine swap cannot steal your funds because the on-chain payment depends on the completion of the off-chain payment and vice versa.
+
+===== Submarine swaps with Loop
+
+((("Loop, submarine swaps with")))((("sweeping funds","submarine swaps with Loop")))One example of a submarine swap service is _Loop_ by Lightning Labs, the same company that builds LND. Loop comes in two variations: Loop In and Loop Out. _Loop In_ accepts a Bitcoin on-chain payment and converts it into a Lightning off-chain payment. _Loop Out_ converts a Lightning payment into a Bitcoin payment.
+
+[NOTE]
+====
+To use the Loop service, you must be running an LND Lightning node.
+====
+
+For the purpose of reducing the balance of your Lightning hot wallet, you would use the Loop Out service. To use the Loop service, you need to install some additional software on your node. The Loop software runs alongside your LND node and provides some command-line tools to execute submarine swaps. You can find the Loop software and installation instructions on https://github.com/lightninglabs/loop[GitHub].
+
+Once you have the software installed and running, a Loop Out operation is as simple as running a single command:
+
+----
+loop out --amt 501000 --conf_target 400
+Max swap fees for 501000 sat Loop Out: 25716 sat
+Regular swap speed requested, it might take up to 30m0s for the swap to be executed.
+CONTINUE SWAP? (y/n), expand fee detail (x): x
+
+Estimated on-chain sweep fee:        149 sat
+Max on-chain sweep fee:              14900 sat
+Max off-chain swap routing fee:      10030 sat
+Max no show penalty (prepay):        1337 sat
+Max off-chain prepay routing fee:    36 sat
+Max swap fee:                        750 sat
+CONTINUE SWAP? (y/n): y
+Swap initiated
+
+Run `loop monitor` to monitor progress.
+----
+
+Note that your maximum fee, which represents a worst-case scenario, will depend on the confirmation target that you select(((range="endofrange", startref="ix_05_node_operations-asciidoc17")))(((range="endofrange", startref="ix_05_node_operations-asciidoc16"))).(((range="endofrange", startref="ix_05_node_operations-asciidoc15")))(((range="endofrange", startref="ix_05_node_operations-asciidoc14")))(((range="endofrange", startref="ix_05_node_operations-asciidoc13")))
+
+=== Lightning Node Uptime and Availability
+
+((("Lightning node operation","uptime and availability", id="ix_05_node_operations-asciidoc18", range="startofrange")))Unlike Bitcoin, Lightning nodes need to be online almost continuously. Your node needs to be online to receive payments, open channels, close channels (cooperatively), and monitor protocol violations. Node availability is such an important requirement in the Lightning Network that it is a metric used by various automatic channel management tools (e.g., +autopilot+) to decide which nodes to open channels with. You can also see "availability" as a node metric on popular node explorers (see <<ln_explorer>>) such as https://1ml.com[1ML].
+
+Node availability is especially important to mitigate and resolve potential protocol violations (i.e., revoked commitments). While you can afford short interruptions from an hour up to one or two days, you cannot have your node offline for longer periods of time without risking loss of funds.
+
+Keeping a node online continuously is not easy, as various bugs and resource limitations can and will occasionally cause downtime. Especially if you run a busy and popular node, you will run into limitations of memory, swap space, number of open files, disk space, and so forth. A whole host of different problems will cause your node or your server to crash.
+
+==== Tolerate Faults and Automate
+
+((("automation, Lightning node")))((("fault tolerance, Lightning node")))((("Lightning node operation","fault toleration and automation")))If you have the time and skills, you should test some basic fault scenarios on the Lightning testnet. On the testnet you will learn valuable lessons without risking any funds. Any step you perform to automate your system will improve your availability:
+
+Automatic computer server restart:: What happens when your server or the operating system crashes? What happens when there is a power outage? Simulate this fault by pressing the "reset" button on your PC or by unplugging the power cable. After a crash, reset, or power failure, the computer should automatically restart itself. Some computers have a setting in their BIOS to specify how the computer should react on power failures. Test it to make sure the computer really restarts automatically on power failure without human intervention.
+
+Automatic node restart:: What happens when your node or one of your nodes crashes? Simulate this fault by killing the corresponding node processes. If a node crashes, it should automatically restart itself. Test it to make sure the node or nodes really restart automatically on failure without human intervention. If this is not the case, most likely your node is not set up correctly as an operating system service.
+
+Automatic network reconnection:: What happens if your network goes down? What happens when your ISP goes down temporarily? What happens when your ISP assigns a new IP address to your router or your computer? When the network comes back, do the nodes you are running automatically reconnect to the network? Simulate this fault by unplugging and later replugging the Ethernet cable from the device hosting your nodes. The nodes should automatically reconnect and continue operation without human intervention.
+
+Configure your logfiles:: All of the preceding failures should leave textual entries behind in the corresponding logfiles. Turn up the verbosity of logging if needed. Find these error entries in the logfiles and use them for monitoring.
+
+==== Monitoring Node Availability
+
+((("Lightning node operation","monitoring node availability")))((("monitoring","node availability")))Monitoring your node is an important part of keeping it running. You need to monitor not only the availability of the computer itself, but also the availability and correct operation of the Lightning node software.
+
+There are a number of ways to do this, but most require some customization. You can use generic infrastructure monitoring or application monitoring tools, but you have to customize them specifically to query the Lightning node API to ensure the node is running, synchronized to the blockchain, and connected to channel peers.
+
+https://lightning.watch[Lightning.watch] provides a specialized service that offers Lightning node monitoring. It uses a Telegram bot to notify you of any interruptions in service. This is a free service, though you can pay (over Lightning, of course) to get faster alerts.
+
+Over time, we expect more third-party services to provide specialized Lightning node monitoring payable via micropayments. Perhaps such services and their APIs will become standardized and will one day be directly supported by Lightning node software.
+
+[[watchtowers]]
+==== Watchtowers
+
+((("Lightning node operation","watchtowers")))((("monitoring","watchtowers")))((("protocol violations, watchtowers and")))((("watchtowers")))_Watchtowers_ are a mechanism for outsourcing the monitoring and penalty resolution of Lightning protocol violations.
+
+As we mentioned in previous chapters, the Lightning protocol maintains security through a penalty mechanism. If one of your channel partners broadcasts an old commitment transaction, your node will need to exercise the revocation clause and broadcast a penalty transaction to avoid losing money. But if your node is down during the protocol violation, you might lose money.
+
+To solve this problem, we can use one or more watchtowers to outsource the job of monitoring protocol violations and issuing penalty transactions. There are two parts to a watchtower setup: a watchtower server (or simply watchtower) that monitors the blockchain and a watchtower client that asks the watchtower server for this monitoring service.
+
+Watchtower technology is still in the early stages of development and is not widely supported. However, in the following passage we list some experimental implementations that you can try.
+
+LND software includes both a watchtower server and a watchtower client. You can activate the watchtower server by adding the following configuration options:
+
+[source, subs="quotes"]
+----
+[watchtower]
+watchtower.active=1
+watchtower.towerdir=_/path_to_watchtower_data_directory_
+----
+
+You can use LND's watchtower client by activating it in the configuration and then using the command line to connect it to a watchtower server. The configuration is:
+
+----
+[wtclient]
+wtclient.active=1
+----
+
+LND's command-line client +lncli+ shows the following options for managing the watchtower client:
+
+----
+$ lncli wtclient
+
+NAME:
+   lncli wtclient - Interact with the watchtower client.
+
+USAGE:
+   lncli wtclient command [command options] [arguments...]
+
+COMMANDS:
+     add     Register a watchtower to use for future sessions/backups.
+     remove  Remove a watchtower to prevent its use for future sessions/backups.
+     towers  Display information about all registered watchtowers.
+     tower   Display information about a specific registered watchtower.
+     stats   Display the session stats of the watchtower client.
+     policy  Display the active watchtower client policy configuration.
+
+OPTIONS:
+   --help, -h  show help
+----
+
+`c-lightning` has the API hooks necessary for a watchtower client plug-in, though no such plug-in has been implemented yet.
+
+Finally, a popular standalone watchtower server is _The Eye of Satoshi_ (TEOS). It can be found on https://github.com/talaia-labs/python-teos[GitHub].(((range="endofrange", startref="ix_05_node_operations-asciidoc18")))
+
+=== Channel Management
+
+((("channel management", id="ix_05_node_operations-asciidoc19", range="startofrange")))((("Lightning node operation","channel management", id="ix_05_node_operations-asciidoc20", range="startofrange")))As a Lightning node operator, one of the recurring tasks you will need to perform is management of your channels. This means opening outbound channels from your node to other nodes, as well as getting other nodes to open inbound channels to your node. In the future, cooperative channel construction may be possible, so you can open symmetric channels that have funds committed on both ends on creation. For now, however, new channels only have funds on one end, on the originator's side. Hence, to make your node _balanced_ with both inbound and outbound capacity, you need to open channels to others and entice others to open channels to your node.
+
+==== Opening Outbound Channels
+
+((("channel management","opening outbound channels", id="ix_05_node_operations-asciidoc21", range="startofrange")))As soon as you get your Lightning node up and running, you can fund its Bitcoin wallet and then start opening channels with those funds.
+
+You must choose channel partners carefully because your node's ability to send payments depends on who your channel partners are and how well connected they are to the rest of the Lightning Network. You also want to have more than one channel to avoid being susceptible to a single point of failure. Since Lightning now supports multipart payments, you can split your initial funds into several channels and route bigger payments by combining their capacity. At the same time, avoid making your channels too small. Since you need to pay Bitcoin transaction fees to open and close a channel, the channel balance should not be so small that the on-chain fees consume a significant portion. It's all about balance!
+
+To summarize:
+
+* Connect to a few well-connected nodes
+* Open more than one channel
+* Don't open too many channels
+* Don't make the channels too small
+
+One way to find well-connected nodes is to open a channel to a popular merchant selling products on the Lightning Network. These nodes tend to be well funded and well connected. So, when you are ready to buy something online via Lightning, you can open a channel directly to the merchant's node. The merchant's node ID will be in the invoice you will receive when you try to buy something. That makes it easy.
+
+Another way to find well-connected nodes is to use a Lightning Explorer (see <<ln_explorer>>) such as https://1ml.com[1ML] and browse the list of nodes sorted by channel capacity and number of channels. Don't go for the biggest nodes, because that encourages centralization. Go for a node in the middle of the list so that you can help them grow. Another factor to consider might be the time span a node has been in operation. Nodes established for more than a year are likely to be more reputable and less risky than nodes that started operation a week ago.
+
+[[autopilot]]
+===== Autopilot
+
+((("autopilot", id="ix_05_node_operations-asciidoc22", range="startofrange")))((("channel management","autopilot for", id="ix_05_node_operations-asciidoc23", range="startofrange")))The task of opening channels can be partially automated with the use of an _autopilot_, which is software that opens channels automatically based on some heuristic rules. Autopilot software is still relatively new, and it doesn't always select the best channel partners for you. Especially in the beginning, it might be better to open channels manually.
+Autopilots currently exist in three forms:
+
+- +lnd+ incorporates an autopilot that is fully integrated with +lnd+ and runs constantly in the background while turned on.
+- +lib_autopilot.py+ can offer autopilot computations for any node implementation based on the gossip and channel data.
+- A +c-lightning+ plug-in based on +lib_autopilot.py+ exists that provides an easy-to-use interface for +c-lightning+ users.
+
+((("lnd autopilot", id="ix_05_node_operations-asciidoc24", range="startofrange")))Be aware that the +lnd+ autopilot will start running in the background as soon as it is turned on via the config file. As a result it will start opening channels immediately if you have on-chain outputs in your +lnd+ wallet.
+If you want to have full control over the bitcoin transactions that you make and the channels that you open, make sure to turn the autopilot off _before_ you load your +lnd+ wallet with bitcoin funds.
+If the autopilot was previously turned on, you might have to restart your +lnd+ before you top up your wallet with an on-chain transaction or before you close channels, which effectively gives you on-chain funds again.
+It is crucial that you set key configuration values if you want to run the autopilot.
+Have a look at this example configuration:
+
+
+----
+[lnd-autopilot]
+autopilot.active=1
+autopilot.maxchannels=40
+autopilot.allocation=0.70
+autopilot.minchansize=500000
+autopilot.maxchansize=5000000
+autopilot.heuristic=top_centrality:1.0
+----
+
+This configuration file would activate the autopilot.
+It would open channels as long as the following two conditions are met:
+
+1. Your node currently has less than 40 channels open.
+2. Less than 70% of your total funds are off-chain in payment channels.
+
+The numbers 40 and 0.7 are chosen completely arbitrarily here because we cannot make any recommendations that are valid for everyone about how many channels you should have open and what percentage of your funds should be off-chain.
+The autopilot in +lnd+ will not take into account on-chain fees. In other words, it will not delay opening channels to a time period when fees are low.
+To reduce fees, you can manually open channels during a time period when fees are low, e.g., during the weekend.
+The autopilot will make channel recommendations whenever the conditions are met and will immediately try to open a channel by using the appropriate current fees.
+According to the preceding configuration file, the channels will be between 5 mBTC (`minchansize` = 500,000 satoshi) and 50 mBTC (`maxchansize` = 5,000,000 satoshi) in size.
+As is common, the amounts in the configuration file are enumerated in satoshi.
+Currently, channels below 1 mBTC are not very useful, and we do not recommend you open channels that are too small and below this amount.
+With the wider adoption of multipart payments, smaller channels are less of a burden. But for the time being, this is our recommendation.
+
+((("c-lightning autopilot plugin")))The +c-lightning+ plug-in, which was originally written by René Pickhardt (a coauthor of this book), works very differently in comparison with the +lnd+ autopilot.
+First, it differs in the algorithms used to make the recommendations. We will not cover this here. Secondly, it differs in its user interface.
+You will need to download the _autopilot plug-in_ from the +c-lightning+ plug-in https://github.com/lightningd/plugins/tree/master/autopilot[repository] and activate it.
+
+[NOTE]
+====
+To activate a plug-in in +c-lightning+, place it into the _~/.lightning/plugins_ directory, ensure that it's executable (e.g., `chmod +x ~/.lightning/plugins/autopilot.py`), then restart +lightningd+.
+
+Alternatively, if you don't want a plug-in to automatically activate when you start +lightningd+, you can place it in a different directory and manually activate it with the +plugin+ argument to +lightningd+:
+
+----
+   lightningd --plugin=~/lightning-plugins/autopilot.py
+----
+
+====
+
+The autopilot in +c-lightning+ is controlled via three configuration values that can be set in the config file or as command-line arguments when you start +lightningd+:
+
+----
+[c-lightning-autopilot]
+autopilot-percent=75
+autopilot-num-channels=10
+autopilot-min-channel-size-msat=100000000msat
+----
+
+These values are the actual default config, and you do not need to set them at all.
+
+The autopilot will not automatically run in the background like in +lnd+.
+Instead, you have to start a run specifically with `lightning-cli autopilot-run-once` if you want the autopilot to open the recommended channels.
+But if you want it to just provide you with recommendations, from which you can handpick the nodes, you can append the optional `dryrun` argument.
+
+A key difference between the +lnd+ and the +c-lightning+ autopilots is that the +c-lightning+ autopilot will also make a recommendation for the channel size.
+For example, if the autopilot recommends opening a channel with a small node that only has small channels, it will not recommend opening a large channel.
+However, if it opens a channel with a well-connected node that also has many large channels, it will probably recommend a larger channel size.
+
+As you can see, the +c-lightning+ autopilot is not as automatic as +lnd+'s, but it gives you a little bit more control.
+These differences reflect personal preferences and could actually be the deciding factor for you to choose one implementation over the other.
+
+Keep in mind that current autopilots will primarily use public information from the gossip protocol about the current topology of the Lightning Network.
+It is obvious that your personal requirements for channels can only be reflected to a certain degree.
+More advanced autopilots would use historical and usage information that your node has gathered when running in the past, including information about routing successes, who you have paid in the past, and who paid you.
+In the future, such improved autopilots might also use this collected data to make recommendations on closing channels and reallocating funds.(((range="endofrange", startref="ix_05_node_operations-asciidoc24")))
+
+Overall, at the time of writing of this book, be cautious not to depend or rely too heavily on autopilots(((range="endofrange", startref="ix_05_node_operations-asciidoc23")))(((range="endofrange", startref="ix_05_node_operations-asciidoc22"))).(((range="endofrange", startref="ix_05_node_operations-asciidoc21")))
+
+==== Getting Inbound Liquidity
+
+((("channel management","getting inbound liquidity")))In the current design of the Lightning Network, it is more typical for users to obtain outbound liquidity _before_ obtaining inbound liquidity.
+They will do so by opening a channel with another node, and more often they'll be able to spend bitcoin before they can receive it.
+There are three typical ways of getting inbound liquidity:
+
+* Open a channel with outbound liquidity and then spend some of those funds. Now the balance is on the other end of the channel, which means that you can receive payments.
+
+* Ask someone to open a channel to your node. Offer to reciprocate, so that both of your nodes become better connected and balanced.
+
+* Use a submarine swap (e.g., Loop In) to exchange on-chain BTC for an inbound channel to your node.
+
+* Pay a third-party service to open a channel with you. Several such services exist. Some charge a fee to provide liquidity, some are free.
+
+[role="pagebreak-before"]
+Here is a list of currently available liquidity providers that will open a channel to your node for a fee:
+
+* https://www.bitrefill.com/thor-lightning-network-channels[Bitrefill's Thor service]
+
+* https://lightningto.me[Lightning To Me]
+
+* https://lnbig.com[LNBig]
+
+* https://lightningconductor.net/channels[Lightning Conductor]
+
+Creating inbound liquidity is challenging from both practical and user experience perspectives. Inbound liquidity does not happen automatically, so you have to find ways to build it for your node. This asymmetry of payment channels is also not intuitive. In most other payment systems, you get paid first (inbound) before you pay others (outbound).
+
+The challenge of creating inbound liquidity is most noticeable if you are a merchant or sell your services for Lightning payments. In that case, you need to be vigilant to ensure that you have enough inbound liquidity to be able to continue to receive payments. What if there is a surge of buyers on your store, but they can't actually pay you because there is no more inbound capacity?
+
+In the future, these challenges can be partially mitigated by the implementation of dual-funded channels, which are funded from both sides and offer balanced inbound and outbound capacity. The burden could also be mitigated by more sophisticated autopilot software, which could request and pay for inbound capacity as needed.
+
+Ultimately, Lightning users need to be strategic and proactive about channel management to ensure that sufficient inbound capacity is available to meet their needs.
+
+==== Closing Channels
+
+((("channel management","closing channels")))As discussed earlier in the book, a _mutual close_ is the preferred way of closing a channel. ((("force close")))However, there are instances where a _force close_ is necessary.
+
+Some examples:
+
+* Your channel partner is offline and cannot be contacted to initiate a mutual close.
+* Your channel partner is online, but is not responding to requests to initiate a mutual close.
+* Your channel partner is online and your nodes are negotiating a mutual close, but they become stuck and cannot reach a resolution.
+
+[[channel_rebalancing]]
+==== Rebalancing Channels
+
+((("channel management","rebalancing channels")))((("rebalancing channels")))In the course of transacting and routing payments on Lightning, the combination of inbound and outbound capacities can become unbalanced.
+
+For example, if one of your channel partners is frequently routing payments through your node, you will exhaust the inbound capacity on that channel, while also exhausting the outbound capacity on the outgoing channels. Once that happens, you can no longer route payments through that route.
+
+There are many ways to rebalance channels, each with different advantages and disadvantages. One way is to use a submarine swap (e.g., Loop Out), as described previously in this chapter. Another way to rebalance is to simply wait for routed payments that flow in the opposite direction. If your node is well connected, when a specific route becomes exhausted in one direction, the same route becomes available in the opposite direction. Other nodes may "discover" that route in the opposite direction and start using it as part of their payment path, thereby rebalancing the funds again.
+
+((("circular route rebalancing")))A third way to rebalance channels is to purposefully create a _circular route_ that sends a payment from your node back to your node, via the Lightning Network. By sending a payment out on a channel with large local capacity and arranging the path so that it returns to your node on a channel with large remote capacity, both of those channels will become more balanced. An example of a circular route rebalancing strategy can be seen in <<circular_rebalancing>>.
+
+[[circular_rebalancing]]
+.Circular route rebalancing
+image::images/mtln_0504.png[]
+
+Circular rebalancing is supported by most Lightning node implementations and can be done on the command line or via one of the web management interfaces such as Ride The Lightning (see <<rtl>>).
+
+Channel rebalancing is a complex issue that is the subject of active research and covered in more detail in <<channel_rebalancing>>.(((range="endofrange", startref="ix_05_node_operations-asciidoc20")))(((range="endofrange", startref="ix_05_node_operations-asciidoc19")))
+
+=== Routing Fees
+
+((("Lightning node operation","routing fees")))((("routing","fees")))Running a Lightning node allows you to earn fees by routing payments across your channels. Routing fees are generally not a significant source of income and dwarfed by the cost of operating a node. For example, on a relatively busy node that routes a dozen payments a day, the fees amount to no more than 2,000 satoshis.
+
+Nodes compete for routing fees by setting their desired fee rate on each channel. Routing fees are set by two parameters on each channel: a fixed _base fee_ that is charged for any payment and an additional variable _fee rate_ that is proportional to the payment amount.
+
+When sending a Lightning payment, a node will select a path so as to minimize fees, minimize hops, or both. As a result, a routing fee market emerges from these interactions. There are currently many nodes that charge very low or no fees for routing, creating downward pressure on the routing fee market.
+
+If you make no choices, your Lightning node will set a default base fee and fee rate for each new channel. The default values depend on the node implementation you use.
+The base fee is set in the unit of _millisatoshi_ (thousandths of a satoshi). The proportional fee rate is set in the unit of _millionths_ and is applied to the payment amount.
+The unit of millionths is often abbreviated with _ppm_ (parts per million).
+For example, a base fee of 1,000 (millisatoshi) and a fee rate of 1,000 ppm (millionths) would result in the following charges for a 100,000 satoshi payment:
+
+[latexmath]
+++++
+\begin{equation}
+\begin{aligned}
+P &= 100,000 \text{ satoshi} \\
+F_{base} &= 1,000 \text{ millisatoshi} = 1 \text{ satoshi} \\
+F_{rate} &= 1,000 \text{ ppm} = 1,000/1,000,000 = 1/1,000 = \text{0.001} = 0.1\% \\
+F_{total} &= F_{base} + ( P * F_{rate} ) \\
+ \Rightarrow  F_{total} &= 1 \text{ satoshi} + ( 100,000/1,000 ) \text{ satoshi} \\
+ \Rightarrow  F_{total} &= 1 \text{ satoshi} + 100 \text{ satoshi} = 101 \text{ satoshi} \\
+\end{aligned}
+\end{equation}
+++++
+
+Broadly speaking, you can take one of two approaches to routing fees. You can route lots of payments with low fees, making up for low fees by high volume. Alternatively, you can choose to charge higher fees. If you choose to set higher fees, your node will be selected only when other cheaper routes don't exist. Therefore, you will route less frequently but earn more per successful routing.
+
+For most nodes, it is usually best to use the default routing fee values. This way, your node is competing on a mostly level playing field with other nodes that use the default values.
+
+You can also use the routing fee settings to rebalance channels. If most of your channels have the default fees but you want to rebalance a particular channel, just decrease the fees on that specific channel to zero or to very low rates. Then sit back and wait for someone to route a payment over your "cheap" route and rebalance your channels for you as a side effect.
+
+=== Node Management
+
+((("Lightning node operation","node management")))((("node management")))Managing your Lightning node on the command line is obviously not easy. It gives you the full flexibility of the node's API and the ability to write your own custom scripts to satisfy your personal requirements. But if you don't want to deal with the complexity of the command line and only need some basic node management capabilities, you should consider installing a web-based user interface that makes node management easier.
+
+There are a number of competing projects that offer web-based Lightning node management. Some of the most popular ones are described in the following section.
+
+[[rtl]]
+==== Ride The Lightning
+
+((("Lightning node operation","Ride The Lightning (RTL)")))((("node management","Ride The Lightning (RTL)")))((("Ride The Lightning (RTL)")))((("RTL (Ride The Lightning)")))Ride The Lightning (RTL) is a graphical web user interface to help users manage Lightning node operations for the three main Lightning node implementations (LND, `c-lightning`, and Eclair). RTL is an open source project developed by Shahana Farooqui and many other contributors. You can find the RTL software on https://github.com/Ride-The-Lightning/RTL[GitHub].
+
+<<rtl-web-interface>> shows an example screenshot of RTL's web interface, as provided on the project repository.
+
+[[rtl-web-interface]]
+.Example RTL web interface
+image::images/mtln_0505.png[]
+
+==== lndmon
+
+((("Lightning node operation","lndmon")))((("lndmon")))((("node management","lndmon")))Lightning Labs, the makers of LND, provide a web-based graphical user interface called +lndmon+ to monitor the various metrics of an LND Lightning node. +lndmon+ only works with LND nodes. It is a read-only interface for monitoring and as such does not allow you to actively manage the node. It cannot open channels or make payments. Find +lndmon+ on https://github.com/lightninglabs/lndmon[GitHub].
+
+==== ThunderHub
+
+((("Lightning node operation","ThunderHub")))((("node management","ThunderHub")))((("ThunderHub")))https://thunderhub.io[ThunderHub] is a very aesthetically pleasing web-based graphical user interface similar to RTL but exclusive to LND. It can be used to make payments, rebalance channels, and manage the node through a variety of features.
+
+=== Conclusion
+
+As you maintain your node and gain experiences, you will learn a lot about the Lightning Network. Being a node operator is a challenging but rewarding task. Mastering these skills will allow you to contribute to the growth and development of this technology and the Lightning Network itself. You will furthermore gain the ability to send and receive Lightning payments with the greatest degree of control and ease. You will play a central role in the network's infrastructure and not just be a participant on the edges.(((range="endofrange", startref="ix_05_node_operations-asciidoc0")))

06_lightning_architecture.asciidoc

@@ -0,0 +1,55 @@
+== Lightning Network Architecture
+
+((("architecture, Lightning Network", id="ix_06_lightning_architecture-asciidoc0", range="startofrange")))In the first part of this book we introduced the main concepts of the Lightning Network and worked through a comprehensive example of routing a payment and setting up the tools we can use to explore further. In the second part of the book we will explore the Lightning Network in a lot more technical detail, dissecting each of the building blocks.
+
+In this section we will outline the components of the Lightning Network in more detail and provide a "big picture" perspective to guide you through the following chapters.
+
+=== The Lightning Network Protocol Suite
+
+((("architecture, Lightning Network","protocol suite")))((("protocol stack")))The Lightning Network is composed of a complex collection of protocols that run on top of the internet. We can broadly classify these protocols into five distinct layers that make up a _protocol stack_, where each layer builds upon and uses the protocols in the layer below. Also, each protocol layer abstracts the underlying layers and "hides" some of the complexity.
+
+The architecture diagram shown in <<lightning_network_protocol_suite>> provides an overview of these layers and their component protocols. 
+
+[[lightning_network_protocol_suite]]
+.The Lightning Network protocol suite
+image::images/mtln_0601.png[]
+
+((("architecture, Lightning Network","layers")))The five layers of the Lightning Network, from the bottom up, are:
+
+Network connection layer:: This contains the protocols that interact directly with the internet core protocols (TCP/IP), overlay protocols (Tor v2/v3), and internet services (DNS). This layer also contains the cryptographic transport protocols that protect Lightning pass:[<span class="keep-together">messages</span>].
+
+Messaging layer:: This layer contains the protocols that nodes use to negotiate features, format messages, and encode message fields.
+
+Peer-to-peer (P2P) layer:: This layer is the primary protocol layer for communication between Lightning nodes and contains all the different messages exchanged between nodes.
+
+Routing layer:: This layer contains the protocols used to route payments between nodes, end-to-end and atomically. This layer contains the core functionality of the Lightning Network: routed payments.
+
+Payment layer:: The highest layer of the network, which presents a reliable payment interface to applications.
+
+=== Lightning in Detail
+
+((("architecture, Lightning Network","outline of details")))Over the next 10 chapters, we will dissect the protocol suite and examine each component of the Lightning Network in detail.
+
+We spent quite some time trying to decide the best order of presenting this detail. It's not an easy choice because there is so much interdependence between different components: as you start explaining one, you find that it pulls in quite a few of the other componenents. Instead of a top-down or bottom-up approach, we ended up choosing a more meandering path that starts with the most fundamental building blocks that are unique to the Lightning Network-Payment Channels and moves outward from there. But since that path is not obvious, we will use the Lightning Protocol Suite shown in <<lightning_network_protocol_suite>> as a map. In each chapter will focus on one or more related components, and you will see them highlighted in the protocol suite. Kind of like a map marker saying "You are here!"
+
+Here's what we will cover:
+
+pass:[<a data-type="xref" href="payment_channels" data-xrefstyle="chap-num-title">#payment_channels</a>]:: In this chapter we will look at how payment channels work, in significantly more depth than we saw in the earlier parts of the book. We will look at the structure and Bitcoin Script of the funding and commitment transactions, and the process used by nodes to negotiate each step in the protocol.
+
+pass:[<a data-type="xref" href="#routing" data-xrefstyle="chap-num-title">#routing</a>]:: Next, we will put together several payment channels in a network and route a payment from one end to the other. In that process we will dive into the hash time-locked contract (HTLC) smart contract and the Bitcoin Script that we use to construct it.
+
+pass:[<a data-type="xref" href="#channel_operation" data-xrefstyle="chap-num-title">#channel_operation</a>]:: Putting together the concepts of a simple payment channel and a routed payment using HTLCs, we will now look at how HTLCs are part of each channel's commitment transaction. We will also look at the protocol for adding, settling, failing, and removing HTLCs from the commitments.
+
+pass:[<a data-type="xref" href="#onion_routing" data-xrefstyle="chap-num-title">#onion_routing</a>]:: Next, we will look at how the HTLC information is propagated across the network inside the onion routing protocol. We will look at the mechanism for layered encryption and decryption that gives the Lightning Network some of its privacy characteristics.
+
+pass:[<a data-type="xref" href="#gossip" data-xrefstyle="chap-num-title">#gossip</a>]:: In this chapter we will look at how Lightning nodes find each other and learn about published channels to construct a channel graph that they can use to find paths across the network.
+
+pass:[<a data-type="xref" href="#path_finding" data-xrefstyle="chap-num-title">>#path_finding</a>]:: Next, we will see how the information from the gossip protocol is used by each node to build a "map" of the entire network, which it can use to find paths from one point to another to route payments. We'll also look at the exiting innovations in pathfinding, such as multipart payments.
+
+pass:[<a data-type="xref" href="#wire_protocol" data-xrefstyle="chap-num-title">#wire_protocol</a>]:: Underpinning the Lightning Network is the peer-to-peer protocol that nodes use to exchange messages about the network and about their channels. In this chapter we look at how those messages are constructed and the extension capabilities built into messages with feature bits and Type-Length-Value (TLV) encoding.
+
+pass:[<a data-type="xref" href="#encrypted_message_transport" data-xrefstyle="chap-num-title">#encrypted_message_transport</a>]:: Moving down to the lower-level part of the network, we will look at the underlying encrypted transport system that ensures the secrecy and integrity of all communications between nodes.(((range="endofrange", startref="ix_06_lightning_architecture-asciidoc0")))
+
+pass:[<a data-type="xref" href="#invoices" data-xrefstyle="chap-num-title">#invoices</a>]:: A key part of the Lightning Network is payment requests, also known as Lightning invoices. In this chapter we dissect the structure and encoding of an invoice.
+
+Let's dive in!

07_payment_channels.asciidoc

@@ -0,0 +1,782 @@
+[[payment_channels]]
+== Payment Channels
+
+((("payment channel", id="ix_07_payment_channels-asciidoc0", range="startofrange")))In this chapter we will dive into payment channels and see how they are constructed. We will start with Alice's node opening a channel to Bob's node, building on the examples presented in the beginning of this book.
+
+[role="pagebreak-after"]
+The messages exchanged by Alice and Bob's nodes are defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md["BOLT #2: Peer Protocol for Channel Management"]. The transactions created by Alice and Bob's nodes are defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md["BOLT #3: Bitcoin Transaction and Script Formats"]. In this chapter we are focusing on the "Channel open and close" and "Channel state machine" parts of the Lightning protocol architecture, highlighted by an outline in the center (peer-to-peer layer) of <<LN_protocol_channel_highlight>>. 
+
+[[LN_protocol_channel_highlight]]
+.Payment channels in the Lightning protocol suite
+image::images/mtln_0701.png["Payment channels in the Lightning protocol suite"]
+
+=== A Different Way of Using the Bitcoin System
+
+((("payment channel","Lightning Network as different way of using Bitcoin system")))The Lightning Network is often described as a "Layer 2 Bitcoin Protocol," which makes it sound distinct from Bitcoin. Another way to describe Lightning is as a "smarter way to use Bitcoin" or just as an "application on top of Bitcoin." Let's explore that. 
+
+Historically, Bitcoin transactions are broadcast to everyone and recorded on the Bitcoin blockchain to be considered valid. As we will see, however, if someone holds a presigned Bitcoin transaction that spends a 2-of-2 multisig output that gives them the exclusive ability to spend that Bitcoin, they effectively own that Bitcoin even if they don't broadcast the transaction.
+
+You can think of the presigned Bitcoin transaction like a postdated check (or cheque), one that can be cashed at any time. Unlike the traditional banking system, however, this transaction is not a "promise" of payment (also known as an IOU), but a verifiable bearer instrument that is equivalent to cash. So long as the bitcoin referenced in the transaction has not already been spent at the time of redemption (or at the time you try to "cash" the check), the Bitcoin system guarantees that this presigned transaction can be broadcast and recorded at any time. This is only true, of course, if this is the only presigned transaction. Within the Lightning Network two or more such presigned transactions exist at the same time; therefore, we need a more sophisticated mechanism to still have the functionality of such a verifiable bearer instrument, as you will also learn in this chapter.
+
+The Lightning Network is simply a different and creative way of using Bitcoin. In the Lightning Network a combination of recorded (on-chain) and presigned but withheld (off-chain) transactions form a "layer" of payments that is a faster, cheaper, and more private way to use Bitcoin. You can see this relationship between on-chain and off-chain Bitcoin transactions in <<on_off_chain>>. 
+
+[[on_off_chain]]
+.Lightning payment channel made of on-chain and off-chain transactions
+image::images/mtln_0702.png["Lightning payment channel made of on-chain and off-chain transactions"]
+
+Lightning is Bitcoin. It's just a different way of using the Bitcoin system.
+
+=== Bitcoin Ownership and Control
+
+((("bitcoin (currency)","ownership and control in payment channels", id="ix_07_payment_channels-asciidoc1", range="startofrange")))((("payment channel","bitcoin ownership and control", id="ix_07_payment_channels-asciidoc2", range="startofrange")))Before we understand payment channels, we have to take a small step back and understand how ownership and control work in Bitcoin.
+
+((("private keys","Bitcoin ownership and")))When someone says they "own" bitcoin, they typically mean that they know the private key of a Bitcoin address that has some unspent transaction outputs (see <<bitcoin_fundamentals_review>>). The private key allows them to sign a transaction to spend that bitcoin by transferring it to a different address. In Bitcoin "ownership" of bitcoin can be defined as the _ability to spend_ that bitcoin.
+
+Keep in mind that the term "ownership" as used in Bitcoin is distinct from the term "ownership" used in a legal sense. A thief who has the private keys and can spend Bitcoin is a _de facto owner_ of that Bitcoin even though they are not a lawful owner.
+
+[TIP]
+====
+Bitcoin ownership is only about control of keys and the ability to spend the Bitcoin with those keys. As the popular Bitcoin saying goes: "Your keys, your coins—not your keys, not your coins."
+====
+
+==== Diversity of (Independent) Ownership and Multisig
+
+((("bitcoin (currency)","diversity of independent ownership and multisig")))Ownership and control of private keys is not always in the hands of one person. That's where things get interesting and complicated. We know that more than one person can come to know the same private key, either through theft or because the original holder of the key makes a copy and gives it to someone else. Are all these people owners? In a practical sense, they are, because any one of the people who know the private key can spend the bitcoin without the approval of any other.
+
+Bitcoin also has multisignature addresses where multiple private keys are needed to sign before spending (see <<multisig>>). From a practical perspective, ownership in a multisignature address depends on the quorum (_K_) and total (_N_) defined in the __K__-of-__N__ scheme. A 1-of-10 multisignature scheme would allow any 1 (_K_) of 10 (_N_) signers to spend a bitcoin amount locked in that address. This is similar to the scenario where 10 people have a copy of the same private key and any of them can independently spend it.
+
+==== Joint Ownership Without Independent Control
+
+((("bitcoin (currency)","joint ownership without independent control")))There is also the scenario where _no one_ has quorum. In a 2-of-2 scheme like that used in the Lightning Network, neither signer can spend the bitcoin without obtaining a signature from the other party. Who owns the bitcoin in that case? No one really has ownership because no one has control. They each own the equivalent of a voting share in the decision, but both votes are needed. A key problem (pun intended) with a 2-of-2 scheme, in both Bitcoin and the law, is what happens if one of the parties is unavailable, or if there is a vote deadlock and any one party refuses to cooperate.
+
+==== Preventing "Locked" and Un-Spendable Bitcoin
+
+((("bitcoin (currency)","preventing locked and un-spendable bitcoin")))If one of the two signers of a 2-of-2 multisig cannot or will not sign, the funds become un-spendable. Not only can this scenario occur accidentally (loss of keys), but it can be used as a form of blackmail by either party: "I won't sign unless you pay me a part of the funds."
+
+Payment channels in Lightning are based on a 2-of-2 multisig address, with the two channel partners as signers in the multisig. At this time, channels are funded only by one of the two channel partners: when you choose to "open" a channel, you deposit funds into the 2-of-2 multisig address with a transaction. Once that transaction is mined and the funds are in the multisig, you can't get them back without cooperation from your channel partner, because you need their signature (also) to spend the bitcoin.
+
+In the next section, as we look at how to open (create) a Lightning channel, we will see how we can prevent loss of funds or any blackmail scenario between the two partners by implementing a fairness protocol for the channel construction with the help of presigned transactions that spend the multisig output in a way that gives the peers in the channel exclusive ability to spend one of the outputs which encodes the amount of bitcoin they own in the channel.(((range="endofrange", startref="ix_07_payment_channels-asciidoc2")))(((range="endofrange", startref="ix_07_payment_channels-asciidoc1")))
+
+
+=== Constructing a Payment Channel
+
+((("payment channel","elements", id="ix_07_payment_channels-asciidoc3", range="startofrange")))In <<what_is_payment_channel>>, we described payment channels as a _financial relationship_ between two Lightning nodes, which is established by funding a 2-of-2 multisignature address from the two channel partners.
+
+Let's assume that Alice wants to construct a payment channel allowing her to connect to Bob's store directly. First, the two nodes (Alice's and Bob's) have to establish an internet connection to each other, so that they can negotiate a payment channel.
+
+==== Node Private and Public Keys
+
+((("node public key")))((("payment channel","node private/public keys")))Every node on the Lightning Network is identified by a _node public key_. The public key uniquely identifies the specific node and is usually presented as a hexadecimal encoding. For example, René Pickhardt currently runs a Lightning Node (+ln.rene-pickhardt.de+) that is identified by the following node public key:
+
+----
+02a1cebfacb2674143b5ad0df3c22c609e935f7bc0ebe801f37b8e9023d45ea7b8
+----
+
+((("private keys","generation of")))((("root private key generation")))Each node generates a root private key when first initialized. The private key is kept private at all times (never shared) and securely stored in the node's wallet. From that private key, the node derives a public key that is the node identifier and shared with the network. Since the key space is enormous, as long as each node generates the private key randomly, it will have a unique public key, which can therefore uniquely identify it on the network.
+
+==== Node Network Address
+
+((("payment channel","node network address")))Additionally, every node also advertises a network address where it can be reached, in one of several possible formats:
+
+TCP/IP:: An IPv4 or IPv6 address and TCP port number
+
+TCP/Tor:: A Tor "onion" address and TCP port number
+
+The network address identifier is written as +Address:Port+, which is consistent with international standards for network identifiers, as used, for example, on the web.
+
+For example, René's node with node public key +02a1ceb...45ea7b8+ currently advertises its network address as the TCP/IP address:
+
+----
+172.16.235.20:9735
+----
+
+[TIP]
+====
+The default TCP port for the Lightning Network is 9735, but a node can choose to listen on any TCP port.
+====
+
+==== Node Identifiers
+
+((("node identifiers")))((("payment channel","node identifiers")))Together, the node public key and network address are written in the following format, separated by an +@+ sign, as __++NodeID@Address:Port++__.
+
+So the full identifier for René's node would be:
+
+----
+02a1cebfacb2674143b5ad0df3c22c609e935f7bc0ebe801f37b8e9023d45ea7b8
+@172.16.235.20:9735
+----
+
+[TIP]
+====
+The alias of René's node is +ln.rene-pickhardt.de+; however, this name exists just for better readability. Every node operator can announce whatever alias they want, and there is no mechanism that prevents node operators from selecting an alias that is already being used. Thus to refer to a node, one must use the __++NodeID@Address:Port++__ schema.
+====
+
+The preceding identifier is often encoded in a QR code, making it easier for users to scan if they want to connect their own node to the specific node identified by that address.
+
+Much like Bitcoin nodes, Lightning nodes advertise their presence on the Lightning Network by "gossiping" their node public key and network address. That way, other nodes can find them and keep an inventory (database) of all the known nodes that they can connect to and exchange the messages that are defined in the Lightning P2P message protocol.
+
+==== Connecting Nodes as Direct Peers
+
+((("payment channel","connecting nodes as direct peers")))In order for Alice's node to connect to Bob's node, she will need Bob's node public key, or the full address containing the public key, IP or Tor address, and port. Because Bob runs a store, Bob's node address can be retrieved from an invoice or a store payment page on the web. Alice can scan a QR code that contains the address and instruct her node to connect to Bob's node.
+
+Once Alice has connected to Bob's node, their nodes are now directly connected peers.
+
+[TIP]
+====
+To open a payment channel, two nodes must first be connected as direct peers by opening a connection over the internet (or Tor).(((range="endofrange", startref="ix_07_payment_channels-asciidoc3")))
+====
+
+=== Constructing the Channel
+
+((("Lightning Peer Protocol for Channel Management", id="ix_07_payment_channels-asciidoc4", range="startofrange")))((("payment channel","construction of", id="ix_07_payment_channels-asciidoc5", range="startofrange")))Now that Alice's and Bob's Lightning nodes are connected, they can begin the process of constructing a payment channel. In this section we will review the communications between their nodes, known as the _Lightning Peer Protocol for Channel Management_, and the cryptographic protocol that they use to build Bitcoin transactions.
+
+[TIP]
+====
+We describe two different protocols in this scenario. First, there is a _message protocol_, which establishes how the Lightning nodes communicate over the internet and what messages they exchange with each other. Second, there is the _cryptographic protocol_, which establishes how the two nodes construct and sign Bitcoin pass:[<span class="keep-together">transactions</span>].
+====
+
+[[peer_protocol_channel_management]]
+==== Peer Protocol for Channel Management
+
+The Lightning Peer Protocol for Channel Management is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md[BOLT #2: Peer Protocol for Channel Management]. In this chapter we will be reviewing the "Channel Establishment" and "Channel Closing" sections of BOLT #2 in more detail.
+
+==== Channel Establishment Message Flow
+
+((("channel establishment message flow", id="ix_07_payment_channels-asciidoc6", range="startofrange")))((("payment channel","channel establishment message flow", id="ix_07_payment_channels-asciidoc7", range="startofrange")))Channel establishment is achieved by the exchange of six messages between Alice and Bob's nodes (three from each peer): +open_channel+, +accept_channel+, +funding_created+, +funding_signed+, +funding_locked+, and +funding_locked+. The six messages are shown as a time-sequence diagram in <<funding_message_flow>>. 
+
+[[funding_message_flow]]
+.The channel establishment message flow
+image::images/mtln_0703.png["The channel establishment message flow"]
+
+In <<funding_message_flow>>, Alice and Bob's nodes are represented by the vertical lines "A" and "B" on either side of the diagram. A time-sequence diagram like this shows time flowing downward, and messages flowing from one side to the other between the two communication peers. The lines are sloped down to represent the elapsed time needed to transmit each message, and the direction of the message is shown by an arrow at the end of each line.
+
+The channel establishment involves three parts. First, the two peers communicate their capabilities and expectations, with Alice initiating a request through +open_channel+ and Bob accepting the channel request through +accept_channel+.
+
+Second, Alice constructs the funding and refund transactions (as we will see later in this section) and sends +funding_created+ to Bob. Another name for the "refund" transaction is a "commitment" transaction, as it commits to the current distribution of balances in the channel. Bob responds by sending back the necessary signatures with +funding_signed+. This interaction is the basis for the _cryptographic protocol_ to secure the channel and prevent theft. Alice will now broadcast the funding transaction (on-chain) to establish and anchor the payment channel. The transaction will need to be confirmed on the Bitcoin blockchain.
+
+[TIP]
+====
+The name of the +funding_signed+ message can be a bit confusing. This message does not contain a signature for the funding transaction, but rather it contains Bob's signature for the refund transaction that allows Alice to claim her bitcoin back from the multisig.
+====
+
+Once the transaction has sufficient confirmations (as defined by the `minimum_depth` field in the `accept_channel` message), Alice and Bob exchange +funding_locked+ messages, and the channel enters normal operating mode.
+
+===== The open_channel message
+
+((("channel establishment message flow","open_channel message")))((("open_channel message")))Alice's node requests a payment channel with Bob's node by sending an +open_channel+ message. The message contains information about Alice's _expectations_ for the channel setup, which Bob may accept or decline.
+
+The structure of the +open_channel+ message (taken from BOLT #2) is shown in <<open_channel_message>>.
+
+[[open_channel_message]]
+.The `open_channel` message
+====
+----
+[chain_hash:chain_hash]
+[32*byte:temporary_channel_id]
+[u64:funding_satoshis]
+[u64:push_msat]
+[u64:dust_limit_satoshis]
+[u64:max_htlc_value_in_flight_msat]
+[u64:channel_reserve_satoshis]
+[u64:htlc_minimum_msat]
+[u32:feerate_per_kw]
+[u16:to_self_delay]
+[u16:max_accepted_htlcs]
+[point:funding_pubkey]
+[point:revocation_basepoint]
+[point:payment_basepoint]
+[point:delayed_payment_basepoint]
+[point:htlc_basepoint]
+[point:first_per_commitment_point]
+[byte:channel_flags]
+[open_channel_tlvs:tlvs]
+----
+====
+
+The fields contained in this message specify the channel parameters that Alice wants, as well as various configuration settings from Alice's nodes that reflect the security expectations for the operation of the channel.
+
+[role="pagebreak-before"]
+Some of the channel construction parameters are listed here:
+
++chain_hash+:: This identifies which blockchain (e.g., Bitcoin mainnet) will be used for this channel. It is usually the hash of the genesis block of that blockchain.
+
++funding_satoshis+:: The amount Alice will use to fund the channel, which is the total channel capacity.
+
++channel_reserve_satoshis+:: The minimum balance, in satoshis, that is reserved on each side of a channel. We will come back to this when we talk about penalties.
+
++push_msat+:: An optional amount that Alice will immediately "push" to Bob as a payment upon channel funding. _Setting this value to anything but 0 means effectively gifting money to your channel partner and should be used with caution._
+
++to_self_delay+:: A very important security parameter for the protocol. The value in the `open_channel` message is used in the responder's commitment transaction, and the `accept_channel` in the initiator's. This asymmetry exists to allow each side to express how long the other side needs to wait to unilaterally claim the funds in a commitment transaction. If Bob at any time unilaterally closes the channel against the will of Alice, he commits to not accessing his own funds for the delay defined here. The higher this value, the more security Alice has, but the longer Bob might have his funds locked.
+
++funding_pubkey+:: The public key that Alice will contribute to the 2-of-2 multisig that anchors this channel.
+
++X_basepoint+:: Master keys, used to derive child keys for various parts of the commitment, revocation, routed payment (HTLCs), and closing transactions. These will be used and explained in subsequent chapters.
+
+[TIP]
+====
+If you want to understand the other fields and Lightning peer protocol messages that we do not discuss in this book, we suggest you look them up in the BOLT specifications. These messages and fields are important, but cannot be covered in enough detail in the scope of this book. We want you to understand the fundamental principles well enough that you can fill in the details by reading the actual protocol specification (BOLTs).
+====
+
+===== The accept_channel message
+
+((("accept_channel message")))((("channel establishment message flow","accept_channel message")))In response to Alice's +open_channel+ message, Bob sends back the +accept_channel+ message shown in <<accept_channel_message>>.
+
+[[accept_channel_message]]
+.The `accept_channel` message
+====
+----
+[32*byte:temporary_channel_id]
+[u64:dust_limit_satoshis]
+[u64:max_htlc_value_in_flight_msat]
+[u64:channel_reserve_satoshis]
+[u64:htlc_minimum_msat]
+[u32:minimum_depth]
+[u16:to_self_delay]
+[u16:max_accepted_htlcs]
+[point:funding_pubkey]
+[point:revocation_basepoint]
+[point:payment_basepoint]
+[point:delayed_payment_basepoint]
+[point:htlc_basepoint]
+[point:first_per_commitment_point]
+[accept_channel_tlvs:tlvs]
+----
+====
+
+As you can see, this is similar to the +open_channel+ message and contains Bob's node expectations and configuration values.
+
+The two most important fields in +accept_channel+ that Alice will use to construct the payment channel are:
+
++funding_pubkey+:: The public key Bob's node contributes for the 2-of-2 multisig address that anchors the channel.
+
++minimum_depth+:: The number of confirmations that Bob's node expects for the funding transaction before it considers the channel "open" and ready to use.(((range="endofrange", startref="ix_07_payment_channels-asciidoc7")))(((range="endofrange", startref="ix_07_payment_channels-asciidoc6")))
+
+==== The Funding Transaction
+
+((("funding transaction")))((("payment channel","funding transaction")))Once Alice's node receives Bob's +accept_channel+ message, it has the information necessary to construct the _funding transaction_ that anchors the channel to the Bitcoin blockchain. As we discussed in earlier chapters, a Lightning payment channel is anchored by a 2-of-2 multisignature address. First, we need to generate that multisignature address to allow us to construct the funding transaction (and the refund transaction as described subsequently).
+
+==== Generating a Multisignature Address
+
+((("multisignature addresses")))((("payment channel","multisignature addresses")))The funding transaction sends some amount of bitcoin (+funding_satoshis+ from the +open_channel+ message) to a 2-of-2 multisignature output that is constructed from Alice and Bob's +funding_pubkey+ public keys.
+
+Alice's node constructs a multisignature script as shown here:
+
+++++
+<pre data-type="programlisting">2 &lt;<em>Alice_funding_pubkey</em>&gt; &lt;<em>Bob_funding_pubkey</em>&gt; 2 CHECKMULTISIG
+</pre>
+++++
+
+Note that, in practice, the funding keys are deterministically _sorted_ (using lexicographical order of the serialized compressed form of the public keys) before being placed in the witness script. By agreeing to this sorted order ahead of time, we ensure that both parties will construct an identical funding transaction output, which is signed by the exchanged commitment transaction signature.
+
+
+This script is encoded as a Pay-to-Witness-Script-Hash (P2WSH) Bitcoin address, which looks something like this:
+
+----
+bc1q89ju02heg32yrqdrnqghe6132wek25p6sv6e564znvrvez7tq5zqt4dn02
+----
+==== Constructing the Funding Transaction
+
+((("payment channel","constructing the funding transaction")))Alice's node can now construct a funding transaction, sending the amount agreed on with Bob (`funding_satoshis`) to the 2-of-2 multisig address. Let's assume that funding_satoshis was 140,000 and Alice is spending a 200,000 satoshi output and creating 60,000 satoshi change. The transaction will look something like <<A_B_funding_Tx>>. 
+
+[[A_B_funding_Tx]]
+.Alice constructs the funding transaction
+image::images/mtln_0704.png["Alice constructs the funding transaction"]
+
+Alice _does not broadcast_ this transaction because doing so would put her 140,000 satoshi at risk. Once spent to the 2-of-2 multisig, there is no way for Alice to recover her money without Bob's signature.
+
+[role="pagebreak-before less_space"]
+.Dual-Funded Payment Channels
+****
+((("dual-funded payment channels")))((("payment channel","dual-funded")))In the current implementation of Lightning, channels are funded only by the node initiating the channel (Alice in our example). Dual-funded channels have been proposed, but not yet implemented. In a dual-funded channel, both Alice and Bob would contribute inputs to the funding transaction. Dual-funded channels require a slightly more complicated message flow and cryptographic protocol, so they have not been implemented yet but are planned for a future update to the Lightning BOLTs. The `c-lightning` implementation includes an experimental version of a variant on dual-funded channels.
+****
+
+==== Holding Signed Transactions Without Broadcasting
+
+((("payment channel","holding signed transactions without broadcasting")))An important Bitcoin feature that makes Lightning possible is the ability to construct and sign transactions, but not broadcast them. The transaction is _valid_ in every way, but until it is broadcast and confirmed on the Bitcoin blockchain it is not recognized and its outputs are not spendable because they have not been created on the blockchain. We will use this capability many times in the Lightning Network, and Alice's node uses the capability when constructing the funding transaction: holding it and not broadcasting it yet.
+
+==== Refund Before Funding
+
+((("payment channel","refund before funding")))To prevent loss of funds, Alice cannot put her bitcoin into a 2-of-2 until she has a way to get a refund if things go wrong. Essentially, she must plan the "exit" from the channel before she enters into this arrangement.
+
+Consider the legal construct of a prenuptial agreement, also known as a "prenup." When two people enter into a marriage their money is bound together by law (depending on jurisdiction). Prior to entering into the marriage, they can sign an agreement that specifies how to separate their assets if they dissolve their marriage through divorce.
+
+We can create a similar agreement in Bitcoin. For example, we can create a refund transaction, which functions like a prenup, allowing the parties decide how the funds in their channel will be divided before their funds are actually locked into the multisignature funding address.
+
+==== Constructing the Presigned Refund Transaction
+
+((("payment channel","constructing the presigned refund transaction")))((("refund transactions")))Alice will construct the refund transaction immediately after constructing (but not broadcasting) the funding transaction. The refund transaction spends the 2-of-2 pass:[<span class="keep-together">multisig</span>] back to Alice's wallet. ((("commitment transactions","refund transactions and")))We call this refund transaction a _commitment transaction_ because it commits both channel partners to distributing the channel balance fairly. Since Alice funded the channel on her own, she gets the entire balance, and both Alice and Bob commit to refunding Alice  with this transaction.
+
+In practice, it is a bit more complicated as we will see in subsequent chapters, but for now let's keep things simple and assume it looks like <<A_B_fund_refund_Tx>>. 
+
+[[A_B_fund_refund_Tx]]
+.Alice also constructs the refund transaction
+image::images/mtln_0705.png["Alice also constructs the refund transaction"]
+
+Later in this chapter we will see how more commitment transactions can be made to distribute the balance of the channel in different amounts.
+
+==== Chaining Transactions Without Broadcasting
+
+((("payment channel","chaining transactions without broadcasting")))So now, Alice has constructed the two transactions shown in <<A_B_fund_refund_Tx>>. But you might be wondering how this is possible.  Alice hasn't broadcast the funding transaction to the Bitcoin blockchain. As far as everyone on the network is concerned, that transaction doesn't exist. The refund transaction is constructed so as to _spend_ one of the outputs of the funding transaction, even though that output doesn't exist yet either. How can you spend an output that hasn't been confirmed on the Bitcoin blockchain?
+
+The refund transaction is not yet a valid transaction. For it to become a valid transaction two things must happen:
+
+* The funding transaction must be broadcast to the Bitcoin network. (To ensure the security of the Lightning Network, we will also require it to be confirmed by the Bitcoin blockchain, though this is not strictly necessary to chain pass:[<span class="keep-together">transactions</span>].)
+* The refund transaction's input needs Alice's and Bob's signatures. 
+
+[role="pagebreak-before"]
+But even though these two things haven't happened, and even though Alice's node hasn't broadcast the funding transaction, she can still construct the refund transaction. She can do so because she can calculate the funding transaction's hash and reference it as an input in the refund transaction.
+
+Notice how Alice has calculated +6da3c2...387710+ as the funding transaction hash? If and when the funding transaction is broadcast, that hash will be recorded as the transaction ID of the funding transaction. Therefore, the `0` output of the funding transaction (the 2-of-2 address output) will then be referenced as output ID +6da3c2...387710:0+. The refund transaction can be constructed to spend that funding transaction output even though it doesn't exist yet, because Alice knows what its identifier will be once confirmed.
+
+This means that Alice can create a chained transaction by referencing an output that doesn't yet exist, knowing that the reference will be valid if the funding transaction is confirmed, making the refund transaction valid too. As we will see in the next section, this "trick" of chaining transactions before they are broadcast requires a very important feature of Bitcoin that was introduced in August of 2017: _Segregated Witness_.
+
+==== Solving Malleability (Segregated Witness)
+
+((("payment channel","Transaction Malleability and Segregated Witness", id="ix_07_payment_channels-asciidoc8", range="startofrange")))((("Segregated Witness (SegWit) protocol", id="ix_07_payment_channels-asciidoc9", range="startofrange")))((("Transaction Malleability", id="ix_07_payment_channels-asciidoc10", range="startofrange")))Alice has to depend on the transaction ID of the funding transaction being known before confirmation. But before the introduction of  Segregated Witness (SegWit) in August 2017, this was not sufficient to protect Alice. Because of the way transactions were constructed with the signatures (witnesses) included in the transaction ID, it was possible for a third party (e.g., Bob) to broadcast an alternative version of a transaction with a _malleated_ (modified) transaction ID. This is known as _transaction malleability_, and prior to SegWit, this problem made it difficult to implement indefinite lifetime payment channels securely.
+
+If Bob could modify Alice's funding transaction before it was confirmed, and produce a replica that had a different transaction ID, Bob could make Alice's refund transaction invalid and hijack her bitcoin. Alice would be at Bob's mercy to get a signature to release her funds and could easily be blackmailed. Bob couldn't steal the funds, but he could prevent Alice from getting them back.
+
+The introduction of SegWit made unconfirmed transaction IDs immutable from the point of view of third parties, meaning that Alice could be sure that the transaction ID of the funding transaction would not change. As a result, Alice can be confident that if she gets Bob's signature on the refund transaction, she has a way to recover her money. She now has a way to implement the Bitcoin equivalent of a "prenup" before locking her funds into the multisig.
+
+[TIP]
+====
+You might have wondered how Bob would be able to alter (malleate) a transaction created and signed by Alice. Bob certainly does not have Alice's private keys. However ECDSA signatures for a message are not unique. Knowing a signature (which is included in a valid transaction) allows one to produce many different-looking signatures that are still valid. Before SegWit removed signatures from the transaction digest algorithm, Bob could replace the signature with an equivalent valid signature that produced a different transaction ID, breaking the chain between the funding transaction and the refund transaction.
+====
+
+===== The funding_created message
+
+((("funding_created message")))((("Segregated Witness (SegWit) protocol","funding_created message and")))Now that Alice has constructed the necessary transactions, the channel construction message flow continues. Alice transmits the +funding_created+ message to Bob. You can see the contents of this message here:
+
+[[funding_created_message]]
+.The funding_created message
+----
+[32*byte:temporary_channel_id]
+[sha256:funding_txid]
+[u16:funding_output_index]
+[signature:signature]
+----
+
+With this message, Alice gives Bob the important information about the funding transaction that anchors the payment channel:
+
++funding_txid+:: This is the transaction ID (TxID) of the funding transaction, and is used to create the channel ID once the channel is established.
+
++funding_output_index+:: This is the output index, so Bob knows which output of the transaction (e.g., output `0`) is the 2-of-2 multisig output funded by Alice. This is also used to form the channel ID.
+
+Finally, Alice also sends the +signature+ corresponding to Alice's `funding_pubkey` and used to spend from the 2-of-2 multisig. This is needed by Bob because he will also need to create his own version of a commitment transaction. That commitment transaction needs a signature from Alice, which she provides to him. Note that the commitment transactions of Alice and Bob look slightly different, thus the signatures will be different. Knowing what the commitment transaction of the other party looks like is crucial and part of the protocol to provide the valid signature.
+
+[TIP]
+====
+In the Lightning protocol we often see nodes sending signatures instead of entire signed transactions. That's because either side can reconstruct the same transaction and therefore only the signature is needed to make it valid. Sending only the signature and not the entire transaction saves a lot of network bandwidth.
+====
+
+
+===== The funding_signed message
+
+((("funding_signed message")))((("Segregated Witness (SegWit) protocol","funding_signed message and")))After receiving the +funding_created+ message from Alice, Bob now  knows the funding transaction ID and output index. The channel ID is made by a bitwise "exclusive or" (XOR) of the funding transaction ID and output index:
+
+----
+channel_id = funding_txid XOR funding_output_index
+----
+
+More precisely, a `channel_id`, which is the 32-byte representation of a funding UTXO, is generated by XORing the lower 2 bytes of the funding TxID with the index of the funding output.
+
+Bob will also need to send Alice his signature for the refund transaction, based on Bob's `funding_pubkey` that formed the 2-of-2 multisig. Although Bob already has his local refund transaction, this will allow Alice to complete the refund transaction with all necessary signatures and be sure her money is refundable in case something goes wrong.
+
+Bob constructs a +funding_signed+ message and sends it to Alice. Here we see the contents of this message:(((range="endofrange", startref="ix_07_payment_channels-asciidoc10")))(((range="endofrange", startref="ix_07_payment_channels-asciidoc9")))(((range="endofrange", startref="ix_07_payment_channels-asciidoc8")))
+
+[[funding_signed_message]]
+.The funding_signed message
+----
+
+[channel_id:channel_id]
+[signature:signature]
+
+----
+
+==== Broadcasting the Funding Transaction
+
+((("payment channel","broadcasting the funding transaction")))Upon receiving the +funding_signed+ message from Bob, Alice now has both signatures needed to sign the refund transaction. Her "exit plan" is now secure, and therefore she can broadcast the funding transaction without fear of having her funds locked. If anything goes wrong, Alice can simply broadcast the refund transaction and get her money back, without any further help from Bob.
+
+Alice now sends the funding transaction to the Bitcoin network so that it can be mined into the blockchain. Both Alice and Bob will be watching for this transaction and waiting for +minimum_depth+ confirmations (e.g., six confirmations) on the Bitcoin blockchain.
+
+[TIP]
+====
+Of course Alice will use the Bitcoin Protocol to verify that the signature that Bob sent her is indeed valid. This step is very crucial. If for some reason Bob was sending wrongful data to Alice, her "exit plan" would be sabotaged.
+====
+
+===== The funding_locked message
+
+((("funding_locked message")))As soon as the funding transaction has reached the required number of confirmations, both Alice and Bob send the +funding_locked+ message to each other and the channel is ready for use.(((range="endofrange", startref="ix_07_payment_channels-asciidoc5")))(((range="endofrange", startref="ix_07_payment_channels-asciidoc4")))
+
+=== Sending Payments Across the Channel
+
+((("payment channel","sending payments across", id="ix_07_payment_channels-asciidoc11", range="startofrange")))The channel has been set up, but in its initial state, all the capacity (140,000 satoshis) is on Alice's side. This means that Alice can send payments to Bob across the channel, but Bob has no funds to send to Alice yet.
+
+In the next few sections we will show how payments are made across the payment channel and how the _channel state_ is updated.
+
+Let's assume that Alice wants to send 70,000 satoshis to Bob to pay her bill at Bob's coffee shop.
+
+==== Splitting the Balance
+
+((("payment channel","splitting the payment balance")))In principle, sending a payment from Alice to Bob is simply a matter of redistributing the balance of the channel. Before the payment is sent, Alice has 140,000 satoshis and Bob has none. After the 70,000 satoshi payment is sent, Alice has 70,000 satoshis pass:[<span class="keep-together">and Bob</span>] has 70,000 satoshis.
+
+((("commitment transactions","splitting balances with")))Therefore, all Alice and Bob have to do is create and sign a transaction that spends the 2-of-2 multisig to two outputs paying Alice and Bob their corresponding balances. We call this updated transaction a _commitment transaction_.
+
+Alice and Bob operate the payment channel by _advancing the channel state_ through a series of commitments. Each commitment updates the balances to reflect payments that have flowed across the channel. Both Alice and Bob can initiate a new commitment to update the channel.
+
+In <<competing_commitments_1>> we see several commitment transactions. 
+
+The first commitment transaction shown in <<competing_commitments_1>> is the refund transaction that Alice constructed before funding the channel. In the diagram, this is Commitment #0. After Alice pays Bob 70,000 satoshis, the new commitment transaction (Commitment #1) has two outputs paying Alice and Bob their respective balances. We have included two subsequent commitment transactions (Commitment #2 and Commitment #3) which represent Alice paying Bob an additional 10,000 satoshis and then 20,000 satoshis, respectively.
+
+Each signed and valid commitment transaction can be used by either channel partner at any time to close the channel by broadcasting it to the Bitcoin network. Since they both have the most recent commitment transaction and can use it at any time, they can also just hold it and not broadcast it. It's their guarantee of a fair exit from the channel.
+
+[[competing_commitments_1]]
+.Multiple commitment transactions
+image::images/mtln_0706.png[Multiple commitment transactions]
+
+==== Competing Commitments
+
+((("commitment transactions","completing commitments")))((("payment channel","completing commitments")))You may be wondering how it is possible for Alice and Bob to have multiple commitment transactions, all of them attempting to spend the same 2-of-2 output from the funding transaction. Aren't these commitment transactions conflicting? ((("double-spending")))Isn't this a "double-spend" that the Bitcoin system is meant to prevent?
+
+It is indeed! In fact, we rely on Bitcoin's ability to _prevent_ a double-spend to make Lightning work. No matter how many commitment transactions Alice and Bob construct and sign, only one of them can actually get confirmed.
+
+As long as Alice and Bob hold these transactions and don't broadcast them, the funding output is unspent. But if a commitment transaction is broadcast and confirmed, it will spend the funding output. If Alice or Bob attempts to broadcast more than one commitment transaction, only one of them will be confirmed and the others will be rejected as attempted (and failed) double-spends.
+
+If more than one commitment transaction is broadcast, there are many factors that will determine which one gets confirmed first: the amount of fees included, the speed of propagation of these competing transactions, network topology, etc. Essentially it becomes a race without a predictable outcome. That doesn't sound very secure. It sounds like someone could cheat.
+
+==== Cheating with Old Commitment Transactions
+
+((("cheating","with old transactions")))((("commitment transactions","cheating with old transactions")))((("payment channel","cheating with old commitment transactions")))Let's look more carefully at the commitment transactions in <<competing_commitments_1>>. All four commitment transactions are signed and valid. But only the last one accurately reflects the most recent channel balances. In this particular scenario, Alice has an opportunity to cheat by broadcasting an older commitment and getting it confirmed on the Bitcoin blockchain. Let's say Alice transmits Commitment #0 and gets it confirmed: she will effectively close the channel and take all 140,000 satoshis herself. In fact, in this particular example any commitment but Commitment #3 improves Alice's position and allows her to "cancel" at least part of the payments reflected in the channel.
+
+In the next section we will see how the Lightning Network resolves this problem—preventing older commitment transactions from being used by the channel partners by a mechanism of revocation and penalties. There are other ways to prevent the transmission of older commitment transactions, such as eltoo channels, but they require an upgrade to Bitcoin called input rebinding (see <<bitcoin_prot_17>>).
+
+==== Revoking Old Commitment Transactions
+
+((("commitment transactions","revoking old transactions")))((("payment channel","revoking old commitment transactions")))Bitcoin transactions do not expire and cannot be "canceled." Neither can they be stopped or censored once they have been broadcast. So how do we "revoke" a transaction that another person holds that has already been signed?
+
+The solution used in Lightning is another example of a fairness protocol. ((("penalty mechanisms")))Instead of trying to control the ability to broadcast a transaction, there is a built-in _penalty mechanism_ that ensures it is not in the best interest of a would-be cheater to transmit an old commitment transaction. They can always broadcast it, but they will most likely lose money if they do so.
+
+[TIP]
+====
+The word "revoke" is a misnomer because it implies that older commitments are somehow made invalid and cannot be broadcast and confirmed. But this is not the case, since valid Bitcoin transactions cannot be revoked. Instead, the Lightning protocol uses a penalty mechanism to punish the channel partner who broadcasts an old commitment.
+====
+
+There are three elements that make up the Lightning protocol's revocation and penalty mechanism:
+
+Asymmetric commitment transactions:: Alice's commitment transactions are slightly different from those held by Bob.
+
+Delayed spending:: The payment to the party holding the commitment transaction is delayed (timelocked), whereas the payment to the other party can be claimed immediately.
+
+Revocation keys:: Used to unlock a penalty option for old commitments.
+
+Let's look at these three elements in turn.
+
+
+==== Asymmetric Commitment Transactions
+
+((("commitment transactions","asymmetric")))((("payment channel","asymmetric commitment transactions")))Alice and Bob hold slightly different commitment transactions. Let's look specifically at Commitment #2 from <<competing_commitments_1>>, in more detail in <<commitment_2>>. 
+
+[[commitment_2]]
+.Commitment transaction #2
+image::images/mtln_0707.png[Commitment transaction #2]
+
+Alice and Bob hold two different variations of this transaction, as illustrated in <<asymmetric_1>>.
+
+[[asymmetric_1]]
+.Asymmetric commitment transactions
+image::images/mtln_0708.png[Asymmetric commitment transactions]
+
+By convention, within the Lightning protocol, we refer to the two channel partners as `self` (also known as `local`) and `remote`, depending on which side we're looking at. The outputs that pay each channel partner are called `to_local` and `to_remote`, respectively.
+
+In <<asymmetric_1>> we see that Alice holds a transaction that pays 60,000 satoshis `to_self` (can be spent by Alice's keys), and 80,000 satoshis `to_remote` (can be spent by Bob's keys).
+
+Bob holds the mirror image of that transaction, where the first output is 80,000 satoshis `to_self` (can be spent by Bob's keys), and 60,000 satoshis `to_remote` (can be spent by Alice's keys).
+
+==== Delayed (Timelocked) Spending to_self
+
+((("payment channel","delayed spending to_self")))Using asymmetric transactions allows the protocol to easily ascribe _blame_ to the cheating party. An invariant that the _broadcasting_ party must always wait ensures that the "honest" party has time to refute the claim and revoke their funds. This asymmetry is manifested in the form of differing outputs for each side: the `to_local` output is always timelocked and can't be spent immediately, whereas the `to_remote` output is not timelocked and can be spent immediately.
+
+In the commitment transaction held by Alice, for example, the `to_local` output that pays her is timelocked for 432 blocks, whereas the `to_remote` output that pays Bob can be spent immediately (see <<asymmetric_delayed_1>>). Bob's commitment transaction for Commitment #2 is the mirror image: his own (`to_local`) output is timelocked and Alice's `to_remote` output can be spent immediately.
+
+[[asymmetric_delayed_1]]
+.Asymmetric and delayed commitment transactions
+image::images/mtln_0709.png[Asymmetric and delayed commitment transactions]
+
+[role="pagebreak-before"]
+That means that if Alice closes the channel by broadcasting and confirming the commitment transaction she holds, she cannot spend her balance for 432 blocks, but Bob can claim his balance immediately. If Bob closes the channel using the commitment transaction he holds, he cannot spend his output for 432 blocks while Alice can immediately spend hers.
+
+The delay is there for one reason: to allow the _remote_ party to exercise a penalty option if an old (revoked) commitment should be broadcast by the other channel partner. Let's look at the revocation keys and penalty option next.
+
+The delay is negotiated by Alice and Bob, during the initial channel construction message flow, as a field called +to_self_delay+. To ensure the security of the channel, the delay is scaled to the capacity of the channel—meaning a channel with more funds has longer delays in the +to_self+ outputs in commitments. Alice's node includes a desired +to_self_delay+ in the +open_channel+ message. If Bob finds this acceptable, his node includes the same value for +to_self_delay+ in the +accept_channel+ message. If they do not agree, then the channel is rejected (see <<theShutdownmessage>>).
+
+==== Revocation Keys
+
+((("payment channel","revocation keys")))((("revocation keys")))As we discussed previously, the word "revocation" is a bit misleading because it implies that the "revoked" transaction cannot be used.
+
+In fact, the revoked transaction can be used, but if it is used, and it has been revoked, then one of the channel partners can take all of the channel funds by creating a penalty transaction.
+
+The way this works is that the `to_local` output is not only timelocked, but it also has two spending conditions in the script: it can be spent by _self_ after the timelock delay _or_ it can be spent by _remote_ immediately with a revocation key for this commitment.
+
+So, in our example, each side holds a commitment transaction that includes a revocation option in the `to_local` output, as shown in <<asymmetric_delayed_revocable_1>>.(((range="endofrange", startref="ix_07_payment_channels-asciidoc11"))) 
+
+[[asymmetric_delayed_revocable_1]]
+.Asymmetric, delayed, and revocable commitments
+image::images/mtln_0710.png["Asymmetric, delayed and revocable commitments"]
+
+[[commitment_transaction]]
+=== The Commitment Transaction
+
+((("commitment transactions", id="ix_07_payment_channels-asciidoc12", range="startofrange")))((("payment channel","commitment transaction", id="ix_07_payment_channels-asciidoc13", range="startofrange")))Now that we understand the structure of commitment transactions and why we need asymmetric, delayed, revocable commitments, let's look at the Bitcoin Script that implements this.
+
+The first (`to_local`) output of a commitment transaction is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md#to_local-output[BOLT #3: Commitment Transaction, `to_local` Output], as follows:
+
+----
+OP_IF
+    # Penalty transaction
+    <revocationpubkey>
+OP_ELSE
+    <to_self_delay>
+    OP_CHECKSEQUENCEVERIFY
+    OP_DROP
+    <local_delayedpubkey>
+OP_ENDIF
+OP_CHECKSIG
+----
+
+This is a conditional script (see <<conditional_scripts>>), which means the output can be spent if _either_ of the two conditions is met. The first clause allows the output to be spent by anyone who can sign for +<revocationpubkey>+. The second clause is timelocked by +<to_self_delay>+ blocks and can only be spent after that many blocks by anyone who can sign for +<local_delayedpubkey>+. In our example, we had set the +<to_self_delay>+ timelock to 432 blocks, but this is a configurable delay that is negotiated by the two channel partners. The +to_self_delay+ timelock duration is usually chosen in proportion to the channel capacity, meaning that larger capacity channels (more funds), have longer +to_self_delay+ timelocks to protect the parties.
+
+The first clause allows the output to be spent by anyone who can sign for +<revocationpubkey>+. A critical requirement to the security of this script is that the remote party _cannot_ unilaterally sign with the `revocationpubkey`. To see why this is important, consider the scenario in which the remote party breaches a previously revoked commitment. If they can sign with this key, then they can simply take the revocation clause _themselves_ and steal all the funds in the channel. Instead, we derive the `revocationpubkey` for _each_ state based on information from _both_ the self (local) and remote party. A clever use of symmetric and asymmetric cryptography is used to allow both sides to compute the `revocationpubkey` public key, but only allow the honest self party to compute the private key given their secret information, as detailed in <<revocation_sidebar>>.
+
+[[revocation_sidebar]]
+.Revocation and Commitment Secret Derivations
+****
+((("payment channel","revocation and commitment secret derivations")))Each side sends a `revocation_basepoint` during the initial channel negotiation messages as well as a `first_per_commitment_point`. The `revocation_basepoint` is static for the lifetime of the channel, while each new channel state will be based off a new `first_per_commitment_point`.
+
+Given this information, the `revocationpubkey` for each channel state is derived via the following series of elliptic curve and hashing operations: 
+
+----
+revocationpubkey = revocation_basepoint * sha256(revocation_basepoint || per_commitment_point) + per_commitment_point * sha256(per_commitment_point || revocation_basepoint)
+----
+
+Due to the commutative property of the abelian groups that elliptic curves are defined over, once the `per_commitment_secret` (the private key for the `per_commitment_point`) is revealed by the remote party, self can derive the private key for the `revocationpubkey` with the following operation: 
+
+----
+revocation_priv = (revocationbase_priv * sha256(revocation_basepoint || per_commitment_point)) + (per_commitment_secret * sha256(per_commitment_point || revocation_basepoint)) mod N
+----
+
+To see why this works in practice, notice that we can _reorder_ (commute) and expand the public key computation of the original formula for `revocationpubkey`:
+```
+revocationpubkey = G*(revocationbase_priv * sha256(revocation_basepoint || per_commitment_point) + G*(per_commitment_secret * sha256(per_commitment_point || revocation_basepoint))
+                 = revocation_basepoint * sha256(revocation_basepoint || per_commitment_point) + per_commitment_point * sha256(per_commitment_point || revocation_basepoint))
+```
+
+In other words, the `revocationbase_priv` can only be derived (and used to sign for the `revocationpubkey`) by the party that knows _both_ the `revocationbase_priv` _and_ the `per_commitment_secret`. This little trick is what makes the public-key-based revocation system used in the Lightning Network secure.
+****
+
+
+[TIP]
+====
+((("relative timelock")))The timelock used in the commitment transaction with +CHECK&#x200b;SE&#x2060;QUENCEVERIFY+ is a _relative timelock_. It counts elapsed blocks from the confirmation of this output. That means it will not be spendable until the +to_self_delay+ block _after_ this commitment transaction is broadcast and confirmed.
+====
+
+The second output (to_remote) output of the commitment transaction is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md#to_remote-output[BOLT #3: Commitment Transaction,  `to_remote` Output], and in the simplest form is a Pay-to-Witness-Public-Key-Hash (P2WPKH) for +<remote_pubkey>+, meaning that it simply pays the owner who can sign for +<remote_pubkey>+.
+
+Now that we've defined the commitment transactions in detail, let's see how Alice and Bob advance the state of the channel, create and sign new commitment transactions, and revoke old commitment transactions.(((range="endofrange", startref="ix_07_payment_channels-asciidoc13")))(((range="endofrange", startref="ix_07_payment_channels-asciidoc12")))
+
+=== Advancing the Channel State
+
+((("channel state","advancing", id="ix_07_payment_channels-asciidoc14", range="startofrange")))((("payment channel","advancing the channel state", id="ix_07_payment_channels-asciidoc15", range="startofrange")))To advance the state of the channel, Alice and Bob exchange two messages: +commitment_signed+ and +revoke_and_ack+ messages. The +commitment_signed+ message can be sent by either channel partner when they have an update to the channel state. The other channel partner then may respond with +revoke_and_ack+ to _revoke_ the old commitment and _acknowledge_ the new commitment.
+
+In <<commitment_message_flow>> we see Alice and Bob exchanging two pairs of +commitment_signed+ and +revoke_and_ack+. The first flow shows a state update initiated by Alice (left to right +commitment_signed+), to which Bob responds (right to left +revoke_and_ack+). The second flow shows a state update initiated by Bob and responded to by Alice.
+
+[[commitment_message_flow]]
+.Commitment and revocation message flow
+image::images/mtln_0711.png[Commitment and revocation message flow]
+
+==== The commitment_signed Message
+
+((("channel state","commitment_signed message")))((("commitment_signed message")))The structure of the +commitment_signed+ message is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#committing-updates-so-far-commitment_signed[BOLT #2: Peer Protocol, `commitment_signed`], and shown here:
+
+[[commitment_signed_message]]
+.The commitment_signed message
+----
+[channel_id:channel_id]
+[signature:signature]
+[u16:num_htlcs]
+[num_htlcs*signature:htlc_signature]
+----
+
++channel_id+:: The identifier of the channel
++signature+:: The signature for the new remote commitment
++num_htlcs+:: The number of updated HTLCs in this commitment
++htlc_signature+:: The signatures for the updates
+
+[NOTE]
+====
+The use of HTLCs to commit updates will be explained in detail in <<htlcs>> and in <<channel_operation>>. 
+====
+
+Alice's +commitment_signed+ message gives Bob the signature needed (Alice's part of the 2-of-2) for a new commitment transaction.
+
+==== The revoke_and_ack Message
+
+((("channel state","revoke_and_ack message")))((("revoke_and_ack message")))Now that Bob has a new commitment transaction, he can revoke the previous commitment by giving Alice a revocation key, and construct the new commitment with Alice's signature.
+
+The +revoke_and_ack+ message is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#completing-the-transition-to-the-updated-state-revoke_and_ack[BOLT #2: Peer Protocol, `revoke_and_ack`], and shown here:
+
+[[revoke_and_ack_message]]
+.The revoke_and_ack message
+----
+
+[channel_id:channel_id]
+[32*byte:per_commitment_secret]
+[point:next_per_commitment_point]
+
+----
+
++channel_id+:: This is the identifier of the channel.
++per_commitment_secret+:: Used to generate a revocation key for the previous (old) commitment, effectively revoking it.
++next_per_commitment_point+:: Used to build a `revocation_pubkey` for the new commitment, so that it can later be revoked.
+
+[[revocation]]
+==== Revoking and Recommitting
+
+((("channel state","revoking and recommitting")))Let's look at this interaction between Alice and Bob more closely.
+
+Alice is giving Bob the means to create a new commitment. In return, Bob is revoking the old commitment to assure Alice that he won't use it. Alice can only trust the new commitment if she has the revocation key to punish Bob for publishing the old commitment. From Bob's perspective, he can safely revoke the old commitment by giving Alice the keys to penalize him, because he has a signature for a new commitment.
+
+When Bob responds with +revoke_and_ack+, he gives Alice a +per_commitment_secret+. This secret can be used to construct the revocation signing key for the old commitment, which allows Alice to seize all channel funds by exercising a penalty.
+
+As soon as Bob has given this secret to Alice, he _must not_ ever broadcast that old commitment. If he does, he will give Alice the opportunity to penalize him by taking the funds. Essentially, Bob is giving Alice the ability to hold him accountable for broadcasting an old commitment, and in effect he has revoked his ability to use that old commitment.
+
+Once Alice has received the +revoke_and_ack+ from Bob, she can be sure that Bob cannot broadcast the old commitment without being penalized. She now has the keys necessary to create a penalty transaction if Bob broadcasts an old commitment.
+
+[[revocation_secret_derivation]]
+==== Cheating and Penalty in Practice
+
+((("channel state","cheating and penalty in practice", id="ix_07_payment_channels-asciidoc16", range="startofrange")))((("cheating","monitoring for", id="ix_07_payment_channels-asciidoc17", range="startofrange")))In practice, both Alice and Bob have to monitor for cheating. They are monitoring the Bitcoin blockchain for any commitment transactions related to any of the channels they are operating. If they see a commitment transaction confirmed on-chain, they will check to see if it is the most recent commitment. If it is an "old" commitment, they must immediately construct and broadcast a penalty transaction. The penalty transaction spends _both_ the +to_local+ and +to_remote+ outputs, closing the channel and sending both balances to the cheated channel partner.
+
+To more easily allow both sides to keep track of the commitment numbers of the passed revoke commitments, each commitment actually _encodes_ the number of the commitment within the lock time and sequence fields in a transition. Within the ((("state hints")))protocol, this special encoding is referred to as _state hints_. Assuming a party knows the current commitment number, they're able to use the state hints to easily recognize if a broadcasted commitment was a revoked one, and if so, which commitment number was breached, as that number is used to easily look up which revocation secret should be used in the revocation secret tree (shachain).
+
+((("obfuscated state hints")))Rather than encode the state hint in plain sight, an _obfuscated_ state hint is used in its place. This obfuscation is achieved by first XORing the current commitment number with a set of random bytes generated deterministically using the funding public keys of both sides of the channel. A total of 6 bytes across the lock time and sequence (24 bits of the locktime and 24 bits of the sequence) are used to encode the state hint within the commitment transaction, so 6 random bytes are needed to use for XORing. To obtain these 6 bytes, both sides obtain the SHA-256 hash of the initiator's funding key concatenated to the responder's funding key. Before encoding the current commitment height, the integer is XORed with this state hint obfuscator, and then encoded in the lower 24 bits of the locktime, and the upper 64 bits of the sequence.
+
+Let's review our channel between Alice and Bob and show a specific example of a penalty transaction. In <<competing_commitments_2>> we see the four commitments on Alice and Bob's channel. Alice has made three payments to Bob:
+
+* 70,000 satoshis paid and committed to Bob with Commitment #1
+* 10,000 satoshis paid and committed to Bob with Commitment #2
+* 20,000 satoshis paid and committed to Bob with Commitment #3
+
+[[competing_commitments_2]]
+.Revoked and current commitments
+image::images/mtln_0712.png[Revoked and current commitments]
+
+With each commitment, Alice has revoked the previous (older) commitment. The current state of the channel and the correct balance is represented by Commitment #3. All previous commitments have been revoked, and Bob has the keys necessary to issue penalty transactions against them, in case Alice tries to broadcast one of them.
+
+Alice might have an incentive to cheat because all the previous commitment transactions would give her a higher proportion of the channel balance than she is entitled to. Let's say for example that Alice tried to broadcast Commitment #1. That commitment transaction would pay Alice 70,000 satoshis and Bob 70,000 satoshis. If Alice was able to broadcast and spend her +to_local+ output, she would effectively be stealing 30,000 satoshis from Bob by rolling back her last two payments to Bob.
+
+Alice decides to take a huge risk and broadcast the revoked Commitment #1, to steal 30,000 satoshis from Bob. In <<cheating_commitment>> we see Alice's old commitment that she broadcasts to the Bitcoin blockchain. 
+
+[[cheating_commitment]]
+.Alice cheating
+image::images/mtln_0713.png[Alice cheating]
+
+As you can see, Alice's old commitment has two outputs, one paying herself 70,000 satoshis (+to_local+ output) and one paying Bob 70,000 satoshis. Alice can't yet spend her 70,000 +to_local+ output because it has a 432 block (3 day) timelock. She is now hoping that Bob doesn't notice for three days.
+
+Unfortunately for Alice, Bob's node is diligently monitoring the Bitcoin blockchain and sees an old commitment transaction broadcast and (eventually) confirmed on-chain.
+
+Bob's node will immediately broadcast a penalty transaction. Since this old commitment was revoked by Alice, Bob has the +per_commitment_secret+ that Alice sent him. He uses that secret to construct a signature for the +revocation_pubkey+. While Alice has to wait for 432 blocks, Bob can spend _both_ outputs immediately. He can spend the +to_remote+ output with his private keys because it was meant to pay him anyway. He can also spend the output meant for Alice with a signature from the revocation key. His node broadcasts the penalty transaction shown in <<penalty_transaction>>. 
+
+[[penalty_transaction]]
+.Cheating and penalty
+image::images/mtln_0714.png[Cheating and penalty]
+
+Bob's penalty transaction pays 140,000 satoshis to his own wallet, taking the entire channel capacity. Alice has not only failed to cheat, she has lost everything in the attempt!(((range="endofrange", startref="ix_07_payment_channels-asciidoc17")))(((range="endofrange", startref="ix_07_payment_channels-asciidoc16")))
+
+==== The Channel Reserve: Ensuring Skin in the Game
+
+((("channel reserve")))((("channel state","channel reserve")))You may have noticed there is a special situation that needs to be dealt with. If Alice could keep spending her balance until it is zero, she would be in a position to close the channel by broadcasting an old commitment transaction without risking a penalty: either the revoked commitment transaction succeeds after the delay, or the cheater gets caught but there's no consequence because the penalty is zero. From a game theory perspective, it is free money to attempt to cheat in this situation. This is why the channel reserve is in play, so a prospective cheater always faces the risk of a penalty.(((range="endofrange", startref="ix_07_payment_channels-asciidoc15")))(((range="endofrange", startref="ix_07_payment_channels-asciidoc14")))
+
+=== Closing the Channel (Cooperative Close)
+
+((("payment channel","closing the channel", id="ix_07_payment_channels-asciidoc18", range="startofrange")))((("payment channel","cooperative close", id="ix_07_payment_channels-asciidoc19", range="startofrange")))So far we've looked at the commitment transactions as one possible way to close a channel, unilaterally. This type of channel closure is not ideal because it forces a timelock on the channel partner that uses it.
+
+A better way to close a channel is a cooperative close. In a cooperative close, the two ((("closing transactions")))channel partners negotiate a final commitment transaction called the _closing transaction_ that pays each party their balance immediately to the destination wallet of their choice. Then, the partner that initiated the channel closing flow will broadcast the closing transaction.
+
+The closing message flow is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#channel-close[BOLT #2: Peer Protocol, Channel Close], and is shown in <<closing_message_flow>>. 
+
+[[closing_message_flow]]
+.The channel close message flow
+image::images/mtln_0715.png[The channel close message flow]
+
+[[theShutdownmessage]]
+==== The Shutdown Message
+
+((("closing the channel","shutdown message")))((("shutdown message")))Channel closing starts with one of the two channel partners sending the +shutdown+ message. The contents of this message are shown here:
+
+[[shutdown_message]]
+.The shutdown message
+----
+
+[channel_id:channel_id]
+[u16:len]
+[len*byte:scriptpubkey]
+
+----
+
++channel_id+:: The channel identifier for the channel we want to close
++len+:: The length of the script of the destination wallet that this channel partner wants to receive their balance
++scriptpubkey+:: A Bitcoin script of the destination wallet, in one of the "standard" Bitcoin address formats (P2PKH, P2SH, P2WPKH, P2WSH, etc.; see the <<glossary>>)
+
+Let's say Alice sends the +shutdown+ message to Bob to close their channel. Alice will specify a Bitcoin script that corresponds to the Bitcoin address of her wallet. She's telling Bob: let's make a closing transaction that pays my balance to this wallet. 
+
+Bob will respond with his own +shutdown+ message indicating that he agrees to cooperatively close the channel. His +shutdown+ message includes the script for his wallet address.
+
+Now both Alice and Bob have each other's preferred wallet address, and they can construct identical closing transactions to settle the channel balance.
+
+==== The closing_signed Message
+
+((("closing the channel","closing_signed message")))((("closing_signed message")))Assuming the channel has no outstanding commitments or updates and the channel partners have exchanged the +shutdown+ messages shown in the previous section, they can now finish this cooperative close.
+
+The _funder_ of the channel (Alice in our example) starts by sending a +closing_signed+ message to Bob. This message proposes a transaction fee for the on-chain transaction, and Alice's signature (the 2-of-2 multisig) for the closing transaction. The +closing_signed+ message is shown here:
+
+[[closing_signed_message]]
+.The closing_signed message
+----
+[channel_id:channel_id]
+[u64:fee_satoshis]
+[signature:signature]
+----
+
++channel_id+:: The channel identifier
++fee_satoshis+:: The proposed on-chain transaction fee, in satoshis
++signature+:: The sender's signature for the closing transaction
+
+When Bob receives this, he can reply with a +closing_signed+ message of his own. If he agrees with the fee, he simply returns the same proposed fee and his own signature. If he disagrees, he must propose a different +fee_satoshis+ fee.
+
+This negotiation may continue with back-and-forth +closing_signed+ messages until the two channel partners agree on a fee.
+
+Once Alice receives a +closing_signed+ message with the same fee as the one she proposed in her last message, the negotiation is complete. Alice signs and broadcasts the closing transaction and the channel is closed.
+
+==== The Cooperative Close Transaction
+
+((("closing the channel","cooperative close transaction")))((("cooperative close transaction")))The cooperative close transaction looks similar to the last commitment transaction that Alice and Bob had agreed on. However, unlike the last commitment transaction, it does not have timelocks or penalty revocation keys in the outputs. Since both parties cooperate to produce this transaction and they won't be making any further commitments, there is no need for the asymmetric, delayed, and revocable elements in this transaction.
+
+Typically the addresses used in this cooperative close transaction are generated freshly for each channel being closed. However, it's also possible for both sides to _lock in_ a "delivery" address to be used to send their cooperatively settled funds to. Within the TLV namespace of both the `open_channel` and `accept_channel` messages, both sides are free to specify an "up-front shutdown script." Commonly, this address is derived from keys that reside in cold storage. This practice serves to increase the security of channels: if a channel partner is somehow hacked, then the hacker isn't able to cooperatively close the channel using an address they control. Instead, the uncompromised honest channel partner will refuse to cooperate on a channel closure if the specified up-front shutdown address isn't used. This feature effectively creates a "closed loop," restricting the flow of funds out of a given channel.
+
+Alice broadcasts a transaction shown in <<closing_transaction>> to close the channel. 
+
+[[closing_transaction]]
+.The cooperative close transaction
+image::images/mtln_0716.png[The cooperative close transaction]
+
+As soon as this closing transaction is confirmed on the Bitcoin blockchain, the channel is closed. Now, Alice and Bob can spend their outputs as they please.(((range="endofrange", startref="ix_07_payment_channels-asciidoc19")))(((range="endofrange", startref="ix_07_payment_channels-asciidoc18")))
+
+=== Conclusion
+
+In this section we looked at payment channels in much more detail. We examined three message flows used by Alice and Bob to negotiate funding, commitments, and closing of the channel. We also showed the structure of the funding, commitment, and closing transactions, and looked at the revocation and penalty mechanisms.
+
+As we will see in the next few chapters, HTLCs are used even for local payments between channel partners. They are not necessary, but the protocol is much simpler if local (one channel) and routed (many channels) payments are done in the same way.
+
+In a single payment channel, the number of payments per second is only bound by the network capacity between Alice and Bob. As long as the channel partners are able to send a few bytes of data back and forth to agree to a new channel balance, they have effectively made a payment. This is why we can achieve a much greater throughput of payments on the Lightning Network (off-chain) than the transaction throughput that can be handled by the Bitcoin blockchain (on-chain).(((range="endofrange", startref="ix_07_payment_channels-asciidoc0")))
+
+In the next few chapters we will discuss routing, HTLCs, and their use in channel operations.

08_routing_htlcs.asciidoc

@@ -0,0 +1,620 @@
+[[routing]]
+== Routing on a Network of pass:[<span class="keep-together">Payment Channels</span>]
+
+((("routing", id="ix_08_routing_htlcs-asciidoc0", range="startofrange")))In this chapter we will finally unpack how payment channels can be connected to form a network of payment channels via a process called _routing_. Specifically, we will look at the first part of the routing layer, the "Atomic and trustless multihop contracts" protocol. This is highlighted by an outline in the protocol suite, shown in <<LN_protocol_routing_highlight>>.
+
+[[LN_protocol_routing_highlight]]
+.Atomic payment routing in the Lightning protocol suite
+image::images/mtln_0801.png["Atomic payment routing in the Lightning protocol suite"]
+
+=== Routing a Payment
+
+((("routing","routing a payment")))In this section we will examine routing from the perspective of Dina, a gamer who receives donations from her fans while she streams her game sessions.
+
+The innovation of routed payment channels allows Dina to receive tips without maintaining a separate channel with every one of her fans who want to tip her.
+As long as there exists a path of well-funded channels from that viewer to Dina, she will be able to receive payment from that fan.
+
+In <<dina_routing_diagram>> we see a possible network layout created by various payment channels between Lightning nodes. Everyone in this diagram can send Dina a payment by constructing a path. Imagine that Fan 4 wants to send Dina a payment. Do you see the path that could allow that to happen? Fan 4 could route a payment to Dina via Fan 3, Bob, and Chan. Similarly, Alice could route a payment to Dina via Bob and Chan.
+
+[[dina_routing_diagram]]
+.Fans connected (in)directly to Dina on the Lightning Network
+image::images/mtln_0802.png["Fans connected (in)directly to Dina on the Lightning Network"]
+
+((("routing nodes")))The nodes along the path from the fan to Dina are intermediaries called _routing nodes_ in the context of routing a payment. There is no functional difference between the routing nodes and the nodes operated by Dina's fans. Any Lightning node is capable of routing payments across its payment channels.
+
+Importantly, the routing nodes are unable to steal the funds while routing a payment from a fan to Dina.
+Furthermore, routing nodes cannot lose money while participating in the routing process.
+Routing nodes can charge a routing fee for acting as an intermediary, although they don't have to and may choose to route payments for free.
+
+Another important detail is that due to the use of onion routing, intermediary nodes are only explicitly aware of the one node preceding them and the one node following them in the route.
+They will not necessarily know who is the sender and recipient of the payment.
+This enables fans to use intermediary nodes to pay Dina, without leaking private information and without risking theft.
+
+This process of connecting a series of payment channels with end-to-end security, and the incentive structure for nodes to _forward_ payments, is one of the key innovations of the Lightning Network.
+
+In this chapter, we'll dive into the mechanism of routing in the Lightning Network, detailing the precise manner in which payments flow through the network. First, we will clarify the concept of routing and compare it to that of pathfinding, because these are often confused and used interchangeably. Next, we will construct the fairness protocol: an atomic, trustless, multihop protocol used to route payments. To demonstrate how this fairness protocol works, we will be using a physical equivalent of transferring gold coins between four people. Finally, we will look at the atomic, trustless, multihop protocol implementation currently used in the Lightning Network, which is called a hash time-locked contract (HTLC).
+
+=== Routing Versus Pathfinding
+
+((("pathfinding","routing versus")))((("routing","pathfinding versus")))It's important to note that we separate the concept of _routing_ from the concept of _pathfinding_. These two concepts are often confused, and the term _routing_ is often used to describe both concepts. Let's remove the ambiguity before we proceed any further.
+
+Pathfinding, which is covered in <<path_finding>>, is the process of finding and choosing a contiguous path made of payment channels that connects sender A to recipient B. The sender of a payment does the pathfinding by examining the _channel graph_ that they have assembled from channel announcements gossiped by other nodes.
+
+Routing refers to the series of interactions across the network that attempt to forward a payment from some point A to another point B, across the path previously selected by pathfinding. Routing is the active process of sending a payment on a path, which involves the cooperation of all the intermediary nodes along that path.
+
+An important rule of thumb is that it's possible for a _path_ to exist between Alice and Bob (perhaps even more than one), yet there may not be an active _route_ on which to send the payment. One example is the scenario in which all the nodes connecting Alice and Bob are currently offline. In this example, one can examine the channel graph and connect a series of payment channels from Alice to Bob, hence a _path_ exists. However, because the intermediary nodes are offline, the payment cannot be sent and so no _route_ exists.
+
+=== Creating a Network of Payment Channels
+
+((("routing","creating a network of payment channels")))Before we dive into the concept of an atomic trustless multihop payment, let's work through an example.
+Let's return to Alice who, in previous chapters, purchased a coffee from Bob with whom she has an open channel.
+Now Alice is watching a live stream from Dina, the gamer, and wants to send Dina a tip of 50,000 satoshis via the Lightning Network. But Alice has no direct channel with Dina. What can Alice do?
+
+Alice could open a direct channel with Dina; however, that would require liquidity and on-chain fees which could be more than the value of the tip itself. Instead, Alice can use her existing open channels to send a tip to Dina _without_ the need to open a channel directly with Dina. This is possible, as long as there exists some path of channels from Alice to Dina with sufficient capacity to route the tip.
+
+As you can see in <<routing_network>>, Alice has an open channel with Bob, the coffee shop owner. Bob, in turn, has an open channel with the software developer Chan who helps him with the point of sale system he uses in his coffee shop. Chan is also the owner of a large software company which develops the game that Dina plays, and they already have an open channel which Dina uses to pay for the game's license and in-game items.
+
+[[routing_network]]
+.A network of payment channels between Alice and Dina
+image::images/mtln_0803.png["A network of payment channels between Alice and Dina"]
+
+It's possible to trace a _path_ from Alice to Dina that uses Bob and Chan as intermediary routing nodes.
+Alice can then craft a _route_ from this outlined path and use it to send a tip of a few thousand satoshis to Dina, with the payment being _forwarded_ by Bob and Chan.
+Essentially, Alice will pay Bob, who will pay Chan, who will pay Dina. No direct channel from Alice to Dina is required.
+
+The main challenge is to do this in a way that prevents Bob and Chan from stealing the money that Alice wants delivered to Dina.
+
+=== A Physical Example of "Routing"
+
+((("routing","real-world physical example", id="ix_08_routing_htlcs-asciidoc1", range="startofrange")))To understand how the Lightning Network protects the payment while being routed, we can compare it to an example of routing physical payments with gold coins in the real world.
+
+Assume Alice wants to give 10 gold coins to Dina, but does not have direct access to Dina. However, Alice knows Bob, who knows Chan, who knows Dina, so she decides to ask Bob and Chan for help. This is shown in <<alice_dina_routing_1>>.
+
+[[alice_dina_routing_1]]
+.Alice wants to pay Dina 10 gold coins
+image::images/mtln_0804.png[]
+
+Alice can pay Bob to pay Chan to pay Dina, but how does she make sure that Bob or Chan don't run off with the coins after receiving them?
+In the physical world, contracts could be used for safely carrying out a series of payments.
+
+Alice could negotiate a contract with Bob, which reads:
+
+____
+_I, Alice, will give you, Bob, 10 gold coins if you pass them on to Chan._
+____
+
+While this contract is nice in the abstract, in the real world, Alice runs the risk that Bob might breach the contract and hope not to get caught.
+Even if Bob is caught and prosecuted, Alice faces the risk that he might be bankrupt and be unable to return her 10 gold coins.
+Assuming these issues are magically solved, it's still unclear how to leverage such a contract to achieve our desired outcome: getting the coins delivered to Dina.
+
+Let's improve our contract to incorporate these considerations:
+
+____
+_I, Alice, will reimburse you, Bob, with 10 gold coins if you can prove to me (for example, via a receipt) that you have delivered 10 gold coins to Chan._
+____
+
+You might ask yourself why should Bob sign such a contract.
+He has to pay Chan but ultimately gets nothing out of the exchange, and he runs the risk that Alice might not reimburse him. Bob could offer Chan a similar contract to pay Dina, but similarly Chan has no reason to accept it either.
+
+Even putting aside the risk, Bob and Chan must _already_ have 10 gold coins to send; otherwise, they wouldn't be able to participate in the contract.
+
+Thus Bob and Chan face both risk and opportunity cost for agreeing to this contract, and they would need to be compensated to accept it.
+
+Alice can then make this attractive to both Bob and Chan by offering them fees of one gold coin each, if they transmit her payment to Dina.
+
+The contract would then read:
+
+____
+_I, Alice, will reimburse you, Bob, with 12 gold coins if you can prove to me (for example, via a receipt) that you have delivered 11 gold coins to Chan._
+____
+
+Alice now promises Bob 12 gold coins. There are 10 to be delivered to Dina and 2 for the fees. She promises 12 to Bob if he can prove that he has forwarded 11 to Chan.
+The difference of one gold coin is the fee that Bob will earn for helping out with this particular payment. In <<alice_dina_routing_2>> we see how this arrangement would get 10 gold coins to Dina via Bob and Chan.
+
+[[alice_dina_routing_2]]
+.Alice pays Bob, Bob pays Chan, Chan pays Dina
+image::images/mtln_0805.png[]
+
+Because there is still the issue of trust and the risk that either Alice or Bob won't honor the contract, all parties decide to use an escrow service.
+At the start of the exchange, Alice could "lock up" these 12 gold coins in escrow that will only be paid to Bob once he proves that he's paid 11 gold coins to Chan.
+
+This escrow service is an idealized one, which does not introduce other risks (e.g., counterparty risk). Later we will see how we can replace the escrow with a Bitcoin smart contract. Let's assume for now that everyone trusts this escrow service.
+
+In the Lightning Network, the receipt (proof of payment) could take the form of a secret that only Dina knows.
+In practice, this secret would be a random number that is large enough to prevent others from guessing it (typically a _very, very_ large number, encoded using 256 bits!).
+
+Dina generates this secret value +R+ from a random number generator.
+
+The secret could then be committed to the contract by including the SHA-256 hash of the secret in the contract itself, as follows:
+
+++++
+<ul class="simplelist">
+<li><em>H</em> = SHA-256(<em>R</em>)</li>
+</ul>
+++++
+
+((("payment hash")))((("payment secret (preimage)")))((("preimage (payment secret)")))We call this hash of the payment's secret the _payment hash_.
+The secret that "unlocks" the payment is called the _payment secret_.
+
+For now, we keep things simple and assume that Dina's secret is simply the text line: `Dinas secret`. This secret message is called the _payment secret_ or _payment preimage_.
+
+To "commit" to this secret, Dina computes the SHA-256 hash, which when encoded in hexadecimal, can be displayed as follows:
+
+----
+0575965b3b44be51e8057d551c4016d83cb1fba9ea8d6e986447ba33fe69f6b3
+----
+
+To facilitate Alice's payment, Dina will create the payment secret and the payment hash, and send the payment hash to Alice. In <<alice_dina_routing_3>> we see that Dina sends the payment hash to Alice via some external channel (dashed line), such as an email or text message.
+
+[[alice_dina_routing_3]]
+.Dina sends the hashed secret to Alice
+image::images/mtln_0806.png["Dina sends the hashed secret to Alice"]
+
+Alice doesn't know the secret, but she can rewrite her contract to use the hash of the secret as a proof of payment:
+
+____
+_I, Alice, will reimburse you, Bob, with 12 gold coins if you can show me a valid message that hashes to:`057596`....
+You can acquire this message by setting up a similar contract with Chan who has to set up a similar contract with Dina.
+To assure you that you will be reimbursed, I will provide the 12 gold coins to a trusted escrow before you set up your next contract._
+____
+
+This new contract now protects Alice from Bob not forwarding to Chan, protects Bob from not being reimbursed by Alice, and ensures that there will be proof that Dina was ultimately paid via the hash of Dina's secret.
+
+After Bob and Alice agree to the contract, and Bob receives the message from the escrow that Alice has deposited the 12 gold coins, Bob can now negotiate a similar contract with Chan.
+
+Note that since Bob is taking a service fee of 1 coin, he will only forward 11 gold coins to Chan once Chan shows proof that he has paid Dina.
+Similarly, Chan will also demand a fee and will expect to receive 11 gold coins once he has proved that he has paid Dina the promised 10 gold coins.
+
+Bob's contract with Chan will read:
+
+____
+_I, Bob, will reimburse you, Chan, with 11 gold coins if you can show me a valid message that hashes to:`057596`....
+You can acquire this message by setting up a similar contract with Dina.
+To assure you that you will be reimbursed, I will provide the 11 gold coins to a trusted escrow before you set up your next contract._
+____
+
+Once Chan gets the message from the escrow that Bob has deposited the 11 gold coins, Chan sets up a similar contract with Dina:
+
+____
+_I, Chan, will reimburse you, Dina, with 10 gold coins if you can show me a valid message that hashes to:`057596`....
+To assure you that you will be reimbursed after revealing the secret, I will provide the 10 gold coins to a trusted escrow._
+____
+
+Everything is now in place.
+Alice has a contract with Bob and has placed 12 gold coins in escrow.
+Bob has a contract with Chan and has placed 11 gold coins in escrow.
+Chan has a contract with Dina and has placed 10 gold coins in escrow.
+It is now up to Dina to reveal the secret, which is the preimage to the hash she has established as proof of payment.
+
+Dina now sends +Dinas secret+ to Chan.
+
+Chan checks that +Dinas secret+ hashes to +057596+.... Chan now has proof of payment and so instructs the escrow service to release the 10 gold coins to Dina.
+
+Chan now provides the secret to Bob. Bob checks it and instructs the escrow service to release the 11 gold coins to Chan.
+
+Bob now provides the secret to Alice.
+Alice checks it and instructs the escrow to release 12 gold coins to Bob.
+
+All the contracts are now settled.
+Alice has paid a total of 12 gold coins, 1 of which was received by Bob, 1 of which was received by Chan, and 10 of which were received by Dina.
+With a chain of contracts like this in place, Bob and Chan could not run away with the money because they deposited it in escrow first.
+
+However, one issue still remains.
+If Dina refused to release her secret preimage, then Chan, Bob, and Alice would all have their coins stuck in escrow but wouldn't be reimbursed.
+And similarly if anyone else along the chain failed to pass on the secret, the same thing would happen.
+So while no one can steal money from Alice, everyone would still have their money stuck in escrow permanently.
+
+Luckily, this can be resolved by adding a deadline to the contract.
+
+We could amend the contract so that if it is not fulfilled by a certain deadline, then the contract expires and the escrow service returns the money to the person who made the original deposit.
+We call this deadline a _timelock_.
+
+The deposit is locked with the escrow service for a certain amount of time and is eventually released even if no proof of payment was provided.
+
+To factor this in, the contract between Alice and Bob is once again amended with a new clause:
+
+____
+_Bob has 24 hours to show the secret after the contract was signed.
+If Bob does not provide the secret by this time, Alice's deposit will be refunded by the escrow service and the contract becomes invalid._
+____
+
+Bob, of course, now has to make sure he receives the proof of payment within 24 hours.
+Even if he successfully pays Chan, if he receives the proof of payment later than 24 hours, he will not be reimbursed. To remove that risk, Bob must give Chan an even shorter deadline.
+
+In turn, Bob will alter his contract with Chan as follows:
+
+____
+_Chan has 22 hours to show the secret after the contract was signed.
+If he does not provide the secret by this time, Bob's deposit will be refunded by the escrow service and the contract becomes invalid._
+____
+
+As you might have guessed, Chan will also alter his contract with Dina:
+
+____
+_Dina has 20 hours to show the secret after the contract was signed.
+If she does not provide the secret by this time, Chan's deposit will be refunded by the escrow service and the contract becomes invalid._
+____
+
+With such a chain of contracts we can ensure that, after 24 hours, the payment will successfully go from Alice to Bob to Chan to Dina, or it will fail and everyone will be refunded.
+Either the contract fails or succeeds, there's no middle ground.
+
+In the context of the Lightning Network, we call this "all or nothing" property _atomicity_.
+
+As long as the escrow is trustworthy and faithfully performs its duty, no party will have their coins stolen in the process.
+
+The precondition to this _route_ working at all is that all parties in the path have enough money to satisfy the required series of deposits.
+
+While this seems like a minor detail, we will see later in this chapter that this requirement is actually one of the more difficult issues for LN nodes.
+It becomes progressively more difficult as the size of the payment increases.
+Furthermore, the parties cannot use their money while it is locked in escrow.
+
+Thus, users forwarding payments face an opportunity cost for locking the money, which is ultimately reimbursed through routing fees, as we saw in the preceding example.
+
+Now that we've seen a physical payment routing example, we will see how this can be implemented on the Bitcoin blockchain, without any need for third-party escrow. To do this we will be setting up the contracts between the participants using Bitcoin Script. We replace the third-party escrow with _smart contracts_ that implement a fairness protocol. Let's break that concept down and implement it!(((range="endofrange", startref="ix_08_routing_htlcs-asciidoc1")))
+
+=== Fairness Protocol
+
+((("fairness protocol","routing and")))((("routing","fairness protocol")))As we saw in the first chapter of this book, the innovation of Bitcoin is the ability to use cryptographic primitives to implement a fairness protocol that substitutes trust in third parties (intermediaries) with a trusted protocol.
+
+In our gold coin example, we needed an escrow service to prevent any one of the parties from reneging on their obligations. The innovation of cryptographic fairness protocols allows us to replace the escrow service with a protocol.
+
+((("fairness protocol","properties")))The properties of the fairness protocol we want to create are:
+
+Trustless operation:: The participants in a routed payment do not need to trust each other, or any intermediary or third party. Instead, they trust the protocol to protect them from cheating.
+
+Atomicity:: Either the payment is fully executed, or it fails and everyone is refunded. There is no possibility of an intermediary collecting a routed payment and not forwarding it to the next hop. Thus, the intermediaries can't cheat or steal.
+
+Multihop:: The security of the system extends end to end for payments routed through multiple payment channels, just as it is for a payment between the two ends of a single payment channel.
+
+An optional, additional property is the ability to split payments into multiple parts while maintaining atomicity for the entire payment. These are called _multipart payments_ (_MPP_) and are explored further in <<mpp>>.
+
+==== Implementing Atomic Trustless Multihop Payments
+
+((("fairness protocol","implementing atomic trustless multihop payments")))((("routing","implementing atomic trustless multihop payments")))Bitcoin Script is flexible enough that there are dozens of ways to implement a fairness protocol that has the properties of atomicity, trustless operation, and multihop security. Choosing a specific implementation is dependent on certain trade-offs among privacy, efficiency, and complexity.
+
+((("hash time-locked contracts (HTLCs)","fairness protocol")))The fairness protocol for routing used in the Lightning Network today is called a hash time-locked contract (HTLC). HTLCs use a hash preimage as the secret that unlocks a payment, as we saw in the gold coin example in this chapter. The recipient of a payment generates a random secret number and calculates its hash. The hash becomes the condition of payment, and once the secret is revealed, all the participants can redeem their incoming payments. HTLCs offer atomicity, trustless operation, and multihop security.
+
+((("Point Time-Locked Contract (PTLC)")))((("PTLC (Point Time-Locked Contract)")))Another proposed mechanism for implementing routing is a _Point Time-Locked Contract_ (_PTLC_). PTLCs also achieve atomicity, trustless operation, and multihop security, but do so with increased efficiency and better privacy.  Efficient implementation of PTLCs depends on a new digital signature algorithm called _Schnorr signatures_, which is expected to be activated in Bitcoin in 2021.
+
+=== Revisiting the Tipping Example
+
+((("routing","real-world physical example")))Let's revisit our example from the first part of this chapter. Alice wants to tip Dina with a Lightning payment. Let's say Alice wants to send Dina 50,000 satoshis as a tip.
+
+For Alice to pay Dina, Alice will need Dina's node to generate a Lightning invoice. We will discuss this in more detail in <<invoices>>. For now, let's assume that Dina has a website that can produce a Lightning invoice for tips.
+
+[TIP]
+====
+Lightning payments can be sent without an invoice using a feature called _keysend_, which we will discuss in more detail in <<keysend>>. For now, we will explain the simpler payment flow using an invoice.
+====
+
+Alice visits Dina's site, enters the amount of 50,000 satoshis in a form, and in response, Dina's Lightning node generates a payment request for 50,000 satoshis in the form of a Lightning invoice. This interaction takes place over the web and outside the Lightning Network, as shown in <<alice_dina_invoice_1>>.
+
+[[alice_dina_invoice_1]]
+.Alice requests an invoice from Dina's website
+image::images/mtln_0807.png["Alice requests an invoice from Dina's website"]
+
+As we saw in previous examples, we assume that Alice does not have a direct payment channel to Dina. Instead, Alice has a channel to Bob, Bob has a channel to Chan, and Chan has a channel to Dina. To pay Dina, Alice must find a path that connects her to Dina. We will discuss that step in more detail in <<path_finding>>. For now, let's assume that Alice is able to gather information about available channels and sees that there is a path from her to Dina, via Bob and Chan.
+
+[NOTE]
+====
+Remember how Bob and Chan might expect a small compensation for routing the payment through their nodes? Alice wants to pay Dina 50,000 satoshis, but as you will see in the following sections she will send Bob 50,200 satoshis. The extra 200 satoshis will pay Bob and Chan 100 satoshis each, as a routing fee.
+====
+
+Now, Alice's node can construct a Lightning payment. In the next few sections, we will see how Alice's node constructs an HTLC to pay Dina and how that HTLC is forwarded along the path from Alice to Dina.
+
+
+==== On-Chain Versus Off-Chain Settlement of HTLCs
+
+((("hash time-locked contracts (HTLCs)","on-chain versus off-chain settlement of")))((("off-chain settlement, on-chain payment versus")))((("on-chain payment","off-chain settlement versus")))((("routing","on-chain versus off-chain settlement of HTLCs")))The purpose of the Lightning Network is to enable _off-chain_ transactions that are trusted just the same as on-chain transactions because no one can cheat. The reason no one can cheat is because at any time, any of the participants can take their off-chain transactions on-chain. Each off-chain transaction is ready to be submitted to the Bitcoin blockchain at any time. Thus, the Bitcoin blockchain acts as a dispute-resolution and final settlement mechanism if necessary.
+
+The mere fact that any transaction can be taken on-chain at any time is precisely the reason that all those transactions can be kept off-chain. If you know you have recourse, you can continue to cooperate with the other participants and avoid the need for on-chain settlement and extra fees.
+
+In all the examples that follow, we will assume that any of these transactions can be made on-chain at any time. The participants will choose to keep them off-chain, but there is no difference in the functionality of the system other than the higher fees and delay imposed by on-chain mining of the transactions. The example works the same if all the transactions are on-chain or off-chain.
+
+[[htlcs]]
+=== Hash Time-Locked Contracts
+
+((("hash time-locked contracts (HTLCs)","mechanism of operation", id="ix_08_routing_htlcs-asciidoc2", range="startofrange")))((("routing","hash time-locked contracts mechanism of operation", id="ix_08_routing_htlcs-asciidoc3", range="startofrange")))In this section we explain how HTLCs work.
+
+The first part of an HTLC is the _hash_. This refers to the use of a cryptographic hash algorithm to commit to a randomly generated secret. Knowledge of the secret allows redemption of the payment. The cryptographic hash function guarantees that while it's infeasible for anyone to guess the secret preimage, it's easy for anyone to verify the hash, and there's only one possible preimage that resolves the payment condition.
+
+In <<alice_dina_invoice_2>> we see Alice getting a Lightning invoice from Dina. Inside that invoice ((("payment hash")))Dina has encoded a _payment hash_, which is the cryptographic hash of a secret that Dina's node produced. ((("payment secret (preimage)")))((("preimage (payment secret)")))Dina's secret is called the _payment preimage_. The payment hash acts as an identifier that can be used to route the payment to Dina. The payment preimage acts as a receipt and proof of payment once the payment is complete.
+
+[[alice_dina_invoice_2]]
+.Alice gets a payment hash from Dina
+image::images/mtln_0808.png["Alice gets a payment hash from Dina"]
+
+In the Lightning Network, Dina's payment preimage won't be a phrase like +Dinas secret+ but a random number generated by Dina's node. Let's call that random number _R_.
+
+Dina's node will calculate a cryptographic hash of _R_, such that:
+
+++++
+<ul class="simplelist">
+<li><em>H</em> = SHA-256(<em>R</em>)</li>
+</ul>
+++++
+
+In this equation, _H_ is the hash, or _payment hash_ and _R_ is the secret or _payment preimage_.
+
+The use of a cryptographic hash function is one element that guarantees _trustless operation_. The payment intermediaries do not need to trust each other because they know that no one can guess the secret or fake it.
+
+==== HTLCs in Bitcoin Script
+
+((("Bitcoin script","HTLCs in")))((("hash time-locked contracts (HTLCs)","Bitcoin Script and")))In our gold coin example, Alice had a contract enforced by escrow like this:
+
+____
+_Alice will reimburse Bob with 12 gold coins if you can show a valid message that hashes to:_ +0575...f6b3+. _Bob has 24 hours to show the secret after the contract was signed. If Bob does not provide the secret by this time, Alice's deposit will be refunded by the escrow service and the contract becomes invalid._
+____
+
+[role="pagebreak-before"]
+Let's see how we would implement this as an HTLC in Bitcoin Script. In <<received_htlc>> we see an HTLC Bitcoin Script as currently used in the Lightning Network. You can find this definition in https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md#offered-htlc-outputs[BOLT #3, Transactions].
+
+[[received_htlc]]
+.HTLC implemented in Bitcoin Script (BOLT #3)
+[source,text,linenums]
+====
+----
+# To remote node with revocation key
+OP_DUP OP_HASH160 <RIPEMD160(SHA256(revocationpubkey))> OP_EQUAL
+OP_IF
+    OP_CHECKSIG
+OP_ELSE
+    <remote_htlcpubkey> OP_SWAP OP_SIZE 32 OP_EQUAL
+    OP_IF
+        # To local node via HTLC-success transaction.
+        OP_HASH160 <RIPEMD160(payment_hash)> OP_EQUALVERIFY
+        2 OP_SWAP <local_htlcpubkey> 2 OP_CHECKMULTISIG
+    OP_ELSE
+        # To remote node after timeout.
+        OP_DROP <cltv_expiry> OP_CHECKLOCKTIMEVERIFY OP_DROP
+        OP_CHECKSIG
+    OP_ENDIF
+OP_ENDIF
+----
+====
+
+Wow, that looks complicated! Don't worry though, we will take it one step at a time and simplify it.
+
+The Bitcoin Script currently used in the Lightning Network is quite complex because it is optimized for on-chain space efficiency, which makes it very compact but difficult to read.
+
+In the following sections, we will focus on the main elements of the script and present simplified scripts that are slightly different from what is actually used in Lightning.
+
+The main part of the HTLC is in line 10 of <<received_htlc>>. Let's build it up from scratch!
+
+==== Payment Preimage and Hash Verification
+
+((("hash time-locked contracts (HTLCs)","payment preimage and hash verification")))((("hash verification")))((("payment secret (preimage)")))((("preimage (payment secret)")))The core of an HTLC is the hash, where payment can be made if the recipient knows the payment preimage. Alice locks the payment to a specific payment hash, and Bob has to present a payment preimage to claim the funds. The Bitcoin system can verify that Bob's payment preimage is correct by hashing it and comparing the result to the payment hash that Alice used to lock the funds.
+
+This part of an HTLC can be implemented in Bitcoin Script as follows:
+
+----
+OP_SHA256 <H> OP_EQUAL
+----
+
+Alice can create a transaction output that pays, 50,200 satoshi with a locking script above, replacing `<H>` with the hash value +0575...f6b3+ provided by Dina. Then, Alice can sign this transaction and offer it to Bob:
+
+.Alice's offers a 50,200 satoshi HTLC to Bob
+----
+OP_SHA256 0575...f6b3 OP_EQUAL
+----
+
+Bob can't spend this HTLC until he knows Dina's secret, so spending the HTLC is conditional on Bob's fulfillment of the payment all the way to Dina.
+
+Once Bob has Dina's secret, Bob can spend this output with an unlocking script containing the secret preimage value _R_.
+
+The unlocking script combined with the locking script would produce:
+
+----
+<R> OP_SHA256 <H> OP_EQUAL
+----
+
+The Bitcoin Script engine would evaluate this script as follows:
+
+1. +R+ is pushed to the stack.
+2. The `OP_SHA256` operator takes the value +R+ off the stack and hashes it, pushing the result +H~R~+ to the stack.
+3. +H+ is pushed to the stack.
+4. The `OP_EQUAL` operator compares +H+ and +H~R~+. If they are equal, the result is +TRUE+, the script is complete, and the payment is verified.
+
+==== Extending HTLCs from Alice to Dina
+
+((("hash time-locked contracts (HTLCs)","extending across a network")))Alice will now extend the HTLC across the network so that it reaches Dina.
+
+In <<alice_dina_htlc_1>>, we see the HTLC propagated across the network from Alice to Dina. Alice has given Bob an HTLC for 50,200 satoshi. Bob can now create an HTLC for 50,100 satoshi and give it to Chan.
+
+Bob knows that Chan can't redeem Bob's HTLC without broadcasting the secret, at which point Bob can also use the secret to redeem Alice's HTLC. This is a really important point because it ensures end-to-end _atomicity_ of the HTLC. To spend the HTLC, one needs to reveal the secret, which then makes it possible for others to spend their HTLC also. Either all the HTLCs are spendable, or none of the HTLCs are spendable: atomicity!
+
+Because Alice's HTLC is 100 satoshi more than the HTLC Bob gave to Chan, Bob will earn 100 satoshi as a routing fee if this payment completes.
+
+Bob isn't taking a risk and isn't trusting Alice or Chan. Instead, Bob is trusting that a signed transaction together with the secret will be redeemable on the Bitcoin blockchain.
+
+[[alice_dina_htlc_1]]
+.Propagating the HTLC across the network
+image::images/mtln_0809.png["Propagating the HTLC across the network"]
+
+Similarly, Chan can extend a 50,000 HTLC to Dina. He isn't risking anything or trusting Bob or Dina. To redeem the HTLC, Dina would have to broadcast the secret, which Chan could use to redeem Bob's HTLC. Chan would also earn 100 satoshis as a routing fee.
+
+==== Back-Propagating the Secret
+
+((("hash time-locked contracts (HTLCs)","back-propagating the secret", id="ix_08_routing_htlcs-asciidoc4", range="startofrange")))Once Dina receives a 50,000 HTLC from Chan, she can now get paid. Dina could simply commit this HTLC on-chain and spend it by revealing the secret in the spending transaction. Or, instead, Dina can update the channel balance with Chan by giving him the secret. There's no reason to incur a transaction fee and go on-chain. So, instead, Dina sends the secret to Chan, and they agree to update their channel balances to reflect a 50,000 satoshi Lightning payment to Dina. In <<alice_dina_htlc_redeem_1>> we see Dina giving the secret to Chan, thereby fulfilling the HTLC.
+
+[[alice_dina_htlc_redeem_1]]
+.Dina settles Chan's HTLC off-chain
+image::images/mtln_0810.png["Dina settles Chan's HTLC off-chain"]
+
+Notice that Dina's channel balance goes from 50,000 satoshi to 100,000 satoshi. Chan's channel balance is reduced from 200,000 satoshi to 150,000 satoshi. The channel capacity hasn't changed, but 50,000 has moved from Chan's side of the channel to Dina's side of the channel.
+
+Chan now has the secret and has paid Dina 50,000 satoshi. He can do this without any risk, because the secret allows Chan to redeem the 50,100 HTLC from Bob. Chan has the option to commit that HTLC on-chain and spend it by revealing the secret on the Bitcoin blockchain. But, like Dina, he'd rather avoid transaction fees. So instead, he sends the secret to Bob so they can update their channel balances to reflect a 50,100 satoshi Lightning payment from Bob to Chan. In <<alice_dina_htlc_redeem_2>> we see Chan sending the secret to Bob and receiving a payment in return.
+
+[[alice_dina_htlc_redeem_2]]
+.Chan settles Bob's HTLC off-chain
+image::images/mtln_0811.png["Chan settles Bob's HTLC off-chain"]
+
+Chan has paid Dina 50,000 satoshi, and received 50,100 satoshi from Bob. So Chan has 100 satoshi more in his channel balances, which he earned as a routing fee.
+
+Bob now has the secret too. He can use it to spend Alice's HTLC on-chain. Or, he can avoid transaction fees by settling the HTLC in the channel with Alice. In <<alice_dina_htlc_redeem_3>> we see that Bob sends the secret to Alice and they update the channel balance to reflect a 50,200 satoshi Lightning payment from Alice to Bob.
+
+[[alice_dina_htlc_redeem_3]]
+.Bob settles Alice's HTLC off-chain
+image::images/mtln_0812.png["Bob settles Alice's HTLC off-chain"]
+
+Bob has received 50,200 satoshi from Alice and paid 50,100 satoshi to Chan, so he has an extra 100 satoshi in his channel balances from routing fees.
+
+Alice receives the secret and has settled the 50,200  satoshi HTLC. The secret can be used as a _receipt_ to prove that Dina got paid for that specific payment hash.
+
+The final channel balances reflect Alice's payment to Dina and the routing fees paid at each hop, as shown in <<alice_dina_htlc_redeem_4>>.(((range="endofrange", startref="ix_08_routing_htlcs-asciidoc4")))
+
+[[alice_dina_htlc_redeem_4]]
+.Channel balances after the payment
+image::images/mtln_0813.png["Channel balances after the payment"]
+
+[[preventing_theft]]
+==== Signature Binding: Preventing Theft of HTLCs
+
+((("hash time-locked contracts (HTLCs)","signature binding to prevent theft of", id="ix_08_routing_htlcs-asciidoc5", range="startofrange")))((("signature binding", id="ix_08_routing_htlcs-asciidoc6", range="startofrange")))There's a catch. Did you notice it?
+
+If Alice, Bob, and Chan create the HTLCs as shown in <<alice_dina_htlc_redeem_4>>, they face a small but not insignificant risk of loss. Any of those HTLCs can be redeemed (spent) by anyone who knows the secret. At first only Dina knows the secret. Dina is supposed to only spend the HTLC from Chan. But Dina could spend all three HTLCs at the same time, or even in a single spending transaction! After all, Dina knows the secret before anyone else. Similarly, once Chan knows the secret, he is only supposed to spend the HTLC offered by Bob. But what if Chan also spends Alice's offered HTLC?
+
+This is not _trustless_! It fails the most important security feature. We need to fix this.
+
+The HTLC script must have an additional condition that binds each HTLC to a specific recipient. We do this by requiring a digital signature that matches the public key of each recipient, thereby preventing anyone else from spending that HTLC. Since only the designated recipient has the ability to produce a digital signature matching that public key, only the designated recipient can spend that HTLC.
+
+Let's look at the scripts again with this modification in mind. Alice's HTLC for Bob is modified to include Bob's public key and the +OP_CHECKSIG+ operator.
+
+Here's the modified HTLC script:
+
+----
+OP_SHA256 <H> OP_EQUALVERIFY <Bob's Pub> OP_CHECKSIG
+----
+
+[TIP]
+====
+Notice that we also changed +OP_EQUAL+ to +OP_EQUALVERIFY+. When an operator has the suffix +VERIFY+, it does not return +TRUE+ or +FALSE+ on the stack. Instead, it _halts_ execution and fails the script if the result is false and continues without any stack output if it is true.
+====
+
+To redeem this HTLC, Bob has to present an unlocking script that includes a signature from Bob's private key as well as the secret payment preimage, like this:
+
+----
+<Bob's Signature> <R>
+----
+
+The unlocking and locking scripts are combined and evaluated by the scripting engine, as follows:
+
+----
+<Bob's Sig> <R> OP_SHA256 <H> OP_EQUALVERIFY <Bob's Pub> OP_CHECKSIG
+----
+
+1. +<Bob's Sig>+ is pushed to the stack.
+2. +R+ is pushed to the stack.
+3. +OP_SHA256+ pops and hashes +R+ from the top of the stack and pushes +H~R~+ to the stack.
+4. +H+ is pushed to the stack.
+5. +OP_EQUALVERIFY+ pops +H+ and +H~R~+ and compares them. If they are not the same, execution halts. Otherwise, we continue without output to the stack.
+6. +<Bob's Pub>+ key is pushed to the stack.
+7. +OP_CHECKSIG+ pops +<Bob's Sig>+ and +<Bob's Pub>+ and verifies the signature. The result (`TRUE/FALSE`) is pushed to the stack.
+
+As you can see, this is slightly more complicated, but now we have fixed the HTLC and made sure only the intended recipient can spend it.(((range="endofrange", startref="ix_08_routing_htlcs-asciidoc6")))(((range="endofrange", startref="ix_08_routing_htlcs-asciidoc5")))
+
+==== Hash Optimization
+
+((("hash time-locked contracts (HTLCs)","hash optimization")))Let's look at the first part of the HTLC script so far:
+
+----
+OP_SHA256 <H> OP_EQUALVERIFY
+----
+
+If we look at this in the preceding symbolic representation, it looks like the +OP_+ operators take up the most space. But that's not the case. Bitcoin Script is encoded in binary, with each operator representing one byte. Meanwhile, the +<H>+ value we use as a placeholder for the payment hash is a 32-byte (256-bit) value. You can find a listing of all the Bitcoin Script operators and their binary and hex encoding in https://en.bitcoin.it/wiki/Script[Bitcoin Wiki: Script], or in https://github.com/bitcoinbook/bitcoinbook/blob/develop/appdx-scriptops.asciidoc[Appendix D, "Transaction Script Language Operators, Constants, and Symbols," in _Mastering Bitcoin_].
+
+Represented in hexadecimal, our HTLC script would look like this:
+
+----
+a8 0575965b3b44be51e8057d551c4016d83cb1fba9ea8d6e986447ba33fe69f6b3 88
+----
+
+In hexadecimal encoding, +OP_SHA256+ is +a8+ and +OP_EQUALVERIFY+ is +88+. The total length of this script is 34 bytes, of which 32 bytes are the hash.
+
+As we've mentioned previously, any participant in the Lightning Network should be able to take an off-chain transaction they hold and put it on-chain if they need to enforce their claim to funds. To take a transaction on-chain, they'd have to pay transaction fees to the miners, and these fees are proportional to the size, in bytes, of the transaction.
+
+Therefore, we want to find ways to minimize the on-chain "weight" of transactions by optimizing the script as much as possible. One way to do that is to add another hash function on top of the SHA-256 algorithm, one that produces smaller hashes. The Bitcoin Script language provides the +OP_HASH160+ operator that "double hashes" a preimage: first the preimage is hashed with SHA-256, and then the resulting hash is hashed again with the RIPEMD160 hash algorithm. The hash resulting from RIPEMD160 is 160 bits or 20 bytes--much more compact. In Bitcoin Script this is a very common optimization that is used in many of the common address formats.
+
+So, let's use that optimization instead. Our SHA-256 hash is +057596...69f6b3+. Putting that through another round of hashing with RIPEMD160 gives us the result:
+
+----
+R = "Dinas secret"
+H256 = SHA256(R)
+H256 = 0575965b3b44be51e8057d551c4016d83cb1fba9ea8d6e986447ba33fe69f6b3
+H160 = RIPEMD160(H256)
+H160 = 9e017f6767971ed7cea17f98528d5f5c0ccb2c71
+----
+
+Alice can calculate the RIPEMD160 hash of the payment hash that Dina provides and use the shorter hash in her HTLC, as can Bob and Chan!
+
+[role="pagebreak-before"]
+The "optimized" HTLC script would look like this:
+
+----
+OP_HASH160 <H160> OP_EQUALVERIFY
+----
+
+Encoded in hex, this is:
+
+----
+a9 9e017f6767971ed7cea17f98528d5f5c0ccb2c71 88
+----
+
+Where +OP_HASH160+ is +a9+ and +OP_EQUALVERIFY+ is +88+. This script is only 22 bytes long! We've saved 12 bytes from every transaction that redeems an HTLC on-chain.
+
+With that optimization, you now see how we arrive at the HTLC script shown in line 10 of <<received_htlc>>:
+
+----
+...
+    # To local node via HTLC-success transaction.
+    OP_HASH160 <RIPEMD160(payment_hash)> OP_EQUALVERIFY...
+----
+
+==== HTLC Cooperative and Timeout Failure
+
+((("cooperative failure")))((("hash time-locked contracts (HTLCs)","cooperative/timeout failure")))((("timeout failure")))So far we looked at the "hash" part of HTLC and how it would work if everyone cooperated and was online at the time of payment.
+
+What happens if someone goes offline or fails to cooperate? What happens if the payment cannot succeed?
+
+We need to ensure a way to "fail gracefully," because occasional routing failures are inevitable. There are two ways to fail: cooperatively and with a time-locked refund.
+
+Cooperative failure is relatively simple: the HTLC is unwound by every participant in the route, removing the HTLC output from their commitment transactions without changing the balance. We'll look at how that works in detail in <<channel_operation>>.
+
+Let's look at how we can reverse an HTLC without the cooperation of one or more participants. We need to make sure that if one of the participants does not cooperate, the funds are not simply locked in the HTLC _forever_. This would give someone the opportunity to ransom the funds of another participant: "I'll leave your funds tied up forever if you don't pay me ransom."
+
+To prevent this, every HTLC script includes a refund clause that is connected to a timelock. Remember our original escrow contract? "Bob has 24 hours to show the secret after the contract is signed. If Bob does not provide the secret by this time, Alice's deposit will be refunded."
+
+The time-locked refund is an important part of the script that ensures _atomicity_, so that the entire end-to-end payment either succeeds or fails gracefully. There is no "half paid" state to worry about. If there is a failure, every participant can either unwind the HTLC cooperatively with their channel partner or put the time-locked refund transaction on-chain unilaterally to get their money back.
+
+To implement this refund in Bitcoin Script, we use a special operator pass:[<code>O&#x2060;P&#x2060;_&#x2060;C&#x2060;H&#x2060;E&#x2060;C&#x2060;K&#x2060;L&#x2060;O&#x2060;C&#x2060;K&#x2060;T&#x2060;I&#x2060;M&#x2060;E&#x200b;V&#x2060;E&#x2060;R&#x2060;I&#x2060;F&#x2060;Y</code>] also known +OP_CLTV+ for short. Here's the script, as seen previously in line 13 of <<received_htlc>>:
+
+----
+...
+	OP_DROP <cltv_expiry> OP_CHECKLOCKTIMEVERIFY OP_DROP
+	OP_CHECKSIG
+...
+----
+
+The +OP_CLTV+ operator takes an expiry time defined as the block height after which this transaction is valid. If the transaction timelock is not set the same as +<cltv_expiry>+, the evaluation of the script fails and the transaction is invalid. Otherwise, the script continues without any output to the stack. Remember, the +VERIFY+ suffix means this operator does not output +TRUE+ or +FALSE+ but instead either halts/fails or continues without stack output.
+
+Essentially, the +OP_CLTV+ acts as a "gatekeeper" preventing the script from proceeding any further if the +<cltv_expiry>+ block height has not been reached on the Bitcoin blockchain.
+
+The +OP_DROP+ operator simply drops the topmost item on the script stack. This is necessary in the beginning because there is a "leftover" item from the previous script lines. It is necessary _after_ +OP_CLTV+ to remove the +<cltv_expiry>+ item from the top of the stack because it is no longer necessary.
+
+Finally, once the stack has been cleaned up, there should be a public key and signature left behind that +OP_CHECKSIG+ can verify. As we saw in <<preventing_theft>>, this is necessary to ensure that only the rightful owner of the funds can claim them, by binding this output to their public key and requiring a signature.
+
+==== Decrementing Timelocks
+
+((("hash time-locked contracts (HTLCs)","decrementing timelocks")))As the HTLCs are extended from Alice to Dina, the time-locked refund clause in each HTLC has a _different_ +cltv_expiry+ value. We will see this in more detail in <<onion_routing>>. But suffice it to say that to ensure an orderly unwinding of a payment that fails, each hop needs to wait a bit less for their refund. The difference between timelocks for each hop is called the +cltv_expiry_delta+, and is set by each node and advertised to the network, as we will see in <<gossip>>.
+
+For example, Alice sets the refund timelock on the first HTLC to a block height of current + 500 blocks ("current" being the current block height). Bob would then set the timelock +cltv_expiry+ on the HTLC to Chan to current + 450 blocks. Chan would set the timelock to current + 400 blocks from the current block height. This way, Chan can get a refund on the HTLC he offered to Dina _before_ Bob gets a refund on the HTLC he offered to Chan. Bob can get a refund of the HTLC he offered to Chan before Alice can get a refund for the HTLC she offered to Bob. The decrementing timelock prevents race conditions and ensures the HTLC chain is unwound backward, from the destination toward the origin.(((range="endofrange", startref="ix_08_routing_htlcs-asciidoc3")))(((range="endofrange", startref="ix_08_routing_htlcs-asciidoc2")))
+
+=== Conclusion
+
+In this chapter we saw how Alice can pay Dina even if she doesn't have a direct payment channel. Alice can find a path that connects her to Dina and route a payment across several payment channels so that it reaches Dina.
+
+To ensure that the payment is atomic and trustless across multiple hops, Alice must implement a fairness protocol in cooperation with all the intermediary nodes in the path. The fairness protocol is currently implemented as an HTLC, which commits funds to a payment hash derived from a secret payment preimage.
+
+Each of the participants in the payment route can extend an HTLC to the next participant, without worrying about theft or stuck funds. The HTLC can be redeemed by revealing the secret payment preimage. Once an HTLC reaches Dina, she reveals the preimage, which flows backward, resolving all the HTLCs offered.
+
+Finally, we saw how a time-locked refund clause completes the HTLC, ensuring that every participant can get a refund if the payment fails but for whatever reason one of the participants doesn't cooperate in unwinding the HTLCs. By always having the option to go on-chain for a refund, the HTLC achieves the fairness goal of atomicity and trustless operation.(((range="endofrange", startref="ix_08_routing_htlcs-asciidoc0")))

09_channel_operation.asciidoc

@@ -0,0 +1,358 @@
+[[channel_operation]]
+== Channel Operation and pass:[<span class="keep-together">Payment Forwarding</span>]
+
+((("payment channel","operation", id="ix_09_channel_operation-asciidoc0", range="startofrange")))In this chapter we will bring together payment channels and hash time-locked contracts (HTLCs). In <<payment_channels>>, we explained the way Alice and Bob construct a payment channel between their two nodes. We also looked at the commitment and penalty mechanisms that secure the payment channel. In <<routing>>, we looked at HTLCs and how these can be used to route a payment across a path made of multiple payment channels. In this chapter we bring the two concepts together by looking at how HTLCs are managed on each payment channel, how the HTLCs are committed to the channel state, and how they are settled to update the channel balances.
+
+Specifically, we will be discussing "Adding, settling, failing HTLCs" and the "Channel state machine" that form the overlap between the peer-to-peer layer and the routing layer, as highlighted by an outline in <<LN_protocol_channelops_highlight>>. 
+
+[[LN_protocol_channelops_highlight]]
+.Channel operation and payment forwarding in the Lightning protocol suite
+image::images/mtln_0901.png["Channel operation and payment forwarding in the Lightning protocol suite"]
+
+
+=== Local (Single Channel) Versus Routed (Multiple Channels)
+
+((("payment channel","local channel versus routed channels")))Even though it is possible to send payments across a payment channel simply by updating the channel balances and creating new commitment transactions, the Lightning protocol uses HTLCs even for "local" payments across a payment channel. The reason for this is to maintain the same protocol design regardless of whether a payment is only one hop (across a single payment channel) or several hops (routed across multiple payment channels).
+
+By maintaining the same abstraction for both local and remote, we not only simplify the protocol design but also improve privacy. For the recipient of a payment there is no discernible difference between a payment made directly by their channel partner and a payment forwarded by their channel partner on behalf of someone else.
+
+=== Forwarding Payments and Updating Commitments pass:[<span class="keep-together">with HTLCs</span>]
+
+((("commitment transactions","updating commitments with HTLCs", id="ix_09_channel_operation-asciidoc1", range="startofrange")))((("hash time-locked contracts (HTLCs)","updating commitments with", id="ix_09_channel_operation-asciidoc2", range="startofrange")))((("payment channel","updating commitments with HTLCs", id="ix_09_channel_operation-asciidoc3", range="startofrange")))We will revisit our example from <<routing>> to demonstrate how the HTLCs from Alice to Dina get committed to each payment channel. As you recall in our example, Alice is paying Dina 50,000 satoshis by routing an HTLC via Bob and Chan. The network is shown in <<alice_dina_htlc_2>>. 
+
+[[alice_dina_htlc_2]]
+.Alice pays Dina with an HTLC routed via Bob and Chan
+image::images/mtln_0809.png["Alice pays Dina with an HTLC routed via Bob and Chan"]
+
+We will focus on the payment channel between Alice and Bob and review the messages and transactions that they use to process this HTLC.
+
+==== HTLC and Commitment Message Flow
+
+((("hash time-locked contracts (HTLCs)","commitment message flow")))The message flow between Alice and Bob (and also between any pair of channel partners) is shown in <<HTLC_commitment_message_flow>>. 
+
+[[HTLC_commitment_message_flow]]
+.The message flow for HTLC commitment between channel partners
+image::images/mtln_0903.png["The message flow for HTLC commitment between channel partners"]
+
+[role="pagebreak-before"]
+We've already seen the +commitment_signed+ and +revoke_and_ack+ in <<payment_channels>>. Now we will see how HTLCs fit into the commitment scheme. The two new messages are +update_add_htlc+, which Alice uses to ask Bob to add an HTLC, and +update_fulfill_htlc+, which Bob uses to redeem the HTLC once he has received the payment secret (Dina's secret).(((range="endofrange", startref="ix_09_channel_operation-asciidoc3")))(((range="endofrange", startref="ix_09_channel_operation-asciidoc2")))(((range="endofrange", startref="ix_09_channel_operation-asciidoc1")))
+
+=== Forwarding Payments with HTLCs
+
+((("hash time-locked contracts (HTLCs)","forwarding payments with", id="ix_09_channel_operation-asciidoc4", range="startofrange")))((("payment forwarding","with HTLCs", id="ix_09_channel_operation-asciidoc5", range="startofrange")))Alice and Bob start with a payment channel that has a 70,000 satoshi balance on each side.
+
+As we saw in <<payment_channels>>, this means that Alice and Bob have negotiated and each hold commitment transactions. These commitment transactions are asymmetric, delayed, and revocable, and look like the example in <<alice_bob_commitment_txs_1>>. 
+
+[[alice_bob_commitment_txs_1]]
+.Alice and Bob's initial commitment transactions
+image::images/mtln_0904.png["Alice and Bob's initial commitment transactions"]
+
+==== Adding an HTLC
+
+((("hash time-locked contracts (HTLCs)","adding an HTLC")))Alice wants Bob to accept an HTLC worth 50,200 satoshis to forward to Dina. To do so, Alice must send the details of this HTLC, including the payment hash and amount, to Bob. Bob will also need to know where to forward it, which is something we discuss in detail in <<onion_routing>>.
+
+To add the HTLC, Alice starts the flow we saw in <<HTLC_commitment_message_flow>> by sending the +update_add_htlc+ message to Bob.
+
+[[update_add_htlc]]
+==== The update_add_HTLC Message
+
+((("hash time-locked contracts (HTLCs)","update_add_HTLC message")))((("update_add_HTLC message")))Alice sends the `update_add_HTLC` Lightning message to Bob. This message is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#adding-an-htlc-update_add_htlc[BOLT #2: Peer Protocol, `update_add_HTLC`], and is shown in Example 9-1. 
+
+[[update_add_HTLC_message_fields]]
+.The `update_add_HTLC` message
+====
+----
+[channel_id:channel_id]
+[u64:id]
+[u64:amount_msat]
+[sha256:payment_hash]
+[u32:cltv_expiry]
+[1366*byte:onion_routing_packet]
+----
+====
+
++channel_id+:: This is the channel that Alice has with Bob where she wants to add the HTLC. Remember that Alice and Bob may have multiple channels with each other.
+
++id+:: This is an HTLC counter and starts at +0+ for the first HTLC offered to Bob by Alice and is incremented for each subsequent offered HTLC. 
+
++amount_msat+:: This is the amount (value) of the HTLC in millisatoshis. In our example this is 50,200,000 millisatoshis (i.e., 50,200 satoshis).
+
++payment_hash+:: This is the payment hash calculated from Dina's invoice. It is _H_ = RIPEMD160(SHA-256(_R_)), where _R_ is Dina's secret that is known only by Dina and will be revealed if Dina is paid.
+
++cltv_expiry+:: This is the expiry time for this HTLC, which will be encoded as a time-locked refund in case the HTLC fails to reach Dina in this time.
+
++onion_routing_packet+:: This is an onion-encrypted route that tells Bob where to forward this HTLC next (to Chan). Onion routing is covered in detail in <<onion_routing>>.
+
+[TIP]
+====
+As a reminder, accounting within the Lightning Network is in units of millisatoshis (thousandths of a satoshi), whereas Bitcoin accounting is in satoshis. Any amounts in HTLCs are millisatoshis, which are then rounded to the nearest satoshi in the Bitcoin commitment transactions.
+====
+
+==== HTLC in Commitment Transactions
+
+((("commitment transactions","HTLC in")))((("hash time-locked contracts (HTLCs)","commitment transactions and")))The received information is enough for Bob to create a new commitment transaction. The new commitment transaction has the same two outputs +to_self+ and +to_remote+ for Alice and Bob's balance, and a _new_ output representing the HTLC offered by Alice.
+
+We've already seen the basic structure of an HTLC in <<routing>>. The complete script of an offered HTLC is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md#offered-htlc-outputs[BOLT #3: Transactions, Offered HTLC Output] and is shown in <<offered_htlc_output_script>>. 
+
+
+[[offered_htlc_output_script]]
+.Offered HTLC output script
+====
+[source,text,linenums]
+----
+# Revocation <1>
+OP_DUP OP_HASH160 <RIPEMD160(SHA256(revocationpubkey))> OP_EQUAL
+OP_IF
+    OP_CHECKSIG
+OP_ELSE
+    <remote_HTLCpubkey> OP_SWAP OP_SIZE 32 OP_EQUAL
+    OP_IF
+        # Redemption <2>
+        OP_HASH160 <RIPEMD160(payment_hash)> OP_EQUALVERIFY
+        2 OP_SWAP <local_HTLCpubkey> 2 OP_CHECKMULTISIG
+    OP_ELSE
+        # Refund <3>
+        OP_DROP <cltv_expiry> OP_CHECKLOCKTIMEVERIFY OP_DROP
+        OP_CHECKSIG
+    OP_ENDIF
+OP_ENDIF
+----
+<1> The first clause of the `OP_IF` conditional is redeemable by Alice with a revocation key. If this commitment is later revoked, Alice will have a revocation key to claim this output in a penalty transaction, taking the whole channel balance.
+<2>  The second clause is redeemable by the preimage (payment secret, or in our example, Dina's secret) if it is revealed. This allows Bob to claim this output if he has the secret from Dina, meaning he has successfully delivered the payment to Dina.
+<3> The third and final clause is a refund of the HTLC to Alice if the HTLC expires without reaching Dina. It is time-locked with the expiration +cltv_expiry+. This ensures that Alice's balance is not "stuck" in an HTLC that can't be routed to Dina.
+====
+
+There are three ways to claim this output. Try to read the script and see if you can figure it out (remember, it is a stack-based language, so things appear "backward").
+
+==== New Commitment with HTLC Output
+
+((("commitment transactions","new commitment with HTLC output", id="ix_09_channel_operation-asciidoc6", range="startofrange")))((("hash time-locked contracts (HTLCs)","new commitment with HTLC output", id="ix_09_channel_operation-asciidoc7", range="startofrange")))Bob now has the necessary information to add this HTLC script as an additional output and create a new commitment transaction. Bob's new commitment will have 50,200 satoshis in the HTLC output. That amount will come from Alice's channel balance, so Alice's new balance will be 19,800 satoshis (70,000 – 50,200 = 19,800). Bob constructs this commitment as a tentative "Commitment #3," shown in <<add_commitment_3b>>. 
+
+[[add_commitment_3b]]
+.Bob's new commitment with an HTLC output
+image::images/mtln_0905.png["Bob's new commitment with an HTLC output"]
+
+[role="pagebreak-before less_space"]
+==== Alice Commits
+
+Shortly after sending the +update_add_htlc+ message, she will commit to the new state of the channel, so that the HTLC can be safely added by Bob. Bob has the HTLC information and has constructed a new commitment but does not yet have this new commitment signed by Alice.
+
+Alice sends +commitment_signed+ to Bob, with the signature for the new commitment and for the HTLC within. We saw the +commitment_signed+ message in <<payment_channels>>, but now we can understand the rest of the fields. As a reminder, it is shown in <<ops_commitment_signed_message>>. 
+
+[[ops_commitment_signed_message]]
+.The `commitment_signed` message
+====
+----
+[channel_id:channel_id]
+[signature:signature]
+[u16:num_htlcs]
+[num_htlcs*signature:htlc_signature]
+----
+====
+
+The fields +num_htlcs+ and +htlc_signature+ now make more sense:
+
++num_htlcs+:: This is the number of HTLCs that are outstanding in the commitment transaction. In our example, just one HTLC, the one Alice offered.
+
++htlc_signature+:: This is an array of signatures (+num_htlcs+ in length), containing signatures for the HTLC outputs.
+
+Alice can send these signatures without hesitation: she can always get a refund if the HTLC expires without being routed to Dina.
+
+Now, Bob has a new signed commitment transaction, as shown in <<signed_commitment_3b>>. 
+
+[[signed_commitment_3b]]
+.Bob has a new signed commitment
+image::images/mtln_0906.png[Bob has a new signed commitment]
+
+==== Bob Acknowledges New Commitment and Revokes Old One
+
+((("hash time-locked contracts (HTLCs)","acknowledging new commitment/revoking old commitment")))Now that Bob has a new signed commitment, he needs to acknowledge it and revoke the old commitment. ((("revoke_and_ack message", id="ix_09_channel_operation-asciidoc8", range="startofrange")))He does so by sending the +revoke_and_ack+ message, as we saw in <<payment_channels>> previously. As a reminder, that message is shown in <<revoke_and_ack_message_2>>. 
+
+[[revoke_and_ack_message_2]]
+.The +revoke_and_ack+ message
+====
+----
+[channel_id:channel_id]
+[32*byte:per_commitment_secret]
+[point:next_per_commitment_point]
+----
+====
+
+Bob sends the +per_commitment_secret+ that allows Alice to construct a revocation key to build a penalty transaction spending Bob's old commitment. Once Bob has sent this, he can never publish "Commitment #2" without risking a penalty transaction and losing all his money. So, the old commitment is effectively revoked.
+
+Bob has effectively moved the channel state forward, as shown in <<revoked_commitment_2b>>. 
+
+[[revoked_commitment_2b]]
+.Bob has revoked the old commitment
+image::images/mtln_0907.png[Bob has revoked the old commitment]
+
+Despite the fact that Bob has a new (signed) commitment transaction and an HTLC output inside, he cannot consider his HTLC as being set up successfully.
+
+He first needs to have Alice revoke her old commitment, because otherwise, Alice can roll back her balance to 70,000 satoshis. Bob needs to make sure that Alice also has a commitment transaction containing the HTLC and has revoked the old commitment.
+
+That is why, if Bob is not the final recipient of the HTLC funds, he should not forward the HTLC yet by offering an HTLC on the next channel with Chan.
+
+Alice has constructed a mirror-image new commitment transaction containing the new HTLC, but it is yet to be signed by Bob. We can see it in <<add_commitment_3a>>. 
+
+[[add_commitment_3a]]
+.Alice's new commitment with an HTLC output
+image::images/mtln_0908.png["Alice's new commitment with an HTLC output"]
+
+As we described in <<payment_channels>>, Alice's commitment is the mirror image of Bob's, as it contains the asymmetric, delayed, revocable construct for revocation and penalty enforcement of old commitments. Alice's 19,800 satoshi balance (after deducting the HTLC value), is delayed and revocable. Bob's 70,000 satoshi balance is immediately redeemable.
+
+Next, the message flow for +commitment_signed+ and +revoke_and_ack+ is now repeated, but in the opposite direction. Bob sends +commitment_signed+ to sign Alice's  new commitment, and Alice responds by revoking her old commitment.(((range="endofrange", startref="ix_09_channel_operation-asciidoc8")))
+
+For completeness sake, let's quickly review the commitment transactions as this round of commitment/revocation happens.
+
+[role="pagebreak-before less_space"]
+==== Bob Commits
+
+Bob now sends a +commitment_signed+ back to Alice, with his signatures for Alice's new commitment transaction, including the HTLC output she has added.
+
+Now Alice has the signature for the new commitment transaction. The state of the channel is shown in <<signed_commitment_3a>>. 
+
+[[signed_commitment_3a]]
+.Alice has a new signed commitment
+image::images/mtln_0909.png[Alice has a new signed commitment]
+
+Alice can now acknowledge the new commitment by revoking the old one. Alice sends the +revoke_and_ack+ message containing the necessary +per_commitment_point+ that will allow Bob to construct a revocation key and penalty transaction. Thus, Alice revokes her old commitment.
+
+The channel state is shown in <<revoked_commitment_2a>>.(((range="endofrange", startref="ix_09_channel_operation-asciidoc7")))(((range="endofrange", startref="ix_09_channel_operation-asciidoc6"))) (((range="endofrange", startref="ix_09_channel_operation-asciidoc5")))(((range="endofrange", startref="ix_09_channel_operation-asciidoc4")))
+
+[[revoked_commitment_2a]]
+.Alice has revoked the old commitment
+image::images/mtln_0910.png[Alice has revoked the old commitment]
+
+=== Multiple HTLCs
+
+((("hash time-locked contracts (HTLCs)","multiple contracts")))At any point in time, Alice and Bob may have dozens or even hundreds of HTLCs across a single channel. Each HTLC is offered and added to the commitment transaction as an additional output. A commitment transaction therefore always has two outputs for the channel partner balances and any number of HTLC outputs, one per HTLC.
+
+As we saw in the +commitment_signed+ message, there is an array for HTLC signatures so that multiple HTLC commitments can be transmitted at the same time.
+
+The current maximum number of HTLCs allowed on a channel is 483 HTLCs to account for the maximum Bitcoin transaction size and ensure that the commitment transactions continue to be valid Bitcoin transactions.
+
+As we will see in the next section, the maximum is only for _pending_ HTLCs because, once an HTLC is fulfilled (or fails due to timeout/error), it is removed from the commitment transaction.
+
+=== HTLC Fulfillment
+
+((("hash time-locked contracts (HTLCs)","fulfillment", id="ix_09_channel_operation-asciidoc9", range="startofrange")))((("payment forwarding","HTLC fulfillment", id="ix_09_channel_operation-asciidoc10", range="startofrange")))Now Bob and Alice both have a new commitment transaction with an additional HTLC output, and we have achieved a major step toward updating a payment pass:[<span class="keep-together">channel</span>].
+
+The new balance of Alice and Bob does not reflect yet that Alice has successfully sent 50,200 satoshis to Bob.
+
+However, the HTLCs are now set up in a way that secure settlement in exchange for the proof of payment will be possible.
+
+==== HTLC Propagation
+
+((("hash time-locked contracts (HTLCs)","propagation", id="ix_09_channel_operation-asciidoc11", range="startofrange")))((("payment forwarding","HTLC propagation", id="ix_09_channel_operation-asciidoc12", range="startofrange")))Let's assume that Bob continues the chain and sets up an HTLC with Chan for 50,100 satoshis. The process will be exactly the same as we just saw between Alice and Bob. Bob will send +update_add_htlc+ to Chan, then they will exchange +commitment_signed+ and +revoke_and_ack+ messages in two rounds, progressing their channel to the next state.
+
+Next, Chan will do the same with Dina: offer a 50,000 satoshi HTLC, commit, and revoke, etc. However, Dina is the final recipient of the HTLC. Dina is the only one that knows the payment secret (the preimage of the payment hash). Therefore, Dina can fulfill the HTLC with Chan immediately!
+
+==== Dina Fulfills the HTLC with Chan
+
+Dina can settle the HTLC by sending an +update_ful&#x2060;fill_&#x200b;htlc+ message to Chan. The +update_fulfill_htlc+ message is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#removing-an-htlc-update_fulfill_htlc-update_fail_htlc-and-update_fail_malformed_htlc[BOLT #2: Peer Protocol, `update_fulfill_htlc`] and is shown here:
+
+[[update_fulfill_htlc_message]]
+.The +update_fulfill_htlc+ message
+----
+[channel_id:channel_id]
+[u64:id]
+[32*byte:payment_preimage]
+----
+
+It's a really simple message:
+
++channel_id+:: The channel ID on which the HTLC is committed.
+
++id+:: The ID of the HTLC (we started with 0 and incremented for each HTLC on the channel).
+
++payment_preimage+:: The secret that proves payment was made and redeems the HTLC. This is the +R+ value that was hashed by Dina to produce the payment hash in the invoice to Alice.
+
+When Chan receives this message, he will immediately check if the `payment_preimage` (let's call it _R_) produces the payment hash (let's call it _H_) in the HTLC that he offered to Dina. He hashes it like this:
+
+++++
+<ul class="simplelist">
+<li><em>H</em> = RIPEMD160(SHA-256 (<em>R</em>))</li>
+</ul>
+++++
+
+If the result _H_ matches the payment hash in the HTLC, Chan can do a little dance of celebration. This long-awaited secret can be used to redeem the HTLC, and will be passed back along the chain of payment channels all the way to Alice, resolving every HTLC that was part of this payment to Dina.
+
+Let's go back to Alice and Bob's channel and watch them unwind the HTLC. To get there, let's assume Dina sent the +update_fulfill_htlc+ to Chan, Chan sent +update_fulfill_htlc+ to Bob, and Bob sent +update_fulfill_htlc+ to Alice. The payment preimage has propagated all the way back to Alice.
+
+==== Bob Settles the HTLC with Alice
+
+When Bob sends the +update_fulfill_htlc+ to Alice, it will contain the same +payment_preimage+ that Dina selected for her invoice. That +payment_preimage+ has traveled backward along the payment path. At each step, the +channel_id+ will be different and +id+ (HTLC ID) may be different. But the preimage is the same!
+
+Alice will also validate the +payment_preimage+ received from Bob. She will compare its hash to the HTLC payment hash in the HTLC she offered Bob. She will also find this preimage matches the hash in Dina's invoice. This is proof that Dina was paid.
+
+The message flow between Alice and Bob is shown in <<htlc_fulfillment_message_flow>>. 
+
+[[htlc_fulfillment_message_flow]]
+.The HTLC fulfillment message flow
+image::images/mtln_0911.png[The HTLC fulfillment message flow]
+
+Both Alice and Bob can now remove the HTLC from the commitment transactions and update their channel balances.
+
+They create new commitments (Commitment #4), as shown in <<htlc_fulfillment_commitments_added>>. 
+
+[[htlc_fulfillment_commitments_added]]
+.The HTLC is removed and balances are updated in new commitments
+image::images/mtln_0912.png[The HTLC is removed and balances are updated in new commitments]
+
+[role="pagebreak-before"]
+Next, they complete two rounds of commitment and revocation. First, Alice sends +commitment_signed+ to sign Bob's new commitment transaction. Bob responds with +revoke_and_ack+ to revoke his old commitment. Once Bob has moved the state of the channel forward, the commitments look like we see in <<htlc_fulfillment_commitments_bob_commit>>. 
+
+[[htlc_fulfillment_commitments_bob_commit]]
+.Alice signs Bob's new commitment and Bob revoked the old one
+image::images/mtln_0913.png[Alice signs Bob's new commitment and Bob revoked the old one]
+
+[role="pagebreak-before"]
+Finally, Bob signs Alice's commitment by sending Alice a +commitment_signed+ message. Then Alice acknowledges and revokes her old commitment by sending +revoke_and_ack+ to Bob. The end result is that both Alice and Bob have moved their channel state to Commitment #4, have removed the HTLC, and have updated their balances. Their current channel state is represented by the commitment transactions that are shown in <<alice_bob_htlc_fulfilled>>(((range="endofrange", startref="ix_09_channel_operation-asciidoc12")))(((range="endofrange", startref="ix_09_channel_operation-asciidoc11"))). (((range="endofrange", startref="ix_09_channel_operation-asciidoc10")))(((range="endofrange", startref="ix_09_channel_operation-asciidoc9")))
+
+[[alice_bob_htlc_fulfilled]]
+.Alice and Bob settle the HTLC and update balances
+image::images/mtln_0914.png[Alice and Bob settle the HTLC and update balances]
+
+[role="pagebreak-before less_space"]
+=== Removing an HTLC Due to Error or Expiry
+
+((("hash time-locked contracts (HTLCs)","removing due to error/expiry")))((("payment forwarding","removing an HTLC due to error/expiry")))If an HTLC cannot be fulfilled, it can be removed from the channel commitment using the same process of commitment and revocation.
+
+Instead of +update_fulfill_htlc+, Bob would send an +update_fail_htlc+ or +update_fail_malformed_htlc+. These two messages are defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#removing-an-htlc-update_fulfill_htlc-update_fail_htlc-and-update_fail_malformed_htlc[BOLT #2: Peer Protocol, Removing an HTLC].
+
+The +update_fail_htlc+ message is shown in the following:
+
+[[update_fail_htlc_message]]
+.The +update_fail_htlc+ message
+----
+[channel_id:channel_id]
+[u64:id]
+[u16:len]
+[len*byte:reason]
+----
+
+It's pretty self-explanatory. The multibyte +reason+ field is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#failure-messages[BOLT #4: Onion Routing], which we will describe in <<onion_routing>>.
+
+If Alice received an +update_fail_htlc+ from Bob, the process would unfold in much the same way: the two channel partners would remove the HTLC, create updated commitment transactions, and go through two rounds of commitment/revocation to move the channel state forward to the next commitment. The only difference: the end balances would revert back to what they were without the HTLC, essentially refunding Alice for the HTLC value.
+
+=== Making a Local Payment
+
+((("hash time-locked contracts (HTLCs)","local payment with")))((("local payments")))((("payment forwarding","local payments")))At this point, you will easily understand why HTLCs are used for both remote and local payments. When Alice pays Bob for a coffee, she doesn't just update the channel balance and commit to a new state. Instead, the payment is made with an HTLC, in the same way Alice paid Dina. The fact that there's only one channel hop makes no difference. It would work like this:
+[start=1]
+. Alice orders a coffee from Bob's shop page.
+. Bob's shop sends an invoice with a payment hash.
+. Alice constructs an HTLC from that payment hash.
+. Alice offers the HTLC to Bob with +update_add_htlc+.
+. Alice and Bob exchange commitments and revocations adding the HTLC to their commitment transactions.
+. Bob sends +update_fulfill_htlc+ to Alice with the payment preimage.
+. Alice and Bob exchange commitments and revocations removing the HTLC and updating the channel balances.
+
+Whether an HTLC is forwarded across many channels or just fulfilled in a single channel "hop," the process is exactly the same
+
+=== Conclusion
+
+In this chapter we saw how commitment transactions (from <<payment_channels>>) and HTLCs (from <<routing>>) work together. We saw how an HTLC is added to a commitment transaction, and how it is fulfilled. We saw how the asymmetric, delayed, revocable system for enforcing channel state is extended to HTLCs.
+
+We also saw how a local payment and a multihop routed payment are handled identically: using HTLCs.(((range="endofrange", startref="ix_09_channel_operation-asciidoc0")))
+
+In the next chapter we will look at the encrypted message routing system called _onion routing_.

10_onion_routing.asciidoc

@@ -0,0 +1,937 @@
+[[onion_routing]]
+== Onion Routing
+
+((("onion routing", id="ix_10_onion_routing-asciidoc0", range="startofrange")))In this chapter we will describe the Lightning Network's onion routing mechanism. The invention of _onion routing_ precedes the Lightning Network by 25 years! Onion routing was invented by U.S. Navy researchers as a communications security protocol. Onion routing is most famously used by Tor, the onion-routed internet overlay that allows researchers, activists, intelligence agents, and everyone else to use the internet privately and anonymously.
+
+In this chapter we are focusing on the "Source-based onion routing (SPHINX)" part of the Lightning protocol architecture, highlighted by an outline in the center (routing layer) of <<LN_protocol_onion_highlight>>. 
+
+[[LN_protocol_onion_highlight]]
+.Onion routing in the Lightning protocol suite
+image::images/mtln_1001.png["Onion routing in the Lightning protocol suite"]
+
+Onion routing describes a method of encrypted communication where a message sender builds successive _nested layers of encryption_ that are "peeled" off by each intermediary node, until the innermost layer is delivered to the intended recipient. The name "onion routing" describes this use of layered encryption that is peeled off one layer at a time, like the skin of an onion.
+
+Each of the intermediary nodes can only "peel" one layer and see who is next in the communication path. Onion routing ensures that no one except the sender knows the destination or length of the communication path. Each intermediary only knows the previous and next hop.
+
+The Lightning Network uses an implementation of onion routing protocol based on pass:[<em>Sphinx</em>],footnote:[George Danezis and Ian Goldberg, "Sphinx: A Compact and Provably Secure Mix Format," in _IEEE Symposium on Security and Privacy_ (New York: IEEE, 2009), 269–282.] developed in 2009 by George Danezis and Ian Goldberg.
+
+The implementation of onion routing in the Lightning Network is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md[BOLT #4: Onion Routing Protocol].
+
+=== A Physical Example Illustrating Onion Routing
+
+((("onion routing","physical example", id="ix_10_onion_routing-asciidoc1", range="startofrange")))There are many ways to describe onion routing, but one of the easiest is to use the physical equivalent of sealed envelopes. An envelope represents a layer of encryption, allowing only the named recipient to open it and read the contents.
+
+Let's say Alice wants to send a secret letter to Dina, indirectly via some intermediaries.
+
+==== Selecting a Path
+
+((("onion routing","selecting a path")))The Lightning Network uses _source routing_, which means that the payment path is selected and specified by the sender, and only the sender. In this example, Alice's secret letter to Dina will be the equivalent of a payment. To make sure the letter reaches Dina, Alice will create a path from her to Dina, using Bob and Chan as intermediaries.
+
+[TIP]
+====
+There may be many paths that make it possible for Alice to reach Dina. We will explain the process of selecting the _optimum_ path in <<path_finding>>. For now, we'll assume that the path selected by Alice uses Bob and Chan as intermediaries to get to Dina.
+====
+
+[role="pagebreak-before"]
+As a reminder, the path selected by Alice is shown in <<alice_dina_path>>. 
+
+[[alice_dina_path]]
+.Path: Alice to Bob to Chan to Dina
+image::images/mtln_1002.png["Alice to Bob to Chan to Dina"]
+
+Let's see how Alice can use this path without revealing information to intermediaries Bob and Chan.
+
+.Source-Based Routing
+****
+((("source-based routing")))Source-based routing is not how packets are typically routed on the internet today, though source routing was possible in the early days.
+Internet routing is based on _packet switching_ at each intermediary routing node. An IPv4 packet, for example, includes the sender and recipient's IP addresses, and every other IP routing node decides how to forward each packet toward the destination.
+However, the lack of privacy in such a routing mechanism, where every intermediary node sees the sender and recipient, makes this a poor choice for use in a payment network.
+****
+
+==== Building the Layers
+
+((("onion routing","building the layers", id="ix_10_onion_routing-asciidoc2", range="startofrange")))Alice starts by writing a secret letter to Dina.  She then seals the letter inside an envelope and writes "To Dina" on the outside (see <<dina_envelope>>). The envelope represents encryption with Dina's public key so that only Dina can open the envelope and read the letter.
+
+[[dina_envelope]]
+.Dina's secret letter, sealed in an envelope
+image::images/mtln_1003.png["Dina's secret letter, sealed in an envelope"]
+
+Dina's letter will be delivered to Dina by Chan, who is immediately before Dina in the "path." So, Alice puts Dina's envelope inside an envelope addressed to Chan (see <<chan_envelope>>). The only part that Chan can read is the destination (routing instructions): "To Dina." Sealing this inside an envelope addressed to Chan represents encrypting it with Chan's public key so that only Chan can read the envelope address. Chan still can't open Dina's envelope. All he sees is the instructions on the outside (the address).
+
+[[chan_envelope]]
+.Chan's envelope, containing Dina's sealed envelope
+image::images/mtln_1004.png["Chan's envelope, containing Dina's sealed envelope"]
+
+Now, this letter will be delivered to Chan by Bob. So Alice puts it inside an envelope addressed to Bob (see <<bob_envelope>>). As before, the envelope represents a message encrypted to Bob that only Bob can read. Bob can only read the outside of Chan's envelope (the address), so he knows to send it to Chan.
+
+[[bob_envelope]]
+.Bob's envelope, containing Chan's sealed envelope
+image::images/mtln_1005.png["Bob's envelope, containing Chan's sealed envelope"]
+
+Now, if we could look through the envelopes (with X-rays!) we would see the envelopes nested one inside the other, as shown in <<nested_envelopes>>. (((range="endofrange", startref="ix_10_onion_routing-asciidoc2")))
+
+[[nested_envelopes]]
+.Nested envelopes
+image::images/mtln_1006.png[Nested envelopes]
+
+==== Peeling the Layers
+
+((("onion routing","peeling the layers")))Alice now has an envelope that says "To Bob" on the outside. It represents an encrypted message that only Bob can open (decrypt). Alice will now begin the process by sending this to Bob. The entire process is shown in <<sending_nested_envelopes>>. 
+
+[[sending_nested_envelopes]]
+.Sending the envelopes
+image::images/mtln_1007.png[Sending the envelopes]
+
+As you can see, Bob receives the envelope from Alice. He knows it came from Alice, but doesn't know if Alice is the original sender or just someone forwarding envelopes. He opens it to find an envelope inside that says "To Chan." Since this is addressed to Chan, Bob can't open it. He doesn't know what's inside it and doesn't know if Chan is getting a letter or another envelope to forward. Bob doesn't know if Chan is the ultimate recipient or not. Bob forwards the envelope to Chan.
+
+Chan receives the envelope from Bob. He doesn't know that it came from Alice. He doesn't know if Bob is an intermediary or the sender of a letter. Chan opens the envelope and finds another envelope inside addressed "To Dina," which he can't open. Chan forwards it to Dina, not knowing if Dina is the final recipient.
+
+Dina receives an envelope from Chan. Opening it she finds a letter inside, so now she knows she's the intended recipient of this message. She reads the letter, knowing that none of the intermediaries know where it came from and no one else has read her secret letter!
+
+This is the essence of onion routing. The sender wraps a message in layers, specifying exactly how it will be routed and preventing any of the intermediaries from gaining any information about the path or payload. Each intermediary peels one layer, sees only a forwarding address, and doesn't know anything other than the previous and next hop in the path.
+
+Now, let's look at the details of the onion routing implementation in the Lightning Network.(((range="endofrange", startref="ix_10_onion_routing-asciidoc1")))
+
+=== Introduction to Onion Routing of HTLCs
+
+((("hash time-locked contracts (HTLCs)","onion routing basics", id="ix_10_onion_routing-asciidoc3", range="startofrange")))((("onion routing","HTLCs", id="ix_10_onion_routing-asciidoc4", range="startofrange")))Onion routing in the Lightning Network appears complex at first glance, but once you understand the basic concept, it is really quite simple.
+
+From a practical perspective, Alice is telling every intermediary node what HTLC to set up with the next node in the path.
+
+((("origin node")))The first node, which is the payment sender or Alice in our example, is called the _origin node_.  ((("final node")))The last node, which is the payment recipient or Dina in our example, is called the _final node_.
+
+((("hop")))Each intermediary node, or Bob and Chan in our example, is called a _hop_. Every hop must set up an _outgoing HTLC_ to the next hop. The information communicated to each hop by Alice is called the _hop payload_ or _hop data_. The message that is routed from Alice to Dina is called an _onion_ and consists of encrypted _hop payload_ or _hop data_ messages encrypted to each hop.
+
+Now that we know the terminology used in Lightning onion routing, let's restate Alice's task: Alice must construct an onion with hop data, telling each hop how to construct an outgoing HTLC to send a payment to the final node (Dina).
+
+==== Alice Selects the Path
+
+((("onion routing","selecting a path")))From <<routing>> we know that Alice will send a 50,000 satoshi payment to Dina via Bob and Chan. This payment is transmitted via a series of HTLCs, as shown in <<alice_dina_htlc_path>>. 
+
+[[alice_dina_htlc_path]]
+.Payment path with HTLCs from Alice to Dina
+image::images/mtln_1008.png[Payment path with HTLCs from Alice to Dina]
+
+As we will see in <<gossip>>, Alice is able to construct this path to Dina because Lightning nodes announce their channels to the entire Lightning Network using the Lightning Gossip Protocol. After the initial channel announcement, Bob and Chan each sent out an additional `channel_update` message with their routing fee and timelock expectations for payment routing.
+
+From the announcements and updates, Alice knows the following information about the channels between Bob, Chan, and Dina:
+
+* A +short_channel_id+ (short channel ID) for each channel, which Alice can use to reference the channel when constructing the path
+
+* A +cltv_expiry_delta+ (timelock delta), which Alice can add to the expiry time for each HTLC
+
+* A +fee_base_msat+ and +fee_proportional_millionths+, which Alice can use to calculate the total routing fee expected by that node for relay on that channel.
+
+In practice, other information is also exchanged, such as the largest (`htlc_maximum_msat`) and smallest (`htlc_minimum_msat`) HTLCs a channel will carry, but these aren't used as directly during onion route construction as the preceding fields are.
+
+This information is used by Alice to identify the nodes, channels, fees, and timelocks for the following detailed path, shown in <<alice_dina_path_detail>>. 
+
+[[alice_dina_path_detail]]
+.A detailed path constructed from gossiped channel and node information
+image::images/mtln_1009.png[A path constructed from gossiped channel and node information]
+
+Alice already knows her own channel to Bob and therefore doesn't need this info to construct the path. Note also that Alice didn't need a channel update from Dina because she has the update from Chan for that last channel in the path.
+
+==== Alice Constructs the Payloads
+
+((("onion routing","payload construction", id="ix_10_onion_routing-asciidoc5", range="startofrange")))There are two possible formats that Alice can use for the information communicated to each hop: ((("hop data")))a legacy fixed-length format called the _hop data_ and a more flexible Type-Length-Value (TLV) based format called the _hop payload_. The TLV message format is explained in more detail in <<tlv>>. It offers flexibility by allowing fields to be added to the protocol at will.
+
+[NOTE]
+====
+Both formats are specified in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#packet-structure[BOLT #4: Onion Routing Protocol, Packet Structure].
+====
+
+Alice will start building the hop data from the end of the path backwards: Dina, Chan, then Bob.
+
+===== Final node payload for Dina
+
+((("final node")))Alice first builds the payload that will be delivered to Dina. Dina will not be constructing an "outgoing HTLC," because Dina is the final node and payment recipient. For this reason, the payload for Dina is different than all the others (uses all zeros for the `short_channel_id`), but only Dina will know this because it will be encrypted in the innermost layer of the onion. Essentially, this is the "secret letter to Dina" we saw in our physical envelope example.
+
+The hop payload for Dina must match the information in the invoice generated by Dina for Alice and will contain (at least) the following fields in TLV format:
+
++amt_to_forward+:: The amount of this payment in millisatoshis. If this is only one part of a multipart payment, the amount is less than the total. Otherwise, this is a single, full payment and it is equal to the invoice amount and +total_msat+ value.
+
++outgoing_cltv_value+:: The payment expiry timelock set to the value +min_final_cltv_expiry+ in the invoice.
+
++payment_secret+:: A special 256-bit secret value from the invoice, allowing Dina to recognize this incoming payment. This also prevents a class of probing that previously made zero-value invoices insecure. Probing by intermediate nodes is mitigated as this value is encrypted to _only_ the recipient, meaning they can't reconstruct a final packet that "looks" legitimate.
+
++total_msat+:: The total amount matching the invoice. This may be omitted if there is only one part, in which case it is assumed to match +amt_to_forward+ and must equal the invoice amount.
+
+The invoice Alice received from Dina specified the amount as 50,000 satoshis, which is 50,000,000 millisatoshis. Dina specified the minimum expiry for the payment +min_final_cltv_expiry+ as 18 blocks (3 hours, given 10-minute on average Bitcoin blocks). At the time Alice is attempting to make the payment, let's say the Bitcoin blockchain has recorded 700,000 blocks. So Alice must set the +outgoing_cltv_value+ to a _minimum_ block height of 700,018.
+
+((("hop payload", id="ix_10_onion_routing-asciidoc6", range="startofrange")))Alice constructs the hop payload for Dina as follows:
+
+----
+amt_to_forward : 50,000,000
+outgoing_cltv_value: 700,018
+payment_secret: fb53d94b7b65580f75b98f10...03521bdab6d519143cd521d1b3826
+total_msat: 50,000,000
+----
+
+Alice serializes it in TLV format, as shown (simplified) in <<dina_onion_payload>>. 
+
+[[dina_onion_payload]]
+.Dina's payload is constructed by Alice
+image::images/mtln_1010.png[Dina's payload is constructed by Alice]
+
+===== Hop payload for Chan
+
+Next, Alice constructs the hop payload for Chan. This will tell Chan how to set up an outgoing HTLC to Dina.
+
+The hop payload for Chan includes three fields: +short_channel_id+, +amt_to_forward+, and +outgoing_cltv_value+:
+
+----
+short_channel_id: 010002010a42be
+amt_to_forward: 50,000,000
+outgoing_cltv_value: 700,018
+----
+
+Alice serializes this payload in TLV format, as shown (simplified) in <<chan_onion_payload>>. 
+
+[[chan_onion_payload]]
+.Chan's payload is constructed by Alice
+image::images/mtln_1011.png[Chan's payload is constructed by Alice]
+
+===== Hop payload for Bob
+
+Finally, Alice constructs the hop payload for Bob, which also contains the same three fields as the hop payload for Chan, but with different values:
+
+----
+short_channel_id: 000004040a61f0
+amt_to_forward: 50,100,000
+outgoing_cltv_value: 700,038
+----
+
+As you can see, the +amt_to_forward+ field is 50,100,000 millisatoshis, or 50,100 satoshis. That's because Chan expects a fee of 100 satoshis to route a payment to Dina. In order for Chan to "earn" that routing fee, Chan's incoming HTLC must be 100 satoshis more than Chan's outgoing HTLC. Since Chan's incoming HTLC is Bob's outgoing HTLC, the instructions to Bob reflect the fee Chan earns. In simple terms, Bob needs to be told to send 50,100 satoshi to Chan, so that Chan can send 50,000 satoshi and keep 100 satoshi.
+
+Similarly, Chan expects a timelock delta of 20 blocks. So Chan's incoming HTLC must expire 20 blocks _later_ than Chan's outgoing HTLC. To achieve this, Alice tells Bob to make his outgoing HTLC to Chan expire at block height 700,038-20 blocks later than Chan's HTLC to Dina.
+
+[TIP]
+====
+Fees and timelock delta expectations for a channel are set by the difference between incoming and outgoing HTLCs. Since the incoming HTLC is created by the _preceding node_, the fee and timelock delta is set in the onion payload to that preceding node. Bob is told how to make an HTLC that meets Chan's fee and timelock expectations.
+====
+
+Alice serializes this payload in TLV format, as shown (simplified) in <<bob_onion_payload>>. 
+
+[[bob_onion_payload]]
+.Bob's payload is constructed by Alice
+image::images/mtln_1012.png[Bob's payload is constructed by Alice]
+
+===== Finished hop payloads
+
+Alice has now built the three hop payloads that will be wrapped in an onion. A simplified view of the payloads is shown in <<onion_hop_payloads>>.(((range="endofrange", startref="ix_10_onion_routing-asciidoc6"))) (((range="endofrange", startref="ix_10_onion_routing-asciidoc5")))
+
+[[onion_hop_payloads]]
+.Hop payloads for all the hops
+image::images/mtln_1013.png[Hop payloads for all the hops]
+
+[role="pagebreak-before less_space"]
+==== Key Generation
+
+((("keys","generating for onion routing", id="ix_10_onion_routing-asciidoc7", range="startofrange")))((("keys","onion routing and", id="ix_10_onion_routing-asciidoc8", range="startofrange")))((("onion routing","key generation", id="ix_10_onion_routing-asciidoc9", range="startofrange")))Alice must now generate several keys that will be used to encrypt the various layers in the onion.
+
+With these keys, Alice can achieve a high degree of privacy and integrity:
+
+* Alice can encrypt each layer of the onion so that only the intended recipient can read it.
+* Every intermediary can check that the message is not modified.
+* No one in the path will know who sent this onion or where it is going. Alice doesn't reveal her identity as the sender or Dina's identity as the recipient of the payment.
+* Each hop only learns about the previous and next hop.
+* No one can know how long the path is, or where  in the path they are.
+
+[WARNING]
+====
+Like a chopped onion, the following technical details may bring tears to your eyes. Feel free to skip to the next section if you get confused. Come back to this and read https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#packet-construction[BOLT #4: Onion Routing, Packet Construction], if you want to learn more.
+====
+
+
+((("shared secret (ss)")))The basis for all the keys used in the onion is a _shared secret_ that Alice and Bob can both generate independently using the Elliptic Curve Diffie–Hellman (ECDH) algorithm. From the shared secret (ss), they can independently generate four additional keys named ++rho++, ++mu++, ++um++, and ++pad++:
+
+++rho++:: Used to generate a stream of random bytes from a stream cipher (used as a
+CSPRNG). These bytes are used to encrypt/decrypt the message body as well as
+filler zero bytes during Sphinx packet processing.
+
+++mu++:: Used in the hash-based message authentication code (HMAC) for integrity/authenticity verification.
+
+++um++:: Used in error reporting.
+
+++pad++:: Used to generate filler bytes for padding the onion to a fixed length.
+
+The relationship between the various keys and how they are generated is diagrammed in <<onion_keygen>>. 
+
+[[onion_keygen]]
+.Onion key generation
+image::images/mtln_1014.png[Onion Key Generation]
+
+[[session_key]]
+===== Alice's session key
+
+((("session key")))To avoid revealing her identity, Alice does not use her own node's public key in building the onion. Instead, Alice creates a temporary 32-byte (256-bit) key called the _session private key_ and corresponding _session public key_. This serves as a temporary "identity" and key _for this onion only_. From this session key, Alice will build all the other keys that will be used in this onion.
+
+[[keygen_details]]
+===== Key generation details
+The key generation, random byte generation, ephemeral keys, and how they are used in packet construction are specified in three sections of BOLT #4:
+
+* https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#key-generation[Key Generation]
+* https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#pseudo-random-byte-stream[Random Byte Stream]
+* https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#packet-construction[Packet Construction]
+
+For simplicity and to avoid getting too technical, we have not included these details in the book. See the preceding links if you want to see the inner workings.
+
+[[shared_secret]]
+===== Shared secret generation
+
+((("shared secret (ss)")))One important detail that seems almost magical is the ability for Alice to create a _shared secret_ with another node simply by knowing their public keys. ((("Diffie-Hellman Key Exchange (DHKE)", id="ix_10_onion_routing-asciidoc10", range="startofrange")))This is based on the invention of Diffie–Hellman key exchange (DH) in the 1970s that revolutionized cryptography. Lightning onion routing uses Elliptic Curve Diffie–Hellman (ECDH) on Bitcoin's +secp256k1+ curve. It's such a cool trick that we try to explain it in simple terms in <<ecdh_explained>>. 
+
+
+// To editor: Maybe put this in an appendix instead of a sidebar?
+
+[[ecdh]]
+[[ecdh_explained]]
+.Elliptic Curve Diffie–Hellman Explained
+****
+((("ECDH (Elliptic Curve Diffie–Hellman)")))((("Elliptic Curve Diffie–Hellman (ECDH)")))Assume Alice's private key is _a_ and Bob's private key is _b_. Using the elliptic curve, Alice and Bob each multiply their private key by the generator point _G_ to produce their public keys _A_ and _B_, respectively:
+
+++++
+<ul class="simplelist">
+<li><em>A</em> = <em>aG</em></li>
+<li><em>B</em> = <em>bG</em></li>
+</ul>
+++++
+
+Now Alice and Bob can use _Elliptic Curve Diffie–Hellman Key Exchange_ to create a shared secret _ss_, a value that they can both calculate independently without exchanging any information
+
+The shared secret _ss_ is calculated by each by multiplying their own private key with the _other's_ public key, such that:
+
+++++
+<ul class="simplelist">
+<li><em>ss</em> = <em>aB</em> = <em>bA</em></li>
+</ul>
+++++
+
+But why would these two multiplications result in the same value _ss_?
+Follow along, as we demonstrate the math that proves this is possible:
+
+++++
+<ul class="simplelist">
+<li><em>ss</em></li>
+<li>= <em>aB</em></li>
+</ul>
+++++
+
+calculated by Alice who knows both _a_ (her private key) and _B_ (Bob's public key)
+
+++++
+<ul class="simplelist">
+<li>= <em>a</em>(<em>bG</em>)</li>
+</ul>
+++++
+
+because we know that _B_ = _bG_, we substitute
+
+++++
+<ul class="simplelist">
+<li> = (<em>ab</em>)<em>G</em></li>
+</ul>
+++++
+
+because of associativity, we can move the parentheses
+
+++++
+<ul class="simplelist">
+<li>= (<em>ba</em>)<em>G</em></li>
+</ul>
+++++
+
+because _xy_ = _yx_ (the curve is an abelian group)
+
+++++
+<ul class="simplelist">
+<li>= <em>b</em>(<em>aG</em>)</li>
+</ul>
+++++
+
+because of associativity, we can move the parentheses
+
+++++
+<ul class="simplelist">
+<li>= <em>bA</em></li>
+</ul>
+++++
+
+and we can substitute _aG_ with _A_.
+
+The result _bA_ can be calculated independently by Bob who knows _b_ (his private key) and _A_ (Alice's public key).
+
+We have therefore shown that:
+
+++++
+<ul class="simplelist">
+<li><em>ss</em> = <em>aB</em> (Alice can calculate this)</li>
+<li><em>ss</em> = <em>bA</em> (Bob can calculate this)</li>
+</ul>
+++++
+
+Thus, they can each independently calculate _ss_ which they can use as a shared key to symmetrically encrypt secrets between the two of them without communicating the shared secret.(((range="endofrange", startref="ix_10_onion_routing-asciidoc10")))
+
+****
+
+A unique trait of Sphinx as a mix-net packet format is that rather than include a distinct session key for each hop in the route, which would increase the size of the mix-net packet dramatically, ((("blinding scheme")))a clever _blinding_ scheme is used to deterministically randomize the session key at each hop.
+
+In practice, this little trick allows us to keep the onion packet as compact as possible while still retaining the desired security properties.
+
+The session key for hop `i` is derived using the node public key, and derived shared secret of hop `i – 1`:
+```
+session_key_i = session_key_{i-1} * SHA-256(node_pubkey_{i-1} || shared_secret_{i-1})
+```
+
+In other words, we take the session key of the prior hop, and multiply it by a value derived from the public key and the derived shared secret for that hop.
+
+As elliptic curve multiplication can be performed on a public key without knowledge of the private key, each hop is able to re-randomize the session key for the next hop in a deterministic fashion.
+
+The creator of the onion packet knows all the shared secrets (as they've encrypted the packet uniquely for each hop), and thus are able to derive all the blinding factors.
+
+This knowledge allows them to derive all the session keys used up front during packet generation.
+
+Note that the very first hop uses the original session key generated because this key is used to kick off the session key blinding by each subsequent hop(((range="endofrange", startref="ix_10_onion_routing-asciidoc9")))(((range="endofrange", startref="ix_10_onion_routing-asciidoc8")))(((range="endofrange", startref="ix_10_onion_routing-asciidoc7"))).(((range="endofrange", startref="ix_10_onion_routing-asciidoc4")))(((range="endofrange", startref="ix_10_onion_routing-asciidoc3")))
+
+
+[[wrapping_the_onion]]
+=== Wrapping the Onion Layers
+
+((("onion routing","wrapping the onion layers", id="ix_10_onion_routing-asciidoc11", range="startofrange")))The process of wrapping the onion is detailed in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#packet-construction[BOLT #4: Onion Routing, Packet Construction].
+
+In this section we will describe this process at a high and somewhat simplified level, omitting certain details.
+
+
+[[fixed_length_onions]]
+==== Fixed-Length Onions
+
+((("onion routing","fixed-length onions")))We've mentioned the fact that none of the "hop" nodes know how long the path is, or where they are in the path. How is this possible?
+
+If you have a set of directions, even if encrypted, can't you tell how far you are from the beginning or end simply by looking at _where_ in the list of directions you are?
+
+The trick used in onion routing is to always make the path (the list of directions) the same length for every node. This is achieved by keeping the onion packet the same length at every step.
+
+At each hop, the hop payload appears at the beginning of the onion payload, followed by _what seem to be_ 19 more hop payloads. Every hop sees itself as the first of 20 hops.
+
+[TIP]
+====
+The onion payload is 1,300 bytes. Each hop payload is 65 bytes or less (padded to 65 bytes if less). So the total onion payload can fit 20 hop payloads (1300 = 20 &times; 65). The maximum onion routed path is therefore 20 hops.
+====
+
+As each layer is "peeled off," more filler data (essentially junk) is added at the end of the onion payload so the next hop gets an onion of the same size and is once again the "first hop" in the onion.
+
+The onion size is 1,366 bytes, structured as shown in <<onion_packet>>: 
+
+1 byte:: A version byte
+33 bytes:: A compressed public session key (<<session_key>>) from which the per-hop shared secret (<<shared_secret>>) can be generated without revealing Alice's identity
+1,300 bytes:: The actual _onion payload_ containing the instructions for each hop
+32 bytes:: An HMAC integrity checksum
+
+[[onion_packet]]
+.The onion packet
+image::images/mtln_1015.png[]
+
+A unique trait of Sphinx as a mix-net packet format is that rather than include a distinct session key for each hop in the route, which would increase the size of the mix-net packet dramatically, instead a clever _blinding_ scheme is used to deterministically randomize the session key at each hop.
+
+In practice, this little trick allows us to keep the onion packet as compact as possible while still retaining the desired security properties.
+
+==== Wrapping the Onion (Outlined)
+
+((("onion routing","outline of wrapping process")))Here is the process of wrapping the onion, outlined next. Come back to this list as we explore each step with our real-world example.
+
+For each hop, the sender (Alice) repeats the same process:
+
+1. Alice generates the per-hop shared secret and the ++rho++, ++mu++, and ++pad++ keys.
+
+2. Alice generates 1,300 bytes of filler and fills the 1,300-byte onion payload field with this filler.
+
+3. Alice calculates the HMAC for the hop payload (zeros for the final hop).
+
+4. Alice calculates the length of the hop payload + HMAC + space to store the length itself.
+
+5. Alice _right-shifts_ the onion payload by the calculated space needed to fit the hop payload. The rightmost "filler" data is discarded, making enough space on the left for the payload.
+
+6. Alice inserts the length + hop payload + HMAC at the front of the payload field in the space made from shifting the filler.
+
+7. Alice uses the ++rho++ key to generate a 1,300-byte one-time pad.
+
+8. Alice obfuscates the entire onion payload by XORing with the bytes generated from ++rho++.
+
+9. Alice calculates the HMAC of the onion payload, using the ++mu++ key.
+
+10. Alice adds the session public key (so that the hop can calculate the shared secret).
+
+11. Alice adds the version number.
+
+12. Alice deterministically re-blinds the session key using a value derived by hashing the shared secret and prior hop's public key.
+
+Next, Alice repeats the process. The new keys are calculated, the onion payload is shifted (dropping more junk), the new hop payload is added to the front, and the whole onion payload is encrypted with the ++rho++ byte stream for the next hop.
+
+For the final hop, the HMAC included in Step #3 over the plain-text instructions is actually _all zero_.
+The final hop uses this signal to determine that it is indeed the final hop in the route.
+Alternatively, the fact that the `short_chan_id` included in the payload to denote the "next hop" is all zero can be used as well.
+
+Note that at each phase the ++mu++ key is used to generate an HMAC over the _encrypted_ (from the point of view of the node processing the payload) onion packet, as well as over the contents of the packet with a single layer of encryption removed.
+This outer HMAC allows the node processing the packet to verify the integrity of the onion packet (no bytes modified).
+The inner HMAC is then revealed during the inverse of the "shift and encrypt" routine described previously, which serves as the _outer_ HMAC for the next hop.
+
+==== Wrapping Dina's Hop Payload
+
+((("onion routing","wrapping hop payloads", id="ix_10_onion_routing-asciidoc12", range="startofrange")))As a reminder, the onion is wrapped by starting at the end of the path from Dina, the final node or recipient. Then the path is built in reverse all the way back to the sender, Alice.
+
+Alice starts with an empty 1,300-byte field, the fixed-length _onion payload_. Then, she fills the onion payload with a pseudorandom byte stream "filler" that is generated from the ++pad++ key.
+
+This is shown in <<onion_payload_filler>>. 
+
+[NOTE]
+====
+Random byte stream generation uses the ChaCha20 algorithm, as a cryptographic secure pseudorandom number generator (CSPRNG). Such an algorithm will generate a deterministic, long, nonrepeating stream of seemingly random bytes from an initial seed. The details are specified in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#pseudo-random-byte-stream[BOLT #4: Onion Routing, Pseudo Random Byte Stream].
+====
+
+[[onion_payload_filler]]
+.Filling the onion payload with a random byte stream
+image::images/mtln_1016.png[]
+
+Alice will now insert Dina's hop payload into the left side of the 1,300-byte array, shifting the filler to the right and discarding anything that overflows. This is visualized in <<onion_add_dina>>. 
+
+[[onion_add_dina]]
+.Adding Dina's hop payload
+image::images/mtln_1017.png[]
+
+Another way to look at this is that Alice measures the length of Dina's hop payload, shifts the filler right to create an equal space in the left side of the onion payload, and inserts Dina's payload in that space.
+
+Next row down we see the result: the 1,300 byte onion payload contains Dina's hop payload and then the filler byte stream filling up the rest of the space.
+
+Next, Alice obfuscates the entire onion payload so that _only Dina_ can read it.
+
+To do this, Alice generates a byte stream using the ++rho++ key (which Dina also knows). Alice uses a bitwise exclusive or (XOR) between the bits of the onion payload and the byte stream created from ++rho++. The result appears like a random (or encrypted) byte stream of 1,300 bytes length. This step is shown in <<onion_obfuscate_dina>>. 
+
+[[onion_obfuscate_dina]]
+.Obfuscating the onion payload
+image::images/mtln_1018.png[]
+
+One of the properties of XOR is that if you do it twice, you get back to the original data. As we will see in <<bobDeobfuscates>>, if Dina applies the same XOR operation with the byte stream generated from ++rho++, it will reveal the original onion payload.
+
+[TIP]
+====
+XOR is an _involutory_ function, which means that if it is applied twice, it undoes itself. Specifically XOR(XOR(_a_, _b_), _b_) = _a_. This property is used extensively in symmetric-key cryptography.
+====
+
+Because only Alice and Dina have the ++rho++ key (derived from Alice and Dina's shared secret), only they can do this. Effectively, this encrypts the onion payload for Dina's eyes only.
+
+Finally, Alice calculates a hash-based message authentication code (HMAC) for Dina's payload, which uses the ++mu++ key as its initialization key. This is shown in <<dina_hop_payload_hmac>>. 
+
+[[dina_hop_payload_hmac]]
+.Adding an HMAC integrity checksum to Dina's hop payload
+image::images/mtln_1019.png[]
+
+===== Onion routing replay protection and detection
+
+((("onion routing","replay protection/detection")))The HMAC acts as a secure checksum and helps Dina verify the integrity of the hop payload. The 32-byte HMAC is appended to Dina's hop payload.
+((("encrypt-then-mac")))Note that we compute the HMAC over the _encrypted_ data rather then over the plain-text data.
+This is known as _encrypt-then-mac_ and is the recommended way to use a MAC, as it provides both plain-text _and_ ciphertext integrity.
+
+((("AD (associated data)")))((("associated data (AD)")))Modern authenticated encryption also allows for the use of an optional set of plaintext bytes to also be authenticated, known as _associated data._
+In practice, this is usually something like a plain-text packet header or other auxiliary information.
+By including this associated data in the payload to be authenticated (MAC'ed), the verifier of the MAC ensures that this associated data hasn't been tampered with (e.g., swapping out the plain-text header on an encrypted packet).
+
+In the context of the Lightning Network, this associated data is used to _strengthen_ the replay protection of this scheme.
+As we'll learn in the following, replay protection ensures that an attacker can't _retransmit_ (replay) a packet into the network and observe its resulting path.
+Instead, intermediate nodes are able to use the defined replay protection measures to detect and reject a replayed packet.
+The base Sphinx packet format uses a log of all the ephemeral secret keys used to detect replays.
+If a secret key is ever used again, then the node can detect it and reject the packet.
+
+The nature of HTLCs in the Lightning Network allows us to further strengthen the replay protection by adding an additional _economic_ incentive.
+Remember that the payment hash of an HTLC can only ever safely be used (for a complete payment) once.
+If a payment hash is used again and traverses a node that has already seen the payment secret for that hash, then they can simply pull the funds and collect the entire payment amount without forwarding!
+We can use this fact to strengthen the replay protection by requiring that the _payment hash_ is included in our HMAC computation as the associated data.
+With this added step, attempting to replay an onion packet also requires the sender to commit to using the _same_ payment hash.
+As a result, on top of the normal replay protection, an attacker also stands to lose the entire amount of the HTLC replayed.
+
+One consideration with the ever-increasing set of session keys stored for replay protection is: are nodes able to reclaim this space?
+In the context of the Lightning Network, the answer is: yes!
+Once again, due to the unique attributes of the HTLC construct, we can make a further improvement over the base Sphinx protocol.
+Given that HTLCs are _time-locked_ contracts based on the absolute block height, once an HTLC has expired, then the contract is effectively permanently closed.
+As a result, nodes can use this CLTV (CHECKLOCKTIMEVERIFY operator) expiration height as an indicator to know when it's safe to garbage collect an entry in the anti-replay log.
+
+==== Wrapping Chan's Hop Payload
+
+In <<chan_onion_wrapping>> we see the steps used to wrap Chan's hop payload in the onion. These are the same steps Alice used to wrap Dina's hop payload.
+
+[[chan_onion_wrapping]]
+.Wrapping the onion for Chan
+image::images/mtln_1020.png[]
+
+Alice starts with the 1,300 onion payload created for Dina. The first 65 (or fewer) bytes of this are Dina's payload obfuscated and the rest is filler. Alice must be careful not to overwrite Dina's payload.
+
+Next, Alice needs to locate the ephemeral public key (which was generated at the very start for each hop) that will be prepended to the routing packet at this hop.
+
+Remember that rather than include a unique ephemeral public key (that the sender and intermediate node use in an ECDH operation to generate a shared secret), Sphinx uses a single ephemeral public key that is deterministically randomized at each hop.
+
+When processing the packet, Dina will use her shared secret and public key to derive the blinding value (`b_dina`) and use that to re-randomize the ephemeral public key, in an identical operation to what Alice performs during initial packet construction.
+
+Alice adds an inner HMAC checksum to Chan's payload and inserts it at the "front" (left side) of the onion payload, shifting the existing payload to the right by an equal amount.
+Remember that there are effectively _two_ HMACs used in the scheme: the outer HMAC and the inner HMAC.
+In this case, Chan's _inner_ HMAC is actually Dina's _outer_ HMAC.
+
+Now Chan's payload is in the front of the onion. When Chan sees this, he has no idea how many payloads came before or after. It looks like the first of 20 hops always!
+
+Next, Alice obfuscates the entire payload by XOR with the byte stream generated from the Alice-Chan ++rho++ key. Only Alice and Chan have this ++rho++ key, and only they can produce the byte stream to obfuscate and de-obfuscate the onion.
+Finally, as we did in the earlier step, we compute Chan's outer HMAC, which is what she'll use to verify the integrity of the encrypted onion packet.
+
+==== Wrapping Bob's Hop Payload
+
+In <<bob_onion_wrapping>> we see the steps used to wrap Bob's hop payload in the onion.
+
+All right, by now this is easy!
+
+Start with the onion payload (obfuscated) containing Chan's and Dina's hop payloads.
+
+Obtain the session key for this hop dervied from the blinding factor generated by the prior hop.
+Include the prior hop's outer HMAC as this hop's inner HMAC.
+Insert Bob's hop payload at the beginning and shift everything else over to the right, dropping a Bob-hop-payload-size chunk from the end (it was filler anyway).
+
+Obfuscate the whole thing XOR with the ++rho++ key from the Alice-Bob shared secret so that only Bob can unwrap this.
+
+Calculate the outer HMAC and stick it on the end of Bob's hop payload.(((range="endofrange", startref="ix_10_onion_routing-asciidoc12")))
+
+[[bob_onion_wrapping]]
+.Wrapping the onion for Bob
+image::images/mtln_1021.png[]
+
+
+==== The Final Onion Packet
+
+((("onion routing","final onion packet")))The final onion payload is ready to be sent to Bob. Alice doesn't need to add any more hop payloads.
+
+Alice calculates an HMAC for the onion payload to cryptographically secure it with a checksum that Bob can verify.
+
+Alice adds a 33-byte public session key that will be used by each hop to generate a shared secret and the ++rho++, ++mu++, and ++pad++ keys.
+
+Finally Alice puts the onion version number (+0+ currently) in the front. This allows for future upgrades of the onion packet format.
+
+The result can be seen in <<onion_packet_2>>. (((range="endofrange", startref="ix_10_onion_routing-asciidoc11")))
+
+[[onion_packet_2]]
+.The onion packet
+image::images/mtln_1015.png[]
+
+=== Sending the Onion
+
+((("onion routing","sending the onion", id="ix_10_onion_routing-asciidoc13", range="startofrange")))In this section we will look at how the onion packet is forwarded and how HTLCs are deployed along the path.
+
+==== The update_add_htlc Message
+
+((("onion routing","update_add_htlc message")))((("update_add_htlc message")))Onion packets are sent as part of the +update_add_htlc+ message. If you recall from <<update_add_htlc>>, in <<channel_operation>>, we saw the contents of the +update_add_htlc+ message are as follows:
+
+----
+[channel_id:channel_id]
+[u64:id]
+[u64:amount_msat]
+[sha256:payment_hash]
+[u32:cltv_expiry]
+[1366*byte:onion_routing_packet]
+----
+
+You will recall that this message is sent by one channel partner to ask the other channel partner to add an HTLC. This is how Alice will ask Bob to add an HTLC to pay Dina. Now you understand the purpose of the last field, +onion_routing_packet+, which is 1,366 bytes long. It's the fully wrapped onion packet we just constructed!
+
+==== Alice Sends the Onion to Bob
+
+Alice will send the +update_add_htlc+ message to Bob. Let's look at what this message will contain:
+
++channel_id+:: This field contains the Alice-Bob channel ID, which in our example is +0000031e192ca1+ (see <<alice_dina_path_detail>>).
+
++id+:: The ID of this HTLC in this channel, starting at +0+.
+
++amount_msat+:: The amount of the HTLC: 50,200,000 millisatoshis.
+
++payment_hash+:: The RIPEMD160(SHA-256) payment hash:
++
++9e017f6767971ed7cea17f98528d5f5c0ccb2c71+.
+
++cltv_expiry+:: The expiry timelock for the HTLC will be 700,058. Alice adds 20 blocks to the expiry set in Bob's payload according to Bob's negotiated +cltv_expiry_delta+.
+
++onion_routing_packet+:: The final onion packet Alice constructed with all the hop payloads!
+
+==== Bob Checks the Onion
+
+As we saw in <<channel_operation>>, Bob will add the HTLC to the commitment transactions and update the state of the channel with Alice.
+
+Bob will unwrap the onion he received from Alice as follows:
+
+1. Bob takes the session key from the onion packet and derives the Alice-Bob shared secret.
+
+2. Bob generates the ++mu++ key from the shared secret and uses it to verify the onion packet HMAC checksum.
+
+Now that Bob has generated the shared key and verified the HMAC, he can start unwrapping the 1,300 byte onion payload inside the onion packet. The goal is for Bob to retrieve his own hop payload and then forward the remaining onion to the next hop.
+
+If Bob extracts and removes his hop payload, the remaining onion will not be 1,300 bytes, it will be shorter! So the next hop will know that they are not the first hop and will be able to detect how long the path is. To prevent this, Bob needs to add more filler to refill the onion.
+
+==== Bob Generates Filler
+
+Bob generates filler in a slightly different way than Alice, but following the same general principle.
+
+First, Bob _extends_ the onion payload by 1,300 bytes and fills them with +0+ values. Now the onion packet is 2,600 bytes long, with the first half containing the data Alice sent and the next half containing zeroes. This operation is shown in <<bob_extends>>. 
+
+[[bob_extends]]
+.Bob extends the onion payload by 1,300 (zero-filled) bytes
+image::images/mtln_1023.png["Bob extends the onion payload by 1,300 (zero-filled) bytes"]
+
+This empty space will become obfuscated and turn into "filler" by the same process that Bob uses to de-obfuscate his own hop payload. Let's see how that works.
+
+[[bobDeobfuscates]]
+==== Bob De-Obfuscates His Hop Payload
+
+Next, Bob will generate the ++rho++ key from the Alice-Bob shared key. He will use this to generate a 2,600 byte stream, using the ChaCha20 algorithm.
+
+[TIP]
+====
+The first 1,300 bytes of the byte stream generated by Bob are exactly the same as those generated by Alice using the ++rho++ key.
+====
+
+Next, Bob applies the 2,600 bytes of the ++rho++ byte stream to the 2,600-byte onion payload with a bitwise XOR operation.
+
+The first 1,300 bytes will become de-obfuscated by this XOR operation, because it is the same operation Alice applied and XOR is involutory. So Bob will _reveal_ his hop payload followed by some data that seems scrambled.
+
+At the same time, applying the ++rho++ byte stream to the 1,300 zeroes that were added to the onion payload will turn them into seemingly random filler data. This operation is shown in <<bob_deobfuscates>>.
+
+[[bob_deobfuscates]]
+.Bob de-obfuscates the onion, obfuscates the filler
+image::images/mtln_1024.png["Bob de-obfuscates the onion, obfuscates the filler"]
+
+==== Bob Extracts the Outer HMAC for the Next Hop
+
+Remember that an inner HMAC is included for each hop, which will then become the outer HMAC for the _next_ hop.
+In this case, Bob extracts the inner HMAC (he's already verified the integrity of the encrypted packet with the outer HMAC), and puts it aside because he'll append it to the de-obfuscated packet to allow Chan to verify the HMAC of his encrypted packet.
+
+==== Bob Removes His Payload and Left-Shifts the Onion
+
+Now Bob can remove his hop payload from the front of the onion and left-shift the remaining data. An amount of data equal to Bob's hop payload from the second-half 1,300 bytes of filler will now shift into the onion payload space. This is shown in <<bob_removes_shifts>>. 
+
+Now Bob can keep the first half 1,300 bytes, and discard the extended (filler) 1,300 bytes.
+
+Bob now has a 1,300-byte onion packet to send to the next hop. It is almost identical to the onion payload that Alice had created for Chan, except that the last 65 or so bytes of filler was put there by Bob and will be different.
+
+[[bob_removes_shifts]]
+.Bob removes the hop payload and left-shifts the rest, filling the gap with new filler
+image::images/mtln_1025.png["Bob removes the hop payload and left-shifts the rest, filling the gap with new filler"]
+
+[role="pagebreak-before"]
+No one can tell the difference between filler put there by Alice and filler put there by Bob. Filler is filler! It's all random bytes anyway. Note that if Bob (or one of Bob's other aliases) is present in the route in two distinct locations, then they can tell the difference because the base protocol always uses the same payment hash across the entire route. Atomic multipath payments (AMPs) and Point Time-Locked Contracts (PTLCs) eliminate the correlation vector by randomizing the payment identifier across each route/hop.
+
+==== Bob Constructs the New Onion Packet
+
+Bob now copies the onion payload into the onion packet, appends the outer HMAC for chan, re-randomizes the session key (the same way Alice the sender does) with the elliptic curve multiplication operation, and appends a fresh version byte.
+
+To re-randomize the session key, Bob first computes the blinding factor for his hop, using his node public key and the shared secret he derived:
+```
+b_bob = SHA-256(P_bob || shared_secret_bob)
+```
+
+With this generated, Bob now re-randomizes the session key by performing an EC multiplication using his session key and the blinding factor:
+```
+session_key_chan = session_key_bob * b_bob
+```
+
+The `session_key_chan` public key will then be appended to the front of the onion packet for processing by Chan.
+
+==== Bob Verifies the HTLC Details
+
+Bob's hop payload contains the instructions needed to create an HTLC for Chan.
+
+In the hop payload, Bob finds a +short_channel_id+, +amt_to_forward+, and +cltv_expiry+.
+
+First, Bob checks to see if he has a channel with that short ID. He finds that he has such a channel with Chan.
+
+Next, Bob confirms that the outgoing amount (50,100 satoshis) is less than the incoming amount (50,200 satoshis), and therefore Bob's fee expectations are met.
+
+Similarly, Bob checks that the outgoing +cltv_expiry+ is less than the incoming +cltv_expiry+, giving Bob enough time to claim the incoming HTLC if there is a breach.
+
+==== Bob Sends the update_add_htlc to Chan
+
+Bob now constructs and HTLC to send to Chan, as follows:
+
++channel_id+:: This field contains the Bob-Chan channel ID, which in our example is +000004040a61f0+ (see <<alice_dina_path_detail>>).
+
++id+:: The ID of this HTLC in this channel, starting at +0+.
+
++amount_msat+:: The amount of the HTLC: 50,100,000 millisatoshis.
+
++payment_hash+:: The RIPEMD160(SHA-256) payment hash:
++
++9e017f6767971ed7cea17f98528d5f5c0&#x200b;ccb2c71+. 
++
+This is the same as the payment hash from Alice's HTLC.
+
++cltv_expiry+:: The expiry timelock for the HTLC will be 700,038.
+
++onion_routing_packet+:: The onion packet Bob reconstructed after removing his hop payload.
+
+==== Chan Forwards the Onion
+
+Chan repeats the exact same process as Bob:
+
+1. Chan receives the +update_add_htlc+ and processes the HTLC request, adding it to commitment transactions.
+
+2. Chan generates the Alice-Chan shared key and the ++mu++ subkey.
+
+3. Chan verifies the onion packet HMAC, then extracts the 1,300-byte onion pass:[<span class="keep-together">payload</span>].
+
+4. Chan extends the onion payload by 1,300 extra bytes, filling it with zeroes.
+
+5. Chan uses the ++rho++ key to produce 2,600 bytes.
+
+6. Chan uses the generated byte stream to XOR and de-obfuscate the onion payload. Simultaneously, the XOR operation obfuscates the extra 1,300 zeroes, turning them into filler. 
+
+7. Chan extracts the inner HMAC in the payload, which will become the outer HMAC for Dina.
+
+8. Chan removes his hop payload and left-shifts the onion payload by the same amount. Some of the filler generated in the 1,300 extended bytes moves into the first-half 1,300 bytes, becoming part of the onion payload.
+
+9. Chan constructs the onion packet for Dina with this onion payload.
+
+10. Chan builds an +update_add_htlc+ message for Dina and inserts the onion packet into it.
+
+11. Chan sends the +update_add_htlc+ to Dina.
+
+12. Chan re-randomizes the session key as Bob did in the prior hop for Dina.
+
+==== Dina Receives the Final Payload
+
+When Dina receives the +update_add_htlc+ message from Chan, she knows from the +payment_hash+ that this is a payment for her. She knows she is the last hop in the onion.
+
+Dina follows the exact same process as Bob and Chan to verify and unwrap the onion, except she doesn't construct new filler and doesn't forward anything. Instead, Dina responds to Chan with +update_fulfill_htlc+ to redeem the HTLC. The +update_fulfill_htlc+ will flow backward along the path until it reaches Alice. All the HTLCs are redeemed and channel balances are updated. The payment is complete!(((range="endofrange", startref="ix_10_onion_routing-asciidoc13")))
+
+=== Returning Errors
+
+((("error return","onion routing and", id="ix_10_onion_routing-asciidoc14", range="startofrange")))((("onion routing","returning errors", id="ix_10_onion_routing-asciidoc15", range="startofrange")))This far we've looked at the forward propagation of the onion establishing the HTLCs and the backward propagation of the payment secret unwinding the HTLCs once payment is successful.
+
+There is another very important function of onion routing: _error return_. If there is a problem with the payment, onion, or hops, we must propagate an error backwards to inform all nodes of the failure and unwind any HTLCs.
+
+Errors generally fall into three categories: onion failures, node failures, and channel failures. These furthermore may be subdivided into permanent and transient errors. Finally, some errors contain channel updates to help with future payment delivery attempts.
+
+[NOTE]
+====
+Unlike messages in the peer-to-peer (P2P) protocol (defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md[BOLT #2: Peer Protocol for Channel Management]), errors are not sent as P2P messages but are wrapped inside onion return packets and follow the reverse of the onion path (back-propagating).
+====
+
+Error return is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#returning-errors[BOLT #4: Onion Routing, Returning Errors].
+
+Errors are encoded by the returning node (the one that discovered an error) in a _return packet_ as follows:
+
+----
+    [32*byte:hmac]
+    [u16:failure_len]
+    [failure_len*byte:failuremsg]
+    [u16:pad_len]
+    [pad_len*byte:pad]
+----
+
+The return packet HMAC verification checksum is calculated with the ++um++ key, generated from the shared secret established by the onion.
+
+[TIP]
+====
+The ++um++ key name is the reverse of the ++mu++ name, indicating the same use but in the opposite direction (back-propagation). 
+====
+
+Next, the returning node generates an +ammag+ (inverse of the word "gamma") key and obfuscates the return packet using an XOR operation with a byte stream generated from +ammag+.
+
+Finally the return node sends the return packet to the hop from which it received the original onion.
+
+Each hop receiving an error will generate an +ammag+ key and obfuscate the return packet again using an XOR operation with the byte stream from +ammag+.
+
+Eventually, the sender (origin node) receives a return packet. It will then generate +ammag+ and ++um++ keys for each hop and XOR de-obfuscate the return error iteratively until it reveals the return packet.
+
+[[failure_messages]]
+==== Failure Messages
+
+((("error return","failure messages", id="ix_10_onion_routing-asciidoc16", range="startofrange")))((("failure messages, onion routing and", id="ix_10_onion_routing-asciidoc17", range="startofrange")))The +failuremsg+ is defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md#failure-messages[BOLT #4: Onion Routing, Failure Messages].
+
+A failure message consists of a two-byte +failure code+ followed by the data applicable to that failure type.
+
+The top byte of the +failure_code+ is a set of binary flags that can be combined (with binary OR):
+
+
+0x8000 (`BADONION`):: Unparsable onion encrypted by sending peer
+0x4000 (`PERM`):: Permanent failure (otherwise transient)
+0x2000 (`NODE`):: Node failure (otherwise channel)
+0x1000 (`UPDATE`):: New channel update enclosed
+
+
+The failure types shown in <<failure_types_table>> are currently defined.
+
+include::failure_types_table.asciidoc[]
+
+[[stuck_payments]]
+===== Stuck payments
+
+((("onion routing","stuck payments")))((("stuck payments")))In the current implementation of the Lightning Network, there is a possibility that a payment attempt becomes _stuck_: neither fulfilled nor cancelled by an error. This can happen due to a bug on an intermediary node, a node going offline while handling HTLCs, or a malicious node holding HTLCs without reporting an error. In all of these cases, the HTLC cannot be resolved until it expires. The timelock (CLTV) that is set on every HTLC helps resolve this condition (among other possible HTLC routing and channel failures).
+
+However, this means that the sender of the HTLC has to wait until expiry, and the funds committed to that HTLC remain unavailable until the HTLC expires. Furthermore, the sender _cannot retry_ that same payment, because if they do, they run the risk of _both_ the original and the retried payment succeeding—the recipient gets paid twice. This is because, once sent, an HTLC cannot be "cancelled" by the sender—it either has to fail or expire. Stuck payments, while rare, create an unwanted user experience, where the user's wallet cannot pay or cancel a payment.
+
+((("Point Time-Locked Contract (PTLC)")))((("PTLC (Point Time-Locked Contract)")))((("stuckless payments")))One proposed solution to this problem is called _stuckless payments_, and it depends on Point Time-Locked Contracts (PTLCs), which are payment contracts that use a different cryptographic primitive than HTLCs (i.e., point addition on the elliptic curve instead of a hash and secret preimage). PTLCs are cumbersome using ECDSA but much easier with Bitcoin's Taproot and Schnorr signature features, which were recently locked in for activation in November 2021. It is expected that PTLCs will be implemented in the Lightning Network after these Bitcoin features become activated(((range="endofrange", startref="ix_10_onion_routing-asciidoc17")))(((range="endofrange", startref="ix_10_onion_routing-asciidoc16"))).(((range="endofrange", startref="ix_10_onion_routing-asciidoc15")))(((range="endofrange", startref="ix_10_onion_routing-asciidoc14")))
+
+[[keysend]]
+=== Keysend Spontaneous Payments
+
+((("keysend spontaneous payments")))((("onion routing","keysend spontaneous payments")))In the payment flow described earlier in the chapter, we assumed that Dina
+received an invoice from Alice "out of band," or obtained it via some mechanism
+unrelated to the protocol (typically copy/paste or QR code scans). This trait
+means that the payment process always takes two steps: first, the sender
+obtains an invoice, and second, uses the payment hash (encoded in the invoice) to
+successfully route an HTLC. The extra round trip required to obtain an invoice
+before making a payment may be a bottleneck in applications that involve
+streaming micropayments over Lightning. What if we could just "push" a payment
+over spontaneously, without having to obtain an invoice from the recipient
+first? The `keysend` protocol is an end-to-end extension (only the sender and
+receiver are aware) to the Lightning protocol that enables spontaneous push
+payments.
+
+==== Custom Onion TLV Records
+
+((("onion routing","custom onion TLV records")))((("Type-Length-Value (TLV) format","custom onion TLV records")))The modern Lightning protocol uses the TLV (Type-Length-Value) encoding in
+the onion to encode information that tells each node _where_ and _how_ to
+forward the payment. Leveraging the TLV format, each piece of routing information
+(like the next node to which to pass the HTLC) is assigned a specific type (or key)
+encoded as a `BigSize` variable length integer (max sized as as 64-bit
+integer). These "essential" (reversed values below `65536`) types are defined
+in BOLT #4, along with the rest of the onion routing details. Onion types with a
+value greater than `65536` are intended to be used by wallets and applications
+as "custom records."
+
+Custom records allow payment applications to attach additional metadata or
+context to a payment as key/value pairs in the onion. Since the custom records
+are included in the onion payload itself, like all other hop contents, the
+records are end-to-end encrypted. As the custom records effectively consume a
+portion of the fixed-size 1300-bytes onion packet, encoding each key and
+value of each custom record reduces the amount of available space for encoding
+the rest of the route. In practice, this means that the more onion space used for custom records, the shorter the route can be. Given that each HTLC
+packet is fixed size, custom records don't _add_ any additional data to an
+HTLC; rather, they reallocate bytes that would have been filled with random data
+otherwise.
+
+==== Sending and Receiving Keysend Payments
+
+((("onion routing","sending/receiving keysend payments")))A `keysend` payment inverts the typical flow of an HTLC where the receiver
+reveals a secret preimage to the sender. Instead, the sender includes the
+preimage _within_ the onion to the receiver, and routes the HTLC to the
+receiver. The receiver then decrypts the onion payload, and uses the included
+preimage (which _must_ match the payment hash of the HTLC) to settle the
+payment. As a result, `keysend` payments can be carried out without first
+obtaining an invoice from the receiver, as the preimage is "pushed" over to
+the receiver. A `keysend` payment uses a TLV custom record type of `5482373484`
+to encode a 32-byte preimage value.
+
+==== Keysend and Custom Records in Lightning Applications
+
+((("onion routing","keysend and custom records in Lightning applications")))Many streaming Lightning applications use the `keysend` protocol to continually
+stream satoshis to a destination identified by its public key in the network.
+Typically, an application will also include metadata such as a
+tipping/donation note or other application-level information in addition to
+the `keysend` record.
+
+=== Conclusion
+
+The Lightning Network's onion routing protocol is adapted from the Sphinx protocol to better serve the needs of a payment network. As such, it offers a huge improvement in privacy and counter-surveillance compared to the public and transparent Bitcoin blockchain.(((range="endofrange", startref="ix_10_onion_routing-asciidoc0")))
+
+In <<path_finding>> we will see how the combination of source routing and onion routing is used by Alice to find a good path and route the payment to Dina. To find a path, Alice first needs to learn about the network topology, which is the topic of <<gossip>>.

11_gossip_channel_graph.asciidoc

@@ -0,0 +1,488 @@
+[[gossip]]
+== Gossip and the Channel Graph
+
+((("channel graph", id="ix_11_gossip_channel_graph-asciidoc0", range="startofrange")))((("gossip protocol", id="ix_11_gossip_channel_graph-asciidoc1", range="startofrange")))In this chapter we will describe the Lightning Network's gossip protocol and how it is used by nodes to construct and maintain a channel graph. We will also review the DNS bootstrap mechanism used to find peers to "gossip" with.
+
+The "Routing fees and Gossip relaying" section is highlighted by an outline spanning the routing layer and peer-to-peer layer of <<LN_protocol_gossip_highlight>>.
+
+[[LN_protocol_gossip_highlight]]
+.Gossip protocol in the Lightning protocol suite
+image::images/mtln_1101.png["Gossip protocol in the Lightning protocol suite"]
+
+As we've learned already, the Lightning Network uses a source-based onion routing protocol to deliver a payment from a sender to the recipient.
+To do this, the sending node must be able to construct a path of payment channels that connects it with the recipient, as we will see in <<path_finding>>.
+Thus, the sender has to be able to map the Lightning Network by constructing a channel graph.
+The _channel graph_ is the interconnected set of publicly advertised channels and the nodes that these channels interlink.
+
+As channels are backed by a funding transaction that is happening on-chain, one might falsely believe that Lightning nodes could just extract the existing channels from the Bitcoin blockchain.
+However this is only possible to a certain extent.
+((("P2WSH (Pay-to-Witness-Script-Hash)")))((("Pay-to-Witness-Script-Hash (P2WSH)")))The funding transactions are Pay-to-Witness-Script-Hash (P2WSH) addresses, and the nature of the script (a 2-of-2 multisig) will only be revealed once the funding transaction output is spent.
+Even if the nature of the script were known, it's important to remember that not all 2-of-2 multisig scripts correspond to payment channels.
+
+There are even more reasons why looking at the Bitcoin blockchain might not be helpful.
+For example, on the Lightning Network, the Bitcoin keys that are used for signing are rotated by the nodes for every channel and update.
+Thus, even if we could reliably detect funding transactions on the Bitcoin blockchain, we would not know which two nodes on the Lightning Network own that particular channel.
+
+The Lightning Network solves this problem by implementing a _gossip protocol_.
+Gossip protocols are typical for peer-to-peer (P2P) networks and allow nodes to share information with the whole network with just a few direct connections to peers.
+Lightning nodes open encrypted peer-to-peer connections to each other and share (gossip) information that they have received from other peers.
+As soon as a node wants to share some information, for example, about a newly created channel, it sends a message to all its peers.
+Upon receiving a message, a node decides if the received message was novel and, if so, forwards the information to its peers.
+In this way, if the peer-to-peer network is well connected, all new information that is necessary for the operation of the network will eventually be propagated to all other peers.
+
+Obviously, if a new peer joins the network for the first time, it needs to know some other peers on the network, so it can connect to others and participate in the network.
+
+In this chapter, we'll explore exactly _how_ Lightning nodes discover each other, discover and update their node status, and communicate with one another.
+
+When most refer to the _network_ part of the Lightning Network, they're referring to the _channel graph_ which itself is a unique authenticated data structure _anchored_ in the base Bitcoin
+blockchain.
+
+However, the Lightning Network is also a peer-to-peer network of nodes that gossip information about payment channels and nodes. Usually, for two peers to maintain a payment channel they need to talk to each other directly, which means that there will be a peer connection between them.
+This suggests that the channel graph is a subnetwork of the peer-to-peer network.
+However, this is not true because payment channels can remain open even if one or both peers go temporarily offline.
+
+Let's revisit some of the terminology that we have used throughout the book, specifically looking at what they mean in terms of the channel graph and the peer-to-peer network (see <<network_terminology>>).
+
+[[network_terminology]]
+.Terminology of the different networks
+[options="header"]
+|===
+| Channel graph  |Peer-to-peer network
+|  channel | connection
+| open | connect
+| close | disconnect
+|  funding transaction | encrypted TCP/IP connection
+| send	|	transmit
+| payment |  message
+|===
+
+Because the Lightning Network is a peer-to-peer network, some initial bootstrapping is required in order for peers to discover each other.  Within this chapter we'll follow the story of a new peer connecting to the network for the first time and examine each step in the bootstrapping process, from initial peer discovery to channel graph syncing and validation.
+
+As an initial step, our new node needs to somehow _discover_ at least _one_ peer that is already connected to the network and has a full channel graph (as we'll see later, there's no canonical version of the channel graph). Using one of many initial bootstrapping protocols to find that first peer, after a connection is established, our new
+peer now needs to _download_ and _validate_ the channel graph. Once the channel graph has been fully validated, our new peer is ready to start opening channels and sending payments on the network.
+
+After initial bootstrap, a node on the network needs to continue to maintain its view of the channel graph by processing new channel routing policy updates, discovering and validating new channels, removing channels that have been closed on-chain, and finally pruning channels that fail to send out a proper "heartbeat" every two weeks pass:[<span class="keep-together">or so</span>].
+
+Upon completion of this chapter, you will understand a key component of
+the peer-to-peer Lightning Network: namely, how peers discover each other and maintain a local copy (perspective) of the channel graph. We'll begin by exploring the story of a new node that has just booted up and needs to find other peers to connect to on the network.(((range="endofrange", startref="ix_11_gossip_channel_graph-asciidoc1")))(((range="endofrange", startref="ix_11_gossip_channel_graph-asciidoc0")))
+
+[role="pagebreak-before less_space"]
+=== Peer Discovery
+
+((("gossip protocol","peer discovery", id="ix_11_gossip_channel_graph-asciidoc2", range="startofrange")))((("peer discovery", id="ix_11_gossip_channel_graph-asciidoc3", range="startofrange")))In this section, we'll begin to follow a new Lightning node that wishes to join the network through three steps:
+
+. Discover a set of bootstrap peers
+. Download and validate the channel graph
+. Begin the process of ongoing maintenance of the channel graph itself
+
+
+==== P2P Bootstrapping
+
+((("bootstrapping","P2P")))((("initial peer bootstrapping")))((("P2P bootstrapping")))((("peer discovery","P2P bootstrapping")))Before doing any thing else, our new node first needs to discover a set of peers who are already part of the network. We call this process initial peer bootstrapping, and it's something that every peer-to-peer network needs to implement properly to ensure a robust, healthy network.
+
+Bootstrapping new peers to existing peer-to-peer networks is a very well studied problem with several known solutions, each with their own distinct trade-offs. The simplest solution to this problem is simply to package a set of _hardcoded_ bootstrap peers into the packaged P2P node software. This is simple in that each new node has a list of bootstrap peers in the software they're running, but rather fragile given that if the set of bootstrap peers goes offline, then no new nodes will be able to join the network. Due to this fragility, this
+option is usually used as a fallback in case none of the other P2P bootstrapping mechanisms work properly.
+
+((("initial peer discovery")))Rather than hardcoding the set of bootstrap peers within the software/binary itself, we can instead allow peers to dynamically obtain a fresh/new set of bootstrap peers they can use to join the network. We'll call this process _initial peer discovery_. Typically we'll leverage
+existing internet protocols to maintain and distribute a set of bootstrapping peers. A nonexhaustive list of protocols that have been used in the past to accomplish initial peer discovery includes:
+
+  * Domain Name Service (DNS)
+  * Internet Relay Chat (IRC)
+  * Hypertext Transfer Protocol (HTTP)
+
+Similar to the Bitcoin protocol, the primary initial peer discovery mechanism used in the Lightning Network happens via DNS. Because initial peer discovery is a critical and universal task for the network, the process has been _standardized_ in https://github.com/lightningnetwork/lightning-rfc/blob/master/10-dns-bootstrap.md[BOLT #10: DNS Bootstrap].
+
+==== DNS Bootstrapping
+
+((("bootstrapping","DNS", id="ix_11_gossip_channel_graph-asciidoc4", range="startofrange")))((("DNS bootstrapping", id="ix_11_gossip_channel_graph-asciidoc5", range="startofrange")))((("peer discovery","DNS bootstrapping", id="ix_11_gossip_channel_graph-asciidoc6", range="startofrange")))The https://github.com/lightningnetwork/lightning-rfc/blob/master/10-dns-bootstrap.md[BOLT #10] document describes a standardized way of implementing peer
+discovery using the DNS. Lightning's flavor of DNS-based bootstrapping uses up to three distinct record types:
+
+  * +SRV+ records for discovering a set of _node public keys_.
+  * +A+ records for mapping a node's public key to its current +IPv4+ address.
+  * +AAA+ records for mapping a node's public key to its current +IPv6+ address.
+
+Those somewhat familiar with the DNS protocol may already be familiar with the +A+ (name to IPv4 address) and +AAA+ (name to IPv6 address) record types, but not the +SRV+ type. The +SRV+ record type is used by protocols built on top of DNS to determine the _location_ for a specified service. In our context, the service in question is a given Lightning node, and the location is its IP address. We need to use this additional record type because, unlike nodes within the Bitcoin protocol, we need both a public key _and_ an IP address to connect to a node. As we see in <<wire_protocol>>, the transport encryption protocol used in the Lightning Network requires knowledge of the public key of a node before connecting, so as to implement identity hiding for nodes in the network.
+
+===== A new peer's bootstrapping workflow
+
+Before diving into the specifics of https://github.com/lightningnetwork/lightning-rfc/blob/master/10-dns-bootstrap.md[BOLT #10], we'll first outline the high-level flow of a new node that wishes to use BOLT #10 to join the network.
+
+First, a node needs to identify a single DNS server or set of DNS servers that understand BOLT #10 so they can be used for P2P bootstrapping.
+
+While BOLT #10 uses _lseed.bitcoinstats.com_ as the seed server, there exists no "official" set of DNS seeds for this purpose, but each of the major implementations maintains their own DNS seed, and they cross-query each other's seeds for redundancy purposes. In <<dns_seeds>> you'll see a nonexhaustive list of some popular DNS seed servers.
+
+[[dns_seeds]]
+.Table of known Lightning DNS seed servers
+[options="header"]
+|===
+| DNS server     | Maintainer
+| _lseed.bitcoinstats.com_ | Christian Decker
+| _nodes.lightning.directory_ | Lightning Labs (Olaoluwa Osuntokun)
+| _soa.nodes.lightning.directory_ | Lightning Labs (Olaoluwa Osuntokun)
+| _lseed.darosior.ninja_ | Antoine Poinsot
+|===
+
+
+DNS seeds exist for both Bitcoin's mainnet and testnet. For the sake
+of our example, we'll assume the existence of a valid BOLT #10 DNS seed at _nodes.lightning.directory_.
+
+Next, our new node will issue an +SRV+ query to obtain a set of _candidate bootstrap peers_. The response to our query will be a series of bech32 encoded public keys. Because DNS is a text-based protocol, we can't send raw binary data, so an encoding scheme is required. BOLT #10 specifies a bech32 encoding due to its use in the wider Bitcoin ecosystem. The number of encoded public keys returned depends on the server returning the query, as well as all the resolvers that stand between the client and the authoritative server.
+
+Using the widely available +dig+ command-line tool, we can query the _testnet_ version of the DNS seed mentioned previously with the following command:
+
+----
+$ dig @8.8.8.8 test.nodes.lightning.directory SRV
+----
+
+We use the +@+ argument to force resolution via Google's nameserver (with IP address 8.8.8.8) because it does not filter large SRV query responses. At the end of the command, we specify that we only want +SRV+ records to be returned. A sample response looks something like <<ex1101>>.
+
+[[ex1101]]
+.Querying the DNS seed for reachable nodes
+====
+----
+$ dig @8.8.8.8 test.nodes.lightning.directory SRV
+
+; <<>> DiG 9.10.6 <<>> @8.8.8.8 test.nodes.lightning.directory SRV
+; (1 server found)
+;; global options: +cmd
+;; Got answer:
+;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43610
+;; flags: qr rd ra; QUERY: 1, ANSWER: 25, AUTHORITY: 0, ADDITIONAL: 1
+
+;; QUESTION SECTION:
+;test.nodes.lightning.directory.	IN	SRV
+
+;; ANSWER SECTION:
+test.nodes.lightning.directory.	59 IN	SRV	10 10 9735 <1>
+ln1qfkxfad87fxx7lcwr4hvsalj8vhkwta539nuy4zlyf7hqcmrjh40xx5frs7.test.nodes.lightning.directory. <2>
+test.nodes.lightning.directory.	59 IN	SRV	10 10 15735 ln1qtgsl3efj8verd4z27k44xu0a59kncvsarxatahm334exgnuvwhnz8dkhx8.test.nodes.lightning.directory.
+
+ [...]
+
+;; Query time: 89 msec
+;; SERVER: 8.8.8.8#53(8.8.8.8)
+;; WHEN: Thu Dec 31 16:41:07 PST 2020
+----
+<1> TCP port number where the LN node can be reached.
+<2> Node public key (ID) encoded as a virtual domain name.
+====
+
+We've truncated the response for brevity and show only two of the returned responses. The responses contain a "virtual" domain name for a target node, then to the left we have the _TCP port_ where this node can be reached. The first response uses the standard TCP port for the Lightning Network: +9735+. The second response uses a custom port, which is permitted by the protocol.
+
+Next, we'll attempt to obtain the other piece of information we need to connect to a node: its IP address. Before we can query for this, however, we'll first _decode_ the bech32 encoding of the public key from the virtual domain name:
+
+----
+ln1qfkxfad87fxx7lcwr4hvsalj8vhkwta539nuy4zlyf7hqcmrjh40xx5frs7
+----
+
+Decoding this bech32 string we obtain the following valid
++secp256k1+ public key:
+
+----
+026c64f5a7f24c6f7f0e1d6ec877f23b2f672fb48967c2545f227d70636395eaf3
+----
+
+Now that we have the raw public key, we'll ask the DNS server to _resolve_ the virtual host given so we can obtain the IP information (+A+ record) for the node, as shown in <<ex1102>>.
+
+++++
+<div id="ex1102" data-type="example">
+<h5>Obtaining the latest IP address for a node</h5>
+
+<pre data-type="programlisting">$ dig ln1qfkxfad87fxx7lcwr4hvsalj8vhkwta539nuy4zlyf7hqcmrjh40xx5frs7.test.nodes.lightning.directory A
+
+; &lt;&lt;&gt;&gt; DiG 9.10.6 &lt;&lt;&gt;&gt; ln1qfkxfad87fxx7lcwr4hvsalj8vhkwta539nuy4zlyf7hqcmrjh40xx5frs7.test.nodes.lightning.directory A
+;; global options: +cmd
+;; Got answer:
+;; -&gt;&gt;HEADER&lt;&lt;- opcode: QUERY, status: NOERROR, id: 41934
+;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
+
+;; OPT PSEUDOSECTION:
+; EDNS: version: 0, flags:; udp: 4096
+;; QUESTION SECTION:
+;ln1qfkxfad87fxx7lcwr4hvsalj8vhkwta539nuy4zlyf7hqcmrjh40xx5frs7.test.nodes.lightning.directory. IN A
+
+;; ANSWER SECTION:
+ln1qfkxfad87fxx7lcwr4hvsalj8vhkwta539nuy4zlyf7hqcmrjh40xx5frs7.test.nodes.lightning.directory. 60 IN A <em>X.X.X.X</em> <a class="co" id="comarker1" href="#c01"><img src="callouts/1.png" alt="1"/></a>
+
+;; Query time: 83 msec
+;; SERVER: 2600:1700:6971:6dd0::1#53(2600:1700:6971:6dd0::1)
+;; WHEN: Thu Dec 31 16:59:22 PST 2020
+;; MSG SIZE  rcvd: 138</pre>
+
+<dl class="calloutlist">
+<dt><a class="co" id="c01" href="#comarker1"><img src="callouts/1.png" alt="1"/></a></dt>
+<dd><p>The DNS server returns an IP address <code><em>X.X.X.X</em></code>. We’ve replaced it with X’s in the text here so as to avoid presenting a real IP address.</p></dd>
+</dl></div>
+++++
+
+In the preceding command, we've queried the server so we can obtain an IPv4 pass:[<span class="keep-together">(<code>A</code> record)</span>] address for our target node (replaced by ++__X.X.X.X__++ in the preceding example). Now that we have the raw public key, IP address, and TCP port, we can connect to the node transport protocol at:
+
+----
+026c64f5a7f24c6f7f0e1d6ec877f23b2f672fb48967c2545f227d70636395eaf3@X.X.X.X:9735
+----
+
+Querying the current DNS +A+ record for a given node can also be used to look up the _latest_ set of addresses. Such queries can be used to more quickly sync the latest addressing information for a node, compared to waiting for address updates on the gossip network (see <<node_announcement>>).
+
+At this point in our journey, our new Lightning node has found its first
+peer and established its first connection! Now we can begin the second phase of new peer bootstrapping: channel graph synchronization and validation.
+
+First, we'll explore more of the intricacies of BOLT #10 itself to take a deeper look into how things work under the hood.(((range="endofrange", startref="ix_11_gossip_channel_graph-asciidoc6")))(((range="endofrange", startref="ix_11_gossip_channel_graph-asciidoc5")))(((range="endofrange", startref="ix_11_gossip_channel_graph-asciidoc4")))
+
+==== SRV Query Options
+
+The https://github.com/lightningnetwork/lightning-rfc/blob/master/10-dns-bootstrap.md[BOLT #10] standard is highly extensible due to its usage of nested
+subdomains as a communication layer for additional query options. The
+bootstrapping protocol allows clients to further specify the _type_ of nodes they're attempting to query for versus the default of receiving a random subset of nodes in the query responses.
+
+The query option subdomain scheme uses a series of key-value pairs where the key itself is a _single letter_ and the remaining set of text is the value itself. The following query types exist in the current version of the https://github.com/lightningnetwork/lightning-rfc/blob/master/10-dns-bootstrap.md[BOLT #10] standards document:
+
++r+:: The _realm_ byte which is used to determine which chain or realm    queries should be returned for. As is, the only value for this key is +0+ which denotes "Bitcoin."
+
++a+:: Allows clients to filter out returned nodes based on the _types_ of addresses they advertise. As an example, this can be used to only obtain nodes that advertise a valid IPv6 address. The value that follows this type is based on a bitfield that _indexes_ into the set of specified address _types_ that are defined in https://github.com/lightningnetwork/lightning-rfc/blob/master/07-routing-gossip.md[BOLT #7]. The default value for this field is +6+, which represents both IPv4 and IPv6 (bits 1 and 2 are set).
+
++l+:: A valid node public key serialized in compressed format. This allows a client to query for a specified node rather than receiving a set of random nodes.
+
++n+:: The number of records to return. The default value for this field is +25+.
+
+An example query with additional query options looks something like the following:
+
+----
+r0.a2.n10.nodes.lightning.directory
+----
+
+Breaking down the query one key-value pair at a time, we gain the following
+insights:
+
++r0+:: The query targets the Bitcoin realm
++a2+:: The query only wants IPv4 addresses to be returned
++n10+:: The query requests
+
+Try some combinations of the various flags using the +dig+ DNS command-line tool yourself(((range="endofrange", startref="ix_11_gossip_channel_graph-asciidoc3")))(((range="endofrange", startref="ix_11_gossip_channel_graph-asciidoc2"))):
+
+----
+dig @8.8.8.8 r0.a6.nodes.lightning.directory SRV
+----
+
+=== The Channel Graph
+
+((("channel graph","structure of")))Now that our new node is able to use the DNS bootstrapping protocol to connect to its very first peer, it can start to sync the channel graph! However, before we sync the channel graph, we'll need to learn exactly _what_ we mean by the channel graph. In this section we'll explore the precise _structure_ of the channel graph and examine the unique aspects of the channel graph compared to the typical abstract "graph" data structure which is well-known/used in the field of computer science.
+
+==== A Directed Graph
+
+((("channel graph","directed graph")))((("directed graph")))A _graph_ in computer science is a special data structure composed of vertices (typically referred to as nodes) and edges (also known as links). Two nodes may be connected by one or more edges. The channel graph is also _directed_ given that a payment is able to flow in either direction over a given edge (a channel). An example of a _directed graph_ is shown in <<directed_graph>>.
+
+[[directed_graph]]
+.A directed graph 
+image::images/mtln_1102.png["A directed graph"]
+
+In the context of the Lightning Network, our vertices are the Lightning nodes themselves, with our edges being the payment channels connecting these nodes. Because we're concerned with _routing payments_, in our model a node with no edges (no payment channels) isn't considered to be a part of the graph since it isn't useful.
+
+Because channels themselves are UTXOs (funded 2-of-2 multisig addresses), we can view the channel graph as a special subset of the Bitcoin UTXO set, on top of which we can add some additional information (the nodes, etc.) to arrive at the final overlay structure, which is the channel graph. This anchoring of fundamental components of the channel graph in the
+base Bitcoin blockchain means that it's impossible to _fake_ a valid channel graph, which has useful properties when it comes to spam prevention as we'll see later.
+
+=== Gossip Protocol Messages
+
+((("channel graph","gossip protocol messages", id="ix_11_gossip_channel_graph-asciidoc7", range="startofrange")))((("gossip protocol","messages", id="ix_11_gossip_channel_graph-asciidoc8", range="startofrange")))The channel graph information is propagated across the Lightning P2P Network as three messages, which are described in https://github.com/lightningnetwork/lightning-rfc/blob/master/07-routing-gossip.md[BOLT #7]:
+
++node_announcement+:: The vertex in our graph which communicates the public key of a node, as well as how to reach the node over the internet and some additional metadata describing the set of _features_ the node supports.
+
++channel_announcement+:: A blockchain anchored proof of the existence of a channel between two individual nodes. Any third party can verify this proof to ensure that a _real_ channel is actually being advertised. Similar to the +node_announcement+, this message also contains information describing the _capabilities_ of the channel, which is useful when attempting to route a payment.
+
++channel_update+:: A _pair_ of structures that describes the set of routing policies for a given channel. +channel_update+ messages come in a _pair_ because a channel is a directed edge, so each side of the channel is able to specify its own custom routing policy.
+
+It's important to note that each component of the channel graph is _authenticated_, allowing a third party to ensure that the owner of a channel/update/node is actually the one sending out an update. This effectively makes the channel graph a unique type of _authenticated data structure_ that cannot be counterfeited. For authentication, we use an +secp256k1+ ECDSA digital signature (or a series of them) over the serialized digest of the message itself. We won't get into the specific of the messaging framing/serialization used in the Lightning Network in this chapter, as we'll cover that information in <<wire_protocol>>.
+
+With the high-level structure of the channel graph laid out, we'll now dive down into the precise structure of each of the three messages used to gossip the channel graph. We'll also explain how one can also verify each message and component of the channel graph.
+
+[[node_announcement]]
+==== The node_announcement Message
+
+((("gossip protocol","node_announcement message", id="ix_11_gossip_channel_graph-asciidoc9", range="startofrange")))((("node_announcement message", id="ix_11_gossip_channel_graph-asciidoc10", range="startofrange")))First, we have the +node_announcement+ message, which serves two primary
+purposes:
+
+ 1. To advertise connection information so other nodes can connect to a node either to bootstrap to the network or to attempt to establish a  new payment channel with that node.
+
+ 2. To communicate the set of protocol-level features (capabilities) a node understands/supports. Feature negotiation between nodes allows developers to add new features independently and support them with any other node on an opt-in basis.
+
+Unlike channel announcements, node announcements are not anchored in
+the base blockchain. Therefore, node announcements are
+only considered valid if they have propagated with a corresponding channel announcement. In other words, we always reject nodes without payment channels to ensure a malicious peer can't flood the network with bogus nodes that are not part of the channel graph.
+
+===== The node_announcement message structure
+
+((("node_announcement message","structure")))The +node_announcement+ is comprised of
+the following fields:
+
+ +signature+:: A valid ECDSA signature that covers the serialized digest of all fields listed below. This signature must correspond to the public key of the advertised node.
+
+ +features+:: A bit vector that describes the set of protocol features that this node understands. We'll cover this field in more detail in <<feature_bits>> on the extensibility of the Lightning protocol. At a high level, this field carries a set of bits that represent the features a node understands. As an example, a node may signal that it understands the latest channel type.
+
++timestamp+:: A Unix epoch encoded timestamp. This allows clients to enforce a partial ordering over the updates to a node's announcement.
+
++node_id+:: The +secp256k1+ public key that this node announcement belongs to. There can only be a single +node_announcement+ for a given node in the channel graph at any given time. As a result, a +node_announcement+ can supersede a prior +node_announcement+ for the same node if it carries a higher (later) timestamp.
+
++rgb_color+:: A field that allows a node to specify an RGB color to be associated with it, often used in channel graph visualizations and node directories.
+
++alias+:: A UTF-8 string to serve as the nickname for a given node. Note that these aliases aren't required to be globally unique, nor are they verified in any way. As a result, they should not be relied on as a form of identity—they can be easily spoofed.
+
++addresses+:: A set of public internet reachable addresses that are to be associated with a given node. In the current version of the protocol, four address types are supported: IPv4 (type: 1), IPv6 (type: 2), Tor v2 (type: 3), and Tor v3 (type: 4). In the +node_announcement+ message, each of these address types is denoted by an integer type which is included in parenthesis after the address type.
+
+===== Validating node announcements
+
+((("node_announcement message","validating")))Validating an incoming +node_announcement+ is straightforward. The following assertions should be upheld when examining a node announcement:
+
+  * If an existing +node_announcement+ for that node is already known, then the +timestamp+ field of a new incoming +node_announcement+ must be greater than the prior one.
+
+    * With this constraint, we enforce a forced level of "freshness."
+
+  * If no +node_announcement+ exists for the given node, then an existing +channel_announcement+ that references the given node (more on that later) must already exist in one's local channel graph.
+
+  * The included +signature+ must be a valid ECDSA signature verified using the included +node_id+ public key and the double–SHA-256 digest of the raw message encoding (minus the signature and frame header) as the message.
+
+  * All included +addresses+ must be sorted in ascending order based on their address identifier.
+
+  * The included +alias+ bytes must be a valid UTF-8 string.(((range="endofrange", startref="ix_11_gossip_channel_graph-asciidoc10")))(((range="endofrange", startref="ix_11_gossip_channel_graph-asciidoc9")))
+
+==== The channel_announcement Message
+
+((("channel_announcement message", id="ix_11_gossip_channel_graph-asciidoc11", range="startofrange")))((("gossip protocol","channel_announcement message", id="ix_11_gossip_channel_graph-asciidoc12", range="startofrange")))Next, we have the +channel_announcement+ message, which is used to _announce_ a new _public_ channel to the wider network. Note that announcing a channel is _optional_. A channel only needs to be announced if it is intended to be used for routing by the Lightning Network. Active routing nodes may wish to announce all their channels. However, certain nodes like mobile nodes likely don't have the
+uptime or desire to be an active routing node. As a result, these
+mobile nodes (which typically use light clients to connect to the Bitcoin P2P network) instead may have purely _unannounced_ (private) channels.
+
+===== Unannounced (private) channels
+
+((("channel_announcement message","unannounced (private) channels")))((("unannounced channels")))An unannounced channel isn't part of the known public channel graph, but can still be used to send/receive payments. An astute reader may now be wondering how a channel which isn't part of the public channel graph is able to receive payments. The solution to this problem is a set of "pathfinding helpers" that we call routing hints. As we'll see in <<invoices>>, invoices created by nodes with unadvertised channels will include information to help the sender route to them, assuming the node has at least a single channel with an existing public routing node.
+
+Due to the existence of unadvertised channels, the _true_ size of the channel graph (both the public and private components) is unknown.
+
+===== Locating a channel on the bitcoin blockchain
+
+((("blockchain","locating a channel on the Bitcoin blockchain")))((("channel_announcement message","locating a channel on the Bitcoin blockchain")))As mentioned earlier, the channel graph is authenticated due to its usage of public key cryptography, as well as the Bitcoin blockchain as a spam prevention system. To have a node accept a new +channel_announcement+, the advertisement must _prove_ that the channel actually exists in the Bitcoin blockchain. This proof system adds an up-front cost to adding a new entry to the channel graph (the on-chain fees one must pay to create the UTXO of the channel). As a result, we mitigate spam and ensure that a dishonest node on the network can't fill up the memory of an honest node at no cost with bogus channels.
+
+Given that we need to construct a proof of the existence of a channel, a
+natural question that arises is: how do we "point to" or reference a given channel for the verifier? Given that a payment channel is anchored in an unspent transaction output (see <<utxo>>), an initial thought might be to first attempt to advertise the full outpoint (+txid:index+) of the channel. Given that the outpoint is globally unique and confirmed in the chain, this sounds like a good idea; however, it has a drawback: the verifier must maintain a full copy of the UTXO set to verify channels. This works fine for Bitcoin full nodes, but clients that rely on lightweight verification don't typically maintain a full UTXO set. Because we want to ensure we can support mobile nodes in the Lightning Network, we're forced to find another solution.
+
+What if rather than referencing a channel by its UTXO, we reference it based on its "location" in the chain? To do this, we'll need a scheme that allows us to reference a given block, then a transaction within that block, and finally a specific output created by that transaction. Such an identifier is described in https://github.com/lightningnetwork/lightning-rfc/blob/master/07-routing-gossip.md[BOLT #7] and is referred to as a _short channel ID_, or +scid+.
+The +scid+ is used in +channel_announcement+ (and +channel_update+) as well as within the onion-encrypted routing packet included within HTLCs, as we learned in <<onion_routing>>.
+
+[[short_channel_id]]
+[[scid]]
+===== The short channel ID
+
+((("blockchain","short channel ID")))Based on the preceding information, we have three pieces of information we need to encode to uniquely reference a given channel. Because we want a compact representation, we'll attempt to encode the information into a _single_ integer. Our integer format of choice is an unsigned 64-bit integer, comprised of 8 bytes.
+
+First, the block height. Using 3 bytes (24 bits) we can encode 16,777,216 blocks. That leaves 5 bytes for us to encode the transaction index and the output index, respectively. We'll use the next 3
+bytes to encode the transaction index _within_ a block. This is more than enough given that it's only possible to fix tens of thousands of transactions in a block at current block sizes. This leaves 2 bytes left for us to encode the output index of the channel within the transaction.
+
+Our final +scid+ format resembles:
+----
+block_height (3 bytes) || transaction_index (3 bytes) || output_index (2 bytes)
+----
+
+Using bit packing techniques, we first encode the most significant 3 bytes as the block height, the next 3 bytes as the transaction index, and the least significant 2 bytes as the output index of that creates the channel UTXO.
+
+A short channel ID can be represented as a single integer
+(+695313561322258433+) or as a more human friendly string: +632384x1568x1+. Here we see the channel was mined in block +632384+, was the ++1568++th transaction in the block, with the channel output as the second (UTXOs are zero-indexed) output produced by the transaction.
+
+Now that we're able to succinctly point to a given channel funding output in the chain, we can examine the full structure of the +channel_announcement+ message, as well as see how to verify the proof-of-existence included within the message.
+
+===== The channel_announcement message structure
+
+((("channel_announcement message","message structure")))A +channel_announcement+ primarily communicates two things:
+
+ 1. A proof that a channel exists between node A and node B with both nodes controlling the mulitsig keys in that channel output.
+
+ 2. The set of capabilities of the channel (what types of HTLCs can it route, etc.).
+
+When describing the proof, we'll typically refer to node +1+ and node +2+. Out of the two nodes that a channel connects, the "first" node is the node that has a "lower" public key encoding when we compare the public key of the two nodes in compressed format hex-encoded in lexicographical order. Correspondingly, in addition to a node public key on the network, each node should also control a public key within the Bitcoin blockchain.
+
+Similar to the +node_announcement+ message, all included signatures of the +channel_announcement+ message should be signed/verified against the raw encoding of the message (minus the header) that follows _after_ the final signature (because it isn't possible for a digital signature to sign itself).
+
+With that said, a +channel_announcement+ message has the following fields:
+
++node_signature_1+:: The signature of the first node over the message digest.
+
++node_signature_2+:: The signature of the second node over the message digest.
+
++bitcoin_signature_1+:: The signature of the multisig key (in the funding output) of the first node over the message digest.
+
++bitcoin_signature_2+::  The signature of the multisig key (in the funding output) of the second node over the message digest.
+
++features+:: A feature bit vector that describes the set of protocol level features supported by this channel.
+
++chain_hash+:: A 32-byte hash which is typically the genesis block hash of the blockchain (e.g., Bitcoin mainnet) the channel was opened in.
+
++short_channel_id+:: The +scid+ that uniquely locates the given channel funding output within the blockchain.
+
++node_id_1+:: The public key of the first node in the network.
+
++node_id_2+:: The public key of the second node in the network.
+
++bitcoin_key_1+:: The raw multisig key for the channel funding output for the first node in the network.
+
++bitcoin_key_2+:: The raw multisig key for the channel funding output for the second node in the network.
+
+===== Channel announcement validation
+
+((("channel_announcement message","validation")))Now that we know what a +channel_announcement+ contains, we can look at how to verify the channel's existence on-chain.
+
+Armed with the information in the +channel_announcement+, any Lightning node (even one without a full copy of the Bitcoin blockchain) can verify the existence and authenticity of the payment channel.
+
+First, the verifier will use the short channel ID to find which Bitcoin block contains the channel funding output. With the block height information, the verifier can request only that specific block from a Bitcoin node. The block can then be linked back to the genesis block by following the block header chain backward (verifying the proof-of-work), confirming that this is in fact a block belonging to the Bitcoin blockchain.
+
+Next, the verifier uses the transaction index number to identify the transaction ID of the transaction containing the payment channel. Most modern Bitcoin libraries will allow indexing into the transaction of a block based on the index of the transaction within the greater block.
+
+Next, the verifier uses a Bitcoin library (in the verifier's language) to extract the relevant transaction according to its index within the block. The verifier will validate the transaction (checking that it is properly signed and produces the same transaction ID when hashed).
+
+Next, the verifier will extract the Pay-to-Witness-Script-Hash (P2WSH) output referenced by the output index number of the short channel ID. This is the address of the channel funding output. Additionally, the verifier will ensure that the size of the alleged channel matches the value of the output produced at the specified output index.
+
+Finally, the verifier will reconstruct the multisig script from +bitcoin_key_1+ and +bitcoin_key_2+ and confirm that it produces the same address as in the output.
+
+The verifier has now independently verified that the payment channel in the announcement is funded and confirmed on the Bitcoin blockchain!(((range="endofrange", startref="ix_11_gossip_channel_graph-asciidoc12")))(((range="endofrange", startref="ix_11_gossip_channel_graph-asciidoc11")))
+
+==== The channel_update Message
+
+((("channel_update message")))((("gossip protocol","channel_update message")))The third and final message used in the gossip protocol is the +channel_update+ message. Two of these are generated for each payment channel (one by each channel partner) announcing their routing fees, timelock expectations, and capabilities.
+
+The +channel_update+ message also contains a timestamp, allowing a node to update its routing fees and other expectations and capabilities by sending a new +channel_update+ message with a higher (later) timestamp that supersedes any older updates.
+
+The +channel_update+ message contains the following fields:
+
+
++signature+:: A digital signature matching the node's public key, to authenticate the source and integrity of the channel update
+
++chain_hash+:: The hash of the genesis block of the chain containing the channel
+
++short_channel_id+:: The short channel ID to identify the channel
+
++timestamp+:: The timestamp of this update, to allow recipients to sequence updates and replace older updates
+
++message_flags+:: A bit field indicating the presence of additional fields in the +channel_update+ message
+
++channel_flags+:: A bit field showing the direction of the channel and other channel options
+
++cltv_expiry_delta+:: The timelock delta expectations of this node for routing (see <<onion_routing>>)
+
++htlc_minimum_msat+:: The minimum HTLC amount that will be routed
+
++fee_base_msat+:: The base fee that will be charged for routing
+
++fee_proportional_millionths+:: The proportional fee rate that will be charged for routing
+
++htlc_maximum_msat+ (+option_channel_htlc_max+):: The maximum amount that will be routed
+
+A node that receives the +channel_update+ message can attach this metadata to the channel graph edge to enable pathfinding,  as we will see in <<path_finding>>.(((range="endofrange", startref="ix_11_gossip_channel_graph-asciidoc8")))(((range="endofrange", startref="ix_11_gossip_channel_graph-asciidoc7")))
+
+=== Ongoing Channel Graph Maintenance
+
+((("channel graph","ongoing maintenance")))The construction of a channel graph is not a one-time event, but rather an ongoing activity. As a node bootstraps into the network it will start receiving "gossip," in the form of the three update messages. It will use these messages to immediately start building a validated channel graph.
+
+The more information a node receives, the better its "map" of the Lightning Network becomes and the more effective it can be at pathfinding and payment delivery.
+
+A node won't only add information to the channel graph. It will also keep track of the last time a channel was updated and will delete "stale" channels that have not been updated in more than two weeks. Finally, if it sees that some node no longer has any channels, it will also remove that node.
+
+The information collected from the gossip protocol is not the only information that can be stored in the channel graph. Different Lightning node implementations may attach other metadata to nodes and channels. For example, some node implementations calculate a "score" that evaluates a node's "quality" as a routing peer. This score is used as part of pathfinding to prioritize or deprioritize paths.
+
+=== Conclusion
+In this chapter, we've learned how Lightning nodes discover each
+other, discover and update their node status, and communicate with one another. We've learned how channel graphs are created and maintained, and we've explored a few ways that the Lightning Network discourages bad actors or dishonest nodes from spamming the network.

12_path_finding.asciidoc

@@ -0,0 +1,455 @@
+[[path_finding]]
+== Pathfinding and Payment Delivery
+
+((("pathfinding", id="ix_12_path_finding-asciidoc0", range="startofrange")))Payment ((("payment delivery", id="ix_12_path_finding-asciidoc1", range="startofrange")))delivery on the Lightning Network depends on finding a path from the sender to the recipient, a process called _pathfinding_. Since the routing is done by the sender, the sender must find a suitable path to reach the destination. This path is then encoded in an onion, as we saw in <<onion_routing>>.
+
+In this chapter we will examine the problem of pathfinding, understand how uncertainty about channel balances complicates this problem, and look at how a typical pathfinding implementation attempts to solve it.
+
+=== Pathfinding in the Lightning Protocol Suite
+
+((("Lightning Network Protocol","pathfinding in")))((("pathfinding","Lightning Protocol Suite and")))Pathfinding, path selection, multipart payments (MPP), and the payment attempt trial-and-error loop occupy the majority of the payment layer at the top of the protocol suite.
+
+These components are highlighted by an outline in the protocol suite, shown in <<LN_protocol_pathfinding_highlight>>.
+
+[[LN_protocol_pathfinding_highlight]]
+.Payment delivery in the Lightning protocol suite
+image::images/mtln_1201.png["Payment delivery in the Lightning protocol suite"]
+
+==== Where Is the BOLT?
+
+((("BOLT (Basis of Lightning Technology) standards documents","pathfinding and")))((("pathfinding","BOLT standard and")))So far we've looked at several technologies that are part of the Lightning Network and we have seen their exact specification as part of a BOLT standard. You may be surprised to find that pathfinding is not part of the BOLTs!
+
+That's because pathfinding isn't an activity that requires any form of coordination or interoperability between different implementations. As we've seen, the path is selected by the sender. Even though the routing details are specified in detail in the BOLTs, the path discovery and selection are left entirely up to the sender. So each node implementation can choose a different strategy/algorithm to find paths. In fact, the different node/client and wallet implementations can even compete and use their pathfinding algorithm as a point of differentiation.
+
+=== Pathfinding: What Problem Are We Solving?
+
+((("pathfinding","nature of problem solved by", id="ix_12_path_finding-asciidoc2", range="startofrange")))The term pathfinding may be somewhat misleading because it implies a search for _a single path_ connecting two nodes. In the beginning, when the Lightning Network was small and not well interconnected, the problem was indeed about finding a way to join payment channels to reach the recipient.
+
+But, as the Lightning Network has grown explosively, the pathfinding problem's nature has shifted. In mid-2021, as we finish this book, the Lightning Network consists of 20,000 nodes connected by at least 55,000 public channels with an aggregate capacity of almost 2,000 BTC. A node has on average 8.8 channels, while the top 10 most connected nodes have between 400 and 2,000 channels _each_. A visualization of just a small subset of the LN channel graph is shown in <<lngraph>>.
+
+[[lngraph]]
+.A visualization of part of the Lightning Network as of July 2021
+image::images/mtln_1202.png[]
+
+[NOTE]
+====
+The network visualization in <<lngraph>> was produced with a simple Python script you can find in code/lngraph in the book's repository.
+====
+
+If the sender and recipient are connected to other well-connected nodes and have at least one channel with adequate capacity, there will be thousands of paths. The problem becomes selecting the _best_ path that will succeed in payment delivery, out of a list of thousands of possible paths.
+
+==== Selecting the Best Path
+
+((("pathfinding","selecting the best path")))To select the best path, we have to first define what we mean by "best." There may be many different criteria, such as:
+
+* Paths with enough liquidity. Obviously if a path doesn't have enough liquidity to route our payment, then it is not a suitable path.
+
+* Paths with low fees. If we have several candidates, we may want to select ones with lower fees.
+
+* Paths with short timelocks. We may want to avoid locking our funds for too long and therefore select paths with shorter timelocks.
+
+All of these criteria may be desirable to some extent, and selecting paths that are favorable across many dimensions is not an easy task. Optimization problems like this may be too complex to solve for the "best" solution, but often can be solved for some approximation of the optimal, which is good news because otherwise pathfinding would be an intractable problem.
+
+
+==== Pathfinding in Math and Computer Science
+
+((("pathfinding","math and computer science")))Pathfinding in the Lightning Network falls under a general category of _graph theory_ in mathematics and the more specific category of _graph traversal_ in computer science.
+
+A network such as the Lightning Network can be represented as a mathematical construct called a _graph_, where _nodes_ are connected to each other by _edges_ (equivalent to the payment channels). ((("directed graph")))The Lightning Network forms a _directed graph_ because the nodes are linked _asymmetrically_, since the channel balance is split between the two channel partners and the payment liquidity is different in each direction. ((("flow network")))A directed graph with numerical capacity constraints on its edges is called a _flow network_, a mathematical construct used to optimize transportation and other similar networks. Flow networks can be used as a framework when solutions need to achieve a specific flow while minimizing cost, known as the minimum cost flow problem (MCFP).
+
+==== Capacity, Balance, Liquidity
+
+((("pathfinding","capacity, balance, and liquidity")))To better understand the problem of transporting satoshis from point A to point B, we need to better define three important terms: capacity, balance, and liquidity. We use these terms to describe a payment channel's ability to route a payment.
+
+In a payment channel connecting A<-->B:
+
+Capacity:: ((("capacity, payment channel")))This is the aggregate amount of satoshis that were funded into the 2-of-2 multisig with the funding transaction. It represents the maximum amount of value held in the channel. The channel capacity is announced by the gossip protocol and is known to nodes.
+
+Balance:: ((("balance, in payment channel")))This is the amount of satoshis held by each channel partner that can be sent to the other channel partner. A subset of the balance of A can be sent in the direction (A->B) toward node B. A subset of the balance of B can be sent in the opposite direction (A<-B).
+
+Liquidity:: ((("liquidity","in payment channel")))The available (subset) balance that can actually be sent across the channel in one direction. Liquidity of A is equal to the balance of A minus the channel reserve and any pending HTLCs committed by A.
+
+The only value known to the network (via gossip announcements) is the aggregate capacity of the channel. Some unknown portion of that capacity is distributed as each partner's balance. Some subset of that balance is available to send across the channel in one direction:
+
+++++
+<ul class="simplelist">
+<li>capacity = balance(A) + balance(B)</li>
+<li>liquidity(A) = balance(A) – channel_reserve(A) – pending_HTLCs(A)</li>
+</ul>
+++++
+
+==== Uncertainty of Balances
+
+((("pathfinding","uncertainty of balances")))If we knew the exact channel balances of every channel, we could compute one or more payment paths using any of the standard pathfinding algorithms taught in good computer science programs. But we don't know the channel balances; we only know the aggregate channel capacity, which is advertised by nodes in channel announcements. In order for a payment to succeed, there must be adequate balance on the sending side of the channel. If we don't know how the capacity is distributed between the channel partners, we don't know if there is enough balance in the direction we are trying to send the payment.
+
+Balances are not announced in channel updates for two reasons: privacy and scalability. First, announcing balances would reduce the privacy of the Lightning Network because it would allow surveillance of payment by statistical analysis of the changes in balances. Second, if nodes announced balances (globally) with every payment, the Lightning Network's scaling would be as bad as that of on-chain Bitcoin transactions which are broadcast to all participants. Therefore, balances are not announced. To solve the pathfinding problem in the face of uncertainty of balances, we need innovative pathfinding strategies. These strategies must relate closely to the routing algorithm that is used, which is source-based onion routing where it is the responsibility of the sender to find a path through the network.
+
+((("range of liquidity")))The uncertainty problem can be described mathematically as a _range of liquidity_, indicating the lower and upper bounds of liquidity based on the information that is known. Since we know the capacity of the channel and we know the channel reserve balance (the minimum allowed balance on each end), the liquidity can be defined as:
+
+++++
+<ul class="simplelist">
+<li>min(liquidity) = channel_reserve</li>
+<li>max(liquidity) = capacity – channel_reserve</li>
+</ul>
+++++
+
+[role="pagebreak-before"]
+or as a range:
+
+++++
+<ul class="simplelist">
+<li>channel_reserve &lt;= liquidity &lt;= (capacity – channel_reserve)</li>
+</ul>
+++++
+
+Our channel liquidity uncertainty range is the range between the minimum and maximum possible liquidity. This is unknown to the network, except the two channel partners. However, as we will see, we can use failed HTLCs returned from our payment attempts to update our liquidity estimate and reduce uncertainty. If, for example, we get an HTLC failure code that tells us that a channel cannot fulfill an HTLC that is smaller than our estimate for maximum liquidity, that means the maximum liquidity can be updated to the amount of the failed HTLC. In simpler terms, if we think the liquidity can handle an HTLC of _N_ satoshis and we find out it fails to deliver _M_ satoshis (where _M_ is smaller), then we can update our estimate to __M__–1 as the upper bound. We tried to find the ceiling and bumped against it, so it's lower than we thought!
+
+==== Pathfinding Complexity
+
+((("pathfinding","complexity")))Finding a path through a graph is a problem modern computers can solve rather efficiently.
+Developers mainly choose breadth-first search if the edges are all of equal weight.
+In cases where the edges are not of equal weight, an algorithm based on ((("Dijkstra&apos;s algorithm")))Dijkstra's algorithm is used, such as https://en.wikipedia.org/wiki/A*_search_algorithm[A* (pronounced "A-star")].
+In our case the weights of the edges can represent the routing fees.
+Only edges with a capacity larger than the amount to be sent will be included in the search.
+In this basic form, pathfinding in the Lightning Network is very simple and straightforward.
+
+However, channel liquidity is unknown to the sender. This turns our easy theoretical computer science problem into a rather complex real-world problem.
+We now have to solve a pathfinding problem with only partial knowledge.
+For example, we suspect which edges might be able to forward a payment because their capacity seems big enough.
+But we can't be certain unless we try it out or ask the channel owners directly.
+Even if we were able to ask the channel owners directly, their balance might change by the time we have asked others, computed a path, constructed an onion, and sent it along.
+Not only do we have limited information but the information we have is highly dynamic and might change at any point in time without our knowledge.
+
+==== Keeping It Simple
+
+((("pathfinding","simplicity")))The pathfinding mechanism implemented in Lightning nodes is to first create a list of candidate paths, filtered and sorted by some function. Then, the node or wallet will probe paths (by attempting to deliver a payment) in a trial-and-error loop until a path is found that successfully delivers the payment.
+
+[NOTE]
+====
+This probing is done by the Lightning node or wallet and is not directly observed by the user of the software.
+However, the user might suspect that probing is taking place if the payment is not completed instantly.
+====
+
+While blind probing is not optimal and leaves ample room for improvement, it should be noted that even this simplistic strategy works surprisingly well for smaller payments and well-connected nodes.
+
+Most Lightning node and wallet implementations improve on this approach by ordering/weighting the list of candidate paths. Some implementations order the candidate paths by cost (fees) or some combination of cost and capacity.(((range="endofrange", startref="ix_12_path_finding-asciidoc2")))
+
+=== Pathfinding and Payment Delivery Process
+
+((("pathfinding","payment delivery process")))((("payment delivery","pathfinding and delivery process")))Pathfinding and payment delivery involves several steps, which we list here. Different implementations may use different algorithms and strategies, but the basic steps are likely to be very similar:
+
+. Create a _channel graph_ from announcements and updates containing the capacity of each channel, and filter the graph, ignoring any channels with insufficient capacity for the amount we want to send.
+
+. Find paths connecting the sender to the recipient.
+
+. Order the paths by some weight (this may be part of the previous step's pass:[<span class="keep-together">algorithm</span>]).
+
+. Try each path in order until payment succeeds (the trial-and-error loop).
+
+. Optionally use the HTLC failure returns to update our graph, reducing pass:[<span class="keep-together">uncertainty</span>].
+
+We can group these steps into three primary activities:
+
+* Channel graph construction
+* Pathfinding (filtered and ordered by some heuristics)
+* Payment attempt(s)
+
+These three activities can be repeated in a _payment round_ if we use the failure returns to update the graph, or if we are doing multipart payments (see <<mpp>>).
+
+In the next sections we will look at each of these steps in more detail, as well as more advanced payment strategies.
+
+=== Channel Graph Construction
+
+((("channel graph","construction of", id="ix_12_path_finding-asciidoc3", range="startofrange")))((("pathfinding","channel graph construction", id="ix_12_path_finding-asciidoc4", range="startofrange")))In <<gossip>> we covered the three main messages that nodes use in their gossip: +node_announcement+, +channel_announcement+, and +channel_update+. These three messages allow any node to gradually construct a "map" of the Lightning Network in the form of a _channel graph_. Each of these messages provides a critical piece of information for the channel graph:
+
++node_announcement+:: ((("node_announcement message")))This contains the information about a node on the Lightning Network, such as its node ID (public key), network address (e.g., IPv4/6 or Tor), capabilities/features, etc.
+
++channel_announcement+:: ((("channel_announcement message","channel graph and")))((("channel_update message")))This contains the capacity and channel ID of a public (announced) channel between two nodes and proof of the channel's existence and ownership.
+
++channel_update+:: This contains a node's fee and timelock (CLTV) expectations for routing an outgoing (from that node's perspective) payment over a specified channel.
+
+In terms of a mathematical graph, the +node_announcement+ is the information needed to create the nodes or _vertices_ of the graph. The +channel_announcement+ allows us to create the _edges_ of the graph representing the payment channels. Since each direction of the payment channel has its own balance, we create a directed graph. The +channel_update+ allows us to incorporate fees and timelocks to set the _cost_ or _weight_ of the graph edges.
+
+Depending on the algorithm we will use for pathfinding, we may establish a number of different cost functions for the edges of the graph.
+
+For now, let's ignore the cost function and simply establish a channel graph showing nodes and channels, using the +node_announcement+ and +channel_announcement+ messages.
+
+In this chapter we will see how Selena attempts to find a path to pay Rashid one million satoshis. To start, Selena is constructing a channel graph using the information from Lightning Network gossip to discover nodes and channels. Selena will then explore her channel graph to find a path to send a payment to Rashid.
+
+This is _Selena's_ channel graph. There is no such thing as _the_ channel graph, there is only ever _a channel graph_, and it is always from the perspective of the node that has constructed it (see <<map_territory_relation>>).
+
+[TIP]
+====
+Selena does not contruct a channel graph only when sending a payment. Rather, Selena's node is _continuously_ building and updating a channel graph. From the moment Selena's node starts and connects to any peer on the network it will participate in the gossip and use every message to learn as much as possible about the network.
+====
+
+[[map_territory_relation]]
+.The Map-Territory Relation
+****
+((("channel graph","map–territory relation")))From Wikipedia's https://en.wikipedia.org/wiki/Map%E2%80%93territory_relation[page on the Map-Territory Relation], "The map-territory relation describes the relationship between an object and a representation of that object, as in the relation between a geographical territory and a map of it."
+
+The map-territory relation is best illustrated in "Sylvie and Bruno Concluded," a short story by Lewis Carroll which describes a fictional map that is a 1:1 scale of the territory it maps, therefore having perfect accuracy but becoming completely useless as it would cover the entire territory if unfolded.
+
+What does this mean for the Lightning Network? The Lightning Network is the territory, and a channel graph is a map of that territory.
+
+While we could imagine a theoretical (Platonic ideal) channel graph that represents the complete, up-to-date map of the Lightning Network, such a map is simply the Lightning Network itself. Each node has its own channel graph which is constructed from announcements and is necessarily incomplete, incorrect, and out-of-date!
+
+The map can never completely and accurately describe the territory.
+****
+
+Selena listens to +node_announcement+ messages and discovers four other nodes (in addition to Rashid, the intended recipient). The resulting graph represents a network of six nodes: Selena and Rashid are the sender and recipient, respectively; Alice, Bob, Xavier, and Yan are intermediary nodes. Selena's initial graph is just a list of nodes, shown in <<channel_graph_nodes>>.
+
+[[channel_graph_nodes]]
+.Node announcements
+image::images/mtln_1203.png[]
+
+Selena also receives seven +channel_announcement+ messages with the corresponding channel capacities, allowing her to construct a basic "map" of the network, shown in <<channel_graph_1>>. (The names Alice, Bob, Selena, Xavier, Yan, and Rashid have been replaced by their initials: A, B, S, X, and R, respectively.)
+
+[[channel_graph_1]]
+.The channel graph
+image::images/mtln_1204.png[]
+
+===== Uncertainty in the channel graph
+
+((("channel graph","uncertainty in")))As you can see from <<channel_graph_1>>, Selena does not know any of the balances of the channels. Her initial channel graph contains the highest level of uncertainty.
+
+But wait: Selena does know _some_ channel balances! She knows the balances of the channels that her own node has connected with other nodes. While this does not seem like much, it is in fact very important information for constructing a path—Selena knows the actual liquidity of her own channels. Let's update the channel graph to show this information. We will use a "?" symbol to represent the unknown balances, as shown in <<channel_graph_2>>.
+
+[[channel_graph_2]]
+.Channel graph with known and unknown balances
+image::images/mtln_1205.png[]
+
+While the "?" symbol seems ominous, a lack of certainty is not the same as complete ignorance. We can _quantify_ the uncertainty and _reduce_ it by updating the graph with the successful/failed HTLCs we attempt.
+
+Uncertainty can be quantified, because we know the maximum and minimum possible liquidity and can calculate probabilities for smaller (more precise) ranges.
+
+Once we attempt to send an HTLC, we can learn more about channel balances: if we succeed, then the balance was _at least_ sufficient to transport the specific amount. Meanwhile if we get a "temporary channel failure" error, the most likely reason is a lack of liquidity for the specific amount.
+
+[TIP]
+====
+You may be thinking, "What's the point of learning from a successful HTLC?" After all, if it succeeded we're "done." But consider that we may be sending one part of a multipart payment. We also may be sending other single-part payments within a short time. Anything we learn about liquidity is useful for the next attempt!
+====
+
+==== Liquidity Uncertainty and Probability
+
+((("channel graph","liquidity uncertainty and probability")))((("liquidity","uncertainty and probability")))To quantify the uncertainty of a channel's liquidity, we can apply probability theory. A basic model of the probability of payment delivery will lead to some rather obvious, but important, conclusions:
+
+* Smaller payments have a better chance of successful delivery across a path.
+
+* Larger capacity channels will give us a better chance of payment delivery for a specific amount.
+
+* The more channels (hops), the lower the chance of success.
+
+While these may be obvious, they have important implications, especially for the use of multipart payments (see <<mpp>>). The math is not difficult to follow.
+
+Let's use probability theory to see how we arrived at these conclusions.
+
+First, let's posit that a channel with capacity _c_ has liquidity on one side with an unknown value in the range of (0, _c_) or "range between 0 and _c_." For example, if the capacity is 5, then the liquidity will be in the range (0, 5). Now, from this we see that if we want to send 5 satoshis, our chance of success is only 1 in 6 (16.66%), because we will only succeed if the liquidity is exactly 5.
+
+More simply, if the possible values for the liquidity are 0, 1, 2, 3, 4, and 5, only one of those six possible values will be sufficient to send our payment. To continue this example, if our payment amount was 3, then we would succeed if the liquidity was 3, 4, or 5. So our chances of success are 3 in 6 (50%). Expressed in math, the success probability function for a single channel is:
+
+[latexmath]
+++++
+$P_c(a) = (c + 1 - a) / (c + 1)$
+++++
+
+where _a_ is the amount and _c_ is the capacity.
+
+From the equation we see that if the amount is close to 0, the probability is close to 1, whereas if the amount exceeds the capacity, the probability is zero.
+
+In other words: "Smaller payments have a better chance of successful delivery" or "Larger capacity channels give us better chances of delivery for a specific amount" and "You can't send a payment on a channel with insufficient capacity."
+
+Now let's think about the probability of success across a path made of several channels. Let's say our first channel has a 50% chance of success (_P_ = 0.5). Then if our second channel has a 50% chance of success (_P_ = 0.5), it is intuitive that our overall chance is 25% (_P_ = 0.25).
+
+We can express this as an equation that calculates the probability of a payment's success as the product of probabilities for each channel in the path(s):
+
+[latexmath]
+++++
+$P_{payment} = \prod_{i=1}^n P_i$
+++++
+
+Where __P__~__i__~ is the probability of success over one path or channel, and __P__~__payment__~ is the overall probability of a successful payment over all the paths/channels.
+
+From the equation we see that since the probability of success over a single channel is always less than or equal to 1, the  probability across many channels will _drop exponentially_.
+
+In other words, "The more channels (hops) you use, the lower the chance of success."
+
+[NOTE]
+====
+There is a lot of mathematical theory and modeling behind the uncertainty of the liquidity in the channels. Fundamental work about modeling the uncertainty intervals of the channel liquidity can be found in the paper https://arxiv.org/abs/2103.08576["Security and Privacy of Lightning Network Payments with Uncertain Channel Balances"] by (coauthor of this book) Pickhardt pass:[<span class="keep-together">et al</span>].
+====
+
+==== Fees and Other Channel Metrics
+
+((("channel graph","fees and other channel metrics", id="ix_12_path_finding-asciidoc5", range="startofrange")))((("fees","channel graph and", id="ix_12_path_finding-asciidoc6", range="startofrange")))Next, our sender will add information to the graph from +channel_update+ messages received from the intermediary nodes. As a reminder, the +channel_update+ contains a wealth of information about a channel and the expectations of one of the channel partners.
+
+In <<channel_graph_3>> we see how Selena can update the channel graph based on +channel_update+ messages from A, B, X, and Y. Note that the channel ID and channel direction (included in +channel_flags+) tell Selena which channel and which direction this update refers to. Each channel partner gossips one or more +channel_update+ messages to announce their fee expectations and other information about the channel. For example, in the top left we see the +channel_update+ sent by Alice for the channel A--B and the direction A-to-B. With this update, Alice tells the network how much she will charge in fees to route an HTLC to Bob over that specific channel. Bob may announce a channel update (not shown in this diagram) for the opposite direction with completely different fee expectations. Any node may send a new +channel_update+ to change the fees or timelock expectations at any time.
+
+[[channel_graph_3]]
+.Channel graph fees and other channel metrics
+image::images/mtln_1206.png[]
+
+The fee and timelock information are very important, not just as path selection metrics. As we saw in <<onion_routing>>, the sender needs to add up fees and timelocks (+cltv_expiry_delta+) at each hop to make the onion. The process of calculating fees happens from the recipient to the sender _backward_ along the path because each intermediary hop expects an incoming HTLC with higher amount and expiry timelock than the outgoing HTLC they will send to the next hop. So, for example, if Bob wants 1,000 satoshis in fees and 30 blocks of expiry timelock delta to send a payment to Rashid, then that amount and expiry delta must be added to the HTLC _from Alice_.
+
+It is also important to note that a channel must have liquidity that is sufficient not only for the payment amount but also for the cumulative fees of all the subsequent hops. Even though Selena's channel to Xavier (S->X) has enough liquidity for a 1M satoshi payment, it _does not_ have enough liquidity once we consider fees. We need to know fees because only paths that have sufficient liquidity for _both payment and all fees_ will be considered(((range="endofrange", startref="ix_12_path_finding-asciidoc6")))(((range="endofrange", startref="ix_12_path_finding-asciidoc5"))).(((range="endofrange", startref="ix_12_path_finding-asciidoc4")))(((range="endofrange", startref="ix_12_path_finding-asciidoc3")))
+
+=== Finding Candidate Paths
+
+((("pathfinding","finding candidate paths")))Finding a suitable path through a directed graph like this is a well-studied computer science problem (known broadly as the _shortest path problem_), which can be solved by a variety of algorithms depending on the desired optimization and resource constraints.
+
+((("Dijkstra&apos;s algorithm")))The most famous algorithm solving this problem was invented by Dutch mathematician E. W. Dijkstra in 1956, known simply as https://en.wikipedia.org/wiki/Dijkstra's_algorithm[_Dijkstra's algorithm_]. In addition to the original Dijkstra's algorithm, there are many variations and optimizations, such as https://en.wikipedia.org/wiki/A*_search_algorithm[A* ("A-star")], which is a heuristic-based algorithm.
+
+As mentioned previously, the "search" must be applied _backward_ to account for fees that are accumulated from recipient to sender. Thus, Dijkstra, A*, or some other algorithm would search for a path from the recipient to the sender, using fees, estimated liquidity, and timelock delta (or some combination) as a cost function for each hop.
+
+Using one such algorithm, Selena calculates several possible paths to Rashid, sorted by shortest path:
+
+1. S->A->B->R
+
+2. S->X->Y->R
+
+3. S->X->B->R
+
+4. S->A->B->X->Y->R
+
+
+But, as we saw previously, the channel S->X does not have enough liquidity for a 1M satoshi payment once fees are considered. So Paths 2 and 3 are not viable. That leaves Paths 1 and 4 as possible paths for the payment.
+
+With two possible paths, Selena is ready to attempt delivery!
+
+=== Payment Delivery (Trial-and-Error Loop)
+
+Selena's ((("payment delivery","trial-and error loop", id="ix_12_path_finding-asciidoc8", range="startofrange")))((("trial-and error loop", id="ix_12_path_finding-asciidoc9", range="startofrange")))node starts the trial-and-error loop by constructing the HTLCs, building the onion, and attempting delivery of the payment. For each attempt, there are three possible outcomes:
+
+[role="pagebreak-before"]
+- A successful result (+update_fulfill_htlc+)
+- An error (+update_fail_htlc+)
+- A "stuck" payment with no response (neither success nor failure)
+
+If the payment fails, it can be retried via a different path by updating the graph (changing a channel's metrics) and recalculating an alternative path.
+
+We looked at what happens if the payment is "stuck" in <<stuck_payments>>. The important detail is that a stuck payment is the worst outcome because we cannot retry with another HTLC since both (the stuck one and the retry one) might go through eventually and cause a double payment.
+
+==== First Attempt (Path #1)
+
+Selena attempts the first path (S->A->B->R). She constructs the onion and sends it, but receives a failure code from Bob's node. Bob reports back a +temporary channel failure+ with a +channel_update+ identifying the channel B->R as the one that can't deliver. This attempt is shown in <<path_1_fail>>.
+
+[[path_1_fail]]
+.Path #1 attempt fails
+image::images/mtln_1207.png[]
+
+===== Learning from failure
+
+From this failure code, Selena will deduce that Bob doesn't have enough liquidity to deliver the payment to Rashid on that channel. Importantly, this failure narrows the uncertainty of the liquidity of that channel! Previously, Selena's node assumed that the liquidity on Bob's side of the channel was somewhere in the range (0, 4M). Now, she can assume that the liquidity is in the range (0, 999999). Similarly, Selena can now assume that the liquidity of that channel on Rashid's side is in the range (1M, 4M), instead of (0, 4M). Selena has learned a lot from this failure.
+
+==== Second Attempt (Path #4)
+
+Now Selena attempts the fourth candidate path (S->A->B->X->Y->R). This is a longer path and will incur more fees, but it's now the best option for delivery of the payment.
+
+Fortunately, Selena receives an +update_fulfill_htlc+ message from Alice, indicating that the payment was successful, as shown in <<path_4_success>>.
+
+[[path_4_success]]
+.Path #4 attempt succeeds
+image::images/mtln_1208.png[]
+
+===== Learning from success
+
+Selena has also learned a lot from this successful payment. She now knows that all the channels on the path S->A->B->X->Y->R  had enough liquidity to deliver the payment. Furthermore, she now knows that each of these channels has moved the HTLC amount (1M &#x2b; fees) to the other end of the channel. This allows Selena to recalculate the range of liquidity on the receiving side of all the channels in that path, replacing the minimum liquidity with 1M &#x2b; fees.
+
+===== Stale knowledge?
+
+Selena now has a much better "map" of the Lightning Network (at least as far as these seven channels go). This knowledge will be useful for any subsequent payments that Selena attempts to make.
+
+However, this knowledge becomes somewhat "stale" as the other nodes send or route payments. Selena will never see any of these payments (unless she is the sender). Even if she is involved in routing payments, the onion routing mechanism means she can only see the changes for one hop (her own channels).
+
+Therefore, Selena's node must consider how long to keep this knowledge before assuming that it is stale and no longer useful(((range="endofrange", startref="ix_12_path_finding-asciidoc9")))(((range="endofrange", startref="ix_12_path_finding-asciidoc8"))).
+
+[[mpp]]
+=== Multipart Payments 
+
+((("multipart payments (MPP)", id="ix_12_path_finding-asciidoc10", range="startofrange")))((("payment delivery","multipart payments", id="ix_12_path_finding-asciidoc11", range="startofrange")))_Multipart payments (MPP)_ are a feature that was introduced in the Lightning Network in 2020 and are already very widely available. Multipart payments allow a payment to be split into multiple _parts_ which are sent as HTLCs over several different paths to the intended recipient, preserving the _atomicity_ of the overall payment. In this context, atomicity means that either all the HTLC parts of a payment are eventually fulfilled or the  entire payment fails and all the HTLC parts fail. There is no possibility of a partially successful payment.
+
+Multipart payments are a significant improvement in the Lightning Network because they make it possible to send amounts that won't "fit" in any single channel by splitting them into smaller amounts for which there is sufficient liquidity. Furthermore, multipart payments have been shown to increase the probability of a successful payment, as compared to a single-path payment.
+
+[TIP]
+====
+Now that MPP is available, it is best to think of a single-path payment as a subcategory of an MPP. Essentially, a single-path is just a multipart of size one. All payments can be considered as multipart payments unless the size of the payment and liquidity available make it possible to deliver with a single part.
+====
+
+==== Using MPP
+
+MPP is not something that a user will select, but rather it is a node pathfinding and payment delivery strategy. The same basic steps are implemented: create a graph, select paths, and the trial-and-error loop. The difference is that during path selection we must also consider how to split the payment to optimize delivery.
+
+In our example we can see some immediate improvements to our pathfinding problem that become possible with MPP. First, we can utilize the S->X channel that has known insufficient liquidity to transport 1M satoshis plus fees. By sending a smaller part along that channel, we can use paths that were previously unavailable. Second, we have the unknown liquidity of the B->R channel, which is insufficient to transport the 1M amount, but might be sufficient to transport a smaller amount.
+
+===== Splitting payments
+
+((("multipart payments (MPP)","splitting payments", id="ix_12_path_finding-asciidoc12", range="startofrange")))((("payment","splitting", id="ix_12_path_finding-asciidoc13", range="startofrange")))The fundamental question is how to split the payments. More specifically, what are the optimal number of splits and the optimal amounts for each split?
+
+This is an area of ongoing research where novel strategies are emerging. Multipart payments lead to a different algorithmic approach than single-path payments, even though single-path solutions can emerge from a multipart optimization (i.e., a single path may be the optimal solution suggested by a multipart pathfinding algorithm).
+
+If you recall, we found that the uncertainty of liquidity/balances leads to some (somewhat obvious) conclusions that we can apply in MPP pathfinding, namely:
+
+* Smaller payments have a higher chance of succeeding.
+
+* The more channels you use, the chance of success becomes (exponentially) lower.
+
+From the first of these insights, we might conclude that splitting a large payment (e.g., 1 million satoshis) into tiny payments increases the chance that each of those smaller payments will succeed. The number of possible paths with sufficient liquidity will be greater if we send smaller amounts.
+
+To take this idea to an extreme, why not split the 1M satoshi payment into one million separate one-satoshi parts? Well, the answer lies in our second insight: since we would be using more channels/paths to send our million single-satoshi HTLCs, our chance of success would drop exponentially.
+
+If it's not obvious, the two preceding insights create a "sweet spot" where we can maximize our chances of success: splitting into smaller payments but not too many splits!
+
+Quantifying this optimal balance of size/number of splits for a given channel graph is out of the scope of this book, but it is an active area of research. Some current implementations use a very simple strategy of splitting the payment in two halves, four quarters, etc.
+
+[NOTE]
+====
+To read more about the optimization problem known as minimum-cost flows involved when splitting payments into different sizes and allocating them to paths, see the paper https://arxiv.org/abs/2107.05322["Optimally Reliable & Cheap Payment Flows on the Lightning Network"] by (coauthor of this book) René Pickhardt and Stefan Richter.
+====
+
+In our example, Selena's node will attempt to split the 1M satoshi payment into 2 parts with 600k and 400k satoshi, respectively, and send them on 2 different paths. This is shown in <<mpp_paths>>.
+
+Because the S->X channel can now be utilized, and (luckily for Selena), the B->R channel has sufficient liquidity for 600k satoshis, the 2 parts are successful along paths that were previously not possible.(((range="endofrange", startref="ix_12_path_finding-asciidoc13")))(((range="endofrange", startref="ix_12_path_finding-asciidoc12")))
+
+[[mpp_paths]]
+.Sending two parts of a multipart payment
+image::images/mtln_1209.png[]
+
+==== Trial and Error over Multiple "Rounds"
+
+((("multipart payments (MPP)","trial-and error over multiple rounds")))((("payment delivery","trial-and error loop")))((("trial-and error loop")))Multipart payments lead to a somewhat modified trial-and-error loop for payment delivery. Because we are attempting multiple paths in each attempt, we have four possible outcomes:
+
+* All parts succeed, the payment is successful
+* Some parts succeed, some fail with errors returned
+* All parts fail with errors returned
+* Some parts are "stuck," no errors are returned
+
+In the second case, where some parts fail with errors returned and some parts succeed, we can now _repeat_ the trial-and-error loop, but _only for the residual amount_.
+
+Let's assume for example that Selena had a much larger channel graph with hundreds of possible paths to reach Rashid. Her pathfinding algorithm might find an optimal payment split consisting of 26 parts of varying sizes. After attempting to send all 26 parts in the first round, 3 of those parts failed with errors.
+
+If those 3 parts consisted of, say 155k satoshis, then Selena would restart the pathfinding effort, only for 155k satoshis. The next round could find completely different paths (optimized for the residual amount of 155k), and split the 155k amount into completely different splits!
+
+[TIP]
+====
+While it seems like 26 split parts are a lot, tests on the Lightning Network have successfully delivered a payment of 0.3679 BTC by splitting it into 345 parts.
+====
+
+Furthermore, Selena's node would update the channel graph using the information gleaned from the successes and errors of the first round to find the most optimal paths and splits for the second round.
+
+Let's say that Selena's node calculates that the best way to send the 155k residual is 6 parts split as 80k, 42k, 15k, 11k, 6.5k, and 500 satoshis. In the next round, Selena gets only one error, indicating that the 11k satoshi part failed. Again, Selena updates the channel graph based on the information gleaned and runs the pathfinding again to send the 11k residual. This time, she succeeds with 2 parts of 6k and 5k satoshis, respectively.
+
+This multiround example of sending a payment using MPP is shown in <<mpp_rounds>>.
+
+[[mpp_rounds]]
+.Sending a payment in multiple rounds with MPP
+image::images/mtln_1210.png[]
+
+In the end, Selena's node used 3 rounds of pathfinding to send the 1M satoshis in 30 parts.(((range="endofrange", startref="ix_12_path_finding-asciidoc11")))(((range="endofrange", startref="ix_12_path_finding-asciidoc10")))
+
+=== Conclusion
+
+In this chapter we looked at pathfinding and payment delivery. We saw how to use the channel graph to find paths from a sender to a recipient. We also saw how the sender will attempt to deliver payments on a candidate path and repeat in a trial-and-error loop.
+
+We also examined the uncertainty of channel liquidity (from the perspective of the sender) and the implications that has for pathfinding. We saw how we can quantify the uncertainty and use probability theory to draw some useful conclusions. We also saw how we can reduce uncertainty by learning from both successful and failed payments.
+
+Finally, we saw how the newly deployed multipart payments feature allows us to split payments into parts, increasing the probability of success even for larger payments(((range="endofrange", startref="ix_12_path_finding-asciidoc1"))).(((range="endofrange", startref="ix_12_path_finding-asciidoc0")))

13_wire_protocol.asciidoc

@@ -0,0 +1,411 @@
+[[wire_protocol]]
+== Wire Protocol: Framing and Extensibility
+
+((("wire protocol", id="ix_13_wire_protocol-asciidoc0", range="startofrange")))In this chapter, we dive into the wire protocol of the Lightning Network
+and also cover all the various extensibility levers that have been built into
+the protocol. By the end of this chapter, an ambitious reader should be able to
+write their very own wire protocol parser for the Lightning Network. In addition
+to being able to write a custom wire protocol parser, a reader of this chapter
+will gain a deep understanding of the various upgrade mechanisms
+that have been built into the protocol.
+
+=== Messaging Layer in the Lightning Protocol Suite
+
+((("Lightning Network Protocol","messaging layer")))((("wire protocol","messaging layer in the Lightning Protocol Suite")))The messaging layer, which is detailed in this chapter, consists of "Framing and message format," "Type-Length-Value" encoding, and "Feature bits." These components are highlighted by an outline in the protocol suite, shown in <<LN_protocol_wire_message_highlight>>.
+
+[[LN_protocol_wire_message_highlight]]
+.Messaging layer in the Lightning protocol suite
+image::images/mtln_1301.png["Messaging layer in the Lightning protocol suite"]
+
+=== Wire Framing
+
+((("wire framing", id="ix_13_wire_protocol-asciidoc1", range="startofrange")))((("wire protocol","wire framing", id="ix_13_wire_protocol-asciidoc2", range="startofrange")))We begin by describing the high-level structure of the wire _framing_
+within the protocol. When we say framing, we mean the way that the bytes are
+packed on the wire to _encode_ a particular protocol message. Without knowledge
+of the framing system used in the protocol, a string of bytes on the wire would
+resemble a series of random bytes because no structure has been imposed. By applying
+proper framing to decode these bytes on the wire, we'll be able to extract
+structure and finally parse this structure into protocol messages within our
+higher-level language.
+
+It's important to note that the Lightning Network is an _end-to-end
+encrypted_ protocol, and the wire framing is itself encapsulated within an
+_encrypted_ message transport layer. As we see in <<encrypted_message_transport>>, the Lightning
+Network uses a custom variant of the Noise Protocol to handle
+transport encryption. Within this chapter, whenever we give an example of wire
+framing, we assume the encryption layer has already been stripped away (when
+decoding), or that we haven't yet encrypted the set of bytes before we send
+them on the wire (encoding).
+
+==== High-Level Wire Framing
+
+((("wire framing","high-level schema")))With that said, we're ready to describe the high-level schema used to
+encode messages on the wire:
+
+  * Messages on the wire begin with a _2-byte_ type field, followed by a
+    message payload.
+  * The message payload itself can be up to 65 KB in size.
+  * All integers are encoded in big-endian (network order).
+  * Any bytes that follow after a defined message can be safely ignored.
+
+Yep, that's it. As the protocol relies on an _encapsulating_ transport protocol
+encryption layer, we don't need an explicit length for each message type. This
+is due to the fact that transport encryption works at the _message_ level, so
+by the time we're ready to decode the next message, we already know the total
+number of bytes of the message itself. Using 2 bytes for the message type
+(encoded in big-endian) means that the protocol can have up to 2^16 – 1 or
+65,535 distinct messages. Continuing, because we know all messages must be less than
+65 KB, this simplifies our parsing as we can use a _fixed-size_ buffer and
+maintain strong bounds on the total amount of memory required to parse an
+incoming wire message.
+
+The final bullet point allows for a degree of _backward_ compatibility  because new nodes are able to provide information in the wire messages that older nodes
+(which may not understand them) can safely ignore. As we see subsequently, this
+feature, combined with a very flexible wire message extensibility format, allows the protocol to achieve _forward_ compatibility as well.
+
+==== Type Encoding
+
+((("wire framing","type encoding")))With this high-level background provided, we now start at the most primitive
+layer: parsing primitive types. In addition to encoding integers, the Lightning
+Protocol also allows for encoding of a vast array of types, including variable-length byte slices, elliptic curve public keys, Bitcoin addresses, and
+signatures. When we describe the _structure_ of wire messages later in this
+chapter, we refer to the high-level type (the abstract type) rather than the
+lower-level representation of said type. In this section, we peel back this
+abstraction layer to ensure that our future wire parser is able to properly
+encode/decode any of the higher-level types.
+
+In <<message_types>>, we map the name of a given message type to the
+high-level routine used to encode/decode the type.
+
+[[message_types]]
+.High-level message types
+[options="header"]
+|===
+| High-level type | Framing | Comment
+| `node_alias` | A 32-byte fixed-length byte slice      | When decoding, reject if contents are not a valid UTF-8 string
+| `channel_id` | A 32-byte fixed-length byte slice that maps an outpoint to a 32-byte value     | Given an outpoint, one can convert it to a `channel_id` by taking the TxID of the outpoint and XORing it with the index (interpreted as the lower 2 bytes)
+| `short_chan_id` | An unsigned 64-bit integer (`uint64`) | Composed of the block height (24 bits), transaction index (24 bits), and output index (16 bits) packed into 8 bytes
+| `milli_satoshi` | An unsigned 64-bit integer (`uint64`) | Represents 1000th of a satoshi
+| `satoshi` | An unsigned 64-bit integer (`uint64`) | The base unit of bitcoin
+| `pubkey`  | An secp256k1 public key encoded in _compressed_ format, occupying 33 bytes | Occupies a fixed 33-byte length on the wire
+| `sig`     | An ECDSA signature of the secp256k1 elliptic curve | Encoded as a _fixed_ 64-byte byte slice, packed as `R \|\| S`
+| `uint8`   | An 8-bit integer  |
+| `uint16`  | A 16-bit integer  |
+| `uint64`  | A 64-bit integer  |
+| `[]byte`  | A variable-length byte slice | Prefixed with a 16-bit integer denoting the length of the bytes
+| `color_rgb` | RGB color encoding | Encoded as a series of 8-bit integers
+| `net_addr` | The encoding of a network address | Encoded with a 1-byte prefix that denotes the type of address, followed by the address body
+|===
+
+In the next section, we describe the structure of each wire message,
+including the prefix type of the message along with the contents of its message
+body.(((range="endofrange", startref="ix_13_wire_protocol-asciidoc2")))(((range="endofrange", startref="ix_13_wire_protocol-asciidoc1")))
+
+[[tlv_message_extensions]]
+=== Type-Length-Value Message Extensions
+
+((("Type-Length-Value (TLV) message extensions","message extensions in wire protocol")))((("wire protocol","TLV message extensions")))Earlier in this chapter we mentioned that messages can be up to 65 KB in size,
+and if while parsing a message, extra bytes are left over, then those bytes
+are to be ignored. At an initial glance, this requirement may appear to be
+somewhat arbitrary; however, this requirement allows for decoupled desynchronized evolution of the Lightning
+Protocol itself. We discuss this more toward the end of the chapter. But first, we turn our attention to exactly what those "extra bytes" at
+the end of a message can be used for.
+
+==== The Protocol Buffers Message Format
+
+((("Protocol Buffers (Protobuf) message serialization format")))((("Type-Length-Value (TLV) message extensions","Protocol Buffers message format")))The Protocol Buffers (Protobuf) message serialization format started out as an
+internal format used at Google and has blossomed into one of the most popular
+message serialization formats used by developers globally. The Protobuf format
+describes how a message (usually some sort of data structure related to an API)
+is encoded on the wire and decoded on the other end. Several "Protobuf
+compilers" exists in dozens of languages which act as a bridge that allows any
+language to encode a Protobuf that will be able to decode by a compliant decode
+in another language. Such cross-language data structure compatibility allows
+for a wide range of innovation because it's possible to transmit structure and even
+typed data structures across language and abstraction boundaries.
+
+Protobufs are also known for their flexibility with respect to how they
+handle changes in the underlying messages structure. As long as the field
+numbering schema is adhered to, then it's possible for a newer write of
+Protobufs to include information within a Protobuf that may be unknown to any
+older readers. When the old reader encounters the new serialized format, if
+there are types/fields that it doesn't understand, then it simply _ignores_
+them. This allows old clients and new clients to coexist because all clients can
+parse some portion of the newer message format.
+
+==== Forward and Backward Compatibility
+
+((("Protocol Buffers (Protobuf) message serialization format")))((("Type-Length-Value (TLV) message extensions","forward/backward compatibility")))Protobufs are extremely popular amongst developers because they have built-in
+support for both forward and backward compatibility. Most developers are
+likely familiar with the concept of backward compatibility. In simple terms,
+the principle states that any changes to a message format or API should be
+done in a manner that doesn't break support for older clients. Within our preceding Protobuf extensibility examples, backward compatibility is achieved by
+ensuring that new additions to the Protobuf format don't break the known portions
+of older readers. Forward compatibility, on the other hand, is just as important
+for desynchronized updates; however, it's less commonly known. For a change to
+be forward compatible, clients are to simply ignore any information
+they don't understand. The soft fork mechanism of upgrading the Bitcoin
+consensus system can be said to be both forward and backward compatible: any
+clients that don't update can still use Bitcoin, and if they encounter any
+transactions they don't understand, then they simply ignore them as their funds
+aren't using those new features.
+
+[[tlv]]
+=== Type-Length-Value Format
+
+((("Type-Length-Value (TLV) format", id="ix_13_wire_protocol-asciidoc3", range="startofrange")))((("Type-Length-Value (TLV) format","wire protocol and", id="ix_13_wire_protocol-asciidoc4", range="startofrange")))((("wire protocol","TLV format", id="ix_13_wire_protocol-asciidoc5", range="startofrange")))To be able to upgrade messages in a manner that is both forward and backward
+compatible, in addition to feature bits (more on that later), the Lightning Network utilizes a custom message serialization format plainly called Type-Length-Value, or TLV for short. The format was inspired by the widely used Protobuf
+format and borrows many concepts by significantly simplifying the
+implementation as well as the software that interacts with message parsing. A
+curious reader might ask, "why not just use Protobufs?" In response, the
+Lightning developers would respond that we're able to have the best of the
+extensibility of Protobufs while also having the benefit of a smaller
+implementation and thus smaller attack. As of version 3.15.6, the Protobuf
+compiler weighs in at over 656,671 lines of code.  In comparison, LND's
+implementation of the TLV message format weighs in at only 2.3k lines of code
+(including tests).
+
+With the necessary background presented, we're now ready to describe the TLV
+format in detail. A TLV message extension is said to be a stream of
+individual pass:[<span class="keep-together">TLV records</span>]. A single TLV record has three components: the type of
+the record, the length of the record, and finally the opaque value of the
+record:
+
+`type`:: An integer representing the name of the record being encoded
+`length`:: The length of the record
+`value`:: The opaque value of the record
+
+Both the `type` and `length` are encoded using a variable-sized integer that's inspired by the variable-sized integer (varint) used in Bitcoin's P2P protocol, called `BigSize` for short.
+
+==== BigSize Integer Encoding
+
+((("BigSize integer encoding")))((("Type-Length-Value (TLV) format","BigSize integer encoding")))In its fullest form, a `BigSize`
+integer can represent value up to 64 bits. In contrast to Bitcoin's varint
+format, the `BigSize` format instead encodes integers using a big-endian byte
+ordering.
+
+The `BigSize` varint has two components: the discriminant and the body. In the
+context of the `BigSize` integer, the discriminant communicates to the decoder
+the size of the variable-sized integer that follows. Remember that the unique thing about
+variable-sized integers is that they allow a parser to use fewer bytes to encode
+smaller integers than larger ones, saving space. Encoding of a `BigSize`
+integer follows one of the four following options:
+
+1. If the value is less than `0xfd` (`253`): Then the discriminant isn't really used, and the encoding is simply the integer itself. This allows us to encode very small integers with no additional overhead.
+
+2. If the value is less than or equal to `0xffff` (`65535`): The discriminant is encoded as `0xfd`, which indicates that the value that follows is larger than `0xfd`, but smaller than `0xffff`. The number is then encoded as a 16-bit integer. Including the discriminant, we can encode a value that is greater than 253, but less than 65,535 using 3 bytes.
+
+3. If the value is less than `0xffffffff` (`4294967295`): The discriminant is encoded as `0xfe`. The body is encoded using a 32-bit integer, including the discriminant, and we can encode a value that's less than `4,294,967,295` using 5 bytes.
+
+4. Otherwise, we just encode the value as a full-size 64-bit integer.
+
+
+====  TLV Encoding Constraints
+
+((("Type-Length-Value (TLV) format","encoding constraints")))Within the context of a TLV message, record types below `2^16` are said to be _reserved_ for future use. Types beyond this
+range are to be used for "custom" message extensions used by higher-level application protocols.
+
+The `value` of a record depends on the `type`. In other words, it can take any form because parsers will attempt to interpret it depending on the context of the type itself.
+
+==== TLV Canonical Encoding
+
+One issue with the Protobuf format is that encodings of the same message may
+output an entirely different set of bytes when encoded by two different
+versions of the compiler. Such instances of a noncanonical encoding are not
+acceptable within the context of Lightning, as many messages contain a
+signature of the message digest. If it's possible for a message to be encoded
+in two different ways, then it would be possible to break the authentication of
+a signature inadvertently by re-encoding a message using a slightly different
+set of bytes on the wire.
+
+To ensure that all encoded messages are canonical, the following
+constraints are defined when encoding:
+
+  * All records within a TLV stream must be encoded in order of strictly
+    increasing type.
+
+  * All records must minimally encode the `type` and `length` fields. In other words, the smallest `BigSize` representation for an integer must be used at all times.
+
+  * Each `type` may only appear once within a given TLV stream.
+
+In addition to these encoding constraints, a series of higher-level
+interpretation requirements is also defined based on the _arity_ of a given `type` integer. We dive further into these details toward the end of the
+chapter once we describe how the Lightning Protocol is upgraded in practice and
+in theory.(((range="endofrange", startref="ix_13_wire_protocol-asciidoc5")))(((range="endofrange", startref="ix_13_wire_protocol-asciidoc4")))(((range="endofrange", startref="ix_13_wire_protocol-asciidoc3")))
+
+[[feature_bits]]
+=== Feature Bits and Protocol Extensibility
+
+((("feature bits", id="ix_13_wire_protocol-asciidoc6", range="startofrange")))((("wire protocol","feature bits/protocol extensibility", id="ix_13_wire_protocol-asciidoc7", range="startofrange")))Because the Lightning Network is a decentralized system, no single entity can enforce a
+protocol change or modification upon all the users of the system. This
+characteristic is also seen in other decentralized networks such as Bitcoin.
+However, unlike Bitcoin, overwhelming consensus _is not_ required to change a
+subset of the Lightning Network. Lightning is able to evolve at will without a
+strong requirement of coordination because, unlike Bitcoin, there is no global consensus required in the Lightning Network. Due to this fact and the several
+upgrade mechanisms embedded in the Lightning Network, only the
+participants that wish to use these new Lightning Network features need to
+upgrade, and then they are able to interact with each other.
+
+In this section, we explore the various ways that developers and users are
+able to design and deploy new features to the Lightning Network. The
+designers of the original Lightning Network knew that there were many possible future directions for the network and the underlying protocol. As a result, they made sure to implement several
+extensibility mechanisms within the system, which can be used to upgrade it partially or fully in a decoupled, desynchronized, and decentralized
+manner.
+
+==== Feature Bits as an Upgrade Discoverability Mechanism
+
+((("feature bits","upgrade discoverability mechanism")))An astute reader may have noticed the various locations where feature bits are
+included within the Lightning Protocol. A _feature bit_ is a bitfield that can
+be used to advertise understanding or adherence to a possible network protocol
+update. Feature bits are commonly assigned in pairs, meaning that each
+potential new feature/upgrade always defines two bits within the bitfield.
+One bit signals that the advertised feature is _optional_, meaning that the
+node knows about the feature and can use it, but doesn't
+consider it required for normal operation. The other bit signals that the
+feature is instead _required_, meaning that the node will not continue
+operation if a prospective peer doesn't understand that feature.
+
+Using these two bits (optional and required), we can construct a simple
+compatibility matrix that nodes/users can consult to determine if a peer is compatible with a desired feature, as shown in <<table1302>>.
+
+[[table1302]]
+.Feature bit compatibility matrix
+[options="header"]
+|===
+|Bit type|Remote optional|Remote required|Remote unknown
+|Local optional|✅|✅|✅
+|Local required|✅|✅|❌
+|Local unknown|✅|❌|❌
+|===
+
+From this simplified compatibility matrix, we can see that as long as the other
+party knows about our feature bit, then we can interact with them using the
+protocol. If the party doesn't even know about what bit we're referring to
+_and_ they require the feature, then we are incompatible with them. Within the
+network, optional features are signaled using an _odd bit number_, while
+required features are signaled using an _even bit number_. As an example, if a peer signals that they know of a feature that uses bit +15+, then we know that
+this is an optional feature, and we can interact with them or respond to
+their messages even if we don't know about the feature. If
+they instead signaled the feature using bit +16+, then we know this is a
+required feature, and we can't interact with them unless our node also
+understands that feature.
+
+The Lightning developers have come up with an easy-to-remember phrase that
+encodes this matrix: "it's OK to be odd." This simple rule  allows for a
+rich set of interactions within the protocol, as a simple bitmask operation
+between two feature bit vectors allows peers to determine if certain
+interactions are compatible with each other or not. In other words, feature
+bits are used as an upgrade discoverability mechanism: they easily allow to
+peers to understand if they are compatible or not based on the concepts of
+optional, required, and unknown feature bits.
+
+Feature bits are found in the `node_announcement`, `channel_announcement`, and
+`init` messages within the protocol. As a result, these three messages can be
+used to signal the knowledge and/or understanding of in-flight protocol
+updates within the network. The feature bits found in the `node_announcement`
+message can allow a peer to determine if their _connections_ are compatible or
+not. The feature bits within the `channel_announcement` messages allow a peer
+to determine if a given payment type or HTLC can transit through a given peer or
+not. The feature bits within the `init` message allow peers to understand if
+they can maintain a connection, and also which features are negotiated for the
+lifetime of a given connection.
+
+==== TLV for Forward and Backward Compatibility
+
+((("feature bits","TLV for forward/backward compatibility")))((("Type-Length-Value (TLV) format","forward/backward compatibility and")))((("wire protocol","TLV for forward/backward compatibility")))As we learned earlier in the chapter, TLV records can be
+used to extend messages in a forward and backward compatible manner.
+Over time, these records have been used to extend existing messages without
+breaking the protocol by utilizing the "undefined" area within a message beyond
+that set of known bytes.
+
+As an example, the original Lightning Protocol didn't have a concept of the
+"largest amount HTLC" that could traverse through a channel as dictated by a routing
+policy. Later on, the `max_htlc` field was added to the `channel_update`
+message to phase in this concept over time. Peers that receive a
+`channel_update` that sets such a field but don't even know the upgrade existed
+are unaffected by the change, but have their HTLCs rejected if they are
+beyond the limit. Newer peers, on the other hand, are able to parse, verify,
+and utilize the new field.
+
+Those familiar with the concept of soft forks in Bitcoin may now see some
+similarities between the two mechanisms.  Unlike Bitcoin consensus-level
+soft forks, upgrades to the Lightning Network don't require overwhelming
+consensus to be adopted. Instead, at minimum, only two peers within the
+network need to understand a new upgrade to start using it. Commonly these two peers may be the recipient and sender of a
+payment, or may be the channel partners of a new payment channel.
+
+==== A Taxonomy of Upgrade Mechanisms
+
+((("Lightning Network (generally)","taxonomy of upgrade mechanisms", id="ix_13_wire_protocol-asciidoc8", range="startofrange")))((("upgrades","taxonomy of upgrade mechanisms", id="ix_13_wire_protocol-asciidoc9", range="startofrange")))((("wire protocol","taxonomy of upgrade mechanisms", id="ix_13_wire_protocol-asciidoc10", range="startofrange")))Rather than there being a single widely utilized upgrade mechanism within the
+network (such as soft forks for Bitcoin), there exist several possible upgrade mechanisms within the Lightning Network. In this
+section, we enumerate these upgrade mechanisms and
+provide a real-world example of their use in the past.
+
+===== Internal network upgrades
+
+((("upgrades","internal network")))We start with the upgrade type that requires the most protocol-level
+coordination: internal network upgrades. An internal network upgrade is
+characterized by one that requires _every single node_ within a prospective payment path to understand the new feature. Such an upgrade is similar to any
+upgrade within the internet that requires hardware-level upgrades within
+the core-relay portion of the upgrade. In the context of the Lightning Network, however, we deal
+with pure software, so such upgrades are easier to deploy, yet they still
+require much more coordination than any other upgrade mechanism in the
+network.
+
+One example of such an upgrade within the network was the introduction of a TLV
+encoding for the routing information encoded within the onion
+packets. The prior format used a hardcoded fixed-length message
+format to communicate information such as the next hop.
+Because this format was fixed, it meant that new protocol-level upgrades  weren't possible. The move to the more flexible TLV
+format meant that after this upgrade, any sort of feature that
+modified the type of information communicated at each hop could be rolled out at will.
+
+It's worth mentioning that the TLV onion upgrade was a sort of "soft" internal
+network upgrade, in that if a payment wasn't using any new feature beyond
+that new routing information encoding, then a payment could be transmitted
+using a mixed set of nodes.
+
+===== End-to-end upgrades
+
+((("upgrades","end-to-end")))To contrast the internal network upgrade, in this section we describe the
+_end-to-end_ network upgrade. This upgrade mechanism differs from the internal
+network upgrade in that it only requires the "ends" of the payment, the sender
+and recipient, to upgrade.
+
+This type of upgrade allows
+for a wide array of unrestricted innovation within the network. Because of the
+onion encrypted nature of payments within the network, those forwarding HTLCs
+within the center of the network may not even know that new features are being
+utilized.
+
+One example of an end-to-end upgrade within the network was the rollout of multipart payments (MPP). MPP is a protocol-level feature that enables a
+single payment to be split into multiple parts or paths, to be assembled at the
+recipient for settlement. The rollout of MPP was coupled with a new
+`node_announcement` level feature bit that indicates that the recipient knows
+how to handle partial payments. Assuming a sender and recipient know about each
+other (possibly via a BOLT #11 invoice), then they're able to use the new
+feature without any further negotiation.
+
+Another example of an end-to-end upgrade are the various types of
+_spontaneous_ payments deployed within the network. One early type of
+spontaneous payments called _keysend_ worked by simply placing the preimage of a payment within the encrypted onion. Upon receipt, the destination would decrypt the
+preimage, then use that to settle the payment. Because the entire packet is end-to-end encrypted, this payment type was safe, since none of the intermediate nodes
+are able to fully unwrap the onion to uncover the payment preimage.
+
+==== Channel Construction-Level Updates
+
+((("wire protocol","channel construction-level updates")))The final broad category of updates are those that happen at
+the channel construction level, but which don't modify the structure of the HTLC used widely within the network. When we say channel construction, we mean
+how the channel is funded or created. As an example, the eltoo channel type
+can be rolled out within the network using a new `node_announcement` level
+feature bit as well as a `channel_announcement` level feature bit. Only the two
+peers on the sides of the channels need to understand and advertise these new
+features. This channel pair can then be used to forward any payment type
+granted the channel supports it.
+
+Another is the _anchor outputs_ channel format which allows the commitment fee to be
+bumped via Bitcoin's Child-Pays-For-Parent (CPFP) fee management mechanism(((range="endofrange", startref="ix_13_wire_protocol-asciidoc10")))(((range="endofrange", startref="ix_13_wire_protocol-asciidoc9")))(((range="endofrange", startref="ix_13_wire_protocol-asciidoc8"))).(((range="endofrange", startref="ix_13_wire_protocol-asciidoc7")))(((range="endofrange", startref="ix_13_wire_protocol-asciidoc6")))
+
+=== Conclusion
+
+Lightning's wire protocol is incredibly flexible and allows for rapid innovation and interoperability without strict consensus. It is one of the reasons that the Lightning Network is experiencing much faster development and is attractive to many developers, who might otherwise find Bitcoin's development style too conservative and slow.(((range="endofrange", startref="ix_13_wire_protocol-asciidoc0")))

14_encrypted_transport.asciidoc

@@ -0,0 +1,650 @@
+[[encrypted_message_transport]]
+== Lightning's Encrypted Message Transport
+
+((("Lightning encrypted transport protocol", id="ix_14_encrypted_transport-asciidoc0", range="startofrange")))In this chapter we will review the Lightning Network's _encrypted message
+transport_, sometimes referred to as the ((("Brontide Protocol")))_Brontide Protocol_, which allows peers to
+establish end-to-end encrypted communication, authentication, and integrity
+checking.
+
+[NOTE]
+====
+Part of this chapter includes some highly technical detail about the encryption protocol and encryption algorithms used in the Lightning encrypted transport. You may decide to skip that section if you are not interested in those details.
+====
+
+=== Encrypted Transport in the Lightning Protocol Suite
+
+((("Lightning encrypted transport protocol","Lightning Protocol Suite and")))The transport component of the Lightning Network and its several components are shown in the leftmost part of the network connection layer in  <<LN_protocol_encrypted_transport_highlight>>.
+
+[[LN_protocol_encrypted_transport_highlight]]
+.Encrypted message transport in the Lightning protocol suite
+image::images/mtln_1401.png["Encrypted message transport in the Lightning protocol suite"]
+
+=== Introduction
+
+Unlike the vanilla Bitcoin P2P network, every node in the Lightning Network is
+identified by a unique public key which serves as its identity. By default, this
+public key is used to end-to-end encrypt _all_ communication within the
+network. Encryption by default at the lowest level of the protocol ensures that
+all messages are authenticated, are immune to man-in-the-middle (MITM) attacks and snooping by third parties, and ensures privacy at the fundamental transport
+level. In this chapter, we'll learn about the encryption protocol used by the
+Lightning Network in detail. Upon completion of this chapter, the reader will
+be familiar with the state of the art in encrypted messaging protocols, as well
+as the various properties such a protocol provides to the network. It's worth
+mentioning that the core of the encrypted message transport is _agnostic_ to
+its usage within the context of the Lightning Network. As a result, the
+custom encrypted message transport that Lightning uses can be dropped into any context
+that requires encrypted communication between two parties.
+
+=== The Channel Graph as Decentralized Public Key Infrastructure
+
+((("channel graph","decentralized public key infrastructure")))((("Lightning encrypted transport protocol","channel graph as decentralized public key infrastructure")))((("PKI (public key infrastructure)")))((("public key infrastructure (PKI)")))As we learned in <<routing>>, every node has a long-term
+identity that is used as the identifier for a vertex during pathfinding and
+also used in the asymmetric cryptographic operations related to the creation of
+onion encrypted routing packets. This public key, which serves as a node's
+long-term identity, is included in the DNS bootstrapping response, as well as
+embedded within the channel graph. As a result, before a node attempts to
+connect out to another node on the P2P network, it already knows the public key
+of the node it wishes to connect to.
+
+Additionally, if the node being connected to already has a series of public
+channels within the graph, then the connecting node is able to further verify the identity of the node. Because the entire channel graph is fully
+authenticated, one can view it as a sort of decentralized public key
+infrastructure (PKI): to register a key, a public channel in the Bitcoin
+blockchain must be opened, and once a node no longer has any public channels, then
+they've effectively been removed from the PKI.
+
+Because Lightning is a decentralized network, it's imperative that no one central
+party is designated the power to provision a public key identity within the
+network. In place of a central party, the Lightning Network uses the Bitcoin
+blockchain as a Sybil mitigation mechanism because gaining an identity on the
+network has a tangible cost: the fee needed to create a channel in the
+blockchain, as well as the opportunity cost of the capital allocated to their
+channels. In the process of essentially rolling a domain-specific PKI, the
+Lightning Network is able to significantly simplify its encrypted transport
+protocol as it doesn't need to deal with all the complexities that come along
+with TLS, the Transport Layer Security protocol.
+
+=== Why Not TLS?
+
+((("Lightning encrypted transport protocol","TLS vulnerabilities/limitations")))((("TLS (Transport Layer Security protocol)")))((("Transport Layer Security protocol (TLS)")))Readers familiar with the TLS system may be wondering at this point: why wasn't
+TLS used in spite of the drawbacks of the existing PKI system? It is indeed a
+fact that "self-signed certificates" can be used to effectively sidestep the
+existing global PKI system by simply asserting to the identity of a given
+public key amongst a set of peers. However, even with the existing PKI system
+out of the way, TLS has several drawbacks that prompted the creators of the Lightning Network
+to instead opt for a more compact custom encryption protocol.
+
+To start with, TLS is a protocol that has been around for several decades and
+as a result has evolved over time as new advances have been made in the space
+of transport encryption. However, over time this evolution has caused the
+protocol to balloon in size and complexity. Over the past few decades, several
+vulnerabilities in TLS have been discovered and patched, with each evolution
+further increasing the complexity of the protocol. As a result of the age of
+the protocol, several versions and iterations exist, meaning a client needs to
+understand many of the prior iterations of the protocol to communicate
+with a large portion of the public internet, further increasing implementation
+complexity.
+
+In the past, several memory safety vulnerabilities have been discovered in
+widely used implementations of SSL/TLS. Packaging such a protocol within every
+Lightning node would serve to increase the attack surface of nodes exposed to the public peer-to-peer network. To increase the security of the
+network as a whole and minimize exploitable attack surface, the creators of
+the Lightning Network instead opted to adopt the Noise Protocol Framework. Noise as a protocol
+internalizes several of the security and privacy lessons learned over time due
+to continual scrutiny of the TLS protocol over decades. In a way, the existence
+of Noise allows the community to effectively "start over," with a more compact,
+simplified protocol that retains all the added benefits of TLS.
+
+=== The Noise Protocol Framework
+
+((("Lightning encrypted transport protocol","Noise Protocol Framework")))((("Noise Protocol Framework","encrypted message transport and")))The Noise Protocol Framework is a modern, extensible, and flexible message
+encryption protocol designed by the creators of the Signal Protocol. The Signal Protocol is one of the most widely used message encryption protocols in the
+world. It's used by both Signal and Whatsapp, which cumulatively are used by
+over a billion people around the world. The Noise framework is the result of
+decades of evolution both within academia as well as the industry of message
+encryption protocols. Lightning uses the Noise Protocol Framework to implement
+a _message-oriented_ encryption protocol used by all nodes to communicate with
+each other.
+
+A communication session using Noise has two distinct phases: the handshake
+phase and the messaging phase. Before two parties can communicate with each
+other, they first need to arrive at a shared secret known only to them which
+will be used to encrypt and authenticate messages sent to each other. ((("handshake","defined")))A flavor
+of an authenticated key agreement is used to arrive at a final shared key
+between the two parties. In the context of the Noise protocol, this
+authenticated key agreement is referred to as a _handshake_. Once that
+handshake has been completed, both nodes can now being to send each other
+encrypted messages. Each time peers need to connect or reconnect to each
+other, a fresh iteration of the handshake protocol is executed, ensuring that
+forward secrecy is achieved (leaking the key of a prior transcript doesn't compromise any
+future transcripts).
+
+Because the Noise Protocol allows a protocol designer to choose from several
+cryptographic primitives, such as symmetric encryption and public key
+cryptography, it's customary that each flavor of the Noise Protocol is referred
+to by a unique name. In the spirit of "Noise," each flavor of the protocol
+selects a name derived from some sort of "noise." In the context of the
+((("Brontide Protocol")))Lightning Network, the flavor of the Noise Protocol used is sometimes referred to
+as Brontide. A _brontide_ is a low billowing noise, similar to what one would
+hear during a thunderstorm when very far away.
+
+=== Lightning Encrypted Transport in Detail
+
+((("Lightning encrypted transport protocol","elements of", id="ix_14_encrypted_transport-asciidoc1", range="startofrange")))In this section we will break down the Lightning encrypted transport protocol and delve into the details of the cryptographic algorithms and protocol used to establish encrypted, authenticated, and integrity-assured communications between peers. Feel free to skip this section if you find this level of detail daunting.
+
+==== Noise_XK: Lightning Network's Noise Handshake
+
+((("Lightning encrypted transport protocol","Noise_XK")))((("Noise Protocol Framework","Noise_XK")))((("Noise_XK")))The Noise Protocol is extremely flexible in that it advertises several
+handshakes, each with different security and privacy properties for a would-be
+protocol implementer to select from. A deep exploration of each of the
+handshakes and their various trade-offs is out of the scope of this chapter.
+With that said, the Lightning Network uses a specific handshake referred to as
+`Noise_XK`. ((("identity hiding")))The unique property provided by this handshake is __identity hiding__: in order for a node to initiate a connection with another node, it
+must first know its public key. Mechanically, this means that the public key
+of the responder is actually never transmitted during the context of the
+handshake. Instead, a clever series of Elliptic Curve Diffie–Hellman (ECDH) and
+message authentication code (MAC) checks are used to authenticate the
+responder.
+
+==== Handshake Notation and Protocol Flow
+
+((("handshake","notation and protocol flow")))((("Lightning encrypted transport protocol","handshake notation and protocol flow")))((("Noise_XK","handshake notation and protocol flow")))Each handshake typically consists of several steps. At each step some
+(possibly) encrypted material is sent to the opposite party, an ECDH (or
+several) is performed, with the result of the handshake being "mixed" into a
+protocol _transcript_. This transcript serves to authenticate each step of the
+protocol and helps thwart a flavor of man-in-the-middle attacks. At the
+end of the handshake, two keys, `ck` and `k`, are produced which are used to
+encrypt messages (`k`) and rotate keys (`ck`) throughout the lifetime of
+the session.
+
+In the context of a handshake, `s` is usually a long-term static public key.
+In our case, the public key crypto system used is an elliptic curve one,
+instantiated with the `secp256k1` curve, which is used elsewhere in Bitcoin.
+Several ephemeral keys are generated throughout the handshake. We use `e` to
+refer to a new ephemeral key. ECDH operations between two keys are notated as
+the concatenation of two keys. As an example, `ee` represents an ECDH operation
+between two ephemeral keys.
+
+==== High-Level Overview
+
+((("Lightning encrypted transport protocol","high-level overview")))((("Noise_XK","high-level overview")))Using the notation laid out earlier, we can succinctly describe the `Noise_XK`
+as pass:[<span class="keep-together">follows</span>]:
+```
+    Noise_XK(s, rs):
+       <- rs
+       ...
+       -> e, e(rs)
+       <- e, ee
+       -> s, se
+```
+
+The protocol begins with the "pretransmission" of the responder's static key
+(`rs`) to the initiator. Before executing the handshake, the initiator is to
+generate its own static key (`s`). During each step of the handshake, all
+material sent across the wire and the keys sent/used are incrementally
+hashed into a _handshake digest_, `h`. This digest is never sent across the
+wire during the handshake, and is instead used as the "associated data" when
+AEAD (authenticated encryption with associated data) is sent across the wire.
+((("AD (associated data)")))((("associated data (AD)")))_Associated data_ (AD) allows an encryption protocol to authenticate additional
+information alongside a cipher text packet. In other domains, the AD may be a
+domain name, or plain-text portion of the packet.
+
+The existence of `h` ensures that if a portion of a transmitted handshake
+message is replaced, then the other side will notice. At each step, a MAC
+digest is checked. If the MAC check succeeds, then the receiving party knows
+that the handshake has been successful up until that point. Otherwise, if a MAC
+check ever fails, then the handshake process has failed, and the connection
+should be terminated.
+
+The protocol also adds a new piece of data to each handshake message: a protocol
+version. The initial protocol version is `0`. At the time of writing, no new
+protocol versions have been created. As a result, if a peer receives a version
+other than `0`, then they should reject the handshake initiation attempt.
+
+As far as cryptographic primitives, SHA-256 is used as the hash function of
+choice, `secp256k1` as the elliptic curve, and `ChaChaPoly-130` as the AEAD
+(symmetric encryption) construction.
+
+Each variant of the Noise Protocol has a unique ASCII string used to refer to it. To ensure that two parties are using the same protocol
+variant, the ASCII string is hashed into a digest, which is used to initialize
+the starting handshake state. In the context of the Lightning Network, the ASCII
+string describing the protocol is `Noise_XK_secp256k1_ChaChaPoly_SHA256`.
+
+==== Handshake in Three Acts
+
+((("Lightning encrypted transport protocol","handshake in three acts", id="ix_14_encrypted_transport-asciidoc2", range="startofrange")))((("Noise_XK","handshake in three acts", id="ix_14_encrypted_transport-asciidoc3", range="startofrange")))The handshake portion can be separated into three distinct "acts."
+The entire handshake takes 1.5 round trips between the initiator and responder.
+At each act, a single message is sent between both parties. The handshake
+message is a fixed-size payload prefixed by the protocol version.
+
+The Noise Protocol uses an object-oriented inspired notation to describe the
+protocol at each step. During setup of the handshake state, each side will
+initialize the following variables:
+
+`ck`:: The _chaining key_. This value is the accumulated hash of all
+   previous ECDH outputs. At the end of the handshake, `ck` is used to derive
+   the encryption keys for Lightning messages.
+
+`h`:: The _handshake hash_. This value is the accumulated hash of _all_
+   handshake data that has been sent and received so far during the handshake
+   process.
+
+`temp_k1`, `temp_k2`, `temp_k3`:: The _intermediate keys_. These are used to
+   encrypt and decrypt the zero-length AEAD payloads at the end of each handshake
+   message.
+
+ `e`:: A party's _ephemeral keypair_. For each session, a node must generate a
+   new ephemeral key with strong cryptographic randomness.
+
+`s`:: A party's _static keypair_ (`ls` for local, `rs` for remote).
+
+Given this handshake plus messaging session state, we'll then define a series of
+functions that will operate on the handshake and messaging state. When
+describing the handshake protocol, we'll use these variables in a manner
+similar to pseudocode to reduce the verbosity of the explanation of
+each step in the protocol. We'll define the _functional_ primitives of the
+handshake as:
+
+`ECDH(k, rk)`:: Performs an Elliptic Curve Diffie–Hellman operation using
+    `k`, which is a valid `secp256k1` private key, and `rk`, which is a valid public key.
++
+The returned value is the SHA-256 of the compressed format of the
+      generated point.
+
+`HKDF(salt,ikm)`:: A function defined in `RFC 5869`,
+    evaluated with a zero-length `info` field.
++
+All invocations of `HKDF` implicitly return 64 bytes of
+       cryptographic randomness using the extract-and-expand component of the
+       `HKDF`.
+
+`encryptWithAD(k, n, ad, plaintext)`:: Outputs `encrypt(k, n, ad, plaintext)`.
++
+Where `encrypt` is an evaluation of `ChaCha20-Poly1305` (Internet Engineering Task Force variant)
+       with the passed arguments, with nonce `n` encoded as 32 zero bits,
+       followed by a _little-endian_ 64-bit value. Note: this follows the Noise
+       Protocol convention, rather than our normal endian.
+
+`decryptWithAD(k, n, ad, ciphertext)`:: Outputs `decrypt(k, n, ad, ciphertext)`.
++
+Where `decrypt` is an evaluation of `ChaCha20-Poly1305` (IETF variant)
+       with the passed arguments, with nonce `n` encoded as 32 zero bits,
+       followed by a _little-endian_ 64-bit value.
+
+`generateKey()`:: Generates and returns a fresh `secp256k1` keypair.
++
+Where the object returned by `generateKey` has two attributes:`.pub`, which returns an abstract object representing the public key; and `.priv`, which represents the private key used to generate the public key
++
+Where the object also has a single method: `.serializeCompressed()`
+
+`a || b`:: This denotes the concatenation of two byte strings `a` and `b`.
+
+===== Handshake session state initialization
+
+((("handshake","session state initialization")))((("Lightning encrypted transport protocol","handshake session state initialization")))((("Noise_XK","handshake session state initialization")))Before starting the handshake process, both sides need to initialize the
+starting state that they'll use to advance the handshake process. To start,
+both sides need to construct the initial handshake digest `h`.
+
+ 1. ++h = SHA-256(__protocolName__)++
++
+Where ++__protocolName__ = "Noise_XK_secp256k1_ChaChaPoly_SHA256"++ encoded as
+      an ASCII string.
+
+ 2. `ck = h`
+
+ 3. ++h = SHA-256(h || __prologue__)++
++
+Where ++__prologue__++ is the ASCII string: `lightning`.
+
+In addition to the protocol name, we also add in an extra "prologue" that is
+used to further bind the protocol context to the Lightning Network.
+
+To conclude the initialization step, both sides mix the responder's public key
+into the handshake digest. Because this digest is used while the associated data with a
+zero-length ciphertext (only the MAC) is sent, this ensures that the initiator
+does indeed know the public key of the responder.
+
+ * The initiating node mixes in the responding node's static public key
+   serialized in Bitcoin's compressed format: `h = SHA-256(h || rs.pub.serializeCompressed())`
+
+ * The responding node mixes in their local static public key serialized in
+   Bitcoin's compressed format: `h = SHA-256(h || ls.pub.serializeCompressed())`
+
+===== Handshake acts
+
+((("handshake","acts", id="ix_14_encrypted_transport-asciidoc4", range="startofrange")))((("Lightning encrypted transport protocol","handshake acts", id="ix_14_encrypted_transport-asciidoc5", range="startofrange")))((("Noise_XK","handshake acts", id="ix_14_encrypted_transport-asciidoc6", range="startofrange")))After the initial handshake initialization, we can begin the actual execution
+of the handshake process. The handshake is composed of a series of
+three messages sent between the initiator and responder, henceforth referred to as
+"acts." Because each act is a single message sent between the parties, a handshake
+is completed in a total of 1.5 round trips (0.5 for each act).
+
+((("Diffie-Hellman Key Exchange (DHKE)")))The first act completes the initial portion of the incremental triple Diffie–Hellman (DH) key exchange (using a new ephemeral key generated by the initiator)
+and also ensures that the initiator actually knows the long-term public key of
+the responder. During the second act, the responder transmits the ephemeral key
+they wish to use for the session to the initiator, and once again incrementally
+mixes this new key into the triple DH handshake. During the third and final
+act, the initiator transmits their long-term static public key to the
+responder and executes the final DH operation to mix that into the final
+resulting shared secret.
+
+====== Act One
+
+```
+    -> e, es
+```
+
+Act One is sent from initiator to responder. During Act One, the initiator
+attempts to satisfy an implicit challenge by the responder. To complete this
+challenge, the initiator must know the static public key of the responder.
+
+The handshake message is _exactly_ 50 bytes: 1 byte for the handshake
+version, 33 bytes for the compressed ephemeral public key of the initiator,
+and 16 bytes for the `poly1305` tag.
+
+Sender actions:
+
+1. `e = generateKey()`
+2. `h = SHA-256(h || e.pub.serializeCompressed())`
++
+The newly generated ephemeral key is accumulated into the running
+       handshake digest.
+3. `es = ECDH(e.priv, rs)`
++
+The initiator performs an ECDH between its newly generated ephemeral
+       key and the remote node's static public key.
+4. `ck, temp_k1 = HKDF(ck, es)`
++
+A new temporary encryption key is generated, which is
+       used to generate the authenticating MAC.
+5. `c = encryptWithAD(temp_k1, 0, h, zero)`
++
+Where `zero` is a zero-length plain text.
+6. `h = SHA-256(h || c)`
++
+Finally, the generated ciphertext is accumulated into the authenticating
+       handshake digest.
+7. Send `m = 0 || e.pub.serializeCompressed() || c` to the responder over the network buffer.
+
+Receiver actions:
+
+1. Read _exactly_ 50 bytes from the network buffer.
+2. Parse the read message (`m`) into `v`, `re`, and `c`:
+    * Where `v` is the _first_ byte of `m`, `re` is the next 33
+      bytes of `m`, and `c` is the last 16 bytes of `m`.
+    * The raw bytes of the remote party's ephemeral public key (`re`) are to be
+      deserialized into a point on the curve using affine coordinates as encoded
+      by the key's serialized composed format.
+3. If `v` is an unrecognized handshake version, then the responder must
+    abort the connection attempt.
+4. `h = SHA-256(h || re.serializeCompressed())`
++
+The responder accumulates the initiator's ephemeral key into the authenticating
+      handshake digest.
+5. `es = ECDH(s.priv, re)`
++
+The responder performs an ECDH between its static private key and the
+      initiator's ephemeral public key.
+6. `ck, temp_k1 = HKDF(ck, es)`
++
+A new temporary encryption key is generated, which will
+      shortly be used to check the authenticating MAC.
+7. `p = decryptWithAD(temp_k1, 0, h, c)`
++
+If the MAC check in this operation fails, then the initiator does _not_
+      know the responder's static public key. If this is the case, then the
+      responder must terminate the connection without any further messages.
+8. `h = SHA-256(h || c)`
++
+The received ciphertext is mixed into the handshake digest. This step serves
+       to ensure the payload wasn't modified by a MITM.
+
+====== Act Two
+
+```
+   <- e, ee
+```
+
+Act Two is sent from the responder to the initiator. Act Two will _only_
+take place if Act One was successful. Act One was successful if the
+responder was able to properly decrypt and check the MAC of the tag sent at
+the end of Act One.
+
+The handshake is _exactly_ 50 bytes: 1 byte for the handshake version, 33
+bytes for the compressed ephemeral public key of the responder, and 16 bytes
+for the `poly1305` tag.
+
+Sender actions:
+
+1. `e = generateKey()`
+2. `h = SHA-256(h || e.pub.serializeCompressed())`
++
+The newly generated ephemeral key is accumulated into the running
+       handshake digest.
+3. `ee = ECDH(e.priv, re)`
++
+Where `re` is the ephemeral key of the initiator, which was received
+       during Act One.
+4. `ck, temp_k2 = HKDF(ck, ee)`
++
+A new temporary encryption key is generated, which is
+       used to generate the authenticating MAC.
+5. `c = encryptWithAD(temp_k2, 0, h, zero)`
++
+Where `zero` is a zero-length plain text.
+6. `h = SHA-256(h || c)`
++
+Finally, the generated ciphertext is accumulated into the authenticating
+       handshake digest.
+7. Send `m = 0 || e.pub.serializeCompressed() || c` to the initiator over the network buffer.
+
+Receiver actions:
+
+1. Read _exactly_ 50 bytes from the network buffer.
+2. Parse the read message (`m`) into `v`, `re`, and `c`:
++
+Where `v` is the _first_ byte of `m`, `re` is the next 33
+      bytes of `m`, and `c` is the last 16 bytes of `m`.
+3. If `v` is an unrecognized handshake version, then the responder must
+    abort the connection attempt.
+4. `h = SHA-256(h || re.serializeCompressed())`
+5. `ee = ECDH(e.priv, re)`
++
+Where `re` is the responder's ephemeral public key.
++
+The raw bytes of the remote party's ephemeral public key (`re`) are to be
+      deserialized into a point on the curve using affine coordinates as encoded
+      by the key's serialized composed format.
+6. `ck, temp_k2 = HKDF(ck, ee)`
++
+A new temporary encryption key is generated, which is
+       used to generate the authenticating MAC.
+7. `p = decryptWithAD(temp_k2, 0, h, c)`
++
+If the MAC check in this operation fails, then the initiator must
+      terminate the connection without any further messages.
+8. `h = SHA-256(h || c)`
++
+The received ciphertext is mixed into the handshake digest. This step serves
+       to ensure the payload wasn't modified by a MITM.
+
+====== Act Three
+
+```
+   -> s, se
+```
+
+Act Three is the final phase in the authenticated key agreement described in
+this section. This act is sent from the initiator to the responder as a
+concluding step. Act Three is executed _if and only if_ Act Two was successful.
+During Act Three, the initiator transports its static public key to the
+responder encrypted with _strong_ forward secrecy, using the accumulated `HKDF`
+derived secret key at this point of the handshake.
+
+The handshake is _exactly_ 66 bytes: 1 byte for the handshake version, 33
+bytes for the static public key encrypted with the `ChaCha20` stream
+cipher, 16 bytes for the encrypted public key's tag generated via the AEAD
+construction, and 16 bytes for a final authenticating tag.
+
+Sender actions:
+
+1. `c = encryptWithAD(temp_k2, 1, h, s.pub.serializeCompressed())`
++
+Where `s` is the static public key of the initiator.
+2. `h = SHA-256(h || c)`
+3. `se = ECDH(s.priv, re)`
++
+Where `re` is the ephemeral public key of the responder.
+4. `ck, temp_k3 = HKDF(ck, se)`
++
+The final intermediate shared secret is mixed into the running chaining key.
+5. `t = encryptWithAD(temp_k3, 0, h, zero)`
++
+Where `zero` is a zero-length plain text.
+6. `sk, rk = HKDF(ck, zero)`
++
+Where `zero` is a zero-length plain text,
+       `sk` is the key to be used by the initiator to encrypt messages to the
+       responder,
+       and `rk` is the key to be used by the initiator to decrypt messages sent by
+       the responder.
++
+The final encryption keys, to be used for sending and
+       receiving messages for the duration of the session, are generated.
+7. `rn = 0, sn = 0`
++
+The sending and receiving nonces are initialized to 0.
+8. Send `m = 0 || c || t` over the network buffer.
+
+Receiver actions:
+
+1. Read _exactly_ 66 bytes from the network buffer.
+2. Parse the read message (`m`) into `v`, `c`, and `t`:
++
+Where `v` is the _first_ byte of `m`, `c` is the next 49
+      bytes of `m`, and `t` is the last 16 bytes of `m`.
+3. If `v` is an unrecognized handshake version, then the responder must
+    abort the connection attempt.
+4. `rs = decryptWithAD(temp_k2, 1, h, c)`
++
+At this point, the responder has recovered the static public key of the
+       initiator.
+5. `h = SHA-256(h || c)`
+6. `se = ECDH(e.priv, rs)`
++
+Where `e` is the responder's original ephemeral key.
+7. `ck, temp_k3 = HKDF(ck, se)`
+8. `p = decryptWithAD(temp_k3, 0, h, t)`
++
+If the MAC check in this operation fails, then the responder must
+       terminate the connection without any further messages.
+9. `rk, sk = HKDF(ck, zero)`
++
+Where `zero` is a zero-length plain text,
+       `rk` is the key to be used by the responder to decrypt the messages sent
+       by the initiator,
+       and `sk` is the key to be used by the responder to encrypt messages to
+       the initiator.
++
+The final encryption keys, to be used for sending and
+       receiving messages for the duration of the session, are generated.
+10. `rn = 0, sn = 0`
++
+The sending and receiving nonces are initialized to 0.(((range="endofrange", startref="ix_14_encrypted_transport-asciidoc6")))(((range="endofrange", startref="ix_14_encrypted_transport-asciidoc5")))(((range="endofrange", startref="ix_14_encrypted_transport-asciidoc4")))
+
+===== Transport message encryption
+
+((("Lightning encrypted transport protocol","transport message encryption")))((("Noise_XK","transport message encryption")))At the conclusion of Act Three, both sides have derived the encryption keys, which
+will be used to encrypt and decrypt messages for the remainder of the
+session.
+
+The actual Lightning Protocol messages are encapsulated within AEAD ciphertexts.
+Each message is prefixed with another AEAD ciphertext, which encodes the total
+length of the following Lightning message (not including its MAC).
+
+The _maximum_ size of _any_ Lightning message must not exceed 65,535 bytes. A
+maximum size of 65,535 simplifies testing, makes memory management easier, and
+helps mitigate memory-exhaustion attacks.
+
+To make traffic analysis more difficult, the length prefix for all
+encrypted Lightning messages is also encrypted. Additionally a 16-byte
+`Poly-1305` tag is added to the encrypted length prefix to ensure that
+the packet length hasn't been modified when in flight and also to avoid
+creating a decryption oracle.
+
+The structure of packets on the wire resembles the diagram in <<noise_encrypted_packet>>.
+
+[[noise_encrypted_packet]]
+.Encrypted packet structure
+image::images/mtln_1402.png["Encrypted Packet Structure"]
+
+The prefixed message length is encoded as a 2-byte big-endian integer, for a
+total maximum packet length of pass:[<span>2 + 16 + 65,535 + 16 = 65,569</span>] bytes.
+
+====== Encrypting and sending messages
+
+To encrypt and send a Lightning message (`m`) to the network stream,
+given a sending key (`sk`) and a nonce (`sn`), the following steps are
+completed:
+
+[role="pagebreak-before"]
+1. Let `l = len(m)`.
++
+Where `len` obtains the length in bytes of the Lightning message.
+2. Serialize `l` into 2 bytes encoded as a big-endian integer.
+3. Encrypt `l` (using `ChaChaPoly-1305`, `sn`, and `sk`), to obtain `lc`
+    (18 bytes).
+    * The nonce `sn` is encoded as a 96-bit little-endian number. As the
+      decoded nonce is 64 bits, the 96-bit nonce is encoded as 32 bits
+      of leading zeros followed by a 64-bit value.
+        * The nonce `sn` must be incremented after this step.
+    * A zero-length byte slice is to be passed as the AD (associated data).
+4. Finally, encrypt the message itself (`m`) using the same procedure used to
+    encrypt the length prefix. Let this encrypted ciphertext be known pass:[<span class="keep-together">as <code>c</code></span>].
++
+The nonce `sn` must be incremented after this step.
+5. Send `lc || c` over the network buffer.
+
+====== Receiving and decrypting messages
+
+To decrypt the _next_ message in the network stream, the following
+steps are completed:
+
+1. Read _exactly_ 18 bytes from the network buffer.
+2. Let the encrypted length prefix be known as `lc`.
+3. Decrypt `lc` (using `ChaCha20-Poly1305`, `rn`, and `rk`) to obtain the size of
+    the encrypted packet `l`.
+    * A zero-length byte slice is to be passed as the AD (associated data).
+    * The nonce `rn` must be incremented after this step.
+4. Read _exactly_ `l + 16` bytes from the network buffer, and let the bytes be
+    known pass:[<span class="keep-together">as <code>c</code></span>].
+5. Decrypt `c` (using `ChaCha20-Poly1305`, `rn`, and `rk`) to obtain decrypted
+    plain-text packet `p`.
++
+The nonce `rn` must be incremented after this step.
+
+===== Lightning message key rotation
+
+((("Lightning encrypted transport protocol","Lightning message key rotation")))((("Noise_XK","Lightning message key rotation")))Changing keys regularly and forgetting previous keys is useful to prevent the
+decryption of old messages, in the case of later key leakage (i.e.,  backward
+secrecy).
+
+Key rotation is performed for _each_ key (`sk` and `rk`) _individually_. A key
+is to be rotated after a party encrypts or decrypts 1,000 times with it (i.e.,
+every 500 messages).  This can be properly accounted for by rotating the key
+once the nonce dedicated to it exceeds 1,000.
+
+Key rotation for a key `k` is performed according to the following steps(((range="endofrange", startref="ix_14_encrypted_transport-asciidoc3")))(((range="endofrange", startref="ix_14_encrypted_transport-asciidoc2"))):(((range="endofrange", startref="ix_14_encrypted_transport-asciidoc1")))
+
+1. Let `ck` be the chaining key obtained at the end of Act Three.
+2. `ck', k' = HKDF(ck, k)`
+3. Reset the nonce for the key to `n = 0`.
+4. `k = k'`
+5. `ck = ck'`
+
+=== Conclusion
+
+Lightning's underlying transport encryption is based on the Noise Protocol and offers strong security guarantees of privacy, authenticity, and integrity for all communications between Lightning peers.
+
+Unlike Bitcoin where peers often communicate "in the clear" (without encryption), all Lightning communications are encrypted peer-to-peer. In addition to transport encryption (peer-to-peer), in the Lightning Network, payments are _also_ encrypted into onion packets (hop-to-hop) and payment details are sent out-of-band between the sender and recipient (end-to-end). The combination of all these security mechanisms is cumulative and provides a layered defense against de-anonymization, man-in-the-middle attacks, and network surveillance.
+
+Of course, no security is perfect and we will see in <<security_and_privacy>> that these properties can be degraded and attacked. However, the Lightning Network significantly improves upon the privacy of Bitcoin.(((range="endofrange", startref="ix_14_encrypted_transport-asciidoc0")))

15_payment_requests.asciidoc

@@ -0,0 +1,221 @@
+[[invoices]]
+== Lightning Payment Requests
+
+((("Lightning invoices", id="ix_15_payment_requests-asciidoc0", range="startofrange")))In this chapter we will look at _Lightning payment requests_, or as they are more commonly known, _Lightning invoices_.
+
+=== Invoices in the Lightning Protocol Suite
+
+((("Lightning invoices","Lightning Protocol suite and")))((("Lightning Network Protocol","Lightning invoices in")))_Payment requests_, aka _invoices_, are part of the payment layer and are shown in the upper left of <<LN_payment_request_highlight>>.
+
+[[LN_payment_request_highlight]]
+.Payment requests in the Lightning protocol suite
+image::images/mtln_1501.png["Payment requests in the Lightning protocol suite"]
+
+=== Introduction
+
+As we've learned throughout the book, minimally two pieces of data are required
+to complete a Lightning payment: a payment hash and a destination. As
+SHA-256 is used in the Lightning Network to implement HTLCs, this information
+requires 32 bytes to communicate. Destinations, on the other hand, are
+simply the `secp256k1` public key of the node that wishes to receive a payment.
+The purpose of a payment request in the context of the Lightning Network is to
+communicate these two pieces of information from sender to receiver. The QR-code-friendly format for communicating the information required
+to complete a payment from receiver to sender is described in https://github.com/lightningnetwork/lightning-rfc/blob/master/11-payment-encoding.md[BOLT #11: Invoice Protocol for Lightning Payments]. In practice, more than just the
+payment hash and destination are communicated in a payment request to
+make the encoding more fully featured.
+
+=== Lightning Payment Requests Versus Bitcoin Addresses
+
+((("Bitcoin addresses, Lightning invoices versus")))((("Lightning invoices","Bitcoin addresses versus")))A commonly asked question when people first encounter a Lightning Payment
+request is: why can't a normal static address format be used instead?
+
+To answer this question, you must first internalize how Lightning
+differs from base layer Bitcoin as a payment method. Compared to a Bitcoin
+address which may be used to make a potentially unbounded number of payments
+(though reusing a Bitcoin address may degrade one's privacy), a Lightning
+payment request should only ever be used _once_.  This is due to the fact that
+sending a payment to a Bitcoin address essentially uses a public key
+cryptosystem to "encode" the payment in a manner that only the true "owner" of
+that Bitcoin address can redeem it.
+
+In contrast, to complete a Lightning payment, the recipient must
+reveal a "secret" to the entire payment route, including the sender. This can be
+interpreted as usage of a kind of domain-specific symmetric cryptography because
+the payment preimage is for practical purposes a nonce (number only used
+once). If the sender attempts to make another payment using that identical
+payment hash, then they risk losing funds because the payment may not actually be
+delivered to the destination. It's safe to assume that after a preimage has
+been revealed, all nodes in the path will keep it around forever, then rather
+than forward the HTLC to collect a routing fee if the payment is
+completed, they can simply settle the payment at that instance and gain the
+entire payment amount in return. As a result, it's unsafe to ever use a payment
+request more than once.
+
+New variants of the original Lightning payment request exist that allow the sender to reuse them as many times as they want. These variants flip the normal payment flow as the sender transmits a preimage within the encrypted onion payload to the receiver, who is the only
+one that is able to decrypt it and settle the payment. Alternatively, assuming
+a mechanism that allows a sender to typically request a new payment request
+from the receiver, then an interactive protocol can be used to allow a
+degree of payment request reuse.
+
+=== BOLT #11: Lightning Payment Request Serialization pass:[<span class="keep-together">and Interpretation</span>]
+
+((("BOLT (Basis of Lightning Technology) standards documents","Lightning payment request serialization/interpretation")))((("Lightning invoices","payment request serialization/interpretation")))In this section, we'll describe the mechanism used to encode the set of
+information required to complete a payment on the Lightning Network. As
+mentioned earlier, the payment hash and destination is the minimum amount of
+information required to complete a payment. However, in practice, more
+information such as timelock information, payment request expiration, and
+possibly an on-chain fallback address are also communicated. The full specification document is https://github.com/lightningnetwork/lightning-rfc/blob/master/11-payment-encoding.md[BOLT #11: Invoice Protocol for Lightning Payments].
+
+==== Payment Request Encoding in Practice
+
+((("Lightning invoices","payment request encoding in practice")))First, let's examine what a real payment request looks like in practice. The
+following is a valid payment request that could have been used to complete a
+payment on the mainnet Lightning Network at the time it was created:
+
+----
+lnbc2500u1pvjluezpp5qqqsyqcyq5rqwzqfqqqsyqcyq5rqwzqfqqqsyqcyq5rqwzqfqypqdq5xysx
+xatsyp3k7enxv4jsxqzpuaztrnwngzn3kdzw5hydlzf03qdgm2hdq27cqv3agm2awhz5se903vruatf
+hq77w3ls4evs3ch9zw97j25emudupq63nyw24cg27h2rspfj9srp
+----
+
+==== The Human-Readable Prefix
+
+((("human-readable prefixes")))((("Lightning invoices","human-readable prefix")))Looking at the string, we can tease out a portion that we can parse with our
+eyes, while the rest of it just looks like a random set of strings. The part
+that is somewhat parsable by a human is referred to as the _human-readable prefix_. It allows a human to quickly extract some relevant information from a
+payment request at a glance. In this case, we can see that this payment is for
+the mainnet instance of the Lightning Network (`lnbc`), and is requesting 2,500
+uBTC (microbitcoin), or 25,000,000 satoshis. The latter potion is referred
+to as the data portion and uses an extensible format to encode the
+information required to complete a payment.
+
+Each version of instance of the Lightning Network (mainnet, testnet, etc.) has
+its own human-readable prefix (see <<table1501>>). This allows client software and also humans to
+quickly determine if a payment request can be satisfied by their node or not.
+
+[role="pagebreak-before less_space"]
+[[table1501]]
+.BOLT #11 network prefixes
+[options="header"]
+|=============================
+|Network       |BOLT #11 prefix
+|mainnet       |`lnbc`
+|testnet       |`lntb`
+|simnet/regtest|`lnbcrt`
+|=============================
+
+
+The first portion of the human-readable prefix is a _compact_ expression of the
+amount of the payment request. The compact amount is encoded in two parts. First, an integer is used as the _base_ amount. This is then followed by a
+multiplier that allows us to specify distinct order of magnitude increases
+offset by the base amount. If we return to our initial example, then we can
+take the `2500u` portion and decrease it by a factor of 1,000 to instead use
+`2500m` or (2,500 mBTC).  As a rule of thumb, to ascertain the amount
+of an invoice at a glance, take the base factor and multiply it by the
+multiplier.
+
+A full list of the currently defined multipliers is given in <<table1502>>.
+
+[[table1502]]
+.BOLT #11 amount multipliers
+[options="header"]
+|==============================================
+|Multiplier|Bitcoin unit|Multiplication factor
+|`m`|milli|0.001
+|`u`|micro|0.000001
+|`n`|nano|0.000000001
+|`p`|pico|0.000000000001
+|==============================================
+
+
+==== bech32 and the Data Segment
+
+((("bech32, Lightning invoices and")))((("Lightning invoices","bech32 and data segment")))If the "unreadable" portion looks familiar, then that's because it uses the
+very same encoding scheme as SegWit compatible Bitcoin addresses use today,
+namely bech32. Describing the bech32 encoding scheme is outside the scope
+of this chapter. In brief, it's a sophisticated way to encode short strings
+that has very good error correction as well as detection properties.
+
+The data portion can be separated into three sections:
+
+  * The timestamp
+  * Zero or more tagged key-value pairs
+  * The signature of the entire invoice
+
+The timestamp is expressed in seconds since the year 1970, or the Unix Epoch. This
+timestamp allows the sender to gauge how old the invoice is, and as we'll see
+later, allows the receiver to force an invoice to only be valid for a period of
+time if they wish.
+
+Similar to the TLV format we learned about in <<tlv>>, the BOLT #11 invoice
+format uses a series of extensible key-value pairs to encode information
+needed to satisfy a payment. Because key-value pairs are used, it's easy to add
+new values in the future if a new payment type or additional
+requirement/functionality is introduced.
+
+Finally, a signature is included that covers the entire invoice signed by the
+destination of the payment. This signature allows the sender to verify that the
+payment request was indeed created by the destination of the payment. Unlike
+Bitcoin payment requests which aren't signed, this allows us to ensure that a
+particular entity signed the payment request. The signature itself is encoded
+using a recovery ID, which allows a more compact signature to be used that
+allows public key extraction. When verifying the signature, the recovery ID
+extracts the public key, then verifies that against the public key included in
+the invoice.
+
+===== Tagged invoice fields
+
+((("Lightning invoices","tagged invoice fields")))The tagged invoice fields are encoded in the main body of the invoice. These
+fields represent different key-value pairs that express either additional
+information that may help complete the payment or information which is
+_required_ to complete the payment. Because a slight variant of bech32 is
+utilized, each of these fields are actually in the "base 5" domain.
+
+A given tag field is comprised of three components:
+
+  * The `type` of the field (5 bits)
+  * The `length` of the data of the field (10 bits)
+  * The `data` itself, which is `length * 5 bytes` in size
+
+A full list of all the currently defined tagged fields is given in <<table1503>>.
+
+[[table1503]]
+.BOLT #11 tagged invoice fields
+[options="header"]
+|===
+|pass:[<span class="keep-together">Field tag</span>]|pass:[<span class="keep-together">Data length</span>]|Usage
+|`p`|`52`|The SHA-256 payment hash.
+|`s`|`52`|A 256-bit secret that increases the end-to-end privacy of a payment by mitigating probing by intermediate nodes.
+|`d`|Variable|The description, a short UTF-8 string of the purpose of the payment.
+|`n`|`53`|The public key of the destination node.
+|`h`|`52`|A hash that represents a description of the payment itself. This can be used to commit to a description that's over 639 bytes in length.
+|`x`|Variable|The expiry time, in seconds, of the payment. The default is 1 hour (3,600) if not specified.
+|`c`|Variable|The `min_cltv_expiry` to use for the final hop in the route. The default is 9 if not specified.
+|`f`|Variable|A fallback on-chain address to be used to complete the payment if the payment cannot be completed over the Lightning Network.
+|`r`|Variable|One or more entries that allow a receiver to give the sender additional ephemeral edges to complete the payment.
+|`9`|Variable|A set of 5-bit values that contain the feature bits that are required in order to complete the payment.
+|===
+
+The elements contained in the field `r` are commonly referred to as _routing hints_. They allow the receiver to communicate an extra set of edges that may
+help the sender complete their payment. The hints are usually used when the
+receiver has some/all private channels, and they wish to guide the sender into
+this "unmapped" portion of the channel graph. A routing hint encodes
+effectively the same information that a normal `channel_update` message does.
+The update is itself packed into a single value with the following fields:
+
+ * The `pubkey` of the outgoing node in the edge (264 bits)
+ * The `short_channel_id` of the "virtual" edge (64 bits)
+ * The base fee (`fee_base_msat`) of the edge (32 bits)
+ * The proportional fee (`fee_proportional_millionths`) (32 bits)
+ * The CLTV expiry delta (`cltv_expiry_delta`) (16 bits)
+
+The final portion of the data segment is the set of feature bits that
+communicate to the sender the functionality needed to complete a
+payment. As an example, if a new payment type is added in the future that isn't
+backward compatible with the original payment type, then the receiver can set
+a _required_ feature bit to communicate that the payer needs to
+understand that feature to complete the payment.
+
+=== Conclusion
+
+As we have seen, invoices are a lot more than just a request for an amount. They contain critical information about _how_ to make the payment, such as routing hints, the destination node's public key, ephemeral keys to increase security, and much more.(((range="endofrange", startref="ix_15_payment_requests-asciidoc0")))

16_security_privacy_ln.asciidoc

@@ -0,0 +1,622 @@
+[[security_and_privacy]]
+== Security and Privacy of the pass:[<span class="keep-together">Lightning Network</span>]
+
+((("security and privacy", id="ix_16_security_privacy_ln-asciidoc0", range="startofrange")))In this chapter, we look at some of the most important issues related to the security and privacy of the Lightning Network. First, we'll consider privacy, what it means, how to evaluate it, and some things you can do to protect your own privacy while using the Lightning Network. Then we'll explore some common attacks and mitigation techniques.
+
+=== Why Is Privacy Important?
+
+((("security and privacy","importance of privacy")))The key value proposition of cryptocurrency is censorship resistant money. Bitcoin offers participants the possibility of storing and transferring their wealth without interference by governments, banks, or corporations. The Lightning Network continues this mission.
+
+Unlike trivial scaling solutions like custodial Bitcoin banks, the Lightning Network aims to scale Bitcoin without compromising on self custody, which should lead to greater censorship resistance in the Bitcoin ecosystem. However, the Lightning Network operates under a different security model, which introduces novel security and privacy challenges.
+
+=== Definitions of Privacy
+
+((("security and privacy","definitions of privacy", id="ix_16_security_privacy_ln-asciidoc1", range="startofrange")))The question, "Is Lightning private?" has no direct answer. Privacy is a complex topic; it is often difficult to precisely define what we mean by privacy, particularly if you are not a privacy researcher. Fortunately, privacy researchers use processes to analyze and evaluate the privacy characteristics of systems, and we can use them too! Let's look at how a security researcher might seek to answer the question, "Is Lightning private?" in two general steps.
+
+First, a privacy researcher would define a _security model_ that specifies what an adversary is capable of and aims to achieve.
+Then, they would describe the relevant properties of the system and check whether it conforms to the requirements.
+
+=== Process to Evaluate Privacy
+
+((("security and privacy","evaluation process for privacy")))((("security assumptions")))A security model is based on a set of underlying _security assumptions_.
+In cryptographic systems, these assumptions are often centered around the mathematical properties of the cryptographic primitives, such as ciphers, signatures, and hash functions.
+The security assumptions of the Lightning Network are that the ECDSA signatures, SHA-256 hash function, and other cryptographic functions used in the protocol behave within their security definitions.
+For example, we assume that it is practically impossible to find a preimage (and second preimage) of a hash function.
+This allows the Lightning Network to rely on the HTLC mechanism (which uses the preimage of a hash function) for the atomicity of multihop payments: nobody except the final recipient can reveal the payment secret and resolve the HTLC.
+We also assume a degree of connectivity in the network, namely that Lightning channels form a connected graph. Therefore, it is possible to find a path from any sender to any receiver. Finally, we assume network messages are propagated within certain timeouts.
+
+Now that we've identified some of our underlying assumptions, let's consider some possible adversaries.
+
+Here are some possible models of adversaries in the Lightning Network.
+An "honest-but-curious" forwarding node can observe payment amounts, the immediately preceding and following nodes, and the graph of announced channels with their capacities.
+A very well-connected node can do the same but to a larger extent.
+For example, consider the developers of a popular wallet who maintain a node that their users connect to by default.
+This node would be responsible for routing a large share of payments to and from the users of that wallet.
+What if multiple nodes are under adversarial control?
+If two colluding nodes happen to be on the same payment path, they would understand that they are forwarding HTLCs belonging to the same payment because HTLCs have the same payment hash.
+
+[NOTE]
+====
+Multipart payments (see <<mpp>>) enable users to obfuscate their payment amounts given their nonuniform split sizes.
+====
+
+What may be the goals of a Lightning attacker?
+Information security is often described in terms of three main properties: confidentiality, integrity, and availability.
+
+Confidentiality:: The information only gets to intended recipients.
+Integrity:: The information does not get altered in transit.
+Availability:: The system is functioning most of the time.
+
+The important properties of the Lightning Network are mostly centered around confidentiality and availability. Some of the most important properties to protect include:
+
+* Only the sender and the receiver know the payment amount.
+* No one can link senders and receivers.
+* An honest user cannot be blocked from sending and receiving payments.
+
+For each privacy goal and security model, there is a certain probability that an attacker succeeds.
+This probability depends on various factors, such as the size and structure of the network.
+Other things being equal, it is generally easier to successfully attack a small network rather than a large one.
+Similarly, the more centralized the network is, the more capable an attacker can be if "central" nodes are under their control.
+Of course, the term centralization must be defined precisely to build security models around it, and there are many possible definitions of how centralized a network is.
+Finally, as a payment network, the Lightning Network depends on economic stimuli.
+The size and structure of fees affect the routing algorithm, and therefore can either aid the attacker by forwarding most payments through their nodes or prevent this from happening.(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc1")))
+
+
+=== Anonymity Set
+
+((("anonymity set")))((("de-anonymization")))((("security and privacy","anonymity set")))What does it mean to de-anonymize someone?
+In simple terms, de-anonymization implies linking some action with a person's real-world identity, such as their name or physical address.
+In privacy research, the notion of de-anonymization is more nuanced.
+First, we are not necessarily talking about names and addresses.
+Discovering someone's IP address or telephone number may also be considered de-anonymization.
+A piece of information that allows linking a user's action to their previous actions is referred to as _identity_.
+Second, de-anonymization is not binary; a user is neither fully anonymous nor completely de-anonymized.
+Instead, privacy research looks at anonymity compared to the anonymity set.
+
+The _anonymity set_ is a central notion in privacy research.
+It refers to the set of identities such that, from an attacker's viewpoint, a given action could correspond to anyone in the set.
+Consider a real-life example.
+Imagine you meet a person on a city street.
+What is their anonymity set from your point of view?
+If you don't know them personally, and without any additional information, their anonymity set roughly equals the city's population, including travelers.
+If you additionally consider their appearance, you may be able to roughly estimate their age and exclude the city residents who are obviously older or younger than the person in question from the anonymity set.
+Furthermore, if you notice that the person walks into the office of Company X using an electronic badge, the anonymity set shrinks to the number pass:[<span class="keep-together">of Company</span>] X's employees and visitors.
+Finally, you may notice the license number of the car they used to arrive at the place.
+If you are a casual observer, this doesn't give you much.
+However, if you are a city official and have access to the database that matches license plate numbers with names, you can narrow down the anonymity set to just a few people: the car owner and any close friends and relatives that may have borrowed the car.
+
+This example illustrates a few important points.
+First, every bit of information may bring the adversary closer to their goal.
+It may not be necessary to shrink the anonymity set to the size of one.
+For instance, if the adversary plans a targeted denial-of-service (DoS) attack and can take down 100 servers, the anonymity set of 100 suffices.
+Second, the adversary can cross-correlate information from different sources.
+Even if a privacy leak looks relatively benign, we never know what it can achieve in combination with other data sources.
+Finally, especially in cryptographic settings, the attacker always has the "last resort" of a brute-force search.
+Cryptographic primitives are designed so that it is practically impossible to guess a secret such as a private key.
+Nevertheless, each bit of information brings the adversary closer to this goal, and at some point, it becomes attainable.
+
+In terms of Lightning, de-anonymizing generally means deriving a correspondence between payments and users identified by node IDs.
+Each payment may be assigned a sender anonymity set and a receiver anonymity set.
+Ideally, the anonymity set consists of all the users of the network.
+This assures that the attacker has no information whatsoever.
+However, the real network leaks information that allows an attacker to narrow down the search.
+The smaller the anonymity set, the higher the chance of successful de-anonymization.
+
+[role="pagebreak-before less_space"]
+=== Differences Between the Lightning Network and Bitcoin in Terms of Privacy
+
+((("security and privacy","differences between Lightning Network and Bitcoin in terms of privacy", id="ix_16_security_privacy_ln-asciidoc2", range="startofrange")))While it's true that transactions on the Bitcoin network do not associate real-world identities with Bitcoin addresses, all transactions are broadcast in cleartext and can be analyzed.
+Multiple companies have been established to de-anonymize users of Bitcoin and other cryptocurrencies.
+
+At first glance, Lightning provides better privacy than Bitcoin because Lightning payments are not broadcast to the whole network.
+While this improves the privacy baseline, other properties of the Lightning protocol may make anonymous payments more challenging.
+For instance, larger payments may have fewer routing options.
+This may allow an adversary who controls well-capitalized nodes to route most large payments and discover payment amounts and probably other details. Over time, as the Lightning Network grows, this may become less of a problem.
+
+Another relevant difference between Lightning and Bitcoin is that Lightning nodes maintain a permanent identity, whereas Bitcoin nodes do not.
+A sophisticated Bitcoin user can easily switch nodes used to receive blockchain data and broadcast transactions.
+A Lightning user, on the contrary, sends and receives payments through the nodes they have used to open their payment channels.
+Moreover, the Lightning protocol assumes that routing nodes announce their IP address in addition to their node ID.
+This creates a permanent link between node IDs and IP addresses, which may be dangerous, considering that an IP address is often an intermediary step in anonymity attacks linked to the user's physical location and, in most cases, real-world identity.
+It is possible to use Lightning over Tor, but many nodes do not use this functionality, as can be seen from https://1ml.com/statistics[statistics collected from node announcements].
+
+A Lightning user, when sending a payment, has its neighbors in its anonymity set.
+Specifically, a routing node only knows the immediately preceding and following nodes.
+The routing node does not know whether its immediate neighbors in the payment route are the ultimate sender or receiver.
+Therefore, the anonymity set of a node in Lightning roughly equals its neighbors (see <<anonymity_set>>).
+
+[[anonymity_set]]
+.The anonymity set of Alice and Bob constitutes their neighbors
+image::images/mtln_1601.png["The anonymity set of Alice and Bob constitutes their neighbors"]
+
+Similar logic applies to payment receivers.
+Many users open only a handful of payment channels, therefore limiting their anonymity sets.
+Moreover, in Lightning, the anonymity set is static or at least slowly changing.
+In contrast, one can achieve significantly larger anonymity sets in on-chain CoinJoin transactions.
+CoinJoin transactions with anonymity sets larger than 50 are quite frequent.
+Typically, the anonymity sets in a CoinJoin transaction correspond to a dynamically changing set of users.
+
+Finally, Lightning users can also be denied service, having their channels blocked or depleted by an attacker.
+Forwarding payments requires capital—a scarce resource!—to be temporarily blocked in HTLCs along the route.
+An attacker may send many payments but fail to finalize them, occupying honest users' capital for long periods.
+This attack vector is not present (or at least not as obvious) in Bitcoin.
+
+In summary, while some aspects of the Lightning Network's architecture suggest that it is a step forward in terms of privacy compared to Bitcoin, other properties of the protocol may make attacks on privacy easier. Thorough research is needed to evaluate what privacy guarantees the Lightning Network provides and improve the state of affairs.
+
+The issues discussed in this part of the chapter summarize research available in mid-2021. However, this area of research and development is growing quickly. We are happy to report that the authors are aware of multiple research teams currently working on Lightning privacy.
+
+Now let's review some of the attacks on LN privacy that have been described in academic literature.(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc2")))
+
+
+=== Attacks on Lightning
+
+((("security and privacy","attacks on Lightning", seealso="breaches of privacy", id="ix_16_security_privacy_ln-asciidoc3", range="startofrange")))Recent research describes various ways in which the security and privacy of the Lightning Network may be compromised.
+
+==== Observing Payment Amounts
+
+((("breaches of privacy","observing payment amounts")))One of the goals for a privacy-preserving payment system is to hide the payment amount from uninvolved parties.
+The Lightning Network is an improvement over Layer 1 in this regard.
+While Bitcoin transactions are broadcast in cleartext and can be observed by anyone, Lightning payments only travel through a few nodes along the payment path.
+However, intermediary nodes do see the payment amount, although this payment amount might not correspond to the actual total payment amount (see <<mpp>>).
+This is necessary to create a new HTLC at every hop.
+The availability of payment amounts to intermediary nodes do not present an immediate threat.
+However, an _honest-but-curious_ intermediary node may use it as a part of a larger attack.
+
+
+==== Linking Senders and Receivers
+
+((("breaches of privacy","linking senders and receivers", id="ix_16_security_privacy_ln-asciidoc4", range="startofrange")))An attacker might be interested in learning the sender and/or the receiver of a payment to reveal certain economic relationships.
+This breach of privacy could harm censorship resistance, as an intermediary node could censor payments to or from certain receivers or senders.
+Ideally, linking senders to receivers should not be possible to anyone other than the sender and the receiver.
+
+In the following sections, we will consider two types of adversaries: the off-path adversary and the on-path adversary.
+An off-path adversary tries to assess the sender and the receiver of a payment without participating in the payment routing process.
+An on-path adversary can leverage any information it might gain by routing the payment of interest.
+
+((("off-path adversary")))First, consider the _off-path adversary_.
+In the first step of this attack scenario, a potent off-path adversary deduces the individual balances in each payment channel via probing (described in a subsequent section) and forms a network snapshot at time __t~1~__. For simplicity's sake, let's make __t~1~__ equal 12:05.
+It then probes the network again at sometime later at time __t~2~__, which we'll make 12:10. The attacker would then compare the snapshots at 12:10 and 12:05 and use the differences between the two snapshots to infer information about payments that took place by looking at paths that have changed.
+In the simplest case, if only one payment occurred between 12:10 and 12:05, the adversary would observe a single path where the balances have changed by the same amounts.
+Thus, the adversary learns almost everything about this payment: the sender, the recipient, and the amount.
+If multiple payment paths overlap, the adversary needs to apply heuristics to identify such overlap and separate the payments.
+
+((("on-path adversary")))Now, we turn our attention to an _on-path adversary_.
+Such an adversary might seem convoluted.
+However, in June 2020, researchers noted that the single most central node https://arxiv.org/pdf/2006.12143.pdf[observed close to 50% of all LN payments], while the four most central nodes https://arxiv.org/pdf/1909.06890.pdf[observed an average of 72% payments].
+These findings emphasize the relevance of the on-path attacker model.
+Even though intermediaries in a payment path only learn their successor and predecessor, there are several leakages that a malicious or honest-but-curious intermediary might use to infer the sender and the receiver.
+
+The on-path adversary can observe the amount of any routed payment as well as timelock deltas (see <<onion_routing>>).
+Hence, the adversary can exclude any nodes from the sender's or the receiver's anonymity set with capacities lower than the routed amount.
+Therefore, we observe a trade-off between privacy and payment amounts.
+Typically, the larger the payment amount is, the smaller the anonymity sets are.
+We note that this leakage could be minimized with multipart payments or with large capacity payment channels.
+Similarly, payment channels with small timelock deltas could be excluded from a payment path.
+More precisely, a payment channel cannot pertain to a payment if the remaining time the payment might be locked for is larger than what the forwarding node would be willing to accept.
+This leakage could be evicted by adhering to the so-called shadow routes.
+
+One of the most subtle and yet powerful leakages an on-path adversary can foster is the timing analysis.
+An on-path adversary can keep a log for every routed payment, along with the amount of time it takes for a node to respond to an HTLC request.
+Before starting the attack, the attacker learns every node's latency characteristics in the Lightning Network by sending them requests.
+Naturally, this can aid in establishing the adversary's precise position in the payment path.
+Even more, as it was recently shown, an attacker can successfully determine the sender and the receiver of a payment from a set of possible senders and receivers using time-based estimators.
+
+Finally, it's important to recognize that unknown or unstudied leakages probably exist that could aid de-anonymizing attempts. For instance, because different Lightning wallets apply different routing algorithms, even knowing the applied routing algorithm could help exclude certain nodes from being a sender and/or receiver of a payment.(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc4")))
+
+
+==== Revealing Channel Balances (Probing)
+
+((("breaches of privacy","revealing channel balances", id="ix_16_security_privacy_ln-asciidoc5", range="startofrange")))((("channel balances, revealing", id="ix_16_security_privacy_ln-asciidoc6", range="startofrange")))((("channel probing", id="ix_16_security_privacy_ln-asciidoc7", range="startofrange")))((("probing attack", id="ix_16_security_privacy_ln-asciidoc8", range="startofrange")))The balances of Lightning channels are supposed to be hidden for privacy and efficiency reasons.
+A Lightning node only knows the balances of its adjacent channels.
+The protocol provides no standard way to query the balance of a remote channel.
+
+However, an attacker can reveal the balance of a remote channel in a _probing attack_.
+In information security, probing refers to the technique of sending requests to a targeted system and making conclusions about its private state based on the received responses.
+
+Lightning channels are prone to probing.
+Recall that a standard Lightning payment starts with the receiver creating a random payment secret and sending its hash to the sender.
+Note that for the intermediary nodes, all hashes look random.
+There is no way to tell whether a hash corresponds to a real secret or was generated randomly.
+
+The probing attack proceeds as follows.
+Say the attacker Mallory wants to reveal Alice's balance of a public channel between Alice and Bob.
+Suppose the total capacity of that channel is 1 million satoshis.
+Alice's balance could be anything from zero to 1 million satoshis (to be precise, the estimate is a bit tighter due to channel reserve, but we don't account for it here for simplicity).
+Mallory opens a channel with Alice with 1 million satoshis and sends 500,000 satoshis to Bob via Alice using a _random number_ as the payment hash.
+Of course, this number does not correspond to any known payment secret.
+Therefore, the payment will fail.
+The question is: how exactly will it fail?
+
+There are two scenarios.
+If Alice has more than 500,000 satoshis on her side of the channel to Bob, she forwards the payment.
+Bob decrypts the payment onion and realizes that the payment is intended for him.
+He looks up his local store of payment secrets and searches for the preimage that corresponds to the payment hash, but does not find one.
+Following the protocol, Bob returns the "unknown payment hash" error to Alice, who relays it back to Mallory.
+As a result, Mallory knows that the payment _could have succeeded_ if the payment hash was real.
+Therefore, Mallory can update her estimation of Alice's balance from "between zero and 1 million" to "between 500,000 and 1 million."
+Another scenario happens if Alice's balance is lower than 500,000 satoshis.
+In that case, Alice is unable to forward the payment and returns the "insufficient balance" error to Mallory.
+Mallory updates her estimation from "between zero and 1 million" to "between zero and 500,000."
+
+Note that in any case, Mallory's estimation becomes twice as precise after just one probing!
+She can continue probing, choosing the next probing amount such that it divides the current estimation interval in half.
+((("binary search")))This well-known search technique is called _binary search_.
+With binary search, the number of probes is _logarithmic_ in the desired precision.
+For example, to obtain Alice's balance in a channel of 1 million satoshis up to a single satoshi, Mallory would only have to perform log~2~ (1,000,000) &asymp; 20 probings.
+If one probing takes 3 seconds, one channel can be precisely probed in only about a minute!
+
+Channel probing can be made even more efficient.
+In its simplest variant, Mallory directly connects to the channel she wants to probe.
+Is it possible to probe a channel without opening a channel to one of its endpoints?
+Imagine Mallory now wants to probe a channel between Bob and Charlie but doesn't want to open another channel, which requires paying on-chain fees and waiting for confirmations of the funding transactions.
+Instead, Mallory reuses her existing channel to Alice and sends a probe along the route Mallory -> Alice -> Bob -> Charlie.
+Mallory can interpret the "unknown payment hash" error in the same way as before: the probe has reached the destination; therefore, all channels along the route have sufficient balances to forward it.
+But what if Mallory receives the "insufficient balance" error?
+Does it mean that the balance is insufficient between Alice and Bob or between Bob and Charlie?
+
+In the current Lightning protocol, error messages report not only _which_ error occurred but also _where_ it happened.
+So, with more careful error handling, Mallory now knows which channel failed.
+If this is the target channel, she updates her estimates; if not, she chooses another route to the target channel.
+She even gets _additional_ information about the balances of intermediary channels, on top of that of the target channel.
+
+The probing attack can be further used to link senders and receivers, as described in the previous section.
+
+At this point, you may ask: why does the Lightning Network do such a poor job at protecting its users' private data?
+Wouldn't it be better to not reveal to the sender why and where the payment has failed?
+Indeed, this could be a potential countermeasure, but it has significant drawbacks.
+Lightning has to strike a careful balance between privacy and efficiency.
+Remember that regular nodes don't know balance distributions in remote channels.
+Therefore, payments can (and often do) fail because of insufficient balance at an intermediary hop.
+Error messages allow the sender to exclude the failing channel from consideration when constructing another route.
+One popular Lightning wallet even performs probing internally to check whether a constructed route can really handle a payment.
+
+There are other potential countermeasures against channel probing.
+First, it is hard for an attacker to target unannounced channels.
+Second, nodes that implement just-in-time (JIT) routing may be less prone to the attack.
+Finally, as multipart payments make the problem of insufficient capacity less severe, the protocol developers may consider hiding some of the error details without harming efficiency.(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc8")))(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc7")))(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc6")))(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc5")))
+
+[[denial_of_service]]
+==== Denial of Service
+
+((("breaches of privacy","denial-of-service attacks", id="ix_16_security_privacy_ln-asciidoc9", range="startofrange")))((("denial-of-service (DoS) attacks", id="ix_16_security_privacy_ln-asciidoc10", range="startofrange")))When resources are made publicly available, there is a risk that attackers may attempt to make that resource unavailable by executing a denial-of-service (DoS) attack.
+Generally, this is achieved by the attacker bombarding a resource with requests, which are indistinguishable from legitimate queries.
+The attacks seldom result in the target suffering financial loss, aside from the opportunity cost of their service being down, and are merely intended to aggrieve the target.
+
+Typical mitigations for DoS attacks require authentication for requests to separate legitimate users from malicious ones. These mitigations incur a trivial cost to regular users but will act as a sufficient deterrent to an attacker launching requests at scale.
+Anti-denial-of-service measures can be seen everywhere on the internet—websites apply rate limits to ensure that no one user can consume all of their server's attention, film review sites require login authentication to keep angry r/prequelmemes (Reddit group) members at bay, and data services sell API keys to limit the number of queries.
+
+===== DoS in bitcoin
+
+((("Bitcoin (system)","DoS attacks")))((("denial-of-service (DoS) attacks","DoS in Bitcoin")))In Bitcoin, the bandwidth that nodes use to relay transactions and the space that they avail to the network in the form of their mempool are publicly available resources.
+Any node on the network can consume bandwidth and mempool space by sending a valid transaction.
+If this transaction is mined in a valid block, they will pay transaction fees, which adds a cost to using these shared network resources.
+
+In the past, the Bitcoin network faced an attempted DoS attack where attackers spammed the network with low-fee transactions.
+Many of these transactions were not selected by miners due to their low transaction fees, so the attackers could consume network resources without paying the fees.
+To address this issue, a minimum transaction relay fee that set a threshold fee that nodes require to propagate transactions was set.
+This measure largely ensured that the transactions that consume network resources will eventually pay their chain fees.
+The minimum relay fee is acceptable to regular users but would hurt attackers financially if they tried to spam the network.
+While some transactions may not make it into valid blocks within high-fee environments, these measures have largely been effective at deterring this type of spam.
+
+===== DoS in Lightning
+
+((("denial-of-service (DoS) attacks","DoS in Lightning")))Similarly to Bitcoin, the Lightning Network charges fees for the use of its public resources, but in this case, the resources are public channels, and the fees come in the form of routing fees.
+The ability to route payments through nodes in exchange for fees provides the network with a large scalability benefit—nodes that are not directly connected can still transact—but it comes at the cost of exposing a public resource that must be protected against DoS attacks.
+
+When a Lightning node forwards a payment on your behalf, it uses data and payment bandwidth to update its commitment transaction, and the amount of the payment is reserved in their channel balance until it is settled or failed.
+In successful payments, this is acceptable because the node is eventually paid out its fees.
+Failed payments do not incur fees in the current protocol.
+This allows nodes to costlessly route failed payments through any channels.
+This is great for legitimate users, who wouldn't like to pay for failed attempts, but also allows attackers to costlessly consume nodes' resources—much like the low-fee transactions on Bitcoin that never end up paying miner fees.
+
+At the time of writing, a discussion is https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-June/002734.html[ongoing] on the lightning-dev mailing list as to how best address this issue.
+
+===== Known DoS attacks
+
+((("denial-of-service (DoS) attacks","known DoS attacks")))There are two known DoS attacks on public LN channels which render a target channel, or a set of target channels, unusable.
+Both attacks involve routing payments through a public channel, then holding them until their timeout, thus maximizing the attack's duration.
+The requirement to fail payments to not pay fees is fairly simple to meet because malicious nodes can simply reroute payments to themselves.
+In the absence of fees for failed payments, the only cost to the attacker is the on-chain cost of opening a channel to dispatch these payments through, which can be trivial in low-fee environments.(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc10")))(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc9")))
+
+==== Commitment Jamming
+
+((("breaches of privacy","commitment jamming")))((("commitment jamming")))Lightning nodes update their shared state using asymmetric commitment transactions, on which HTLCs are added and removed to facilitate payments.
+Each party is limited to a total of https://github.com/lightningnetwork/lightning-rfc/blob/c053ce7afb4cbf88615877a0d5fc7b8dbe2b9ba0/02-peer-protocol.md#the-open_channel-message[483] HTLCs in the commitment transaction at a time.
+A channel jamming attack allows an attacker to render a channel unusable by routing 483 payments through the target channel and holding them until they time out.
+
+It should be noted that this limit was chosen in the specification to ensure that all the HTLCs can be swept in a https://github.com/lightningnetwork/lightning-rfc/blob/master/05-onchain.md#penalty-transaction-weight-calculation[single justice transaction].
+While this limit _may_ be increased, transactions are still limited by the block size, so the number of slots available is likely to remain limited.
+
+==== Channel Liquidity Lockup
+
+((("breaches of privacy","channel liquidity lockup")))((("channel liquidity lockup")))A channel liquidity lockup attack is comparable to a channel jamming attack in that it routes payments through a channel and holds them so that the channel is unusable.
+Rather than locking up slots on the channel commitment, this attack routes large HTLCs through a target channel, consuming all the channel's available bandwidth.
+This attack's capital commitment is higher than the commitment jamming attack because the attacking node needs more funds to route failed payments through the target.(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc3")))
+
+=== Cross-Layer De-Anonymization
+
+((("breaches of privacy","cross-layer de-anonymization", id="ix_16_security_privacy_ln-asciidoc11", range="startofrange")))((("cross-layer de-anonymization", id="ix_16_security_privacy_ln-asciidoc12", range="startofrange")))((("security and privacy","cross-layer de-anonymization", id="ix_16_security_privacy_ln-asciidoc13", range="startofrange")))Computer networks are often layered.
+Layering allows for separation of concerns and makes the whole system manageable.
+No one could design a website if it required understanding all the TCP/IP stack up to the physical encoding of bits in an optical cable.
+Every layer is supposed to provide the functionality to the layer above in a clean way.
+Ideally, the upper layer should perceive a lower layer as a black box.
+In reality, though, implementations are not ideal, and the details _leak_ into the upper layer.
+This is the problem of leaky abstractions.
+
+In the context of Lightning, the LN protocol relies on the Bitcoin protocol and the LN P2P network.
+Up to this point, we only considered the privacy guarantees offered by the Lightning Network in isolation.
+However, creating and closing payment channels are inherently performed on the Bitcoin blockchain.
+Consequently, for a complete analysis of the Lightning Network's privacy provisions, one needs to consider every layer of the technological stack users might interact with.
+Specifically, a de-anonymizing adversary can and will use off-chain and on-chain data to cluster or link LN nodes to corresponding Bitcoin addresses.
+
+Attackers attempting to de-anonymize LN users may have various goals, in a cross-layer context:
+
+  * Cluster Bitcoin addresses owned by the same user (Layer 1). We call these Bitcoin entities.
+  * Cluster LN nodes owned by the same user (Layer 2).
+  * Unambiguously link sets of LN nodes to the sets of Bitcoin entities that control them.
+
+There are several heuristics and usage patterns that allow an adversary to cluster Bitcoin addresses and LN nodes owned by the same LN users.
+Moreover, these clusters can be linked across layers using other powerful cross-layer linking heuristics.
+The last type of heuristics, cross-layer linking techniques, emphasizes the need for a holistic view of privacy. Specifically, we must consider privacy in the context of both layers together.
+
+
+==== On-Chain Bitcoin Entity Clustering
+((("Bitcoin entities","entity clustering")))((("cross-layer de-anonymization","on-chain Bitcoin entity clustering")))((("on-chain Bitcoin entity clustering")))Lightning Network blockchain interactions are permanently reflected in the Bitcoin entity graph.
+Even if a channel is closed, an attacker can observe which address funded the channel and where the coins are spent after closing it.
+For this analysis, let's consider four separate entities.
+Opening a channel causes a monetary flow from a _source entity_ to a _funding entity_; closing a channel causes a flow from a _settlement entity_ to a _destination entity_.
+
+In early 2021, https://arxiv.org/pdf/2007.00764.pdf[Romiti et al.] identified four heuristics that allow the clustering of these entities.
+Two of them capture certain leaky funding behavior and two describe leaky settlement behaviors.
+
+Star heuristic (funding):: If a component contains one source entity that forwards funds to one or more funding entities, these funding entities are likely controlled by the same user.
+Snake heuristic (funding):: If a component contains one source entity that forwards funds to one or more entities, which themselves are used as source and funding entities, then all these entities are likely controlled by the same user.
+Collector heuristic (settlement):: If a component contains one destination entity that receives funds from one or more settlement entities, these settlement entities are likely controlled by the same user.
+Proxy heuristic (settlement):: If a component contains one destination entity that receives funds from one or more entities, which themselves are used as settlement and destination entities, then these entities are likely controlled by the same user.
+
+It is worthwhile pointing out that these heuristics might produce false positives.
+For instance, if transactions of several unrelated users are combined in a CoinJoin transaction, then the star or the proxy heuristic can produce false positives.
+This could happen if users are funding a payment channel from a CoinJoin transaction.
+Another potential source of false positives could be that an entity could represent several users if clustered addresses are controlled by a service (e.g., exchange) or on behalf of their users (custodial wallet).
+However, these false positives can effectively be filtered out.
+
+===== Countermeasures
+If outputs of funding transactions are not reused for opening other channels, the snake heuristic does not work.
+If users refrain from funding channels from a single external source and avoid collecting funds in a single external destination entity, the other heuristics would not yield any significant results.
+
+==== Off-Chain Lightning Node Clustering
+((("cross-layer de-anonymization","off-chain Lightning node clustering")))((("Lightning node clustering")))((("off-chain Lightning node clustering")))LN nodes advertise aliases, for instance, _LNBig.com_.
+Aliases can improve the usability of the system.
+However, users tend to use similar aliases for their own different nodes.
+For example, _LNBig.com Billing_ is likely owned by the same user as the node with alias _LNBig.com_.
+Given this observation, one can cluster LN nodes by applying their node aliases.
+Specifically, one clusters LN nodes into a single address if their aliases are similar with respect to some string similarity metric.
+
+Another method to cluster LN nodes is applying their IP or Tor addresses.
+If the same IP or Tor addresses correspond to different LN nodes, these nodes are likely controlled by the same user.
+
+===== Countermeasures
+For more privacy, aliases should be sufficiently different from one another.
+While the public announcement of IP addresses may be unavoidable for those nodes that wish to have incoming channels in the Lightning Network, linkability across nodes of the same user can be mitigated if the clients for each node are hosted with different service providers and thus IP addresses.
+
+==== Cross-Layer Linking: Lightning Nodes and Bitcoin Entities
+((("Bitcoin entities","cross-layer linking to Lightning nodes")))((("breaches of privacy","cross-layer linking: Lightning nodes and Bitcoin entities")))((("cross-layer de-anonymization","cross-layer linking: Lightning nodes and Bitcoin entities")))((("Lightning node operation","cross-layer linking to Bitcoin entities")))Associating LN nodes to Bitcoin entities is a serious breach of privacy that is exacerbated by the fact that most LN nodes publicly expose their IP addresses.
+Typically, an IP address can be considered as a unique identifier of a user.
+Two widely observed behavior patterns reveal links between LN nodes and Bitcoin entities:
+
+Coin reuse:: Whenever users close payment channels, they get back their corresponding coins. However, many users reuse those coins in opening a new channel.
+Those coins can effectively be linked to a common LN node.
+
+Entity reuse:: Typically, users fund their payment channels from Bitcoin addresses corresponding to the same Bitcoin entity.
+
+These cross-layer linking algorithms could be foiled if users possess multiple unclustered addresses or use multiple wallets to interact with the Lightning Network.
+
+The possible de-anonymization of Bitcoin entities illustrates how important it is to consider the privacy of both layers simultaneously instead of one at a time.(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc13")))(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc12")))(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc11")))
+
+//TODO from author:  maybe here we should/could include the corresponding figures from the Romiti et al. paper. it would greatly improve and help the understanding of the section
+
+=== Lightning Graph
+
+((("Lightning graph", id="ix_16_security_privacy_ln-asciidoc14", range="startofrange")))((("security and privacy","Lightning graph", id="ix_16_security_privacy_ln-asciidoc15", range="startofrange")))The Lightning Network, as the name suggests, is a peer-to-peer network of payment channels.
+Therefore, many of its properties (privacy, robustness, connectivity, routing efficiency) are influenced and characterized by its network nature.
+
+In this section, we discuss and analyze the Lightning Network from the point of view of network science.
+We are particularly interested in understanding the LN channel graph, its robustness, connectivity, and other important characteristics.
+
+==== How Does the Lightning Graph Look in Reality?
+((("Lightning graph","reality versus theoretical appearance of", id="ix_16_security_privacy_ln-asciidoc16", range="startofrange")))One could have expected that the Lightning Network is a random graph, where edges are randomly formed between nodes.
+If this was the case, then the Lightning Network's degree distribution would follow a Gaussian normal distribution.
+In particular, most of the nodes would have approximately the same degree, and we would not expect nodes with extraordinarily large degrees.
+This is because the normal distribution exponentially decreases for values outside of the interval around the average value of the distribution.
+The depiction of a random graph (as we saw in <<lngraph>>) looks like a mesh network topology.
+It looks decentralized and nonhierarchical: every node seems to have equal importance.
+Additionally, random graphs have a large diameter.
+In particular, routing in such graphs is challenging because the shortest path between any two nodes is moderately long.
+
+However, in stark contrast, the LN graph is completely different.
+
+===== Lightning graph today
+Lightning is a financial network.
+Thus, the growth and formation of the network are also influenced by economic incentives.
+Whenever a node joins the Lightning Network, it may want to maximize its connectivity to other nodes in order to increase its routing efficiency. This phenomenon is called preferential attachment.
+These economic incentives result in a fundamentally different network than a random graph.
+
+Based on snapshots of publicly announced channels, the degree distribution of the Lightning Network follows a power-law function.
+In such a graph, the vast majority of nodes have very few connections to other nodes, while only a handful of nodes have numerous connections.
+At a high level, this graph topology resembles a star: the network has a well-connected core and a loosely connected periphery.
+Networks with power-law degree distribution are also called scale-free networks.
+This topology is advantageous for routing payments efficiently but prone to certain topology-based attacks.
+
+===== Topology-based attacks
+
+((("Lightning graph","topology-based attacks")))((("topology-based attacks")))An adversary might want to disrupt the Lightning Network and may decide its goal is to dismantle the whole network into many smaller components, making payment routing practically impossible in the whole network.
+A less ambitious, but still malicious and severe goal might be to only take down certain network nodes.
+Such a disruption might occur on the node level or on the edge level.
+
+Let's suppose an adversary can take down any node in the Lightning Network.
+For instance, it can attack them with a distributed denial of service (DDoS) attack or make them nonoperational by any means.
+It turns out that if the adversary chooses nodes randomly, then scale-free networks like the Lightning Network are robust against node-removal attacks.
+This is because a random node lies on the periphery with a small number of connections, therefore playing a negligible role in the network's connectivity.
+However, if the adversary is more prudent, it can target the most well-connected nodes.
+Not surprisingly, the Lightning Network and other scale-free networks are _not_ robust against targeted node-removal attacks.
+
+On the other hand, the adversary could be more stealthy.
+Several topology-based attacks target a single node or a single payment channel.
+For example, an adversary might be interested in exhausting a certain payment channel's capacity on purpose.
+More generally, an adversary can deplete all the outgoing capacity of a node to knock it down from the routing market.
+This could be easily obtained by routing payments through the victim node with amounts equalling the outgoing capacity of each payment channel.
+After completing this so-called node isolation attack, the victim cannot send or route payments anymore unless it receives a payment or rebalances its channels.
+
+To conclude, even by design, it is possible to remove edges and nodes from the routable Lightning Network.
+However, depending on the utilized attack vector, the adversary may have to provide more or fewer resources to carry out the attack.
+
+
+===== Temporality of the Lightning Network
+
+((("Lightning graph","temporality of Lightning Network and")))((("temporality of Lightning Network")))The Lightning Network is a dynamically changing, permissionless network.
+Nodes can freely join or leave the network, they can open and create payment channels anytime they want.
+Therefore, a single static snapshot of the LN graph is misleading. We need to consider the temporality and ever-changing nature of the network. For now, the LN graph is growing in terms of the number of nodes and payment channels.
+Its effective diameter is also shrinking; that is, nodes become closer to each other, as we can see in <<temporal_ln>>.
+
+[[temporal_ln]]
+.The steady growth of the Lightning Network in nodes, channels, and locked capacity (as of September 2021)
+image::images/mtln_1602.png["The steady growth of the Lightning Network in terms of nodes, channels, and locked capacity (as of September 2021)"]
+
+In social networks, triangle closing behavior is common.
+Specifically, in a graph where nodes represent people and friendships are represented as edges, it is somewhat expected that triangles will emerge in the graph.
+A triangle, in this case, represents pairwise friendships between three people.
+For instance, if Alice knows Bob and Bob knows Charlie, then it is likely that at some point Bob will introduce Alice to Charlie.
+However, this behavior would be strange in the Lightning Network.
+Nodes are simply not incentivized to close triangles because they could have just routed payments instead of opening a new payment channel.
+Surprisingly, triangle closing is a common practice in the Lightning Network.
+The number of triangles was steadily growing before the implementation of multipart payments.
+This is counterintuitive and surprising given that nodes could have just routed payments through the two sides of the triangle instead of opening the third channel.
+This may mean that routing inefficiencies incentivized users to close triangles and not fall back on routing.
+Hopefully, multipart payments will help increase the effectiveness of payment routing(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc16"))).(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc15")))(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc14")))
+
+=== Centralization in the Lightning Network
+
+((("betweenness centrality")))((("central point dominance")))((("centralization, Lightning Network and")))((("security and privacy","centralization in Lightning Network")))A common metric to assess the centrality of a node in a graph is its _betweenness centrality_. Central point dominance is a metric derived from betweenness centrality, used to assess the centrality of a network.
+For a precise definition of central point dominance, the reader is referred to https://doi.org/10.2307/3033543[Freeman's work].
+
+The larger the central point dominance of a network is, the more centralized the network is.
+We can observe that the Lightning Network has a greater central point dominance (i.e., it is more centralized) than a random graph (Erdős–Rényi graph) or a scale-free graph (Barabási–Albert graph) of equal size.
+
+In general, our understanding of the dynamic nature of the LN channel graph is rather limited.
+It is fruitful to analyze how protocol changes like multipart payments can affect the dynamics of the Lightning Network.
+It would be beneficial to explore the temporal nature of the LN graph in more depth.
+
+=== Economic Incentives and Graph Structure
+
+((("Lightning graph","economic incentives and graph structure")))((("security and privacy","economic incentives and graph structure")))The LN graph forms spontaneously, and nodes connect to each other based on mutual interest.
+As a result, incentives drive graph development.
+Let's look at some of the relevant incentives:
+
+  * Rational incentives:
+    - Nodes establish channels to send, receive, and route payments (earn fees).
+    - What makes a channel more likely to be established between two nodes that act rationally?
+  * Altruistic incentives:
+    - Nodes establish channels "for the good of the network."
+    - While we should not base our security assumptions on altruism, to a certain extent, altruistic behavior drives Bitcoin (accepting incoming connections, serving blocks).
+    - What role does it play in Lightning?
+
+In the early stages of the Lightning Network, many node operators have claimed that the earned routing fees do not compensate for the opportunity costs stemming from liquidity lock-up. This would indicate that operating a node may be driven mostly by altruistic incentives "for the good of the network."
+This might change in the future if the Lightning Network has significantly larger traffic or if a market for routing fees emerges.
+On the other hand, if a node wishes to optimize its routing fees, it would minimize the average shortest path lengths to every other node.
+Put differently, a profit-seeker node will try to locate itself in the _center_ of the channel graph or close pass:[<span class="keep-together">to it</span>].
+
+=== Practical Advice for Users to Protect Their Privacy
+
+((("security and privacy","practical advice for users to protect privacy")))We're still in the early stages of the Lightning Network.
+Many of the concerns listed in this chapter are likely to be addressed as it matures and grows.
+In the meantime, there are some measures that you can take to guard your node against malicious users; something as simple as updating the default parameters that your node runs with can go a long way in hardening your node.
+
+=== Unannounced Channels
+
+((("payment channel","unannounced channels")))((("security and privacy","unannounced channels")))((("unannounced channels")))If you intend to use the Lightning Network to send and receive funds between nodes and wallets you control, and have no interest in routing other users' payments, there is little need to announce your channels to the rest of the network.
+You could open a channel between, say, your desktop PC running a full node and your mobile phone running a Lightning wallet, and simply forgo the channel announcement discussed in <<ch03_How_Lightning_Works>>.
+These are sometimes called "private" channels; however, it is more correct to refer to them as "unannounced" channels because they are not strictly private.
+
+Unannounced channels will not be known to the rest of the network and won't normally be used to route other users' payments.
+They can still be used to route payments if other nodes are made aware of them; for example, an invoice could contain routing hints which suggests a path with an unannounced channel.
+However, assuming that you've only opened an unannounced channel with yourself, you do gain some measure of privacy.
+Since you are not exposing your channel to the network, you lower the risk of a denial-of-service attack on your node.
+You can also more easily manage the capacity of this channel, since it will only be used to receive or send directly to your node.
+
+There are also advantages to opening an unannounced channel with a known party that you transact with frequently.
+For example, if Alice and Bob frequently play poker for bitcoin, they could open a channel to send their winnings back and forth.
+Under normal conditions, this channel will not be used to route payments from other users or collect fees.
+And since the channel will not be known to the rest of the network, any payments between Alice and Bob cannot be inferred by tracking changes in the channel's routing capacity.
+This confers some privacy to Alice and Bob; however, if one of them decides to make other users aware of the channel, such as by including it in the routing hints of an invoice, then this privacy is lost.
+
+It should also be noted that to open an unannounced channel, a public transaction must be made on the Bitcoin blockchain.
+Hence it is possible to infer the existence and size of the channel if a malicious party is monitoring the blockchain for channel opening transactions and attempting to match them to channels on the network.
+Furthermore, when the channel is closed, the final balance of the channel will be made public once it's committed to the Bitcoin blockchain.
+However, since the opening and commitment transactions are pseudonymous, it will not be a simple matter to connect it back to Alice or Bob.
+In addition, the Taproot update of 2021 makes it difficult to distinguish between channel opening and closing transactions and other specific kinds of Bitcoin transactions.
+Hence, while unannouned channels are not completely private, they do provide some privacy benefits when used carefully.
+
+[[routing_considerations]]
+=== Routing Considerations
+
+((("denial-of-service (DoS) attacks","protecting against")))((("routing","security/privacy considerations")))((("security and privacy","routing considerations")))As covered in <<denial_of_service>>, nodes that open public channels expose themselves to the risk of a series of attacks on their channels.
+While mitigations are being developed on the protocol level, there are many steps that a node can take to protect against denial of service attacks on their public channels:
+
+Minimum HTLC size:: On channel open, your node can set the minimum HTLC size that it will accept.
+Setting a higher value ensures that each of your available channel slots cannot be occupied by a very small payment.
+Rate limiting:: Many node implementations allow nodes to dynamically accept or reject HTLCs that are forwarded through your node.
+Some useful guidelines for a custom rate limiter are as follows:
++
+** Limit the number of commitment slots a single peer may consume
+** Monitor failure rates from a single peer, and rate limit if their failures spike suddenly
+Shadow channels:: Nodes that wish to open large channels to a single target can instead open a single public channel to the target and support it with further private channels called pass:[<a href='https://anchor.fm/tales-from-the-crypt/episodes/197-Joost-Jager-ekghn6'>shadow channels</a>]. These channels can still be used for routing but are not announced to potential attackers.
+
+==== Accepting Channels
+((("routing","accepting channels")))At present, Lightning nodes struggle with bootstrapping inbound liquidity. While there are some paid
+solutions to acquiring inbound liquidity, like swap services, channel markets, and paid channel opening services from known hubs, many nodes will gladly accept any legitimate looking channel opening request to increase their inbound liquidity.
+
+Stepping back to the context of Bitcoin, this can be compared to the way that Bitcoin Core treats its incoming and outgoing connections differently out of concern that the node may be eclipsed.
+If a node opens an incoming connection to your Bitcoin node, you have no way of knowing whether the initiator randomly selected you or is specifically targeting your node with malicious intent.
+Your outgoing connections do not need to be treated with such suspicion because either the node was selected randomly from a pool of many potential peers or you intentionally connected to the peer manually.
+
+The same can be said in Lightning.
+When you open a channel, it is done with intention, but when a remote party opens a channel to your node, you have no way of knowing whether this channel will be used to attack your node or not.
+As several papers note, the relatively low cost of spinning up a node and opening channels to targets is one of the significant factors that make attacks easy.
+If you accept incoming channels, it is prudent to place some restrictions on the nodes you accept incoming channels from.
+Many implementations expose channel acceptance hooks that allow you to tailor your channel acceptance policies to your preferences.
+
+The question of accepting and rejecting channels is a philosophical one.
+What if we end up with a Lightning Network where new nodes cannot participate because they cannot open any channels?
+Our suggestion is not to set an exclusive list of "mega-hubs" from which you will accept channels, but rather to accept channels in a manner that suits your risk preference.
+
+Some potential strategies are:
+
+No risk:: Do not accept any incoming channels.
+Low risk:: Accept channels from a known set of nodes that you have previously had successful channels open with.
+Medium risk:: Only accept channels from nodes that have been present in the graph for a longer period and have some long-lived channels.
+Higher risk:: Accept any incoming channels, and implement the mitigations described in <<routing_considerations>>.
+
+=== Conclusion
+In summary, privacy and security are nuanced, complex topics, and while many researchers and developers are looking for network-wide improvements, it's important for everyone participating in the network to understand what they can do to protect their own privacy and increase security on an individual node level.
+
+=== References and Further Reading
+
+In this chapter, we used many references from ongoing research on Lightning security. You may find these useful articles and papers listed by topic in the following lists.
+
+===== Privacy and probing attacks
+
+* Jordi Herrera-Joancomartí et al. https://eprint.iacr.org/2019/328["On the Difficulty of Hiding the Balance of Lightning Network Channels"]. _Asia CCS '19: Proceedings of the 2019 ACM Asia Conference on Computer and Communications Security_ (July 2019): 602–612.
+* Utz Nisslmueller et al. "Toward Active and Passive Confidentiality Attacks on Cryptocurrency Off-Chain Networks." arXiv preprint, https://arxiv.org/abs/2003.00003[] (2020).
+* Sergei Tikhomirov et al. "Probing Channel Balances in the Lightning Network." arXiv preprint, https://arxiv.org/abs/2004.00333[] (2020).
+* George Kappos et al. "An Empirical Analysis of Privacy in the Lightning Network." arXiv preprint, https://arxiv.org/abs/2003.12470[] (2021).
+* https://github.com/LN-Zap/zap-desktop/blob/v0.7.2-beta/services/grpc/router.methods.js[Zap source code with the probing function].
+
+===== Congestion attacks
+
+* Ayelet Mizrahi and Aviv Zohar. "Congestion Attacks in Payment Channel Networks." arXiv preprint, https://arxiv.org/abs/2002.06564[] (2020).
+
+===== Routing considerations
+
+* Marty Bent, interview with Joost Jager, _Tales from the Crypt_, podcast audio, October 2, 2020, https://anchor.fm/tales-from-the-crypt/episodes/197-Joost-Jager-ekghn6[].(((range="endofrange", startref="ix_16_security_privacy_ln-asciidoc0")))
+
+

17_conclusion.asciidoc

@@ -0,0 +1,81 @@
+[[conclusion_chapter]]
+== Conclusion
+
+((("innovations in Lightning", id="ix_17_conclusion-asciidoc0", range="startofrange")))In just a few years, the Lightning Network has gone from a whitepaper to a rapidly growing global network. As Bitcoin's second layer, it has delivered on the promise of fast, inexpensive, and private payments. Additionally, it has started a tsunami of innovation, as it unleashes developers from the constraints of lockstep consensus that exist in Bitcoin development.
+
+Innovation in the Lightning Network is happening in several different levels:
+
+* At Bitcoin's Core protocol, providing use and demand for new Bitcoin Script opcodes, signing algorithms, and optimizations
+* At the Lightning protocol level, with new features deployed rapidly network wide
+* At the payment channel level, with new channel constructs and enhancements
+* As distinct opt-in features deployed end-to-end by independent implementations that senders and recipients can use if they want
+* With new and exciting Lightning Applications (LApps) build on top of the clients and protocols
+
+Let's look at how these innovations are changing Lightning now and in the near future.
+
+=== Decentralized and Asynchronous Innovation
+
+((("innovations in Lightning","decentralized/asynchronous nature of")))Lightning isn't bound by lockstep consensus, as is the case with Bitcoin. That means that different Lightning clients can implement different features and negotiate their interactions (see <<feature_bits>>). As a result, innovation in the Lightning Network is occurring at a much faster rate than in Bitcoin.
+
+Not only is Lightning advancing rapidly, but it is creating demand for new features in the Bitcoin system. Many recent and planned innovations in Bitcoin are both motivated and justified by their use in the Lightning Network. In fact, the Lightning Network is often mentioned as an example use case for many of the new features.
+
+[[bitcoin_prot_17]]
+==== Bitcoin Protocol and Bitcoin Script Innovation
+
+((("Bitcoin (system)","innovations motivated by Lightning Network use cases")))((("Bitcoin script","innovations motivated by Lightning Network use cases")))((("innovations in Lightning","Bitcoin innovations motivated by Lightning Network use cases")))The Bitcoin system is, by necessity, a conservative system that has to preserve compatibility with consensus rules to avoid unplanned forks of the blockchain or partitions of the P2P network. As a result, new features require a lot of coordination and testing before they are implemented in mainnet, the live production system.
+
+Here are some of the current or proposed innovations in Bitcoin that are motivated by use cases in the Lightning Network:
+
+Neutrino:: A lightweight client protocol with improved privacy features over the legacy SPV protocol. Neutrino is mostly used by Lightning clients to access the Bitcoin blockchain.
+
+Schnorr signatures:: Introduced as part of the _Taproot_ soft fork, Schnorr signatures will enable flexible Point Time-Locked Contracts (PTLCs) for channel construction in Lightning. This might in particular make use of revocable signatures instead of revocable transactions.
+
+Taproot:: Also part of the November 2021 soft fork that introduces Schnorr signatures, Taproot allows complex scripts to appear as single-payer, single-payee payments, and indistinguishable from the most common type of payment on Bitcoin. This will allow Lightning channel cooperative (mutual) closure transactions to appear indistinguishable from simple payments and increase privacy for LN users.
+
+Input rebinding:: Also known by the names SIGHAS_NOINPUT or SIGHASH_ANYPREVOUT, this planned upgrade to the Bitcoin Script language is primarily motivated by advanced smart contracts such as the eltoo channel protocol.
+
+Covenants:: Currently in the early stages of research, covenants allow transactions to create outputs that constrain future transactions which spend them. This mechanism could increase security for Lightning channels by making it possible to enforce address whitelisting in commitment transactions.
+
+==== Lightning Protocol Innovation
+
+((("innovations in Lightning","Lightning P2P protocol")))((("Lightning Network Protocol","innovations in")))The Lightning P2P protocol is highly extensible and has undergone a lot of change since its inception. The "It's OK to be odd" rule used in feature bits (see <<feature_bits>>) ensures that nodes can negotiate the features they support, enabling multiple independent upgrades to the protocol.
+
+==== TLV Extensibility
+
+((("innovations in Lightning","TLV extensibility")))((("Type-Length-Value (TLV) format","innovations in")))The Type-Length-Value (see <<tlv>>) mechanism for extending the messaging protocol is extremely powerful and has already enabled the introduction of several new capabilities in Lightning while maintaining both forward and backward compatibility.
+A prominent example, which is currently being developed and makes use of this, is path blinding and trampoline payments. This allows a recipient to hide itself from the sender, but also allows mobile clients to send payments without the necessity of storing the full channel graph on their devices by using a third party to which they don't need to reveal the final recipient.
+
+==== Payment Channel Construction
+
+((("innovations in Lightning","payment channel construction")))((("payment channel","innovations in construction")))Payment channels are an abstraction that is operated by two channel partners. As long as those two are willing to run new code, they can implement a variety of channel mechanisms simultaneously. In fact, recent research suggests that channels could even be upgraded to a new mechanism dynamically, without closing the old channel and opening a new channel type.
+
+eltoo:: A proposed channel mechanism that uses input-rebinding to significantly simplify the operation of payment channels and remove the need for the penalty mechanism. It needs a new Bitcoin signature type before it can be implemented
+
+==== Opt-In End-to-End Features
+
+((("innovations in Lightning","opt-in end-to-end features")))Point Time-Locked Contracts:: A different approach to HTLCs, PTLCs can increase privacy, reduce information leaked to intermediary nodes, and operate more efficiently than HTLC-based channels.
+
+Large channels:: Large or _Wumbo_ channels were introduced in a dynamic way to the network without requiring coordination. Channels that support large payments are advertized as part of the channel announcement messages and can be used in an opt-in manner.
+
+Multipart payments (MPP):: MPP was also introduced in an opt-in manner, but even better only requires the sender and recipient of a payment to be able to do MPP. The rest of the network simply routes HTLCs as if they are single-part payments.
+
+JIT routing:: An optional method that can be used by routing nodes to increase their reliability and to protect themselves from being spammed.
+
+Keysend:: An upgrade introduced independently by Lightning client implementations, it allows the sender to send money in an "unsolicited" and asynchronous way without requiring an invoice first.
+
+HODL invoicesfootnote:[The word _HODL_ comes from an excited misspelling of the word "HOLD" shouted in a forum to encourage people not to sell bitcoin in a panic.]:: Payments where the final HTLC is not collected, committing the sender to the payment, but allowing the recipient to delay collection until some other condition is satisfied, or cancel the invoice without collection. This was also implemented independently by different Lightning clients and can be used in an opt-in manner.
+
+Onion routed message services:: The onion routing mechanism and the underlying public key database of nodes can be used to send data that is unrelated to payments, such as text messages or forum posts. The use of Lightning to enable paid messaging as a solution to spam posts and Sybil attacks (spam) is another innovation that was implemented independently of the core protocol.
+
+Offers:: Currently proposed as BOLT #12 but already implemented by some nodes, this is a communication protocol to request (recurring) invoices from remote nodes via Onion messages.
+
+[[lapps]]
+=== Lightning Applications (LApps)
+
+((("innovations in Lightning","Lightning Applications")))((("Lightning Applications (LApps)")))While still in their infancy, we are already seeing the emergence of interesting Lightning Applications. Broadly defined as an application that uses the Lightning Protocol or a Lightning client as a component, LApps are the application layer of Lightning. 
+A prominent example is LNURL, which provides a similar functionality as BOLT #12 offers, but just over HTTP and Lightning addresses. It works on top of offers to provide users with an email-style address to which others can send funds while the software in the background requests an invoice against the LNURL endpoint of the node.
+Further LApps are being built for simple games, messaging applications, microservices, payable APIs, paid dispensers (e.g., fuel pumps), derivative trading systems, and much more.
+
+=== Ready, Set, Go!
+
+The future is looking bright. The Lightning Network is taking Bitcoin to new unexplored markets and applications. Equipped with the knowledge in this book, you can explore this new frontier or maybe even join as a pioneer and forge a new path.(((range="endofrange", startref="ix_17_conclusion-asciidoc0")))

CONTRIBUTING.md

@@ -78,6 +78,10 @@ All submission should use Unix-like line endings: LF (not CR, not CR/LF). All th
 
 If you are unsure or your OS makes things difficult, consider using a developer's text editor such as Atom.
 
+## Spelling
+
+US English spelling standards should be used for contributions to the text.
+
 ## Thanks
 
 We are very grateful for the support of the entire Lightning Network community. With your help, this will be a great book that can help thousands of developers get started and eventually "master" LN. Thank you!

LICENSE.md

@@ -4,4 +4,12 @@
 
 <a rel="license" href="http://creativecommons.org/licenses/by-nc-nd/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-nd/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" property="dct:title">Mastering the Lightning Network</span> by <a xmlns:cc="http://creativecommons.org/ns#" href="https://lnbook.info/authors" property="cc:attributionName" rel="cc:attributionURL">Andreas M. Antonopoulos, Olaoluwa Osuntokun, Rene Pickhardt</a> is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-nd/4.0/">Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License</a>.
 
-It is expected that the book will be released under a more permissive CC-BY-SA license within a year of publication.
+## Translations and Derivatives (eg. PDF, HTML, EPUB ebooks)
+
+The current license does **not permit derivative or commercial work**. This means, it does **not** permit independent translations without pemission/license from O'Reilly Media (publisher). It also does not permit the production and circulation of PDF, HTML or other derivative renderings of the source ASCIIDOC, (with the exception for personal use only, not shared with others).
+
+You can't translate this book without permission. You can't create ebooks in PDF, HTML EPUB or any other format unless it is for personal use only and not shared/distributed.
+
+## Licensing change in 12 months
+
+It is expected that the book will be released under a more permissive CC-BY-SA license within a year of publication (i.e. Jan 2023). At that time we will provide a Transifex project for volunteer translations and it will be allowed to produce and share PDF or other ebook versions of the book. 

README.md

@@ -1,53 +1,66 @@
 # Mastering the Lightning Network
-
 [![Build Status](https://travis-ci.com/lnbook/lnbook.svg?branch=develop)](https://travis-ci.com/lnbook/lnbook)
 
+STATUS: First Edition published on Dec 21, 2021
+
 <img src="images/cover_thumb.png" width=200 alt="Mastering Lightning Cover">
 
 ## About
-Mastering the Lightning Network is an O'Reilly Media book, due for publication in Q4'2020, and announced on August 28th by authors Andreas M. Antonopoulos ([@aantonop](https://twitter.com/aantonop)), Olaoluwa Osuntokun ([@roasbeef](https://twitter.com/roasbeef)), Rene Pickhardt ([@renepickhardt](https://twitter.com/renepickhardt)).
+Mastering the Lightning Network is an O'Reilly Media book, by authors Andreas M. Antonopoulos ([@aantonop](https://twitter.com/aantonop)), Olaoluwa Osuntokun ([@roasbeef](https://twitter.com/roasbeef)), Rene Pickhardt ([@renepickhardt](https://twitter.com/renepickhardt)). It was published on Dec 21, 2021, in paperback and e-book, by O'Reilly Media. It is available everywhere that books are sold. This repository contains the manuscript of the book as published by O'Reilly Media, tagged as [firstedition_firstprint](https://github.com/lnbook/lnbook/releases/tag/firstedition_firstprint).
 
 The book describes the Lightning Network (LN), a Peer-to-Peer protocol running on top of Bitcoin and other blockchains, which provides near-instant, secure, micro-payments.
 
 The book is suitable for technical readers with an understanding of the fundamentals of Bitcoin and other open blockchains.
 
-## Status
+## Contents
 
-The current status of the book is "IN PROGRESS". See below for status of specific chapters and read the contribution guide to learn how and where to contribute.
+### Preface
 
-### Legend
+*  [Cover](cover.html)
+*  [Titlepage](titlepage.html)
+*  [Copyright](copyright.html)
+*  [Table of Contents](toc.html)
+*  [Preface](preface.asciidoc)
 
-* :arrows_clockwise:  LIVE EDITS - Continuously changing: Submit focused/small Issues and PRs
-* :mag: REVIEW - Ready for review: Submit Issues and PRs as needed
-* :lock_with_ink_pen: EARLY DRAFT - In progress, changing often: Submit issues only, NO PRs or fixes
-* :bookmark_tabs: OUTLINE - Rough outline - Please contribute! PRs welcome.
-* :thought_balloon: PLANNED - Planned section  - Do nothing yet.
+### Part 1
 
-| Section | Length (Word Count) |  Status |
-|-------|------|:------:|
-| [Preface and Acknowledgments](preface.asciidoc) | ### | :arrows_clockwise: |
-| [Glossary](glossary.asciidoc) | ############# | :arrows_clockwise: |
-| [Introduction](01_introduction.asciidoc) | ##### | :mag: |
-| [Getting Started](02_getting_started.asciidoc) | ########### | :mag: |
-| [LN Basics (How LN Works)](03_how_ln_works.asciidoc) | ########################## | :mag: |
-| [Intro to LN Routing (HTLCs)](routing.asciidoc) | ###################### | :lock_with_ink_pen: |
-| [Nodes (LN Clients)](node_client.asciidoc) | #################### | :mag: |
-| [Operating a Node](node_operations.asciidoc) | ######### | :bookmark_tabs: |
-| [P2P Communication](p2p.asciidoc) | # | :bookmark_tabs: |
-| [Channel Construction in Detail](channel-construction.asciidoc) | ######### | :lock_with_ink_pen: |
-| [Channel Graph and Gossip Layer](channel-graph.asciidoc) | # | :bookmark_tabs: |
-| [Payment Path Finding](path-finding.asciidoc) | ########## | :bookmark_tabs: |
-| [End-to-End Payment Presentation Layer](e2e-presentation-layer.asciidoc) | ## | :bookmark_tabs: |
-| [Lightning Applications (LApps)]() | # | :thought_balloon: |
-| [LN's Future]() | # | :thought_balloon: |
+*  [Part 1 - Intro](part_1_divider.asciidoc)
+*  [Introduction](01_introduction.asciidoc)
+*  [Getting Started](02_getting_started.asciidoc)
+*  [How the Lightning Network Works](03_how_ln_works.asciidoc)
+*  [Lightning Node Software](04_node_client.asciidoc)
+*  [Operating a Lightning Network Node](05_node_operations.asciidoc)
 
-Total Word Count: 64775
+### Part 2
 
-Target Word Count: 100,000-120,000
+*  [Part 2 - Intro](part_2_divider.asciidoc)
+*  [Lightning Network Architecture](06_lightning_architecture.asciidoc)
+*  [Payment Channels](07_payment_channels.asciidoc)
+*  [Routing on a Network of Payment Channels](08_routing_htlcs.asciidoc)
+*  [Channel Operation and Payment Forwarding](09_channel_operation.asciidoc)
+*  [Onion Routing](10_onion_routing.asciidoc)
+*  [Gossip and the Channel Graph](11_gossip_channel_graph.asciidoc)
+*  [Pathfinding and Payment Delivery](12_path_finding.asciidoc)
+*  [Wire Protocol: Framing and Extensibility](13_wire_protocol.asciidoc)
+*  [Lightning's Encrypted Message Transport](14_encrypted_transport.asciidoc)
+*  [Lightning Payment Requests](15_payment_requests.asciidoc)
+*  [Security and Privacy of the Lightning Network](16_security_privacy_ln.asciidoc)
+*  [Conclusion](17_conclusion.asciidoc)
 
-## Contributing
+### Appendices
 
-The authors welcome contributions to this book! Read the [Guide to Contributing](CONTRIBUTING.md)
+*  [Bitcoin Fundamentals Review](appendix_bitcoin_fundamentals_review.asciidoc)
+*  [Docker Basics](appendix_docker_basics.asciidoc)
+*  [Protocol Messages](appendix_protocol_messages.asciidoc)
+
+### Glossary
+
+*  [Glossary](glossary.asciidoc)
+
+### Author Bios and Colophon
+
+*  [Author Bios](author_bio.html)
+*  [Colophon](colo.html)
 
 ## Source and license
 
@@ -57,4 +70,12 @@ Mastering the Lightning Network is released under the Creative Commons CC-BY-NC-
 
 <span xmlns:dct="http://purl.org/dc/terms/" property="dct:title">Mastering the Lightning Network</span> by <a xmlns:cc="http://creativecommons.org/ns#" href="https://lnbook.info/" property="cc:attributionName" rel="cc:attributionURL">Andreas M. Antonopoulos, Olaoluwa Osuntokun, Rene Pickhardt</a> is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc-nd/4.0/">Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License</a>.
 
-It is expected that the book will be released under a more permissive CC-BY-SA license within a year of publication.
+## Translations and Derivatives (eg. PDF, HTML, EPUB ebooks)
+
+The current license does **not permit derivative or commercial work**. This means, it does **not** permit independent translations without pemission/license from O'Reilly Media (publisher). It also does not permit the production and circulation of PDF, HTML or other derivative renderings of the source ASCIIDOC, (with the exception for personal use only, not shared with others).
+
+You can't translate this book without permission. You can't create ebooks in PDF, HTML EPUB or any other format unless it is for personal use only and not shared/distributed.
+
+## Licensing change in 12 months
+
+It is expected that the book will be released under a more permissive CC-BY-SA license within a year of publication (i.e. Jan 2023). At that time we will provide a Transifex project for volunteer translations and it will be allowed to produce and share PDF or other ebook versions of the book.

appendix_bitcoin_fundamentals_review.asciidoc

@@ -0,0 +1,521 @@
+[appendix]
+[[bitcoin_fundamentals_review]]
+== Bitcoin Fundamentals Review
+
+((("Bitcoin (system)","fundamentals", id="ix_appendix-bitcoin-fundamentals-review-asciidoc0", range="startofrange")))The Lightning Network is capable of running above multiple blockchains, but is primarily anchored on Bitcoin. To understand the Lightning Network, you need a fundamental understanding of Bitcoin and its building blocks.
+
+There are many good resources that you can use to learn more about Bitcoin, including the companion book _Mastering Bitcoin_, 2nd Edition, by Andreas M. Antonopoulos, which you can find on GitHub under https://github.com/bitcoinbook/bitcoinbook[an open source license]. However, you do not need to read a whole other book to be ready for this one!
+
+In this chapter, we've collected the most important concepts you need to know about Bitcoin and explained them in the context of the Lightning Network. This way you can learn exactly what you need to know to grasp the Lightning Network without any distractions.
+
+This chapter covers several important concepts from Bitcoin, including:
+
+* Keys and digital signatures
+* Hash functions
+* Bitcoin transactions and their structure
+* Bitcoin transaction chaining
+* Transaction outpoints
+* Bitcoin Script: locking and unlocking scripts
+* Basic locking scripts
+* Complex and conditional locking scripts
+* Timelocks
+
+
+=== Keys and Digital Signatures
+
+((("Bitcoin (system)","keys and digital signatures", id="ix_appendix-bitcoin-fundamentals-review-asciidoc1", range="startofrange")))((("Bitcoin (system)","private keys", id="ix_appendix-bitcoin-fundamentals-review-asciidoc2", range="startofrange")))((("keys", id="ix_appendix-bitcoin-fundamentals-review-asciidoc3", range="startofrange")))((("private keys", id="ix_appendix-bitcoin-fundamentals-review-asciidoc4", range="startofrange")))You may have heard that bitcoin is based on _cryptography_, which is a branch of mathematics used extensively in computer security. Cryptography can also be used to prove knowledge of a secret without revealing that secret (digital signature), or prove the authenticity of data (digital fingerprint). These types of cryptographic proofs are the mathematical tools critical to Bitcoin and used extensively in Bitcoin applications.
+
+Ownership of bitcoin is established through _digital keys_, _bitcoin addresses_, and _digital signatures_. The digital keys are not actually stored in the network, but are instead created and stored by users in a file, or simple database, called a _wallet_. The digital keys in a user's wallet are completely independent of the Bitcoin Protocol and can be generated and managed by the user's wallet software without reference to the blockchain or access to the internet.
+
+Most Bitcoin transactions require a valid digital signature to be included in the blockchain, which can only be generated with a secret key; therefore, anyone with a copy of that key has control of the bitcoin.  The digital signature used to spend funds is also referred to as a _witness_, a term used in cryptography. The witness data in a bitcoin transaction testifies to the true ownership of the funds being spent. Keys come in pairs consisting of a private (secret) key and a public key. Think of the public key as similar to a bank account number and the private key as similar to the secret PIN.
+
+==== Private and Public Keys
+
+A private key is simply a number, picked at random. In practice, and to make managing many keys easy, most Bitcoin wallets generate a sequence of private keys from a single random _seed_ using a deterministic derivation algorithm. Simply put, a single random number is used to produce a repeatable sequence of seemingly random numbers that are used as private keys. This allows users to only back up the seed and be able to _derive_ all the keys they need from that seed.
+
+((("elliptic curve")))Bitcoin, like many other cryptocurrencies and blockchains, uses _elliptic curves_ for security. In Bitcoin, elliptic curve multiplication on the _secp256k1_ elliptic curve is used as a ((("one-way function")))_one-way function_. Simply put, the nature of elliptic curve math makes it trivial to calculate the scalar multiplication of a point but impossible to calculate the inverse (division, or discrete logarithm).
+
+((("Bitcoin (system)","public keys")))((("public keys")))Each private key has a corresponding _public key_, which is calculated from the private key, using scalar multiplication on the elliptic curve. In simple terms, with a private key _k_, we can multiply it with a constant _G_ to produce a public key _K_:
+
+++++
+<ul class="simplelist">
+<li><em>K</em> = <em>k</em>*<em>G</em></li>
+</ul>
+++++
+
+It is impossible to reverse this calculation. Given a public key _K_, one cannot calculate the private key _k_. Division by _G_ is not possible in elliptic curve math. Instead, one would have to try all possible values of _k_ in an exhaustive process called a _brute-force attack_. Because _k_ is a 256-bit number, exhausting all possible values with any classical computer would require more time and energy than available in this universe.
+
+==== Hashes
+
+((("Bitcoin (system)","hashes", id="ix_appendix-bitcoin-fundamentals-review-asciidoc5", range="startofrange")))((("cryptographic hash functions", id="ix_appendix-bitcoin-fundamentals-review-asciidoc6", range="startofrange")))((("hashes", id="ix_appendix-bitcoin-fundamentals-review-asciidoc7", range="startofrange")))Another important tool used extensively in Bitcoin, and in the Lightning Network, are _cryptographic hash functions_, and specifically the SHA-256 hash function.
+
+((("digest function")))((("hash function, defined")))A hash function, also known as a _digest function_, is a function that takes arbitrary length data and transforms it into a fixed length result, called the _hash_, _digest_, or _fingerprint_ (see <<SHA256>>). Importantly, hash functions are _one-way_ functions, meaning that you can't reverse them and calculate the input data from the fingerprint.
+
+[[SHA256]]
+.The SHA-256 cryptographic hash algorithm
+image::images/mtln_aa01.png["The SHA-256 cryptographic hash algorithm"]
+
+[role="pagebreak-before"]
+For example, if we use a command-line terminal to feed the text "Mastering the Lightning Network" into the SHA-256 function, it will produce a fingerprint as follows:
+
+----
+$ echo -n "Mastering the Lightning Network" | shasum -a 256
+
+ce86e4cd423d80d054b387aca23c02f5fc53b14be4f8d3ef14c089422b2235de  -
+----
+
+[TIP]
+====
+The input used to calculate a hash is also called a _preimage_.
+====
+
+The length of the input can be much bigger, of course. Let's try the same thing with the https://bitcoin.org/bitcoin.pdf[PDF file of the Bitcoin whitepaper] from Satoshi Nakamoto:
+
+----
+$ wget http://bitcoin.org/bitcoin.pdf
+$ cat bitcoin.pdf | shasum -a 256
+b1674191a88ec5cdd733e4240a81803105dc412d6c6708d53ab94fc248f4f553  -
+----
+
+While it takes longer than a single sentence, the SHA-256 function processes the 9-page PDF, "digesting" it into a 256-bit fingerprint.
+
+Now at this point you might be wondering how it is possible for a function that digests data of unlimited size to produce a unique fingerprint that is a fixed-size number?
+
+In theory, since there is an infinite number of possible preimages (inputs) and only a finite number of fingerprints, there must be many preimages that produce the same 256-bit fingerprint. ((("collision")))When two preimages produce the same hash, this is known as a _collision_.
+
+In practice, a 256-bit number is so large that you will never find a collision on purpose. Cryptographic hash functions work on the basis that a search for a collision is a brute-force effort that takes so much energy and time that it is not practically possible.
+
+Cryptographic hash functions are broadly used in a variety of applications because they have some useful features. They are:
+
+Deterministic:: The same input always produces the same hash.
+
+Irreversible:: It is not possible to compute the preimage of a hash.
+
+Collision-proof:: It is computationally infeasible to find two messages that have the same hash.
+
+Uncorrelated:: A small change in the input produces such a big change in the output that the output seems uncorrelated to the input.
+
+Uniform/random:: A cryptographic hash function produces hashes that are uniformly distributed across the entire 256-bit space of possible outputs. The output of a hash appears to be random, though it is not truly random.
+
+Using these features of cryptographic hashes, we can build some interesting pass:[<span class="keep-together">applications</span>]:
+
+Fingerprints:: A hash can be used to fingerprint a file or message so that it can be uniquely identified. Hashes can be used as universal identifiers of any data set.
+
+Integrity proof:: A fingerprint of a file or message demonstrates its integrity because the file or message cannot be tampered with or modified in any way without changing the fingerprint. This is often used to ensure software has not been tampered with before installing it on your computer.
+
+Commitment/nonrepudiation:: You can commit to a specific preimage (e.g., a number or message) without revealing it by publishing its hash. Later, you can reveal the secret, and everyone can verify that it is the same thing you committed to earlier because it produces the published hash.
+
+Proof-of-work/hash grinding:: You can use a hash to prove you have done computational work by showing a nonrandom pattern in the hash which can only be produced by repeated guesses at a preimage. For example, the hash of a Bitcoin block header starts with a lot of zero bits. The only way to produce it is by changing a part of the header and hashing it trillions of times until it produces that pattern by chance.
+
+Atomicity:: You can make a secret preimage a prerequisite of spending funds in several linked transactions. If any one of the parties reveals the preimage in order to spend one of the transactions, all the other parties can now spend their transactions too. All or none become spendable, achieving atomicity across several transactions.(((range="endofrange", startref="ix_appendix-bitcoin-fundamentals-review-asciidoc7")))(((range="endofrange", startref="ix_appendix-bitcoin-fundamentals-review-asciidoc6")))(((range="endofrange", startref="ix_appendix-bitcoin-fundamentals-review-asciidoc5")))
+
+==== Digital Signatures
+
+((("Bitcoin (system)","digital signatures")))((("digital signatures")))The private key is used to create signatures that are required to spend bitcoin by proving ownership of funds used in a transaction.
+
+A _digital signature_ is a number that is calculated from the application of the private key to a specific message.
+
+Given a message _m_ and a private key _k_, a signature function __F~sign~__ can produce a signature _S_:
+
+[latexmath]
+++++
+$ S = F_{sign}(m, k) $
+++++
+
+This signature _S_ can be independently verified by anyone who has the public key _K_ (corresponding to private key _k_), and the message:
+
+[latexmath]
+++++
+$ F_{verify}(m, K, S) $
+++++
+
+If __F~verify~__ returns a true result, then the verifier can confirm that the message _m_ was signed by someone who had access to the private key _k_. Importantly, the digital signature proves the possession of the private key _k_ at the time of signing, without revealing _k_.
+
+Digital signatures use a cryptographic hash algorithm. The signature is applied to a hash of the message, so that the message _m_ is "summarized" to a fixed-length hash _H_(_m_) that serves as a fingerprint.
+
+By applying the digital signature on the hash of a transaction, the signature not only proves the authorization, but also "locks" the transaction data, ensuring its integrity. A signed transaction cannot be modified because any change would result in a different hash and invalidate the signature.
+
+==== Signature Types
+
+((("signature hash type")))Signatures are not always applied to the entire transaction. To provide signing flexibility, a Bitcoin digital signature contains a prefix called the signature hash type, which specifies which part of the transaction data is included in the hash. This allows the signature to commit or "lock" all, or only some of, the data in the transaction. The most common signature hash type is +SIGHASH_ALL+ which locks everything in the transaction by including all the transaction data in the hash that is signed. By comparison, +SIGHASH_SINGLE+ locks all the transaction inputs, but only one output (more about inputs and outputs in the next section). Different signature hash types can be combined to produce six different "patterns" of transaction data that are locked by the signature.
+
+More information about signature hash types can be found in https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch06.asciidoc#sighash_types[the section "Signature Hash Types" in Chapter 6 of _Mastering Bitcoin_, Second Edition].(((range="endofrange", startref="ix_appendix-bitcoin-fundamentals-review-asciidoc4")))(((range="endofrange", startref="ix_appendix-bitcoin-fundamentals-review-asciidoc3")))(((range="endofrange", startref="ix_appendix-bitcoin-fundamentals-review-asciidoc2")))(((range="endofrange", startref="ix_appendix-bitcoin-fundamentals-review-asciidoc1")))
+
+=== Bitcoin Transactions
+
+((("Bitcoin (system)","transactions", id="ix_appendix-bitcoin-fundamentals-review-asciidoc8", range="startofrange")))((("Bitcoin transactions", id="ix_appendix-bitcoin-fundamentals-review-asciidoc9", range="startofrange")))_Transactions_ are data structures that encode the transfer of value between participants in the bitcoin system.
+
+[[utxo]]
+==== Inputs and Outputs
+
+((("Bitcoin transactions","inputs and outputs", id="ix_appendix-bitcoin-fundamentals-review-asciidoc10", range="startofrange")))The fundamental building block of a bitcoin transaction is a transaction output. ((("transaction outputs")))_Transaction outputs_ are indivisible chunks of bitcoin currency, recorded on the blockchain, and recognized as valid by the entire network. A transaction spends inputs and creates outputs. ((("transaction inputs")))Transaction _inputs_ are simply references to outputs of previously recorded transactions. This way, each transaction spends the outputs of previous transactions and creates new outputs (see <<transaction_structure>>).
+
+[[transaction_structure]]
+.A transaction transfers value from inputs to outputs
+image::images/mtln_aa02.png["transaction inputs and outputs"]
+
+((("unspent transaction outputs (UTXOs)")))((("UTXOs (unspent transaction outputs)")))Bitcoin full nodes track all available and spendable outputs, known as _unspent transaction outputs_ (UTXOs). The collection of all UTXOs is known as the UTXO set, which currently numbers in the millions of UTXOs. The UTXO set grows as new UTXOs are created and shrinks when UTXOs are consumed. Every transaction represents a change (state transition) in the UTXO set, by consuming one or more UTXOs as _transaction inputs_ and creating one or more UTXOs as its _transaction outputs_.
+
+For example, let's assume that a user Alice has a 100,000 satoshi UTXO that she can spend. Alice can pay Bob 100,000 satoshi by constructing a transaction with one input (consuming her existing 100,000 satoshi input) and one output that "pays" Bob 100,000 satoshi. Now Bob has a 100,000 satoshi UTXO that he can spend, creating a new transaction that consumes this new UTXO and spends it to another UTXO as a payment to another user, and so on (see <<alice_100ksat_to_bob>>).
+
+[[alice_100ksat_to_bob]]
+.Alice pays 100,000 satoshis to Bob
+image::images/mtln_aa03.png["Alice pays 100,000 satoshis to Bob"]
+
+A transaction output can have an arbitrary (integer) value denominated in satoshis. Just as dollars can be divided down to two decimal places as cents, bitcoin can be divided down to eight decimal places as satoshis. Although an output can have any arbitrary value, once created it is indivisible. This is an important characteristic of outputs that needs to be emphasized: outputs are discrete and indivisible units of value, denominated in integer satoshis. An unspent output can only be consumed in its entirety by a transaction.
+
+So what if Alice wants to pay Bob 50,000 satoshi, but only has an indivisible 100,000 satoshi UTXO? Alice will need to create a transaction that consumes (as its input) the 100,000 satoshi UTXO and has two outputs: one paying 50,000 satoshi to Bob and one paying 50,000 satoshi _back_ to Alice as "change" (see <<alice_50ksat_to_bob_change>>).
+
+[[alice_50ksat_to_bob_change]]
+.Alice pays 50k sat to Bob and 50k sat to herself as change
+image::images/mtln_aa04.png["Alice pays 50,000 satoshis to Bob and 50,000 satoshis to herself as change"]
+
+[TIP]
+====
+There's nothing special about a change output or any way to distinguish it from any other output. It doesn't have to be the last output. There could be more than one change output, or no change outputs. Only the creator of the transaction knows which outputs are to others and which outputs are to addresses they own and therefore "change."
+====
+
+Similarly, if Alice wants to pay Bob 85,000 satoshi but has two 50,000 satoshi UTXOs available, she has to create a transaction with two inputs (consuming both her 50,000 satoshi UTXOs) and two outputs, paying Bob 85,000 and sending 15,000 satoshi back to herself as change (see <<tx_twoin_twoout>>).
+
+[[tx_twoin_twoout]]
+.Alice uses two 50k inputs to pay 85k sat to Bob and 15k sat to herself as change
+image::images/mtln_aa05.png["Alice uses two 50k inputs to pay 85k sat to Bob and 15k sat to herself as change"]
+
+The preceding illustrations and examples show how a Bitcoin transaction combines (spends) one or more inputs and creates one or more outputs. A transaction can have hundreds or even thousands of inputs and outputs.
+
+[TIP]
+====
+While the transactions created by the Lightning Network have multiple outputs, they do not have "change" per se, because the entire available balance of a channel is split between the two channel partners.(((range="endofrange", startref="ix_appendix-bitcoin-fundamentals-review-asciidoc10")))
+====
+
+==== Transaction Chains
+
+((("Bitcoin transactions","transaction chains")))((("transaction chains")))Every output can be spent as an input in a subsequent transaction. So, for example, if Bob decided to spend 10,000 satoshi in a transaction paying Chan, and Chan spent 4,000 satoshi to pay Dina, it would play out as shown in <<tx_chain>>.
+
+An output is considered _spent_ if it is referenced as an input in another transaction that is recorded on the blockchain. An output is considered _unspent_ (and available for spending) if no recorded transaction references it.
+
+The only type of transaction that doesn't have inputs is a special transaction created by Bitcoin miners called the _coinbase transaction_. The coinbase transaction has only outputs and no inputs because it creates new bitcoin from mining. Every other transaction spends one or more previously recorded outputs as its inputs.
+
+Since transactions are chained, if you pick a transaction at random, you can follow any one of its inputs backward to the previous transaction that created it. If you keep doing that, you will eventually reach a coinbase transaction where the bitcoin was first mined.
+
+[[tx_chain]]
+.Alice pays Bob who pays Chan who pays Dina
+image::images/mtln_aa06.png["Alice pays Bob who pays Chan who pays Dina"]
+
+
+==== TxID: Transaction Identifiers
+
+((("Bitcoin transactions","transaction identifiers")))((("TxID (transaction identifiers)")))Every transaction in the Bitcoin system is identified by a unique identifier (assuming the existence of BIP-0030), called the _transaction ID_ or _TxID_ for short. To produce a unique identifier, we use the SHA-256 cryptographic hash function to produce a hash of the transaction's data. This "fingerprint" serves as a universal identifier. A transaction can be referenced by its transaction ID, and once a transaction is recorded on the Bitcoin blockchain, every node in the Bitcoin network knows that this transaction is valid.
+
+For example, a transaction ID might look like this:
+
+.A transaction ID produced from hashing the transaction data
+----
+e31e4e214c3f436937c74b8663b3ca58f7ad5b3fce7783eb84fd9a5ee5b9a54c
+----
+
+This is a real transaction (created as an example for the _Mastering Bitcoin_ book) that can be found on the Bitcoin blockchain. Try to find it by entering this TxID into a block explorer:
+
+++++
+<ul class="simplelist">
+<li><a href="https://blockstream.info/tx/e31e4e214c3f436937c74b8663b3ca58f7ad5b3fce7783eb84fd9a5ee5b9a54c"><em>https://blockstream.info/tx/e31e4e214c3f436937c74b8663b3ca58f7ad5b3fce7783eb84fd9a5ee5b9a54c</em></a></li></ul>
+++++
+
+or use the short link (case-sensitive):
+
+++++
+<ul class="simplelist">
+<li><a href="http://bit.ly/AliceTx"><em>http://bit.ly/AliceTx</em></a></li>
+</ul>
+++++
+
+==== Outpoints: Output Identifiers
+
+((("Bitcoin transactions","outpoints (output identifiers)")))((("outpoints (output identifiers)")))Because every transaction has a unique ID, we can also identify a transaction output within that transaction uniquely by reference to the TxID and the output index number. The first output in a transaction is output index 0, the second output is output index 1, and so on. An output identifier is commonly known as an _outpoint_.
+
+By convention we write an outpoint as the TxID, a colon, and the output index number:
+
+.A outpoint: identifying an output by TxID and index number
+----
+7957a35fe64f80d234d76d83a2a8f1a0d8149a41d81de548f0a65a8a999f6f18:0
+----
+
+Output identifiers (outpoints) are the mechanisms that link transactions together in a chain. Every transaction input is a reference to a specific output of a previous transaction. That reference is an outpoint: a TxID and output index number. So a transaction "spends" a specific output (by index number) from a specific transaction (by TxID) to create new outputs that themselves can be spent by reference to the outpoint.
+
+<<tx_chain_vout>> presents the chain of transactions from Alice to Bob to Chan to Dina, this time with outpoints in each of the inputs.
+
+[[tx_chain_vout]]
+.Transaction inputs refer to outpoints forming a chain
+image::images/mtln_aa07.png["Transaction inputs refer to outpoints forming a chain"]
+
+The input in Bob's transaction references Alice's transaction (by TxID) and the 0 indexed output.
+
+The input in Chan's transaction references Bob's transaction's TxID and the first indexed output, because the payment to Chan is output #1. In Bob's payment to Chan, Bob's change is output #0.footnote:[Recall that change doesn't have to be the last output in a transaction and is in fact indistinguishable from other outputs.]
+
+Now, if we look at Alice's payment to Bob, we can see that Alice is spending an outpoint that was the third (output index #2) output in a transaction whose ID is 6a5f1b3[...]. We don't see that referenced transaction in the diagram, but we can deduce these details from the outpoint.(((range="endofrange", startref="ix_appendix-bitcoin-fundamentals-review-asciidoc9")))(((range="endofrange", startref="ix_appendix-bitcoin-fundamentals-review-asciidoc8")))
+
+=== Bitcoin Script
+
+((("Bitcoin (system)","script", id="ix_appendix-bitcoin-fundamentals-review-asciidoc11", range="startofrange")))((("Bitcoin script", id="ix_appendix-bitcoin-fundamentals-review-asciidoc12", range="startofrange")))The final element of Bitcoin that is needed to complete our understanding is the scripting language that controls access to outpoints. So far, we've simplified the description by saying "Alice signs the transaction to pay Bob." Behind the scenes, however, there is some hidden complexity that makes it possible to implement more complex spending conditions. The simplest and most common spending condition is "present a signature matching the following public key." A spending condition like this is recorded in each output as _locking script_ written in a scripting language called _Bitcoin Script_.
+
+Bitcoin Script is an extremely simple stack-based scripting language. It does not contain loops or recursion and therefore is _Turing incomplete_ (meaning it cannot express arbitrary complexity and has predictable execution). Those familiar with the (now ancient) programming language FORTH will recognize the syntax and style.
+
+==== Running Bitcoin Script
+
+((("Bitcoin script","running")))In simple terms, the Bitcoin system evaluates Bitcoin Script by running the script on a stack; if the final result is +TRUE+, it considers the spending condition satisfied and the transaction valid.
+
+Let's look at a very simple example of Bitcoin Script, which adds the numbers 2 and 3 and then compares the result to the number 5:
+
+----
+2 3 ADD 5 EQUAL
+----
+
+In <<figa08>>, we see how this script is executed (from left to right).
+
+[[figa08]]
+.Example of Bitcoin Script execution
+image::images/mtln_aa08.png["Example of Bitcoin Script execution"]
+
+[role="pagebreak-before less_space"]
+==== Locking and Unlocking Scripts
+
+((("Bitcoin script","locking/unlocking")))Bitcoin Script is made up of two parts:
+
+Locking scripts:: ((("locking scripts")))These are embedded in transaction outputs, setting the conditions that must be fulfilled to spend that output. For example, Alice's wallet adds a locking script to the output paying Bob, that sets the condition that Bob's signature is required to spend it.
+
+Unlocking scripts:: ((("unlocking scripts")))These are embedded in transaction inputs, fulfilling the conditions set by the referenced output's locking script. For example, Bob can unlock the preceding output by providing an unlocking script containing a digital signature.
+
+Using a simplified model, for validation, the unlocking script and locking script are concatenated and executed (P2SH and SegWit are exceptions). For example, if someone locked a transaction output with the locking script +"3 ADD 5 EQUAL"+, we could spend it with the unlocking script "+2+" in a transaction input. Anyone validating that transaction would concatenate our unlocking script (+2+) and the locking script (+3 ADD 5 EQUAL+) and run the result through the Bitcoin Script execution engine. They would get +TRUE+ and we would be able to spend the output.
+
+Obviously, this simplified example would make a very poor choice for locking an actual Bitcoin output because there is no secret, just basic arithmetic. Anyone could spend the output by providing the answer "2." Most locking scripts therefore require demonstrating knowledge of a secret.
+
+==== Locking to a Public Key (Signature)
+
+((("Bitcoin script","locking to a public key (signature)")))((("locking scripts","locking to a public key (signature)")))((("signatures, locking to a public key")))The simplest form of a locking script is one that requires a signature. Let's consider Alice's transaction that pays Bob 50,000 satoshis. The output Alice creates to pay Bob will have a locking script requiring Bob's signature and would look like this:
+
+[[bob_locking_script]]
+.A locking script that requires a digital signature from Bob's private key
+----
+<Bob Public Key> CHECKSIG
+----
+
+The operator `CHECKSIG` takes two items from the stack: a signature and a public key. As you can see, Bob's public key is in the locking script, so what is missing is the signature corresponding to that public key. This locking script can only be spent by Bob, because only Bob has the corresponding private key needed to produce a digital signature matching the public key.
+
+To unlock this locking script, Bob would provide an unlocking script containing only his digital signature:
+
+[[bob_unlocking_script]]
+.An unlocking script containing (only) a digital signature from Bob's private key
+----
+<Bob Signature>
+----
+
+In <<locking_unlocking_chain>> you can see the locking script in Alice's transaction (in the output that pays Bob) and the unlocking script (in the input that spends that output) in Bob's transaction.
+
+[[locking_unlocking_chain]]
+.A transaction chain showing the locking script (output) and unlocking script (input)
+image::images/mtln_aa09.png["A transaction chain showing the locking script (output) and unlocking script (input)"]
+
+To validate Bob's transaction, a Bitcoin node would do the following:
+
+. Extract the unlocking script from the input (+<Bob Signature>+).
+. Look up the outpoint it is attempting to spend (+a643e37...3213:0+). This is Alice's transaction and would be found on the blockchain.
+. Extract the locking script from that outpoint (+<Bob PubKey> CHECKSIG+).
+. Concatenate into one script, placing the unlocking script in front of the locking script (+<Bob Signature> <Bob PubKey> CHECKSIG+).
+. Execute this script on the Bitcoin Script execution engine to see what result is produced.
+. If the result is +TRUE+, deduce that Bob's transaction is valid because it was able to fulfill the spending condition to spend that outpoint.
+
+==== Locking to a Hash (Secret)
+
+((("hashlock")))((("locking scripts","locking to a hash (secret)")))Another type of locking script, one that is used in the Lightning Network, is a _hashlock_. To unlock it, you must know the secret _preimage_ to the hash.
+
+To demonstrate this, let's have Bob generate a random number +R+ and keep it secret:
+
+----
+R = 1833462189
+----
+
+[role="pagebreak-before"]
+Now, Bob calculates the SHA-256 hash of this number:
+
+----
+H = SHA256(R) =>
+H = SHA256(1833462189) =>
+H = 0ffd8bea4abdb0deafd6f2a8ad7941c13256a19248a7b0612407379e1460036a
+----
+
+Now, Bob gives the hash +H+ we calculated previously to Alice, but keeps the number +R+ secret. Recall that because of the properties of cryptographic hashes, Alice can't "reverse" the hash calculation and guess the number +R+.
+
+Alice creates an output paying 50,000 satoshi with the locking script:
+
+----
+HASH256 H EQUAL
+----
+
+where +H+ is the actual hash value (+0ffd8...036a+) that Bob gave to Alice.
+
+Let's explain this script:
+
+The +HASH256+ operator pops a value from the stack and calculates the SHA-256 hash of that value. Then it pushes the result onto the stack.
+
+The +H+ value is pushed onto the stack, and then the +EQUAL+ operator checks if the two values are the same and pushes +TRUE+ or +FALSE+ onto the stack accordingly.
+
+Therefore, this locking script will only work if it is combined with an unlocking script that contains +R+, so that when concatenated, we have:
+
+----
+R HASH256 H EQUAL
+----
+
+Only Bob knows +R+, so only Bob can produce a transaction with an unlocking script revealing the secret value +R+.
+
+Interestingly, Bob can give the +R+ value to anyone else, who can then spend that Bitcoin. This makes the secret value +R+ almost like a bitcoin "voucher," since anyone who has it can spend the output Alice created. We'll see how this is a useful property for the Lightning Network!
+
+[[multisig]]
+==== Multisignature Scripts
+
+((("Bitcoin script","multisignature scripts")))((("multisignature scripts")))The Bitcoin Script language provides a multisignature building block (primitive), that can be used to build escrow services and complex ownership configurations between several stakeholders. ((("K-of-N scheme")))((("multisignature scheme")))An arrangement that requires multiple signatures to spend Bitcoin is called a _multisignature scheme_, further specified as a _K-of-N_ scheme, where:
+
+* _N_ is the total number of signers identified in the multisignature scheme, and
+* _K_ is the _quorum_ or _threshold_: the minimum number of signatures to authorize spending.
+
+[role="pagebreak-before"]
+The script for an __K__-of-__N__ multisignature is:
+
+----
+K <PubKey1> <PubKey2> ... <PubKeyN> N CHECKMULTISIG
+----
+
+where _N_ is the total number of listed public keys (Public Key 1 through Public Key _N_) and _K_ is the threshold of required signatures to spend the output.
+
+The Lightning Network uses a 2-of-2 multisignature scheme to build a payment channel. For example, a payment channel between Alice and Bob would be built on a 2-of-2 multisignature like this:
+
+----
+2 <PubKey Alice> <PubKey Bob> 2 CHECKMULTISIG
+----
+
+The preceding locking script can be satisfied with an unlocking script containing a pair of signatures:footnote:[The first argument (0) does not have any meaning but is required due to a bug in Bitcoin's multisignature implementation. This issue is described in _Mastering Bitcoin_, https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch07.asciidoc[Chapter 7].]
+
+----
+0 <Sig Alice> <Sig Bob>
+----
+The two scripts together would form the combined validation script:
+
+----
+0 <Sig Alice> <Sig Bob> 2 <PubKey Alice> <PubKey Bob> 2 CHECKMULTISIG
+----
+
+A multisignature locking script can be represented by a Bitcoin address, encoding the hash of the locking script. For example, the initial funding transaction of a Lightning payment channel is a transaction that pays to an address that encodes a locking script of a 2-of-2 multisig of the two channel partners.
+
+==== Timelock Scripts
+
+((("Bitcoin script","timelock scripts")))((("timelock scripts")))Another important building block that exists in Bitcoin and is used extensively in the Lightning Network is a _timelock_. A timelock is a restriction on spending that requires that a certain time or block height has elapsed before spending is allowed. It is a bit like a postdated check drawn from a bank account that can't be cashed before the date on the check.
+
+Bitcoin has two levels of timelocks: transaction-level timelocks and output-level timelocks.
+
+((("transaction-level timelock")))A _transaction-level timelock_ is recorded in the transaction `nLockTime` field of the transaction and prevents the entire transaction from being accepted before the timelock has passed. Transaction-level timelocks are the most commonly used timelock mechanism in Bitcoin today.
+
+((("output-level timelock")))An _output-level timelock_ is created by a script operator. There are two types of output timelocks: absolute timelocks and relative timelocks.
+
+((("absolute timelock")))Output-level _absolute timelocks_ are implemented by the operator +CHECKLOCKTIMEVERIFY+, which is often shortened in conversation as _CLTV_. Absolute timelocks implement a time constraint with an absolute timestamp or blockheight, expressing the equivalent of "not spendable before block 800,000."
+
+((("relative timelock")))Output-level _relative timelocks_ are implemented by the operator +CHECKSEQUENCEVERIFY+, often shortened in conversation as _CSV_. Relative timelocks implement a spending constraint that is relative to the confirmation of the transaction, expressing the equivalent of "can't be spent until 1,024 blocks after confirmation."
+
+[[conditional_scripts]]
+==== Scripts with Multiple Conditions
+
+((("Bitcoin script","scripts with multiple conditions")))((("conditional clauses")))One ((("flow control", id="ix_appendix-bitcoin-fundamentals-review-asciidoc13", range="startofrange")))of the more powerful features of Bitcoin Script is flow control, also known as conditional clauses. You are probably familiar with flow control in various programming languages that use the construct +IF...THEN...ELSE+. Bitcoin conditional clauses look a bit different, but are essentially the same construct.
+
+At a basic level, bitcoin conditional opcodes allow us to construct a locking script that has two ways of being unlocked, depending on a +TRUE+/+FALSE+ outcome of evaluating a logical condition. For example, if x is +TRUE+, the locking script is A +ELSE+ the locking script is B.
+
+Additionally, bitcoin conditional expressions can be _nested_ indefinitely, meaning that a conditional clause can contain another within it, which contains another, etc. Bitcoin Script flow control can be used to construct very complex scripts with hundreds or even thousands of possible execution paths. There is no limit to nesting, but consensus rules impose a limit on the maximum size, in bytes, of a script.
+
+Bitcoin implements flow control using the +IF+, +ELSE+, +ENDIF+, and +NOTIF+ opcodes. Additionally, conditional expressions can contain boolean operators such as +BOOLAND+, pass:[<span class="keep-together"><code>BOOLOR</code></span>], and +NOT+.
+
+At first glance, you may find Bitcoin's flow control scripts confusing. That is because Bitcoin Script is a stack language. The same way that the arithmetic operation latexmath:[$1 + 1$] looks "backward" when expressed in Bitcoin Script as +1 1 ADD+, flow control clauses in
+Bitcoin also look "backward."
+
+In most traditional (procedural) programming languages, flow control looks like this:
+
+.Pseudocode of flow control in most programming languages
+----
+if (condition):
+  code to run when condition is true
+else:
+  code to run when condition is false
+code to run in either case
+----
+
+In a stack-based language like Bitcoin Script, the logical condition comes _before_ the +IF+, which makes it look "backward," like this:
+
+.Bitcoin Script flow control
+----
+condition
+IF
+  code to run when condition is true
+ELSE
+  code to run when condition is false
+ENDIF
+code to run in either case
+----
+
+When reading Bitcoin Script, remember that the condition being evaluated comes _before_ the +IF+ opcode.
+
+==== Using Flow Control in Scripts
+
+((("Bitcoin script","using flow control in")))A very common use for flow control in Bitcoin Script is to construct a locking script that offers multiple execution paths, each a different way of redeeming the UTXO.
+
+Let's look at a simple example, where we have two signers, Alice and Bob, and either one is able to redeem. With multisig, this would be expressed as a 1-of-2 multisig script. For the sake of demonstration, we will do the same thing with an +IF+ clause:
+
+----
+IF
+ <Alice's Pubkey> CHECKSIG
+ELSE
+ <Bob's Pubkey> CHECKSIG
+ENDIF
+----
+
+Looking at this locking script, you may be wondering: "Where is the condition? There is nothing preceding the +IF+ clause!"
+
+The condition is not part of the locking script. Instead, the condition will be _offered in the unlocking script_, allowing Alice and Bob to "choose" which execution path they want.
+
+Alice redeems this with the unlocking script:
+----
+<Alice's Sig> 1
+----
+
+The +1+ at the end serves as the condition (+TRUE+) that will make the +IF+ clause execute the first redemption path for which Alice has a signature.
+
+For Bob to redeem this, he would have to choose the second execution path by giving a +FALSE+ value to the +IF+ clause:
+
+----
+<Bob's Sig> 0
+----
+
+Bob's unlocking script puts a +0+ on the stack, causing the +IF+ clause to execute the second (+ELSE+) script, which requires Bob's signature.
+
+Because each of the two conditions also requires a signature, Alice can't use the second clause and Bob can't use the first clause; they don't have the necessary signatures for that!
+
+Since conditional flows can be nested, so can the +TRUE+ / +FALSE+ values in the unlocking script, to navigate a complex path of conditions.
+
+In <<htlc_script_example>> you can see an example of the kind of complex script that is used in the Lightning Network, with multiple conditions.footnote:[From https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md[BOLT #3].] The scripts used in the Lightning Network are highly optimized and compact, to minimize the on-chain footprint, so they are not easy to read and understand.(((range="endofrange", startref="ix_appendix-bitcoin-fundamentals-review-asciidoc13"))) Nevertheless, see if you can identify some of the Bitcoin Script concepts we learned about in this chapter.(((range="endofrange", startref="ix_appendix-bitcoin-fundamentals-review-asciidoc12")))(((range="endofrange", startref="ix_appendix-bitcoin-fundamentals-review-asciidoc11")))(((range="endofrange", startref="ix_appendix-bitcoin-fundamentals-review-asciidoc0")))
+
+[[htlc_script_example]]
+.A complex script used in the Lightning Network
+====
+----
+# To remote node with revocation key
+DUP HASH160 <RIPEMD160(SHA256(revocationpubkey))> EQUAL
+IF
+    CHECKSIG
+ELSE
+    <remote_htlcpubkey> SWAP SIZE 32 EQUAL
+    NOTIF
+        # To local node via HTLC-timeout transaction (timelocked).
+        DROP 2 SWAP <local_htlcpubkey> 2 CHECKMULTISIG
+    ELSE
+        # To remote node with preimage.
+        HASH160 <RIPEMD160(payment_hash)> EQUALVERIFY
+        CHECKSIG
+    ENDIF
+ENDIF
+----
+====

appendix_docker_basics.asciidoc

@@ -0,0 +1,111 @@
+[appendix]
+[[appendix_docker]]
+== Docker Basic Installation and Use
+
+((("Docker","basic installation and use", id="ix_appendix_docker_basics-asciidoc0", range="startofrange")))This book contains a number of examples that run inside Docker containers for standardization across different operating systems.
+
+This section will help you install Docker and familiarize yourself with some of the most commonly used Docker commands, so that you can run the book's example containers.
+
+
+=== Installing Docker
+
+((("Docker","installing")))Before we begin, you should install the Docker container system on your computer. Docker is an open system that is distributed for free as a _Community Edition_ for many different operating systems including Windows, macOS, and Linux. The Windows and Macintosh versions are called _Docker Desktop_ and consist of a GUI desktop application and command-line tools. The Linux version is called _Docker Engine_ and is comprised of a server daemon and command-line tools. We will be using the command-line tools, which are identical across all platforms.
+
+Go ahead and install Docker for your operating system by following the instructions to "Get Docker" from the https://docs.docker.com/get-docker[Docker website].
+
+Select your operating system from the list and follow the installation instructions.
+
+[TIP]
+====
+If you install on Linux, follow the post-installation instructions to ensure you can run Docker as a regular user instead of user root. Otherwise, you will need to prefix all +docker+ commands with +sudo+, running them as root like: +sudo docker+.
+====
+
+Once you have Docker installed, you can test your installation by running the demo container +hello-world+ like this:
+
+[[docker-hello-world]]
+----
+$ docker run hello-world
+
+Hello from Docker!
+This message shows that your installation appears to be working correctly.
+
+[...]
+----
+
+=== Basic Docker Commands
+
+((("Docker","basic commands")))In this appendix, we use Docker quite extensively. We will be using the following Docker commands and arguments.
+
+==== Building a Container
+
+++++
+<pre data-type="programlisting">docker build [-t <em>tag</em>] [<em>directory</em>]</pre>
+++++
+
+((("Docker","building a container")))((("Docker containers","building a container")))++__tag__++ is how we identify the container we are building, and ++__directory__++ is where the container's context (folders and files) and definition file (+Dockerfile+) are found.
+
+==== Running a Container
+
+++++
+<pre data-type="programlisting">docker run -it [--network <em>netname</em>] [--name <em>cname</em>] <em>tag</em></pre>
+++++
+
+((("Docker containers","running a container")))++__netname__++ is the name of a Docker network, ++__cname__++ is the name we choose for this container instance, and ++__tag__++ is the name tag we gave the container when we built it.
+
+==== Executing a Command in a Container
+
+++++
+<pre data-type="programlisting">docker exec <em>cname command</em></pre>
+++++
+
+((("Docker containers","executing a command in a container")))++__cname__++ is the name we gave the container in the +run+ command, and ++__command__++ is an executable or script that we want to run inside the container.
+
+==== Stopping and Starting a Container
+
+((("Docker containers","stopping/starting a container")))In most cases, if we are running a container in an _interactive_ as well as _terminal_ mode, i.e., with the +i+ and +t+ flags (combined as +-it+) set, the container can be stopped by simply pressing Ctrl-C or by exiting the shell with +exit+ or Ctrl-D. If a container does not terminate, you can stop it from another terminal like this:
+
+++++
+<pre data-type="programlisting">docker stop <em>cname</em></pre>
+++++
+
+To resume an already existing container, use the `start` command like so:
+
+++++
+<pre data-type="programlisting">docker start <em>cname</em></pre>
+++++
+
+==== Deleting a Container by Name
+
+((("Docker containers","deleting a container by name")))If you name a container instead of letting Docker name it randomly, you cannot reuse that name until the container is deleted. Docker will return an error like this:
+
+----
+docker: Error response from daemon: Conflict. The container name "/bitcoind" is already in use...
+----
+
+To fix this, delete the existing instance of the container:
+
+++++
+<pre data-type="programlisting">docker rm <em>cname</em></pre>
+++++
+
+++__cname__++ is the name assigned to the container (+bitcoind+ in the example error message).
+
+==== Listing Running Containers
+
+----
+docker ps
+----
+
+((("Docker containers","list running containers")))This command shows the current running containers and their names.
+
+==== Listing Docker Images
+
+----
+docker image ls
+----
+
+((("Docker containers","list Docker images")))This command shows the Docker images that have been built or downloaded on your computer.
+
+=== Conclusion
+
+These basic Docker commands will be enough to get you started and will allow you to run all the examples in this book.(((range="endofrange", startref="ix_appendix_docker_basics-asciidoc0")))

appendix_license_notices.asciidoc

@@ -1,16 +1,30 @@
 [appendix]
-== License Notices
+[[sources_licenses]]
+== Sources and License Notices
 
-This appendix contains license notices for material included by permission granted via open licenses.
+This appendix contains attribution and license notices for material included by permission granted via open licenses.
 
+=== Sources
+
+Material was sourced from various public and open-licensed sources:
+
+* https://wiki.ion.radar.tech[ION Lightning Network Wiki]
+* https://medium.com/suredbits/lightning-101-what-is-a-lightning-invoice-d527db1a77e6["Lightning 101: What Is a Lightning Invoice?" by Suredbits]
+* https://github.com/lightningnetwork/lightning-rfc[Lightning Network In-Progress Specifications GitHub]; Creative Commons Attribution (CC-BY 4.0)
+* https://w.wiki/4QCL[Wikipedia page, "Elliptic-curve Diffie–Hellman"]
+* https://w.wiki/4QCX[Wikipedia page, "Digital signature"]
+* https://w.wiki/4QCb[Wikipedia page, "Cryptographic hash function"]
+* https://w.wiki/4QCc[Wikipedia page, "Onion routing"]
+* https://w.wiki/4QCd[Wikimedia Commons, "Lightning Network Protocol Suite"]
+* https://w.wiki/4QCf[Wikimedia Commons, "Introduction to the Lightning Network Protocol and the Basics of Lightning Technology"]
+
+[role="pagebreak-before less_space"]
 === BTCPay Server
 
-BTCPay Server Logo, screenshots and other images used with permission under the MIT License:
+BTCPay Server https://github.com/btcpayserver/btcpayserver-media[logo, screenshots, and other images] used with permission under the https://github.com/btcpayserver/btcpayserver-media/blob/master/LICENSE[MIT License]:
 
-https://github.com/btcpayserver/btcpayserver-media
-https://github.com/btcpayserver/btcpayserver-media/blob/master/LICENSE
-
-++++
+[quote]
+____
 MIT License
 
 Copyright (c) 2018 BTCPay Server
@@ -25,11 +39,15 @@ furnished to do so, subject to the following conditions:
 The above copyright notice and this permission notice shall be included in all
 copies or substantial portions of the Software.
 
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+THE SOFTWARE IS PROVIDED "AS IS," WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 SOFTWARE.
-++++
+____
+
+=== Lamassu Industries AG
+
+Images of the https://lamassu.is/product/gaia[_Gaia_ Bitcoin ATM] seen in <<bitcoin-atm>> are used with permission of Lamassu Industries AG. The use of these images is not an endorsement of the product or company, but are provided as a visual example of a Bitcoin ATM.

appendix_protocol_messages.asciidoc

@@ -0,0 +1,736 @@
+[appendix]
+[[wire_protocol_enumeration]]
+[[protocol_messages]]
+[[messages]]
+== Wire Protocol Messages
+
+((("wire protocol messages", id="ix_appendix_protocol_messages-asciidoc0", range="startofrange")))This appendix lists all the currently defined message types used in the Lightning P2P protocol. Additionally, we show the structure of each message, grouping the messages into logical groupings based on the protocol flows.
+
+[NOTE]
+====
+Lightning Protocol messages are extensible and their structure may change during network-wide upgrades. For the authoritative information, consult the latest version of the BOLTs found in the https://github.com/lightningnetwork/lightning-rfc[GitHub Lightning-RFC repository].
+====
+
+=== Message Types
+
+((("wire protocol messages","message types", id="ix_appendix_protocol_messages-asciidoc1", range="startofrange")))Currently defined message types are listed in <<apdx_message_types>>.
+
+[[apdx_message_types]]
+.Message types
+[options="header"]
+|===
+| Type integer | Message name | Category
+| 16  | `init`             |  Connection Establishment
+| 17  | `error`             | Error Communication
+| 18  | `ping`             | Connection Liveness
+| 19  | `pong`             | Connection Liveness
+| 32  | `open_channel`             |          Channel Funding
+| 33  | `accept_channel`             |          Channel Funding
+| 34  | `funding_created`             |          Channel Funding
+| 35  | `funding_signed`             |          Channel Funding
+| 36  | `funding_locked`             |          Channel Funding + Channel Operation
+| 38  | `shutdown`             | Channel Closing
+| 39  | `closing_signed`             |         Channel Closing
+| 128 | `update_add_htlc`             |          Channel Operation
+| 130 | `update_fulfill_hltc`             |          Channel Operation
+| 131 | `update_fail_htlc`             |          Channel Operation
+| 132 | `commit_sig`             |          Channel Operation
+| 133 | `revoke_and_ack`             |          Channel Operation
+| 134 | `update_fee`             |          Channel Operation
+| 135 | `update_fail_malformed_htlc`             |          Channel Operation
+| 136 | `channel_reestablish`             |         Channel Operation
+| 256 | `channel_announcement`             |          Channel Announcement
+| 257 | `node_announcement`             |          Channel Announcement
+| 258 | `channel_update`             |          Channel Announcement
+| 259 | `announce_signatures`             |          Channel Announcement
+| 261 | `query_short_chan_ids`             |          Channel Graph Syncing
+| 262 | `reply_short_chan_ids_end`             |          Channel Graph Syncing
+| 263 | `query_channel_range`             |          Channel Graph Syncing
+| 264 | `reply_channel_range`             |          Channel Graph Syncing
+| 265 | `gossip_timestamp_range`             |          Channel Graph Syncing
+|===
+
+In <<message_types>>, the `Category` field allows us to quickly categorize a
+message based on its functionality within the protocol itself. At a high level,
+we place a message into one of eight (nonexhaustive) buckets including:
+
+Connection Establishment:: Sent when a peer-to-peer connection is first
+   established. Also used to negotiate the set of features supported
+   by a new connection.
+
+Error Communication:: Used by peers to communicate the occurrence of
+   protocol level errors to each other.
+
+Connection Liveness:: Used by peers to check that a given transport
+  connection is still live.
+
+Channel Funding:: Used by peers to create a new payment channel. This
+   process is also known as the channel funding process.
+
+Channel Operation:: The act of updating a given channel off-chain. This
+   includes sending and receiving payments, as well as forwarding payments
+   within the network.
+
+Channel Announcement:: The process of announcing a new public channel to
+   the wider network so it can be used for routing purposes.
+
+Channel Graph Syncing:: The process of downloading and verifying the channel
+  graph.
+
+
+Notice how messages that belong to the same category typically share an
+adjacent _message type_ as well. This is done on purpose to group
+semantically similar messages together within the specification itself.(((range="endofrange", startref="ix_appendix_protocol_messages-asciidoc1")))
+
+=== Message Structure
+
+((("wire protocol messages","message structure", id="ix_appendix_protocol_messages-asciidoc2", range="startofrange")))We now detail each message category in order to define
+the precise structure and semantics of all defined messages within the LN
+protocol.
+
+==== Connection Establishment Messages
+
+((("wire protocol messages","connection establishment messages")))Messages in this category are the very first message sent between peers once
+they establish a transport connection. At the time of writing this chapter,
+there exists only a single message within this category, the `init` message.
+The `init` message is sent by _both_ sides of the connection once it has been
+first established. No other messages are to be sent before the `init` message
+has been sent by both parties.
+
+
+[[apdx_init_message]]
+===== The init message
+
+((("wire protocol messages","init message")))The structure of the `init` message is defined as follows:
+
+ * Type: 16
+ * Fields:
+    ** `uint16`: `global_features_len`
+    ** `global_features_len*byte`: `global_features`
+    ** `uint16`: `features_len`
+    ** `features_len*byte`: `features`
+    ** `tlv_stream_tlvs`
+
+Structurally, the `init` message is composed of two variable size bytes slices
+that each store a set of _feature bits_. ((("feature bits","defined")))As we see in <<feature_bits>>, feature bits are a
+primitive used within the protocol to advertise the set of protocol
+features a node either understands (optional features) or demands (required
+features).
+
+Note that modern node implementations will only use the `features` field, with
+items residing within the `global_features` vector for primarily _historical_
+purposes (backward compatibility).
+
+What follows after the core message is a series of Type-Length-Value (TLV) records that can be used to extend the message in a forward- and backward-compatible manner in the future. We'll cover what TLV records are and how
+they're used later in this appendix.
+
+An `init` message is then examined by a peer to determine if the
+connection is well-defined based on the set of optional and required feature
+bits advertised by both sides.
+
+An optional feature means that a peer knows about a feature, but they don't
+consider it critical to the operation of a new connection. An example of one
+would be something like the ability to understand the semantics of a newly
+added field to an existing message.
+
+On the other hand, required features indicate that if the other peer doesn't
+know about the feature, then the connection isn't well defined. An example of
+such a feature would be a theoretical new channel type within the protocol: if
+your peer doesn't know of this feature, then you don't want to keep the
+connection because they're unable to open your new preferred channel type.
+
+==== Error Communication Messages
+
+((("wire protocol messages","error communication messages")))Messages in this category are used to send connection level errors between two
+peers. Another type of error exists in the protocol: an
+HTLC forwarding level error. Connection level errors may signal things like
+feature bit incompatibility or the intent to _force close_ (unilaterally
+broadcast the latest signed commitment).
+
+[[apdx_error_message]]
+===== The error message
+
+((("wire protocol messages","error message")))The sole message in this category is the `error` message.
+
+ * Type: 17
+ * Fields:
+  ** `channel_id` : `chan_id`
+  ** `uint16` : `data_len`
+  ** `data_len*byte` : `data`
+
+An `error` message can be sent within the scope of a particular channel by
+setting the `channel_id` to the `channel_id` of the channel undergoing this
+new error state. Alternatively, if the error applies to the connection in
+general, then the `channel_id` field should be set to all zeroes. This all zero
+`channel_id` is also known as the connection level identifier for an error.
+
+Depending on the nature of the error, sending an `error` message to a peer you
+have a channel with may indicate that the channel cannot continue without
+manual intervention, so the only option at that point is to force close the
+channel by broadcasting the latest commitment state of the channel.
+
+==== Connection Liveness
+
+((("wire protocol messages","connection liveness messages")))Messages in this section are used to probe to determine if a connection is
+still live or not. Because the LN protocol somewhat abstracts over the underlying
+transport being used to transmit the messages, a set of protocol level ((("wire protocol messages","ping message")))((("wire protocol messages","pong message")))`ping`
+and `pong` messages are defined.
+
+[[apdx_ping_message]]
+===== The ping message
+
+The `ping` message is used to check whether the other party in a connection is "live." It contains the following fields:
+
+ * Type: 18
+ * Fields:
+  ** `uint16` : `num_pong_bytes`
+  ** `uint16` : `ping_body_len`
+  ** `ping_body_len*bytes` : `ping_body`
+
+Next its companion, the `pong` message.
+
+[[apdx_pong_message]]
+===== The pong message
+
+The +pong+ message is sent in response to the +ping+ message and contains the following fields:
+
+ * Type: 19
+ * Fields:
+  ** `uint16` : `pong_body_len`
+  ** `ping_body_len*bytes` : `pong_body`
+
+A `ping` message can be sent by either party at any time.
+
+The `ping` message includes a `num_pong_bytes` field that is used to instruct
+the receiving node with respect to how large the payload it sends in its `pong`
+message is. The `ping` message also includes a `ping_body` opaque set of bytes
+which can be safely ignored. It only serves to allow a sender to pad out `ping`
+messages they send, which can be useful in attempting to thwart certain
+de-anonymization techniques based on packet sizes on the wire.
+
+A `pong` message should be sent in response to a received `ping` message. The
+receiver should read a set of `num_pong_bytes` random bytes to send back as the
+`pong_body` field. Clever use of these fields/messages may allow a privacy
+conscious routing node to attempt to thwart certain classes of network
+de-anonymization attempts because they can create a "fake" transcript that
+resembles other messages based on the packet sizes sent across. Remember that by
+default the Lightning Network uses an _encrypted_ transport, so a passive network monitor
+cannot read the plain-text bytes and thus only has timing and packet sizes to go
+off of.
+
+==== Channel Funding
+
+((("wire protocol messages","channel funding", id="ix_appendix_protocol_messages-asciidoc3", range="startofrange")))As we go on, we enter into the territory of the core messages that govern the
+functionality and semantics of the Lightning Protocol. In this section, we
+explore the messages sent during the process of creating a new channel. We'll
+only describe the fields used, as we leave an in-depth analysis of the
+funding process to <<payment_channels>>.
+
+Messages that are sent during the channel funding flow belong to the following
+set of five messages: `open_channel`, `accept_channel`, `funding_created`,
+`funding_signed`, and `funding_locked`.
+
+The detailed protocol flow using these messages is described in <<payment_channels>>.
+
+[[apdx_open_channel_message]]
+===== The open_channel message
+
+The +open_channel+ message starts the channel funding process and contains the following fields:
+
+ * Type: 32
+ * Fields:
+  ** `chain_hash` : `chain_hash`
+  ** `32*byte` : `temp_chan_id`
+  ** `uint64` : `funding_satoshis`
+  ** `uint64` : `push_msat`
+  ** `uint64` : `dust_limit_satoshis`
+  ** `uint64` : `max_htlc_value_in_flight_msat`
+  ** `uint64` : `channel_reserve_satoshis`
+  ** `uint64` : `htlc_minimum_msat`
+  ** `uint32` : `feerate_per_kw`
+  ** `uint16` : `to_self_delay`
+  ** `uint16` : `max_accepted_htlcs`
+  ** `pubkey` : `funding_pubkey`
+  ** `pubkey` : `revocation_basepoint`
+  ** `pubkey` : `payment_basepoint`
+  ** `pubkey` : `delayed_payment_basepoint`
+  ** `pubkey` : `htlc_basepoint`
+  ** `pubkey` : `first_per_commitment_point`
+  ** `byte` : `channel_flags`
+  ** `tlv_stream` : `tlvs`
+
+((("open_channel message")))((("wire protocol messages","open_channel message")))This is the first message sent when a node wishes to execute a new funding flow
+with another node. This message contains all the necessary information required
+for both peers to construct both the funding transaction as well as the
+commitment transaction.
+
+At the time of writing this chapter, a single TLV record is defined within
+the set of optional TLV records that may be appended to the end of a defined
+message:
+
+ * Type: 0
+ * Data:  `upfront_shutdown_script`
+
+The `upfront_shutdown_script` is a variable-sized byte slice that must be a
+valid public key script as accepted by the Bitcoin network's consensus
+algorithm. By providing such an address, the sending party is able to
+effectively create a "closed loop" for their channel, as neither side will sign
+off an cooperative closure transaction that pays to any other address. In
+practice, this address is usually one derived from a cold storage wallet.
+
+The `channel_flags` field is a bitfield of which, at the time of writing, only
+the _first_ bit has any sort of significance. If this bit is set, then this channel is to be advertised to the public network as a routable channel. Otherwise, the channel is considered to be unadvertised, also
+commonly referred to as a private channel.
+
+[[apdx_accept_channel_message]]
+===== The accept_channel message
+
+((("accept_channel message")))((("wire protocol messages","accept_channel message")))The `accept_channel` message is the response to the `open_channel` message.
+
+[role="pagebreak-before"]
+ * Type: 33
+ * Fields:
+  ** `32*byte` : `temp_chan_id`
+  ** `uint64` : `dust_limit_satoshis`
+  ** `uint64` : `max_htlc_value_in_flight_msat`
+  ** `uint64` : `channel_reserve_satoshis`
+  ** `uint64` : `htlc_minimum_msat`
+  ** `uint32` : `minimum_depth`
+  ** `uint16` : `to_self_delay`
+  ** `uint16` : `max_accepted_htlcs`
+  ** `pubkey` : `funding_pubkey`
+  ** `pubkey` : `revocation_basepoint`
+  ** `pubkey` : `payment_basepoint`
+  ** `pubkey` : `delayed_payment_basepoint`
+  ** `pubkey` : `htlc_basepoint`
+  ** `pubkey` : `first_per_commitment_point`
+  ** `tlv_stream` : `tlvs`
+
+The `accept_channel` message is the second message sent during the funding flow
+process. It serves to acknowledge an intent to open a channel with a new remote
+peer. The message mostly echoes the set of parameters that the responder wishes
+to apply to their version of the commitment transaction. In <<payment_channels>>,
+when we go into the funding process in detail, we explore
+the implications of the various parameters that can be set when opening a new
+channel.
+
+[[apdx_funding_created_message]]
+===== The funding_created message
+
+((("funding_created message")))((("wire protocol messages","funding_created message")))In response, the initiator will send the `funding_created` message.
+
+ * Type: 34
+ * Fields:
+  ** `32*byte` : `temp_chan_id`
+  ** `32*byte` : `funding_txid`
+  ** `uint16` : `funding_output_index`
+  ** `sig` : `commit_sig`
+
+Once the initiator of a channel receives the `accept_channel` message from the
+responder, they have all the materials they need to construct the
+commitment transaction, as well as the funding transaction. As channels by
+default are single funder (only one side commits funds), only the initiator
+needs to construct the funding transaction. As a result, to allow the
+responder to sign a version of a commitment transaction for the initiator, the
+initiator only needs to send the funding outpoint of the channel.
+
+[[apdx_funding_signed_message]]
+===== The funding_signed message
+
+((("funding_signed message")))((("wire protocol messages","funding_signed message")))To conclude, the responder sends the `funding_signed` message.
+
+ * Type: 34
+ * Fields:
+  ** `channel_id` : `channel_id`
+  ** `sig` : `signature`
+
+To conclude after the responder receives the `funding_created` message, they
+now own a valid signature of the commitment transaction by the initiator. With
+this signature they're able to exit the channel at any time by signing their
+half of the multisig funding output and broadcasting the transaction. This is
+referred to as a force close. Conversely, to give the initiator the ability to close the channel, the responder also signs the initiator's commitment transaction.
+
+Once this message has been received by the initiator, it's safe for them to
+broadcast the funding transaction because they're now able to exit the channel
+agreement unilaterally.
+
+[[apdx_funding_locked_message]]
+===== The funding_locked message
+
+((("funding_locked message")))((("wire protocol messages","funding_locked message")))Once the funding transaction has received enough confirmations, the
+`funding_locked` message is sent.
+
+ * Type: 36
+ * Fields:
+  ** `channel_id` : `channel_id`
+  ** `pubkey` : `next_per_commitment_point`
+
+Once the funding transaction obtains a `minimum_depth` number of confirmations,
+then the `funding_locked` message is to be sent by both sides. Only after this
+message has been received and sent can the channel begin to be used.(((range="endofrange", startref="ix_appendix_protocol_messages-asciidoc3")))
+
+==== Channel Closing
+
+((("wire protocol messages","channel closing")))Channel closing is a multistep process. ((("wire protocol messages","shutdown message")))One node initiates by sending the `shutdown` message. The two channel partners then exchange a series of `closing_signed` messages to negotiate mutually acceptable fees for the closing transaction. ((("closing_signed message")))((("wire protocol messages","closing_signed message")))The channel funder sends the first `closing_signed` message, and the other side can accept by sending a `closing_signed` message with the same fee values.
+
+[[apdx_shutdown_message]]
+===== The shutdown message
+
+The +shutdown+ message initiates the process of closing a channel and contains the following fields:
+
+ * Type: 38
+ * Fields:
+  ** `channel_id` : `channel_id`
+  ** `u16` : `len`
+  ** `len*byte` : `scriptpubkey`
+
+[[apdx_closing_signed_message]]
+===== The closing_signed message
+
+The +closing_signed+ message is sent by each channel partner until they agree on fees. It contains the following fields:
+
+ * Type: 39
+ * Fields:
+  ** `channel_id` : `channel_id`
+  ** `u64` : `fee_satoshis`
+  ** `signature` : `signature`
+
+==== Channel Operation
+
+((("wire protocol messages","channel operation", id="ix_appendix_protocol_messages-asciidoc4", range="startofrange")))In this section, we briefly describe the set of messages used to allow
+nodes to operate a channel. By operation, we mean being able to send, receive,
+and forward payments for a given channel.
+
+To send, receive, or forward a payment over a channel, an HTLC must
+first be added to both commitment transactions that comprise a channel link.
+
+[role="pagebreak-before less_space"]
+[[apdx_update_add_htlc_message]]
+===== The update_add_htlc message
+
+((("channel operation","update_add_htlc message")))((("update_add_htlc message")))((("wire protocol messages","update_add_htlc message")))The `update_add_htlc` message allows either side to add a new HTLC to the
+opposite commitment transaction.
+
+ * Type: 128
+ * Fields:
+  ** `channel_id` : `channel_id`
+  ** `uint64` : `id`
+  ** `uint64` : `amount_msat`
+  ** `sha256` : `payment_hash`
+  ** `uint32` : `cltv_expiry`
+  ** `1366*byte` : `onion_routing_packet`
+
+Sending this message allows one party to initiate either sending a new payment
+or forwarding an existing payment that arrived via an incoming channel. The
+message specifies the amount (`amount_msat`) along with the payment hash that
+unlocks the payment itself. The set of forwarding instructions of the next hop
+are onion encrypted within the `onion_routing_packet` field. In <<onion_routing>>, on
+multihop HTLC forwarding, we cover the onion routing protocol used in the
+Lightning Network in detail.
+
+Note that each HTLC sent uses an automatically incrementing ID which is used by any
+message which modifies an HTLC (settle or cancel) to reference the HTLC in a
+unique manner scoped to the channel.
+
+[[apdx_update_fulfill_hltc_message]]
+===== The update_fulfill_hltc message
+
+((("channel operation","update_fulfill_hltc message")))((("update_fulfill_hltc message")))The `update_fulfill_hltc` message allows redemption (receipt) of an active HTLC.
+
+ * Type: 130
+ * Fields:
+  ** `channel_id` : `channel_id`
+  ** `uint64` : `id`
+  ** `32*byte` : `payment_preimage`
+
+This message is sent by the HTLC receiver to the proposer to redeem an
+active HTLC. The message references the `id` of the HTLC in question, and also
+provides the preimage (which unlocks the HLTC).
+
+[[apdx_update_fail_htlc_message]]
+===== The update_fail_htlc message
+
+((("channel operation","update_fail_htlc message")))((("update_fail_htlc message")))The `update_fail_htlc` message is sent to remove an HTLC from a commitment transaction.
+
+ * Type: 131
+ * Fields:
+  ** `channel_id` : `channel_id`
+  ** `uint64` : `id`
+  ** `uint16` : `len`
+  ** `len*byte` : `reason`
+
+The `update_fail_htlc` message is the opposite of the `update_fulfill_hltc` message in that
+it allows the receiver of an HTLC to remove the very same HTLC. This message is
+typically sent when an HTLC cannot be properly routed upstream and needs to be
+sent back to the sender to unravel the HTLC chain. As we explore in
+<<failure_messages>>, the message contains an _encrypted_ failure reason (`reason`) which
+may allow the sender to either adjust their payment route or terminate if the
+failure itself is a terminal one.
+
+[[apdx_commitment_signed_message]]
+===== The commitment_signed message
+
+((("channel operation","commitment_signed message")))((("commitment_signed message")))The `commitment_signed` message is used to stamp the creation of a new commitment transaction.
+
+ * Type: 132
+ * Fields:
+  ** `channel_id` : `channel_id`
+  ** `sig` : `signature`
+  ** `uint16` : `num_htlcs`
+  ** `num_htlcs*sig` : `htlc_signature`
+
+In addition to sending a signature for the next commitment transaction, the
+sender of this message also needs to send a signature for each HTLC that's
+present on the commitment transaction.
+
+[role="pagebreak-before less_space"]
+[[apdx_revoke_and_ack_message]]
+===== The revoke_and_ack message
+
+((("channel operation","revoke_and_ack message")))((("revoke_and_ack message")))The `revoke_and_ack` is sent to revoke a dated commitment.
+
+ * Type: 133
+ * Fields:
+  ** `channel_id` : `channel_id`
+  ** `32*byte` : `per_commitment_secret`
+  ** `pubkey` : `next_per_commitment_point`
+
+Because the Lightning Network uses a replace-by-revoke commitment transaction, after
+receiving a new commitment transaction via the `commit_sig` message, a party
+must revoke their past commitment before they're able to receive another one.
+While revoking a commitment transaction, the revoker then also provides the
+next commitment point that's required to allow the other party to send them a
+new commitment state.
+
+[[apdx_update_fee_message]]
+===== The update_fee message
+
+((("channel operation","update_fee message")))((("update_fee message")))The `update_fee` is sent to update the fee on the current commitment
+transactions.
+
+ * Type: 134
+ * Fields:
+  ** `channel_id` : `channel_id`
+  ** `uint32` : `feerate_per_kw`
+
+This message can only be sent by the initiator of the channel; they're the ones
+that will pay for the commitment fee of the channel as along as it's open.
+
+[[apdx_update_fail_malformed_htlc_message]]
+===== The update_fail_malformed_htlc message
+
+((("channel operation","update_fail_malformed_htlc message")))((("update_fail_malformed_htlc message")))The `update_fail_malformed_htlc` message is sent to remove a corrupted HTLC.
+
+ * Type: 135
+ * Fields:
+  ** `channel_id` : `channel_id`
+  ** `uint64` : `id`
+  ** `sha256` : `sha256_of_onion`
+  ** `uint16` : `failure_code`
+
+This message is similar to the `update_fail_htlc` message, but it's rarely used in
+practice. As mentioned previously, each HTLC carries an onion encrypted routing
+packet that also covers the integrity of portions of the HTLC itself. If a
+party receives an onion packet that has somehow been corrupted along the way,
+then it won't be able to decrypt the packet. As a result, it also can't properly
+forward the HTLC; therefore, it'll send this message to signify that the HTLC
+has been corrupted somewhere along the route back to the sender.(((range="endofrange", startref="ix_appendix_protocol_messages-asciidoc4")))
+
+==== Channel Announcement
+
+((("channel_announcement message", id="ix_appendix_protocol_messages-asciidoc5", range="startofrange")))((("wire protocol messages","channel announcement", id="ix_appendix_protocol_messages-asciidoc6", range="startofrange")))Messages in this category are used to announce components of the channel graph
+authenticated data structure to the wider network. The channel graph has a
+series of unique properties due to the condition that all data added to the
+channel graph must also be anchored in the base Bitcoin blockchain. As a
+result, to add a new entry to the channel graph, an agent must be an
+on-chain transaction fee. This serves as a natural spam deterrent for the
+Lightning Network.
+
+
+[[apdx_channel_announcement_message]]
+===== The channel_announcement message
+
+The `channel_announcement` message is used to announce a new channel to the wider
+network.
+
+ * Type: 256
+ * Fields:
+  ** `sig` : `node_signature_1`
+  ** `sig` : `node_signature_2`
+  ** `sig` : `bitcoin_signature_1`
+  ** `sig` : `bitcoin_signature_2`
+  ** `uint16` : `len`
+  ** `len*byte` : `features`
+  ** `chain_hash` : `chain_hash`
+  ** `short_channel_id` : `short_channel_id`
+  ** `pubkey` : `node_id_1`
+  ** `pubkey` : `node_id_2`
+  ** `pubkey` : `bitcoin_key_1`
+  ** `pubkey` : `bitcoin_key_2`
+
+The series of signatures and public keys in the message serves to create a
+_proof_ that the channel actually exists within the base Bitcoin blockchain. As
+we detail in <<scid>>, each channel is uniquely identified by a locator
+that encodes its _location_ within the blockchain. This locator is called this
+`short_channel_id` and can fit into a 64-bit integer.
+
+[[apdx_node_announcement_message]]
+===== The node_announcement message
+
+((("channel_announcement message","node_announcement message")))((("node_announcement message")))The `node_announcement` message allows a node to announce/update its vertex within the
+greater channel graph.
+
+ * Type: 257
+ * Fields:
+  ** `sig` : `signature`
+  ** `uint64` : `flen`
+  ** `flen*byte` : `features`
+  ** `uint32` : `timestamp`
+  ** `pubkey` : `node_id`
+  ** `3*byte` : `rgb_color`
+  ** `32*byte` : `alias`
+  ** `uint16` : `addrlen`
+  ** `addrlen*byte` : `addresses`
+
+Note that if a node doesn't have any advertised channel within the channel
+graph, then this message is ignored to ensure that adding an item to
+the channel graph bears an on-chain cost. In this case, the on-chain cost will be
+the cost of creating the channel to which this node is connected.
+
+In addition to advertising its feature set, this message also allows a node to
+announce/update the set of network `addresses` where it can be reached.
+
+[[apdx_channel_update_message]]
+===== The channel_update message
+
+((("channel_announcement message","channel_update message")))((("channel_update message")))The `channel_update` message is sent to update the properties and policies of
+an active channel edge within the channel graph.
+
+ * Type: 258
+ * Fields:
+  ** `signature` : `signature`
+  ** `chain_hash` : `chain_hash`
+  ** `short_channel_id` : `short_channel_id`
+  ** `uint32` : `timestamp`
+  ** `byte` : `message_flags`
+  ** `byte` : `channel_flags`
+  ** `uint16` : `cltv_expiry_delta`
+  ** `uint64` : `htlc_minimum_msat`
+  ** `uint32` : `fee_base_msat`
+  ** `uint32` : `fee_proportional_millionths`
+  ** `uint16` : `htlc_maximum_msat`
+
+In addition to being able to enable/disable a channel, this message allows a
+node to update its routing fees as well as other fields that shape the type of
+payment that is permitted to flow through this channel.
+
+[[apdx_announce_signatures_message]]
+===== The announce_signatures message
+
+((("announce_signatures message")))((("channel_announcement message","announce_signatures message")))The `announce_signatures` message is exchanged by channel peers to
+assemble the set of signatures required to produce a `channel_announcement`
+message.
+
+ * Type: 259
+ * Fields:
+  ** `channel_id` : `channel_id`
+  ** `short_channel_id` : `short_channel_id`
+  ** `sig` : `node_signature`
+  ** `sig` : `bitcoin_signature`
+
+After the `funding_locked` message has been sent, if both sides wish to
+advertise their channel to the network, then they'll each send the
+`announce_signatures` message which allows both sides to emplace the four
+signatures required to generate an `announce_signatures` message.(((range="endofrange", startref="ix_appendix_protocol_messages-asciidoc6")))(((range="endofrange", startref="ix_appendix_protocol_messages-asciidoc5")))
+
+==== Channel Graph Syncing
+
+Nodes create a local perspective of the channel graph using five messages: +query_short_chan_ids+, +reply_short_chan_ids_end+, +query_channel_range+, +reply_channel_range+, and +gossip_timestamp_range+.
+
+[[apdx_query_short_chan_ids_message]]
+===== The query_short_chan_ids message
+
+((("channel graph syncing messages", id="ix_appendix_protocol_messages-asciidoc7", range="startofrange")))((("wire protocol messages","channel graph syncing", id="ix_appendix_protocol_messages-asciidoc8", range="startofrange")))The ((("channel graph syncing messages","query_short_chan_ids message")))((("query_short_chan_ids message")))`query_short_chan_ids` message allows a peer to obtain the channel information
+related to a series of short channel IDs.
+
+ * Type: 261
+ * Fields:
+  ** `chain_hash` : `chain_hash`
+  ** `u16` : `len`
+  ** `len*byte` : `encoded_short_ids`
+  ** `query_short_channel_ids_tlvs` : `tlvs`
+
+As we learn in <<gossip>>, these channel IDs may be a series of channels
+that were new to the sender or were out-of-date, which allows the sender to
+obtain the latest set of information for a set of channels.
+
+[[apdx_reply_short_chan_ids_end_message]]
+===== The reply_short_chan_ids_end message
+
+((("channel graph syncing messages","reply_short_chan_ids_end message")))((("reply_short_chan_ids_end message")))The `reply_short_chan_ids_end` message is sent after a peer finishes responding
+to a prior `query_short_chan_ids` message.
+
+ * Type: 262
+ * Fields:
+  ** `chain_hash` : `chain_hash`
+  ** `byte` : `full_information`
+
+This message signals to the receiving party that if they wish to send another
+query message, they can now do so.
+
+[[apdx_query_channel_range_message]]
+===== The query_channel_range message
+
+((("channel graph syncing messages","query_channel_range message")))((("query_channel_range message")))The `query_channel_range` message allows a node to query for the set of channels
+opened within a block range.
+
+ * Type: 263
+ * Fields:
+  ** `chain_hash` : `chain_hash`
+  ** `u32` : `first_blocknum`
+  ** `u32` : `number_of_blocks`
+  ** `query_channel_range_tlvs` : `tlvs`
+
+
+As channels are represented using a short channel ID that encodes the location
+of a channel in the chain, a node on the network can use a block height as a
+sort of _cursor_ to seek through the chain in order to discover a set of newly
+opened channels.
+
+[[apdx_reply_channel_range_message]]
+===== The reply_channel_range message
+
+((("channel graph syncing messages","reply_channel_range message")))((("reply_channel_range message")))The `reply_channel_range` message is the response to the `query_channel_range` message and
+includes the set of short channel IDs for known channels within that range.
+
+ * Type: 264
+ * Fields:
+  ** `chain_hash` : `chain_hash`
+  ** `u32` : `first_blocknum`
+  ** `u32` : `number_of_blocks`
+  ** `byte` : `sync_complete`
+  ** `u16` : `len`
+  ** `len*byte` : `encoded_short_ids`
+  ** `reply_channel_range_tlvs` : `tlvs`
+
+As a response to `query_channel_range`, this message sends back the set of
+channels that were opened within that range. This process can be repeated with
+the requester advancing their cursor further down the chain to
+continue syncing the channel graph.
+
+[[apdx_gossip_timestamp_range_message]]
+===== The gossip_timestamp_range message
+
+((("channel graph syncing messages","gossip_timestamp_range message")))((("gossip_timestamp_range message")))The `gossip_timestamp_range` message allows a peer to start receiving new
+incoming gossip messages on the network.
+
+ * Type: 265
+ * Fields:
+  ** `chain_hash` : `chain_hash`
+  ** `u32` : `first_timestamp`
+  ** `u32` : `timestamp_range`
+
+Once a peer has synced the channel graph, they can send this message if they
+wish to receive real-time updates on changes in the channel graph. They can
+also set the `first_timestamp` and `timestamp_range` fields if they wish to
+receive a backlog of updates they may have missed while they were(((range="endofrange", startref="ix_appendix_protocol_messages-asciidoc8")))(((range="endofrange", startref="ix_appendix_protocol_messages-asciidoc7"))) down(((range="endofrange", startref="ix_appendix_protocol_messages-asciidoc2"))).(((range="endofrange", startref="ix_appendix_protocol_messages-asciidoc0")))

atlas.json

@@ -1,13 +1,35 @@
 {
   "branch": "master",
   "files": [
+    "cover.html",
     "titlepage.html",
     "copyright.html",
     "toc.html",
     "preface.asciidoc",
-    "glossary.asciidoc",
+    "part_1_divider.asciidoc",
     "01_introduction.asciidoc",
     "02_getting_started.asciidoc",
+    "03_how_ln_works.asciidoc",
+    "04_node_client.asciidoc",
+    "05_node_operations.asciidoc",
+    "part_2_divider.asciidoc",
+    "06_lightning_architecture.asciidoc",
+    "07_payment_channels.asciidoc",
+    "08_routing_htlcs.asciidoc",
+    "09_channel_operation.asciidoc",
+    "10_onion_routing.asciidoc",
+    "11_gossip_channel_graph.asciidoc",
+    "12_path_finding.asciidoc",
+    "13_wire_protocol.asciidoc",
+    "14_encrypted_transport.asciidoc",
+    "15_payment_requests.asciidoc",
+    "16_security_privacy_ln.asciidoc",
+    "17_conclusion.asciidoc",
+    "appendix_bitcoin_fundamentals_review.asciidoc",
+    "appendix_docker_basics.asciidoc",
+    "appendix_protocol_messages.asciidoc",
+    "appendix_license_notices.asciidoc",
+    "glossary.asciidoc",
     "ix.html",
     "author_bio.html",
     "colo.html"
@@ -21,7 +43,7 @@
       "syntaxhighlighting": true,
       "show_comments": false,
       "trim_size": "7inx9.1875in",
-      "antennahouse_version": "AHFormatterV62_64-MR4"
+      "antennahouse_version": "AHFormatterV71_64-MR2"
     },
     "epub": {
       "index": true,
@@ -29,7 +51,7 @@
       "epubcheck": true,
       "syntaxhighlighting": true,
       "show_comments": false,
-      "downsample_images": false,
+      "downsample_images": true,
       "mathmlreplacement": false
     },
     "mobi": {

author_bio.html

@@ -1,4 +1,11 @@
-<section data-type="colophon" class="abouttheauthor">
-  <h1>About the Author(s)</h1>
-  <p></p>
+<section class="abouttheauthor" data-type="colophon" xmlns="http://www.w3.org/1999/xhtml">
+<h1>About the Authors</h1>
+
+<p><strong>Andreas M. Antonopoulos</strong> is a best-selling author, speaker, educator, and highly sought after expert in Bitcoin and open blockchain technologies. He is known for making complex subjects easy to understand and highlighting both the positive and negative impacts these technologies can have on our global societies.</p>
+
+<p>Andreas has written two more best-selling technical books for programmers with O’Reilly Media, <em>Mastering Bitcoin</em> and <em>Mastering Ethereum</em>. He has also published <em>The Internet of Money</em> series of books, which focus on the social, political, and economic importance and implications of these technologies. Andreas produces free educational content on his YouTube channel weekly and teaches virtual workshops on his website. Learn more at <a href='http://aantonop.com'><em>aantonop.com</em></a>.</p>
+
+<p><strong>Olaoluwa Osuntokun</strong> is the cofounder and CTO of Lightning Labs, and also the lead developer of lnd, one of the main implementations of Lightning. He received his BS and MS in computer science from UCSB and was a member of the Forbes 30 Under 30 class of 2019. During his graduate studies he focused on the field of applied cryptography, specifically encrypted search.  He has been an active Bitcoin developer for over five years, and is an author of several Bitcoin Improvement Proposals (BIP-157 and 158). These days, his primary focus lies in building, designing, and evolving private, scalable off-chain blockchain protocols, such as Lightning.</p>
+
+<p><strong>René Pickhardt</strong> is a trained mathematician and data science consultant who uses his statistical knowledge to do research with NTNU about pathfinding, privacy, reliability of payments, and service-level agreements of the Lightning Network. René maintains a technical and developer-oriented <a href='https://www.youtube.com/renepickhardt'>YouTube channel</a> about the Lightning Network and has answered roughly half of the questions about the Lightning Network on the Bitcoin Stack Exchange, making him the go-to point for almost all new developers who want to join the space. René has held numerous workshops about the Lightning Network in public and private, including teaching students of the 2019 Chaincode Labs residency together with other core Lightning developers.</p>
 </section>

channel-construction.asciidoc

@@ -1,508 +0,0 @@
-Payment channels are the core and most fundamental building block of the Lightning Network.
-Of course, every detail of a technology exists for a reason and is important.
-However the Lightning Network is literally built around the idea and concept of payment channels.
-In the previous chapters you have already learned that payment channels
-
-* allow two peers who created it to send and receive Bitcoin up to the amount specified by the capacity of the channel as often as they want to.
-* split the capacity of the channel into a balance between the two peers which - as long as the channel is open is only known by the owners of the channel and increases privacy.
-* do not require peers to do any additional onchain transactions other than the one needed to open and - potentially at a later state - to close the channel.
-* can stay open for an arbitrary time. Potentially in the best case forever.
-* do not require peers to trust each other as any attempt by a peer to cheat would enable the other peer to receive all the funds in the channel as a panality.
-* can be connected to a network and allow peers to send money a long a path of connected channels without the necessity to trust the intermediary nodes as they have no ability to steal the Bitcoin that are being forwarded.
-*
-
-In this chapter we will dig deeper into the protocol details that are needed to open and close payment channels.
-Working through this rather technical chapter you will be able to understand how the protocol design achieves the properties of payment channels.
-Where necessary some information from the first chapters of this book will be repeated.
-If you are new to the topic we highly encourage you to start there first.
-If you however already know a fair share about bitcoin script, OP_CODES and protocol design it might be sufficient to skip the previous chapter and start here with us.
-This books follows the construction of payment channels as described in BOLT 02 which is titled `peer protocol` and describes how two peers communicate to open, maintain and close a channel.
-There will be one big difference though.
-We will only discuss opening and closing a channel.
-The operation and maintainance of a channel which means either making or forwarding a payment is discussed in our chapter about routing.
-Also other constructions of payment channels are known and being discussed by the developers.
-Historically speaking these are the Duplex Micropayment channels introduced by Christian Decker during his time as a PhD student at ETH Zuric and the eltoo channels which where also introduced by Christian Decker.
-The eltoo channels are certainly a more elgant and cleaner why of achieving payment channels with the afore mentioned properties.
-However they require the activation of BIP 118 and a softfork and are - at the time of writing - a potential future protocol change.
-Thus this chapter will only focus on the pentalty based channels as described in the Lightning Network Whitepaper and specified in BOLT 02 which are currently supported by the protocol and the implementations.
-
-[NOTE]
-====
-The Lightning Network does not need consensus of features across it's participants.
-If the Bitcoin Softfork related to BIP 118 activates and people implement eltoo channels nodes that support eltoo can create payment channels and the onion routing of payments a long a path of channels would work just fine even if some of the channels are the modern eltoo channels or some channels are the legacy channels.
-Actually when Lightning network connections are established nodes signal feature bits of global and local features that they support.
-Thus havning the ability to create eltoo channels would just be an additional feature bit.
-In this sense upgrading the Lightnign Network is much easier than upgrading Bitcoin where consensus among all stakeholders is needed.
-====
-
-Let's quickly summarize what you should already know about payment channels on a technical level and for what you will learn the details in this chapter.
-A payment channel is encoded as an unspent 2-of-2 multisignature transaction output.
-The capacity of the channel relates to the amount that is bound to the unspent 2-of-2 multisignature transaction output.
-It is opened with the help of a funding transaction that sends bitcoin to a 2-of-2 multisignature output together with a communication protocol that helps to initialize and maintain its state.
-The balance of the channel encodes how the capacity is split between the two peers who maintain the channel.
-Technically the balance is encoded by a the most recent pair of a sequence of pairs of similar (but not equal) presigned commitment transactions.
-
-.Bitcoin, Lightning and "Ownership" of funds
-****
-When someone says they 'own' bitcoin they typically mean that they know the private key of a bitcoin address that has some unspent transaction outputs (UTXOs).
-The private key allows the owner to produce a signature for a transaction that spends the bitcoin by sending it to a different address.
-Thus 'ownership' of bitcoin can be defined as the ability to spend that bitcoin.
-
-If you have an unpublished but signed transaction from a 2-of-2 multisignature address, where some outputs are sent to an address you own, and additionally you own one of the private keys of the multisignature address, then you effectively own the bitcoin of that output.
-Without your help no other transaction from the 2-of-2 multisignature address can be produced.
-However, without the help of anybody else you can transfer the funds to an address which you control.
-On the Lightning Network ownership of your funds is almost always encoded with you having a pre-signed transaction spending from a 2-of-2 multisignature address.
-
-****
-
-These commitment transactions should never hit the blockchain and serve as a safty net for the participants in case the channel partner becomes unresponsive of disappears.
-They are also the reason why the Lightning Network is called an offchain scaling solution.
-Each channel partner has both signatures for one of the commitment transactions from the sequence of pairs.
-The split of the capacity is realized by a `to_local` and a `to_remote` output that is part of every commitment transaction.
-The `to_local` output goes to an address that is controlled by the peer that holds this fully signed commitment transaction.
-`to_local` outputs, which also exist in the second stage HTLC transactions - which we discuss in the routing chapter - have two spending conditions.
-The `to_local` output can be spent either at any time with the help of a revocation secrete or after a timelock with the secret key that is controled by the peer holding this commitment transaction.
-The revocation secrete is necessary to economically disincentivice peers to publish previous commitment transactions.
-Addresses and revokation secretes change with every new pair of commitment transactions that are being negotiated.
-The Lightning Network as a protocol defines the communication protocols that are necessary to achieve this.
-
-### Security of a Payment channel
-While the BOLTs introduce payment channels directly with the opening protocol we have decided to talk about the security model first.
-The security of payment channels come through a penalty based revocation system which help two parties to split the capacity of the payment channel into a balance sheet without the necessity to trust each other.
-In this chapter we start from an insecure approach of creating a payment channel and explain why it is insecure.
-We will then explain how time locks are being used to create revokable sequence maturity contracts that create the penality based revokation system which economically incentivizes people maintain the most recent state.
-After you understood these concepts we will quickly walk you through the technical details of opening and closing a channel.
-
-Any known payment channel constuction uses a 2-of-2 multisgnature output as the basis of the payment channel.
-We call the amount that is attached to this output the capacity of the channel.
-In every case, both channel partners hold 1 secret key of the multisignature address which means that they can only collaboratively control the funds.
-
-#### An example for a highly insecure payment channel construction
-Let us assume Alice does not know the details about the Lightning Network and naivly tries to open a payment channel in a way that will likely lead to the loss of her funds.
-Alice has heard that payment channel are 2-of-2 multisignature outputs.
-As she wants to have a channel with Bob and since she knows a public key from Bob she decides to open a channel by sending money to a 2-of-2 multisignature address that comes from Bob's and her key.
-We call the transaction that Alices used a **funding transaction** as it is supposed to fund the payment channel.
-However signing and broadcasting this funding transaction would be a huge mistake.
-As we have discussed the Bitcoins from the resulting UTXO can only be spent if Alice and Bob work together and both provide a signature for a transaction spending those coins.
-If Bob would not respond to Alice in future Alice would have lost her Bitcoins forever.
-That is because the coins would be stuck in the 2-of-2 multisignature address to which she has just sent them.
-
-Luckily Alice has previously read Mastering Bitcoin and she knows all the properties of Bitcoin script and is aware of the risks that are involved with sending Bitcoins to a 2-of-2 multisignature address to which she does not control both keys.
-She is also aware of the "Don't trust. Verify" principle that Bitcoiners follow and doesn't want to trust Bob to help her moving or accessing her coins.
-She would much more prefere to keep control over her coins even though they shall be stored in this 2-of-2 multisignature address.
-While this seems like an impossible problem, Alice has an idea:
-
-    What if she could already prepare a refund transaction (which we call commitment transaction in future) that sends all the bitcoin back to an address that she controls?
-
-Before broadcasting her funding transaction she already prepared and finishes it so that she knows the transaction id.
-She can now create the commitment transaction that spends the output of the funding transaction and ask Bob to provide a signature.
-At that time Bob has nothing to loose by signing the commitment transaction.
-He did not have Coins at the multisig address anyway.
-Even if he did Alice intends to spend from an output which Bob never was involved in.
-Thus at that point for Bob it is perfectly reasonable to sign the commitment transaction that spends the funding transaction.
-On the other side you as a reader might think:
-
-Why would Alice send money to a multisignature address just to prepare a transaction that sends the money back to her?
-
-We really hope you have wondered about this because this is really the point where the innovation begins.
-Just because in general people are expected to broadcast a transaction to the bitcoin network as soon as they have signed it noone forces you to do that.
-
-As Alice would loose access of her Bitcoins once she sends it to a 2-of-2 multisignature output for which she only controls one key, she needs to make sure that she will be able to regain access of her coins in case Bob becomes unresponisive.
-Thus before Alice publishes the funding transaction she will create another transaction that sends all the bitcoin from the 2-of-2 multisignature output back to an address which she controls.
-
-.The situation can be seen in the following picture
-image:channel-construction-opening-1.png[]
-Of course for the commitment transaction Alice would need to get a signature from Bob before she can safely broadcast the funding transaction.
-After publishing the funding transaction instead of braodcasting the commitment transaction she will keep it in a safe place.
-For this to work Alice needs to be sure that the funding transaction could not be published with a different transaction id.
-This malleability was possible before the Segwit upgrade of Bitcoin.
-We will discuss the details later but didn't want to leave them out here.
-
-
-[NOTE]
-====
-This entire process might be surprising (... comparison with HTTP server push and AJAX...)
-
-====
-Having Segwit and this first commitment transaction is actually secure for Alice.
-We have seen the first of three main properties that commit transactions fulfill:
-
-    Commitment Transactions refund channel participants in case the other side becomes irresponsive.
-
-The second purpose was implicitely defined by the first purpose:
-
-    Commitment Transactions split the capacity of the channel into a balance which is owned by each partner.
-
-Initially this split means that all the capacity is naturally on the side of the partner who funded the channel.
-Of course during the lifetime of the channel the balance could change.
-For example Alice might want to send some funds to Bob.
-This could happen because she wants to pay Bob or because she wants Bob to forward the funds through a path of channels to another merchant that she wants to pay.
-Let us assume as an example that Alice wants to send 30k Satoshi to Bob.
-For now we can assume that through some communication protocol Alice and Bob would negotiate a double spent of the funding transaction output of 100k satoshi.
-The new commitment transaction for which Alice and Bob would exchange signatures would send 70k satoshi to Alice and 30k Satoshi to Bob.
-The situation can be seen in the following picture
-image:channel-construction-opening-2.png[]
-Whenever Alice and Bob want to change the balance of the payment channel they will negotiate a new commitment transaction.
-Effectively they double spend the funding transaction output.
-But as the commitment transactions are not broadcasted - as long as the channel stays open - they will be able to do that.
-
-At this point we want to emphasize that the section was labeled in a way that suggests that this construction is insecure.
-So the main question reads:
-
-    What can go wrong with the insecure payment channel?
-
-The thing that goes and makes this construction insecure lies within the mechanics of Bitcoin.
-The key innovation of Bitcoin was to prevent the double spending problem of electronic coins.
-After Alice and Bob have exchanged signatures for the second commitment transaction Bob cannot rely on the fact that he really owns 30k satoshi.
-Of course he could close the channel by publishing the second commitment transaction assigning 30k satoshi to an address that he controls.
-But similarly Alice could broadcast the first commitment transaction and transfer the entire capacity of the channel back to an address that she controls.
-As Bitcoin prevents double spending of the funding transaction miners will include only one of the two commitment transactions.
-Thus we need to adapt the idea with the commitment transactions to create the ability to revoke an old commitment transaction.
-Regarding the fact that Bob and Alice both have a copy of the transaction and that Bob cannot control the data that Alice has stored on her hardware, it seems pretty hopeless.
-Luckily, the scripting language in Bitcoin allows at least for changing commitment transactions in a way that economically disincentivises channel partners from publish an outdated balances after they have negotated a new balance.
-
-#### Secure Payment channels via Revokable Commitment transactions
-
-
-[NOTE]
-====
-In summary we can conclude that commitment transactions fulfill three purposes:
-1. Refund channel participants in case the other side becomes irresponsive
-2. Split the capacity of the channel into the current balance that peers have agreed upon.
-3. Allow revocation of old state through the means of a penality via a revocable sequence maturity contract.
-====
-
-We have not yet explained how channel partners actually communicate to negotiate a new balance.
-Because it seems pretty amazing that we can make this swap revocation secret for signature atomic.
-In order to understand this we first need to understand the general communication of how a channel is opened.
-The actual negotiation of the new state is also done with HTLCs.
-That is why we only explain this in the routing chapter and ask you to stay patient.
-
-[NOTE]
-====
-*TODO: Move this note to routing chapter?*
-HTLCS fullfill the following purposes:
-1. Make a conditional payment.
-2. Help to update a new balance in a channel
-3. Make payments through a path of channel atomic, meaning that peers along the path cannot steal funds.
-====
-
-### Opening a payment channel
-We call the process of creating a new payment channel "opening a payment channel".
-Currently a payment channel can only exists between exactly two peers.
-Therefore you might be surprised to learn that even though two users are owning and maintaining the channel the current construction requires only one user to open the channel.
-This does not mean that only one peer is needed to open a channel.
-It does however mean that the user who opens the channel also has to provide the bitcoins to fund the channel.
-
-Let us stick to our example where Alice opens a channel with Bob with a capacity of 100k satoshi.
-This means that Alice provides 100k satoshi.
-Alice will do that by creating a so called funding transaction.
-This transaction sends 100k satoshi from an address that she - or her lightning node software controls - to a 2-of-2 multisig address for which she and Bob know 1 secret key each.
-The amount of Bitcoin that is sent to the multisig output by Alice is called the capacity of the payment channel.
-Thus for the reminder of the chapter in all examples we assume the payment channels that we use as examples already magically exist and the two peers Alice and Bob already have all the necessary data at hand.
-
-[NOTE]
-====
-Even though Alice and Bob both have a public node key to which they own the private secret opening a payment channel is not as easy as sending bitcoins to the 2 out of 2 multisignature output that belongs to the public keys of Alice and Bob.
-Let us assume for a moment that Alice would send 100k Satoshi to the Multisig address resulting from hers and Bob's public node id.
-In that case Alice will never be able to maintain her funds back without the help of Bob.
-Of course we want our payment channels to work in a way that Alice does not need to trust Bob.
-Bob could however refuse to sign a transaction that sends all those outputs back to an address that is controled by Alice.
-He would be able to blackmail Alice to assign a significant amount of those Bitcoin to an output address that is controled by him.
-Thus Bob can't steel the coins from Alice directly but he can threten Alice to have her coins lost forever.
-This example shows that unfortunatelly opening a channel will be a little bit more complex than just sending Bitcoins to a multisignature address.
-====
-
-
-
-[NOTE]
-====
-The importance of the segwit upgrade.
-
-
-====
-
-
-
-In order to avoid the reuse of addresses Alice and Bob will generate a new set of keys for the multisig address that they use to open the channel.
-Alice needs to inform Bob which key she intends to use for their channel and ask him which key he intends to use.
-She will do that by sending Bob and `open_channel` message signaling her interest to open a channel.
-This message contains a lot of additional data fields.
-Most of them specify meta data which is necessary for the channel operation and can be be safely ignored for now.
-We will only look at the following ones:
-
-* [chain_hash:chain_hash]
-* [32*byte:temporary_channel_id]
-* [u64:funding_satoshis]
-* [point:funding_pubkey]
-* [point:revocation_basepoint], [point:payment_basepoint], [point:delayed_payment_basepoint], [point:htlc_basepoint], [point:first_per_commitment_point]
-
-With the `chain_hash` Alice signals that she intends to open the channel on the Bitcoin blockchain.
-While the Lightning Network was certainly invented to scale the amount of payments that can be conducted on the Bitcoin Network it is interesting to note that the network is designed in a way that allows to build channels over various currencies.
-If a node has channels with more than one currency it is even possible to route payments through multi asset channels.
-However this turns out to be a little bit tricky in reality as the exchange rate between currencies might change which might lead the forwarding node to wait for a better exchange rate to settle or to abort the payment process.
-For the opening process the final channel id cannot be determined yet thus Alice needs to select a random channel id with Bob that she can use to identify the messages for this channel during the opening phase.
-This design descision allows multiple channels to exist between two nodes - though currently only LND supports this feature.
-Alice tells Bob for how many satoshis she wishes to open the channel.
-This information is necessary to construct the commitment transaction ...
-
-
-Once the channel is open Alice will be able to send 99k satoshi along this channel.
-Bob on the other side will be able to receive 99k satoshi along that channel.
-This means that initially Alice will not be able to recieve Bitcoins on this channel and that Bob initially will not be able to send Bitcoin along that channel.
-
-=== Forwarding payments with HTLCs
-In previous chapters we have seen that payment channels are maintained by two nodes by keeping two disjoint sequences of commitment transactions.
-The pair of latest commitment transactions in both sequences encodes the current, agreed upon balance in the channel.
-We have stated that two channel partners negotiate a new commitment transaction in order to change the balance and conduct a payment from one to another.
-We are finally at the point to explain the communications protocol via Lightning messages and the usage of HTLCs that is executed within a payment channel to change the balance.
-The same protocol will be executed along a path of channels if the network of channels is being utilized to make a payment between two participants without requiring them to have a dedicated payment channel connecting them directly.
-
-Let us start with the payment channel with a capacity of 100 mBTC between Alice and Bob.
-at its current state Alice and Bob have agreed that 20 mBTC belong to Bob and 80 mBTC belong to Alice.
-As Alice bought a coffee flatrate for the week she has to pay 15 mBTC to Bob and wants to use this channel.
-Just creating a new pair of commitment transactions and signing them is not so easy as the old ones have to be invalidated by sharing the revocation secret.
-This process should be executed in a way that it is atomic meaning the nodes will either be able to negotiate a new state without giving the other side the chance to play tricks or it should fail.
-
-[[routing-setup-htlc-0]]
-.Let us look at the initial pair of most recent commitment transactions for Alice and Bob:
-image:images/routing-setup-htlc-0.png[]
-
-Alice sends the `update_add_HTLC` Lightning message to Bob.
-The message type is 128 and has the following data fields:
-
-* [`channel_id`:`channel_id`]
-* [`u64`:`id`]
-* [`u64`:`amount_msat`]
-* [`sha256`:`payment_hash`]
-* [`u32`:`cltv_expiry`]
-* [`1366*byte`:`onion_routing_packet`]
-
-As Bob and Alice might have more than one channel thus the `channel_id` is included to the message.
-The `id` counter counts starts with 0 for the first HTLC that Alice offers to Bob and is increased by 1 with every subsequent offer.
-The id of the HTLC is used to compute the derivation path of the bitcoin key that is used for the output of this particular HTLC.
-In this way addresses changes with every payment and cannot be monitored by a third party.
-Next the amount that Alice wants to send to Bob is entered to the `amount_msat` field.
-As the name suggests the amount is depicted in millisatoshi even those cannot be enforced within the commitment transaction and within bitcoin.
-Still Lightning nodes keep track of subsatoshi amounts to avoid rounding issues.
-As in the offline example Alice includes the `payment_hash` in the next data field.
-This was told to Alice by Bob in case she wants to just send money to him.
-If Alice was to send Money to Gloria the Payment hash would have been given to Alice by Gloria.
-We discussed the potential of time lock or deadline of the contract.
-This is encoded in the `cltv_expiry`.
-cltv stands for OP_CHECKTIMELOCKVERIFY and is the OP_CODE that will be used in the HTLC output and serve as the deadline in which the contract is valid.
-Finally in the last data field there are 1336 Bytes of data included which is an `onion routing packet`.
-The format of this packet will be discussed in the last section of this chapter.
-For now it is important to note that it includes encrypted routing hints and information of the payment path that can only be partially decrypted by the recipient of the onion routing packet to extract information to whom to forward the payment or to learn that one as the final recipient.
-In any case the onion roting packet is always of the same size preventing the possibility to guess the position of an intermediary node within a path.
-In our particular case Bob will be able to decrypt the first couple bytes of the onion routing packet and learn that the payment is not to be forwarded but intended to be for him.
-
-The received information is enough for Bob to create a new commitment transaction.
-This commitment transaction now has not only 2 outputs encoding the balance between Alice and Bob but a third output which encodes the hashed time locked contract.
-
-[[routing-setup-htlc-1]]
-.Lets look at the newly created commitment transaction for Bob:
-image:images/routing-setup-htlc-1.png[]
-
-We can see that Bob Assumes that Alice will agree to lock 15 mBTC of her previous balance and assign it to the HTLC output.
-Creating this HTLC output can be compared to giving Alice golden coins to the escrow service.
-In our situation the bitcoin network can enforce the HTLC as Bob and Alice have agreed upon.
-Bob's Balance has not changed yet.
-In Bitcoin outputs are mainly described by scripts.
-The received HTLC in Bob's commitment transaction will use the following bitcoin script to define the output:
-
-
-   # To remote node with revocation key
-    OP_DUP OP_HASH160 <RIPEMD160(SHA256(revocationpubkey))> OP_EQUAL
-    OP_IF
-        OP_CHECKSIG
-    OP_ELSE
-        <remote_HTLCpubkey> OP_SWAP OP_SIZE 32 OP_EQUAL
-        OP_IF
-            # To local node via HTLC-success transaction.
-            OP_HASH160 <RIPEMD160(payment_hash)> OP_EQUALVERIFY
-            2 OP_SWAP <local_HTLCpubkey> 2 OP_CHECKMULTISIG
-        OP_ELSE
-            # To remote node after timeout.
-            OP_DROP <cltv_expiry> OP_CHECKLOCKTIMEVERIFY OP_DROP
-            OP_CHECKSIG
-        OP_ENDIF
-    OP_ENDIF
-
-We can see that there are basically three conditions to claim the output.
-
-1. Directly if a revocation key is known. This would happen if at a later state Bob fraudulently publishes this particular commitment transaction. As a newer state could only be agreed upon if Alice has learnt Bob's half of the revocation secret she could directly claim the funds and keep them even if Bob was later able to provide a proof of payment. This is mainly described in this line `OP_DUP OP_HASH160 <RIPEMD160(SHA256(revocationpubkey))> OP_EQUAL` and can be down by using `<revocation_sig> <revocationpubkey> as a witness script.
-2. If Bob has successfully delivered the payment and learnt the preimage he can spend the HTLC output with the help of the preimage and his `local_HTLC_secret`. This is to make sure that only Bob can spend this output if the commitment transaction hits the chain and not any other third party who might know the preimage because they had been included in the routing process. Claiming this output requires an HTLC-success transaction which we describe later.
-3. Finally Alice can use her `remote_HTLC_secret` to spend the HTLC output after the timeout of `cltv_expiry` was passed by using the following witness script `<remoteHTLCsig> 0`
-
-As the commitment transaction spends the 2 out of 2 multisig funding transaction Bob needs two signatures after he constructed this commitment transaction.
-He can obviously compute his own signature but he needs also the signature from Alice.
-As Alice initiated the payment and wanted the HTLC to be set up she will be reluctant to provide a signature.
-
-
-[[routing-setup-htlc-2]]
-.Alice sends the `commitment_signed` Lightning Message to Bob:
-image:images/routing-setup-htlc-2.png[]
-
-We can see in the diagram that Bob now has two valid commitment transactions.
-Let us have a quick look at the `commitment_signed` Lightning message which has the type 132.
-It has 4 data fields:
-
-* [`channel_id`:`channel_id`]
-* [`signature`:`signature`]
-* [`u16`:`num_HTLCs`]
-* [`num_HTLCs*signature`:`HTLC_signature`]
-
-First it again states which for which of the channels between Alice and Bob this message is intended.
-Then it has included a signature for the entire commitment transaction.
-As commitment transactions can have several HTLCs and HTLC success transactions need signatures which might not be provided at the time when they are needed those signatures are all already send over to Bob.
-If all signatures are valid Bob has a new commitment transaction.
-At this time he would be able to publish either the old one or the new one without getting a penality as the old one is not yet revoked and invalidated.
-However this is save for Alice as Bob has less money in this old state and is economically not incentivised to publish the old commitment transaction.
-Alice on the other side has no problem if Bob publishes the new commitment transaction as she wanted to send him money.
-If Bob can provide the preimage he is by their agreement and expectation entitled to claim the HTLC output.
-Should Bob decide to sabotage to future steps of the protocol Alice can either publish her commitment transaction without Bob being able to punish her.
-He will just not have received the funds from Alice.
-This is important!
-Despite the fact that Bob has a new commitment transaction with two valid signatures and an HTLC output inside he cannot seen his HTLC as being set up successfully.
-He first needs to have Alice invalidate her old state.
-That is why - in the case that he is not the final recipient of the funds - he should not forward the HTLC yet by setting up a new HTLC on the next channel with Wei.
-Alice will not invalidate her commitment transaction yet as she has to first get her new commitment transaction and she wants Bob to invalidate his old commitment transaction which he can safely do at this time.
-
-[[routing-setup-htlc-3]]
-.Bob sends a `revoke_and_ack` Lighting message to Alice:
-image:images/routing-setup-htlc-3.png[]
-
-The `revoke_and_ack` Lightning message contains three data fields.
-* [`channel_id`:`channel_id`]
-* [`32*byte`:`per_commitment_secret`]
-* [`point`:`next_per_commitment_point`]
-
-While it is really simple and straight forward it is very crucial.
-Bob shares the the `per_commitment_secret` of the old commitment transaction which serves as the revocation key and would allow Alice in future to penalize Bob if he publishes the old commitment transaction without the HTLC output.
-As in a future Alice and Bob might want to negotiate additional commitment transactions he already shares back the `next_per_commitment_point` that he will use in his next commitment transaction.
-
-Alice checks that the `per_commitment_secret` produces the last `per_commitment_point` and constructs her new commitment transaction with the HTLC output.
-Alice's version of the HTLC output is slightly different to the one that Bob had.
-The reason is the asymmetries of the penalty based payment channel construction protocol.
-Alice is offering in her commitment transaction an HTLC to the `remote` partner of the channel while Bob as accepting and offered HTLC to himself the `local` partner of the channel.
-Thus the Bitcoin script is adopted slightly.
-It is a very good exercise to go through both scripts and see where they differ.
-You could also try to use Bob's HTLC output script to come up with Alice's and vice versa and check your result with the following script.
-
-    # To remote node with revocation key
-    OP_DUP OP_HASH160 <RIPEMD160(SHA256(revocationpubkey))> OP_EQUAL
-    OP_IF
-        OP_CHECKSIG
-    OP_ELSE
-        <remote_HTLCpubkey> OP_SWAP OP_SIZE 32 OP_EQUAL
-        OP_NOTIF
-            # To local node via HTLC-timeout transaction (timelocked).
-            OP_DROP 2 OP_SWAP <local_HTLCpubkey> 2 OP_CHECKMULTISIG
-        OP_ELSE
-            # To remote node with preimage.
-            OP_HASH160 <RIPEMD160(payment_hash)> OP_EQUALVERIFY
-            OP_CHECKSIG
-        OP_ENDIF
-    OP_ENDIF
-
-Bob can redeem the HTLC with `<remoteHTLCsig> <payment_preimage>` as the witness script and in case the commitment transaction is revoked but published by Alice, Bob can trigger the penality by spending this output immediately with the following witness script `<revocation_sig> <revocationpubkey>`.
-
-[[routing-setup-htlc-4]]
-.Bob knows how Alice's commitment transaction will look like and sends over the necessary signatures.
-image:images/routing-setup-htlc-4.png[]
-
-This process is completely symmetrical to the one where Alice sent her signatures for Bob's new commitment transaction.
-Now Alice is the one having two valid commitment transactions.
-Technically she can still abort the payment by publishing her old commitment transaction to the bitcoin network.
-No one would lose anything as Bob knows that the contract is still being set up and not fully set up yet.
-This is a little bit different than how the situation would look like in a real world scenario.
-Recall Alice and Bob both have set up a new commitment transaction and have exchanged signatures.
-In the real world one would argue that this contract is now valid.
-
-[[routing-setup-htlc-5]]
-.However Bob knows that Alice has to invalidate her previous commitment transaction which she does
-image:images/routing-setup-htlc-5.png[]
-
-Now Bob and Alice both have a new commitment transaction with and additional HTLC output and we have achieved a major step towards updating a payment channel.
-The new Balance of Alice and Bob does not reflect yet that Alice has successfully send 15 mBTC to Bob.
-However the hashed time locked contracts are now set up in a way that secure settlement in exchange for the proof of payment will be possible.
-This yields another round of communication with Lightning messages and setting up additional commitment transactions which in case of good cooperation remove the outstanding HTLCs.
-Interestingly enough the `commitment_signed` and `revoke_and_ack` mechanism that we described to add an HTLC can be reused to update the commitment transaction.
-
-If Bob was the recipient of the 15 mBTC and knows the preimage to the payment hash Bob can settle the HTLCs by sending and `update_fulfill_htlc` Lightning message to Alice.
-This message has the type 130 and only 3 data fields:
-
-* [`channel_id`:`channel_id`]
-* [`u64`:`id`]
-* [`32*byte`:`payment_preimage`]
-
-As other messages Bob uses the `channel_id` field to indicates for which channel he returns the preimage.
-The htlc that is being removed is identified by the same `id` that was used to set up the HTLC in the commitment transaction initially.
-You might argue that Alice would not need to know the id of the HTLC for which Bob releases the preimage as the preimage and payment hash could be unique.
-However with this design the protocol supports that a payment channel has several htlcs with the same preimage but only settles one.
-One could argue that this does not make too much sense and it is good to be critical but this is how the protocol is designed and what it supports.
-Finally in the last field Bob provides the `payment_preimage` which Alice can check hashes to the payment hash.
-
-[WARNING]
-====
-When designing, implementing or studying a protocol one should ask: Is it safe to this or that in this moment of the protocol and can this be abused. We discussed for example the messages that where necessary for an HTLC to become valid. We pointed out that Bob should not see the received HTLC as valid even though he already has a new commitment transaction with signatures and invalidated his old commitment transaction before Alice also revoked her old commitment transaction. We also saw that no one is able to mess with the protocol of setting up a commitment transaction as in the worst case the protocol could be aborted and any dispute could be resolved by the Bitcoin Network. In the same way we should ask ourselves is it safe for Bob to just send out and release the preimage even though neither he nor alice have created the new pair of commitment transactions in which the HTLCs are removed. It is important to take a short break and ask yourself if Bob will in any case be able to claim the funds from the HTLC if the preimage is correct?
-====
-
-It is safe for Bob to tell Alice the preimage.
-Imagine Alice decides that she would not want to pay Bob anymore and does not respond anymore to create a new pair of commitment transactions with the removed HTLC and the Balance on Bob's end.
-In that case Bob could just force close the channel and publish his latest version of the commitment transaction.
-As the time lock of the HTLC is not over yet with an onchain success transaction Bob would be able to claim and settle his 15 mBTC as he is the only person who is able to spend the HTLC output in the commitment transaction.
-The other way around meaning Bob and Alice would negotiate a new commitment transaction with the removed HTLC would never be save for Alice.
-If the signatures for the new commitment transaction are exchanged Bob has received the money and could decide not to release the preimage.
-
-[NOTE]
-====
-Isn't it remarkable that even though the process of exchanging funds for an preimage seems to be happening concurrently at the same moment in time in reality it is actually happening one step after another but in the right order.
-====
-
-
-[NOTE]
-====
-The current construction could be generalized to multiparty channels and channel factories.
-However the communication protocol would suffer from increased complexity.
-====
-
-Chapter overview:
-  * describes how channels are put together at the script+transaction level
-  * details how a channel if funded in the protocol
-  ** including Key derrivation!
-  * details how a channel is updated in the protocol (moved to routing!)
-  * describes what needs to happen when a channel is force closed
-
-Relevant questions to answer:
-  * Channel construction:
-      * What's the difference between a replace-by-revocation based and a replace-by-versioning commitment format?
-      * What does the funding output script look like, what security guarantees does it give us?
-      * What's the difference between CSV and CLTV? How do both of these use the existing fields of the transaction to enforce new behavior?
-      * How do we implement revocation in our channel format?
-      * What does the script on the commitment to the broadcaster look like?
-      * What does the script on the commitment for the party that didn't broadcast look like?
-      * How are HTLCs constructed? What are second-level HTLCs?
-      * How has the commitment format in LN changed over time? What are some of the changes to the commitment format that've happened?
-  * Funding flow and messages:
-      * What are the messages exchanged to initiate a new channel with another peer?
-      * What do the parameters such as the max in flight do?
-      * How should the CSV values and the number of blocks until a channel is considered confirmed change with the size of the channel?
-      * What are wumbo channels? How are they enabled?
-      * What is an upfront shutdown address? What security does it offer?
-      * Is it possible to open multiple channels in a single transaction?
-  * Channel state machine:
-      * What does Medium Access Control mean in the context of network protocols?
-      * At a high level, how does the MAC protocol for 802.11 work?
-      * What steps need to happen for a new commitment state to be proposed and irrevocably committed for both parties?
-      * When is it safe for a party to forward a new HTLC to another peer? (may be out of scope for this chapter)
-      * Is it possible to commit a
-      * How does the current MAC protocol for the LN work?
-        * What does an htlc_add message contain?
-        * How are HTLCs cancelled or settled?
-        * Can both parties propose updates at the same time?
-        * Is it possible for a party to add a batch of HTLCs in a single go?
-        * What constraints exist that both parties need to adhere to?
-      * How are fees paid in a channel? Who pays which fees? Has this changed with newer commitment formats?
-      * How would the MAC protocol need to change if we used channels with symmetric state?

channel-graph.asciidoc

@@ -1,21 +0,0 @@
-Chapter overview:
-  * explains the channel graph, and how it's modified+verified
-
-Relevant questions to answer:
-  * Gossip announcements:
-      * How does a peer announce a new channel to the network?
-      * How do nodes verify a channel announcement? Why should they verify one in the first place?
-      * How does a node control _how_ a payment is routed through its channel?
-      * What knobs exist for a node to set in their channel updates?
-      * How often are channel updates sent?
-      * How does a node update its node in the channel graph? Do we we need to verify this?
-      * How quickly does an update propagate?
-      * What are "zombie" channels? Why do they matter?
-  * Channel graph syncing:
-      * What are the various ways a node can sync the channel graph?
-      * Which is the most efficient?
-      * What is the "gossip query" system?
-      * Does a node need to keep up with all gossip updates? Does this change if they're a routing node or mobile client?
-  * Protocol Extensions via Feature Bits and TLV:
-      * How can the channel graph be upgraded using feature bits and TLV fields?
-      * How does a receiver signal that they can accept MPP/AMP payments?

code/docker/Makefile

@@ -1,16 +1,126 @@
+#!make
+#
+# Makefile to help with building, pulling and pushing containers
+#
+# NOTE: You cannot push to the container registry unless you are authorized
+# in the lnbook organization (i.e. one of the authors or maintainers)
+#
+# Targets:
+#
+# make build # Build all containers
+# make pull # Pull all containers from the registry
+# make build-bitcoind # Build a specific container
+# make clean # remove all images and containers
+# make push # push updated images to Docker Hub (authors/maintainers only)
 
+
+# Latest tested versions of Bitcoin and Lightning clients
+
+# OS base image
+OS=ubuntu
+OS_VER=focal
+
+# bitcoind version
+BITCOIND_VER=0.21.0
+
+# LND version
+GO_VER=1.13
+LND_VER=v0.13.1-beta
+
+# c-lightning version
+CL_VER=0.10.1
+
+# Eclair version
+ECLAIR_VER=0.4.2
+ECLAIR_COMMIT=52444b0
+
+
+
+
+# Docker registry for lnbook
 REGISTRY=docker.com
-NAME=lnbook
+ORG=lnbook
+
+# List of containers
 CONTAINERS=bitcoind lnd eclair c-lightning
 
-all: build-all push-all
+.DEFAULT: pull
 
-build-all:
+
+
+
+
+build-bitcoind:
+	docker build \
+	--build-arg OS=${OS} \
+	--build-arg OS_VER=${OS_VER} \
+	--build-arg BITCOIND_VER=${BITCOIND_VER} \
+	-t ${ORG}/bitcoind:${BITCOIND_VER} \
+	bitcoind -f bitcoind/Dockerfile
+	docker image tag ${ORG}/bitcoind:${BITCOIND_VER} ${ORG}/bitcoind:latest
+
+
+build-cl: build-bitcoind
+	docker build \
+	--build-arg OS=${OS} \
+	--build-arg OS_VER=${OS_VER} \
+	--build-arg CL_VER=${CL_VER} \
+	-t ${ORG}/c-lightning:${CL_VER} \
+	c-lightning -f c-lightning/Dockerfile
+	docker image tag ${ORG}/c-lightning:${CL_VER} ${ORG}/c-lightning:latest
+
+
+build-lnd:
+	docker build \
+	--build-arg OS=${OS} \
+	--build-arg OS_VER=${OS_VER} \
+	--build-arg LND_VER=${LND_VER} \
+	--build-arg GO_VER=${GO_VER} \
+	-t ${ORG}/lnd:${LND_VER}_golang_${GO_VER} \
+	lnd -f lnd/Dockerfile
+	docker image tag ${ORG}/lnd:${LND_VER}_golang_${GO_VER} ${ORG}/lnd:latest
+
+
+build-eclair:
+	docker build \
+	--build-arg OS=${OS} \
+	--build-arg OS_VER=${OS_VER} \
+	--build-arg ECLAIR_VER=${ECLAIR_VER} \
+	--build-arg ECLAIR_COMMIT=${ECLAIR_COMMIT} \
+	-t ${ORG}/eclair:${ECLAIR_VER}-${ECLAIR_COMMIT} \
+	eclair -f eclair/Dockerfile
+	docker image tag ${ORG}/eclair:${ECLAIR_VER}-${ECLAIR_COMMIT} ${ORG}/eclair:latest
+
+
+push-bitcoind: build-bitcoind
+	docker push ${ORG}/bitcoind:${BITCOIND_VER}
+	docker push ${ORG}/bitcoind:latest
+
+push-lnd: build-lnd
+	docker push ${ORG}/lnd:${LND_VER}_golang_${GO_VER}
+	docker push ${ORG}/lnd:latest
+
+push-cl: build-cl
+	docker push ${ORG}/c-lightning:${CL_VER}
+	docker push ${ORG}/c-lightning:latest
+
+push-eclair: build-eclair
+	docker push ${ORG}/eclair:${ECLAIR_VER}-${ECLAIR_COMMIT}
+	docker push ${ORG}/eclair:latest
+
+build: build-bitcoind build-lnd build-cl build-eclair
+
+push: push-bitcoind push-lnd push-cl push-eclair
+
+pull:
 	for container in ${CONTAINERS}; do \
-		docker build -t ${NAME}/$$container $$container -f $$container/Dockerfile; \
+		docker pull ${ORG}/$$container:latest ;\
 	done
 
-push-all:
-	for container in ${CONTAINERS}; do \
-		docker push ${NAME}/$$container; \
-	done
+clean:
+	# Try 'make clean-confirm' if you are sure you want to do this.
+	# CAUTION: ALL docker containers and images on your computer will be removed.
+
+clean-confirm:
+	docker rm -f `docker ps -qa`
+	docker rmi -f `docker image ls -qa`

code/docker/bitcoind/Dockerfile

@@ -1,34 +1,47 @@
-FROM ubuntu:20.04 AS bitcoind-base
+ARG OS=ubuntu
+ARG OS_VER=focal
+FROM ${OS}:${OS_VER} as os-base
 
-RUN apt update && apt install -yqq \
-	curl gosu jq bash-completion
+# Install dependencies
+RUN DEBIAN_FRONTEND=noninteractive \
+    apt-get update -qq && apt-get install -yqq \
+		curl unzip jq bash-completion
 
-ENV BITCOIND_VERSION 0.20.0
-# Install binaries for Bitcoin Core
-ADD https://bitcoincore.org/bin/bitcoin-core-${BITCOIND_VERSION}/bitcoin-${BITCOIND_VERSION}-x86_64-linux-gnu.tar.gz /usr/local
-RUN cd /usr/local/ \
-	&& tar -zxf bitcoin-${BITCOIND_VERSION}-x86_64-linux-gnu.tar.gz \
-	&& cd bitcoin-${BITCOIND_VERSION} \
-	&& install bin/* /usr/local/bin \
-	&& install include/* /usr/local/include \
-	&& install -v lib/* /usr/local/lib
+FROM os-base as bitcoind-install
 
+ARG BITCOIND_VER=0.21.0
+# Install Bitcoin Core binaries and libraries
+RUN cd /tmp && \
+	curl -# -sLO https://bitcoincore.org/bin/bitcoin-core-${BITCOIND_VER}/bitcoin-${BITCOIND_VER}-x86_64-linux-gnu.tar.gz && \
+	tar -zxf bitcoin-${BITCOIND_VER}-x86_64-linux-gnu.tar.gz && \
+	cd bitcoin-${BITCOIND_VER} && \
+	install -vD bin/* /usr/bin && \
+	install -vD lib/* /usr/lib && \
+	cd /tmp && \
+	rm bitcoin-${BITCOIND_VER}-x86_64-linux-gnu.tar.gz && \
+	rm -rf bitcoin-${BITCOIND_VER}
+
+# Install runtime scripts, bash-completion and configuration files
+
+# bash completion for bitcoind and bitcoin-cli
 ENV GH_URL https://raw.githubusercontent.com/bitcoin/bitcoin/master/
 ENV BC /usr/share/bash-completion/completions/
 ADD $GH_URL/contrib/bitcoin-cli.bash-completion $BC/bitcoin-cli
 ADD $GH_URL/contrib/bitcoind.bash-completion $BC/bitcoind
 ADD $GH_URL/contrib/bitcoin-tx.bash-completion $BC/bitcoin-tx
 
-FROM bitcoind-base AS bitcoind
-
-ADD bitcoind /bitcoind
+# Copy bitcoind configuration directory
+COPY bitcoind /bitcoind
 RUN ln -s /bitcoind /root/.
 
-ADD bashrc /root/.bashrc
-ADD bitcoind-entrypoint.sh /usr/local/bin
+# Copy support scripts
+COPY bashrc /root/.bashrc
+COPY bitcoind-entrypoint.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/bitcoind-entrypoint.sh
-ADD mine.sh /usr/local/bin
+COPY mine.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/mine.sh
+COPY cli /usr/local/bin
+RUN chmod +x /usr/local/bin/cli
 
 # bitcoind P2P
 EXPOSE 18444/tcp

code/docker/bitcoind/bitcoind-entrypoint.sh

@@ -1,22 +1,36 @@
 #!/bin/bash
 set -Eeuo pipefail
 
-echo Starting bitcoind...
+
+# Start bitcoind
+echo "Starting bitcoind..."
 bitcoind -datadir=/bitcoind -daemon
-until bitcoin-cli -datadir=/bitcoind getblockchaininfo  > /dev/null 2>&1
+
+# Wait for bitcoind startup
+echo -n "Waiting for bitcoind to start"
+until bitcoin-cli -datadir=/bitcoind -rpcwait getblockchaininfo  > /dev/null 2>&1
 do
+	echo -n "."
 	sleep 1
 done
-echo bitcoind started
+echo
+echo "bitcoind started"
+
+
+# Load private key into wallet
 export address=`cat /bitcoind/keys/demo_address.txt`
 export privkey=`cat /bitcoind/keys/demo_privkey.txt`
+
+# If restarting the wallet already exists, so don't fail if it does,
+# just load the existing wallet:
+bitcoin-cli -datadir=/bitcoind createwallet regtest > /dev/null || bitcoin-cli -datadir=/bitcoind loadwallet regtest > /dev/null
+bitcoin-cli -datadir=/bitcoind importprivkey $privkey > /dev/null || true
+
 echo "================================================"
-echo "Importing demo private key"
+echo "Imported demo private key"
 echo "Bitcoin address: " ${address}
 echo "Private key: " ${privkey}
 echo "================================================"
-bitcoin-cli -datadir=/bitcoind importprivkey $privkey
 
 # Executing CMD
-echo "$@"
 exec "$@"

code/docker/bitcoind/cli

@@ -0,0 +1,5 @@
+#!/bin/bash
+#
+# Helper script used as an alias for bitcoin-cli with the necessary arguments
+#
+/usr/bin/bitcoin-cli -datadir=/bitcoind -regtest $@

code/docker/bitcoind/mine.sh

@@ -2,18 +2,17 @@
 set -Eeuo pipefail
 
 export address=`cat /bitcoind/keys/demo_address.txt`
-export privkey=`cat /bitcoind/keys/demo_privkey.txt`
 echo "================================================"
-echo "Bitcoin address: " ${address}
-echo "Private key: " ${privkey}
 echo "Balance:" `bitcoin-cli -datadir=/bitcoind getbalance`
 echo "================================================"
 echo "Mining 101 blocks to unlock some bitcoin"
 bitcoin-cli -datadir=/bitcoind generatetoaddress 101 $address
-echo "Mining 1 block every 10 seconds"
-while sleep 10; do \
-	bitcoin-cli -datadir=/bitcoind generatetoaddress 1 $address; \
-	echo "Balance:" `bitcoin-cli -datadir=/bitcoind getbalance`; \
+echo "Mining 6 blocks every 10 seconds"
+while echo "Balance:" `bitcoin-cli -datadir=/bitcoind getbalance`;
+do
+	bitcoin-cli -datadir=/bitcoind generatetoaddress 6 $address; \
+	sleep 10; \
+
 done
 
 # If loop is interrupted, stop bitcoind

code/docker/c-lightning/Dockerfile

@@ -1,31 +1,60 @@
-FROM lnbook/bitcoind AS c-lightning-base
+ARG OS=ubuntu
+ARG OS_VER=focal
+FROM ${OS}:${OS_VER} as os-base
 
-# Install software-properties-common to add apt repositories
-RUN apt update && apt install -yqq \
-	software-properties-common
+# Install dependencies
+RUN DEBIAN_FRONTEND=noninteractive \
+    apt-get update -qq && apt-get install -yqq \
+		curl unzip jq bash-completion
 
-# c-lightning
-RUN add-apt-repository -u ppa:lightningnetwork/ppa
-RUN apt-get install -yqq \
- 	lightningd
+FROM os-base as cl-install
+COPY --from=lnbook/bitcoind:latest /usr/bin/bitcoin-cli /usr/bin
+
+# Set CL_VER ENV from ARG
+ARG CL_VER=0.10.1
+ENV CL_VER=${CL_VER}
+
+RUN apt-get update -qq && apt-get install -yqq \
+		gpg xz-utils libpq5 libsodium23 && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+RUN cd /tmp && \
+    curl -# -sLO https://github.com/ElementsProject/lightning/releases/download/v${CL_VER}/clightning-v${CL_VER}-Ubuntu-20.04.tar.xz
+
+# Verify developer signatures. The `gpg --verify` command will print a
+# couple of warnings about the key not being trusted. That's ok. The
+# important part is that it doesn't error and reports "Good
+# signature".
+ADD devkeys.pem /tmp/devkeys.pem
+RUN gpg --import /tmp/devkeys.pem
+ADD https://github.com/ElementsProject/lightning/releases/download/v0.10.1/SHA256SUMS /tmp/SHA256SUMS
+ADD https://github.com/ElementsProject/lightning/releases/download/v0.10.1/SHA256SUMS.asc /tmp/SHA256SUMS.asc
+RUN cd /tmp && \
+    gpg -q --verify SHA256SUMS.asc SHA256SUMS && \
+    cat SHA256SUMS && \
+    sha256sum --ignore-missing -c SHA256SUMS
+
+RUN tar -xvf /tmp/clightning-v${CL_VER}-Ubuntu-20.04.tar.xz -C /
 
 ADD https://raw.githubusercontent.com/ElementsProject/lightning/master/contrib/lightning-cli.bash-completion /usr/share/bash-completion/completions/lightning-cli
 
-FROM c-lightning-base AS c-lightning-run
-
-ADD lightningd /lightningd
+COPY lightningd /lightningd
 WORKDIR /lightningd
 RUN ln -s /lightningd /root/.lightning
 
-ADD bashrc /root/.bashrc
-ADD c-lightning-entrypoint.sh /usr/local/bin
+COPY bashrc /root/.bashrc
+COPY c-lightning-entrypoint.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/c-lightning-entrypoint.sh
-ADD fund-c-lightning.sh /usr/local/bin
+COPY fund-c-lightning.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/fund-c-lightning.sh
-ADD logtail.sh /usr/local/bin
+COPY logtail.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/logtail.sh
-ADD wait-for-bitcoind.sh /usr/local/bin
+COPY wait-for-bitcoind.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/wait-for-bitcoind.sh
+COPY cli /usr/local/bin
+RUN chmod +x /usr/local/bin/cli
+
 
 EXPOSE 9735 9835
 ENTRYPOINT ["/usr/local/bin/c-lightning-entrypoint.sh"]

code/docker/c-lightning/bashrc

@@ -1,4 +1,4 @@
-alias lightning-cli="lightning-cli --network regtest --lightning-dir=/lightningd"
+alias lightning-cli="lightning-cli --lightning-dir=/lightningd"
 
 [[ $PS1 && -f /usr/share/bash-completion/bash_completion ]] && \
     . /usr/share/bash-completion/bash_completion

code/docker/c-lightning/c-lightning-entrypoint.sh

@@ -6,7 +6,7 @@ source /usr/local/bin/wait-for-bitcoind.sh
 echo Starting c-lightning...
 lightningd --lightning-dir=/lightningd --daemon
 
-until lightning-cli --lightning-dir=/lightningd --network regtest getinfo > /dev/null 2>&1
+until lightning-cli --lightning-dir=/lightningd getinfo > /dev/null 2>&1
 do
 	sleep 1
 done
@@ -15,5 +15,4 @@ sleep 2
 echo "Funding c-lightning wallet"
 source /usr/local/bin/fund-c-lightning.sh
 
-echo "$@"
 exec "$@"

code/docker/c-lightning/cli

@@ -0,0 +1,5 @@
+#!/bin/bash
+#
+# Helper script used as an alias for lightning-cli with the necessary arguments
+#
+/usr/bin/lightning-cli --lightning-dir=/lightningd $@

code/docker/c-lightning/fund-c-lightning.sh

@@ -1,22 +1,17 @@
 #!/bin/bash
 set -Eeuo pipefail
 
-# Generate a new receiving address for LND wallet
-address=$(lightning-cli --lightning-dir=/lightningd --network regtest newaddr | jq .address)
+# Generate a new receiving address for c-lightning wallet
+address=$(lightning-cli --lightning-dir=/lightningd --network regtest newaddr | jq '.bech32' -r)
 
 # Ask Bitcoin Core to send 10 BTC to the address, using JSON-RPC call
-curl --user regtest:regtest \
-     -H 'content-type: text/plain;' \
-	 http://bitcoind:18443/ \
-	 --data-binary @- <<EOF
-	{
-	  "jsonrpc": "1.0",
-	  "id": "c-lightning-container",
-	  "method": "sendtoaddress",
-	  "params": [
-	    ${address},
-	    10,
-	    "funding c-lightning"
-	  ]
-	}
-EOF
+until bitcoin-cli \
+    --rpcuser=regtest \
+    --rpcpassword=regtest \
+    --rpcconnect=bitcoind:18443 \
+    --regtest \
+    sendtoaddress ${address} 10 "funding c-lightning"
+do
+	sleep 1;
+	echo Retrying funding...
+done

code/docker/c-lightning/logtail.sh

@@ -2,4 +2,5 @@
 set -Eeuo pipefail
 
 # Show LND log from beginning and follow
+touch /lightningd/lightningd.log
 tail -n +1 -f /lightningd/lightningd.log || true

code/docker/c-lightning/wait-for-bitcoind.sh

@@ -2,14 +2,14 @@
 set -Eeuo pipefail
 
 echo Waiting for bitcoind to start...
-until bitcoin-cli -rpcconnect=bitcoind -rpcport=18443 -rpcuser=regtest -rpcpassword=regtest getblockchaininfo  > /dev/null 2>&1
+until curl --silent --user regtest:regtest --data-binary '{"jsonrpc": "1.0", "id": "cl-node", "method": "getblockchaininfo", "params": []}' -H 'content-type: text/plain;' http://bitcoind:18443/ | jq -e ".result.blocks > 0" > /dev/null 2>&1
 do
 	echo -n "."
 	sleep 1
 done
 
 echo Waiting for bitcoind to mine blocks...
-until bitcoin-cli -rpcconnect=bitcoind -rpcport=18443 -rpcuser=regtest -rpcpassword=regtest getbalance  | jq -e ". > 0" > /dev/null 2>&1
+until curl --silent --user regtest:regtest --data-binary '{"jsonrpc": "1.0", "id": "cl-node", "method": "getbalance", "params": ["*", 6]}' -H 'content-type: text/plain;' http://bitcoind:18443/ | jq -e ".result > 0" > /dev/null 2>&1
 do
 	echo -n "."
 	sleep 1

code/docker/check-versions.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+# a small script to help sanity check the versions of the different node implementations
+dockerfiles=$(find . -name 'Dockerfile')
+# print location of dockerfiles
+echo $dockerfiles
+# print variables
+awk '/ARG/ && /VER|COMMIT/' $dockerfiles

code/docker/docker-compose.yml

@@ -14,6 +14,7 @@ services:
       - "18443"
       - "12005"
       - "12006"
+    restart: always
 
   Alice:
     container_name: Alice
@@ -24,6 +25,7 @@ services:
       - lnnet
     expose:
       - "9735"
+    restart: always
 
   Bob:
     container_name: Bob
@@ -34,9 +36,10 @@ services:
       - lnnet
     expose:
       - "9735"
+    restart: always
 
-  Wei:
-    container_name: Wei
+  Chan:
+    container_name: Chan
     build:
         context: eclair
     image: lnbook/eclair:latest
@@ -44,9 +47,10 @@ services:
       - lnnet
     expose:
       - "9735"
+    restart: always
 
-  Gloria:
-    container_name: Gloria
+  Dina:
+    container_name: Dina
     build:
         context: lnd
     image: lnbook/lnd:latest
@@ -54,3 +58,4 @@ services:
       - lnnet
     expose:
       - "9735"
+    restart: always

code/docker/eclair/Dockerfile

@@ -1,20 +1,29 @@
-FROM ubuntu:20.04 AS eclair-base
+ARG OS=ubuntu
+ARG OS_VER=focal
+FROM ${OS}:${OS_VER} as os-base
 
-RUN apt update && apt install -yqq \
-	curl gosu jq bash-completion
+# Install dependencies
+RUN DEBIAN_FRONTEND=noninteractive \
+    apt-get update -qq && apt-get install -yqq \
+		curl unzip jq bash-completion
 
-RUN apt update && apt install -yqq \
-	openjdk-11-jdk unzip
-
-COPY --from=lnbook/bitcoind /usr/local/ /usr/local/
+# Install default Java Runtime Environment
+RUN DEBIAN_FRONTEND=noninteractive \
+    apt-get update -qq && apt-get install -yqq \
+	  default-jre-headless && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
 
 # Install eclair
-ENV ECLAIR_VER 0.4
-ENV ECLAIR_COMMIT 69c538e
 WORKDIR /usr/src
-RUN curl -sLO https://github.com/ACINQ/eclair/releases/download/v${ECLAIR_VER}/eclair-node-${ECLAIR_VER}-${ECLAIR_COMMIT}-bin.zip \
-	&& unzip eclair-node-${ECLAIR_VER}-${ECLAIR_COMMIT}-bin.zip \
-	&& install eclair-node-${ECLAIR_VER}-${ECLAIR_COMMIT}/bin/eclair-cli /usr/local/bin
+
+ARG ECLAIR_VER=0.4.2
+ARG ECLAIR_COMMIT=52444b0
+RUN cd /usr/src && \
+	curl -# -sLO https://github.com/ACINQ/eclair/releases/download/v${ECLAIR_VER}/eclair-node-${ECLAIR_VER}-${ECLAIR_COMMIT}-bin.zip && \
+	unzip eclair-node-${ECLAIR_VER}-${ECLAIR_COMMIT}-bin.zip && \
+	install eclair-node-${ECLAIR_VER}-${ECLAIR_COMMIT}/bin/eclair-cli /usr/local/bin && \
+	rm eclair-node-${ECLAIR_VER}-${ECLAIR_COMMIT}-bin.zip
 
 ADD https://raw.githubusercontent.com/ACINQ/eclair/master/contrib/eclair-cli.bash-completion /usr/share/bash-completion/completions/eclair-cli
 
@@ -29,6 +38,11 @@ ADD logtail.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/logtail.sh
 ADD wait-for-bitcoind.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/wait-for-bitcoind.sh
+COPY cli /usr/local/bin
+RUN chmod +x /usr/local/bin/cli
+
+ENV ECLAIR_VER=$ECLAIR_VER
+ENV ECLAIR_COMMIT=$ECLAIR_COMMIT
 
 EXPOSE 9735
 ENTRYPOINT ["/usr/local/bin/eclair-entrypoint.sh"]

code/docker/eclair/cli

@@ -0,0 +1,5 @@
+#!/bin/bash
+#
+# Helper script used as an alias for eclair-cli with the necessary arguments
+#
+/usr/local/bin/eclair-cli -s -j -p eclair $@

code/docker/eclair/eclair-entrypoint.sh

@@ -18,5 +18,4 @@ echo Eclair node started
 sleep 2
 
 # Executing CMD
-echo "$@"
 exec "$@"

code/docker/eclair/eclair/eclair.conf

@@ -24,5 +24,17 @@ eclair {
     zmqtx = "tcp://bitcoind:12006"
   }
 
-  on-chain-fees.max-feerate-mismatch = 100
+  on-chain-fees {
+    feerate-tolerance {
+      ratio-low = 0.000001
+      ratio-high = 1000000
+    }
+  }
+
+  node-alias = "eclair"
+
+  router {
+    channel-exclude-duration = 1 seconds
+    broadcast-interval = 1 seconds
+  }
 }

code/docker/eclair/logtail.sh

@@ -2,4 +2,5 @@
 set -Eeuo pipefail
 
 # Show LND log from beginning and follow
+touch /eclair/eclair.log
 tail -n +1 -f /eclair/eclair.log || true

code/docker/eclair/wait-for-bitcoind.sh

@@ -2,14 +2,14 @@
 set -Eeuo pipefail
 
 echo Waiting for bitcoind to start...
-until bitcoin-cli -rpcconnect=bitcoind -rpcport=18443 -rpcuser=regtest -rpcpassword=regtest getblockchaininfo  > /dev/null 2>&1
+until curl --silent --user regtest:regtest --data-binary '{"jsonrpc": "1.0", "id": "eclair-node", "method": "getblockchaininfo", "params": []}' -H 'content-type: text/plain;' http://bitcoind:18443/ | jq -e ".result.blocks > 0" > /dev/null 2>&1
 do
 	echo -n "."
 	sleep 1
 done
 
 echo Waiting for bitcoind to mine blocks...
-until bitcoin-cli -rpcconnect=bitcoind -rpcport=18443 -rpcuser=regtest -rpcpassword=regtest getbalance  | jq -e ". > 0" > /dev/null 2>&1
+until curl --silent --user regtest:regtest --data-binary '{"jsonrpc": "1.0", "id": "eclair-node", "method": "getbalance", "params": ["*", 6]}' -H 'content-type: text/plain;' http://bitcoind:18443/ | jq -e ".result > 0" > /dev/null 2>&1
 do
 	echo -n "."
 	sleep 1

code/docker/lnd/Dockerfile

@@ -1,38 +1,59 @@
-FROM golang:1.13 as lnd-base
+ARG OS=ubuntu
+ARG OS_VER=focal
+ARG GO_VER=1.13
+# Define base images with ARG versions
+FROM ${OS}:${OS_VER} as os
+FROM golang:${GO_VER} as go
 
-ENV GOPATH /go
-WORKDIR $GOPATH/src
+# OS image with command-line utilities
+FROM os AS os-base
 
-# LND
-RUN go get -d github.com/lightningnetwork/lnd
-WORKDIR $GOPATH/src/github.com/lightningnetwork/lnd
-RUN make && make install
+# Install dependencies
+RUN DEBIAN_FRONTEND=noninteractive \
+    apt-get update -qq && apt-get install -yqq \
+		curl unzip jq bash-completion
 
-FROM ubuntu:20.04 AS lnd-run
+# Go image for building LND
+FROM go as lnd-build
 
-RUN apt update && apt install -yqq \
-	curl gosu jq bash-completion
+ENV GO_VER=${GO_VER}
+ENV GOPATH=/go
 
-COPY --from=lnd-base /go /go
-COPY --from=lnbook/bitcoind /usr/local/ /usr/local/
+# Build LND
+ARG LND_VER=v0.13.1-beta
+ENV LND_VER=${LND_VER}
+RUN	mkdir -p ${GOPATH}/src && \
+	cd ${GOPATH}/src && \
+	go get -v -d github.com/lightningnetwork/lnd && \
+	cd ${GOPATH}/src/github.com/lightningnetwork/lnd && \
+	git checkout tags/${LND_VER} && \
+	make clean && make && make install
 
-RUN cp /go/src/github.com/lightningnetwork/lnd/contrib/lncli.bash-completion \
+# Runtime image for running LND
+FROM os-base as lnd-run
+
+# Copy only the executables
+COPY --from=lnd-build /go/bin /go/bin
+
+ADD https://raw.githubusercontent.com/lightningnetwork/lnd/master/contrib/lncli.bash-completion \
 	   /usr/share/bash-completion/completions/lncli
 
 ENV GOPATH /go
 ENV PATH $PATH:$GOPATH/bin
 
-ADD lnd /lnd
+COPY lnd /lnd
 RUN ln -s /lnd /root/.lnd
-ADD fund-lnd.sh /usr/local/bin
+COPY fund-lnd.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/fund-lnd.sh
-ADD bashrc /root/.bashrc
-ADD lnd-entrypoint.sh /usr/local/bin
+COPY bashrc /root/.bashrc
+COPY lnd-entrypoint.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/lnd-entrypoint.sh
-ADD logtail.sh /usr/local/bin
+COPY logtail.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/logtail.sh
-ADD wait-for-bitcoind.sh /usr/local/bin
+COPY wait-for-bitcoind.sh /usr/local/bin
 RUN chmod +x /usr/local/bin/wait-for-bitcoind.sh
+COPY cli /usr/local/bin
+RUN chmod +x /usr/local/bin/cli
 
 # LND RPC
 EXPOSE 10009/tcp

code/docker/lnd/cli

@@ -0,0 +1,5 @@
+#!/bin/bash
+#
+# Helper script used as an alias for lncli with the necessary arguments
+#
+/go/bin/lncli --lnddir=/lnd -n regtest $@

code/docker/lnd/lnd-entrypoint.sh

@@ -14,5 +14,4 @@ echo "Startup complete"
 echo "Funding lnd wallet"
 source /usr/local/bin/fund-lnd.sh
 
-echo "$@"
 exec "$@"

code/docker/lnd/wait-for-bitcoind.sh

@@ -2,14 +2,14 @@
 set -Eeuo pipefail
 
 echo Waiting for bitcoind to start...
-until bitcoin-cli -rpcconnect=bitcoind -rpcport=18443 -rpcuser=regtest -rpcpassword=regtest getblockchaininfo  > /dev/null 2>&1
+until curl --silent --user regtest:regtest --data-binary '{"jsonrpc": "1.0", "id": "lnd-node", "method": "getblockchaininfo", "params": []}' -H 'content-type: text/plain;' http://bitcoind:18443/ | jq -e ".result.blocks > 0" > /dev/null 2>&1
 do
 	echo -n "."
 	sleep 1
 done
 
 echo Waiting for bitcoind to mine blocks...
-until bitcoin-cli -rpcconnect=bitcoind -rpcport=18443 -rpcuser=regtest -rpcpassword=regtest getbalance  | jq -e ". > 0" > /dev/null 2>&1
+until curl --silent --user regtest:regtest --data-binary '{"jsonrpc": "1.0", "id": "lnd-node", "method": "getbalance", "params": ["*", 6]}' -H 'content-type: text/plain;' http://bitcoind:18443/ | jq -e ".result > 0" > /dev/null 2>&1
 do
 	echo -n "."
 	sleep 1

code/docker/run-payment-demo.sh

@@ -0,0 +1,177 @@
+#!/bin/bash
+
+#
+# Helper functions
+#
+
+
+# run-in-node: Run a command inside a docker container, using the bash shell
+function run-in-node () {
+	docker exec "$1" /bin/bash -c "${@:2}"
+}
+
+# wait-for-cmd: Run a command repeatedly until it completes/exits successfuly
+function wait-for-cmd () {
+		until "${@}" > /dev/null 2>&1
+		do
+			echo -n "."
+			sleep 1
+		done
+		echo
+}
+
+# wait-for-node: Run a command repeatedly until it completes successfully, inside a container
+# Combining wait-for-cmd and run-in-node
+function wait-for-node () {
+	wait-for-cmd run-in-node $1 "${@:2}"
+}
+
+
+# Start the demo
+echo "Starting Payment Demo"
+
+echo "======================================================"
+echo
+echo "Waiting for nodes to startup"
+echo -n "- Waiting for bitcoind startup..."
+wait-for-node bitcoind "cli getblockchaininfo | jq -e \".blocks > 101\""
+echo -n "- Waiting for bitcoind mining..."
+wait-for-node bitcoind "cli getbalance | jq -e \". > 50\""
+echo -n "- Waiting for Alice startup..."
+wait-for-node Alice "cli getinfo"
+echo -n "- Waiting for Bob startup..."
+wait-for-node Bob "cli getinfo"
+echo -n "- Waiting for Chan startup..."
+wait-for-node Chan "cli getinfo"
+echo -n "- Waiting for Dina startup..."
+wait-for-node Dina "cli getinfo"
+echo "All nodes have started"
+
+echo "======================================================"
+echo
+echo "Getting node IDs"
+alice_address=$(run-in-node Alice "cli getinfo | jq -r .identity_pubkey")
+bob_address=$(run-in-node Bob "cli getinfo | jq -r .id")
+chan_address=$(run-in-node Chan "cli getinfo| jq -r .nodeId")
+dina_address=$(run-in-node Dina "cli getinfo | jq -r .identity_pubkey")
+
+# Show node IDs
+echo "- Alice:  ${alice_address}"
+echo "- Bob:    ${bob_address}"
+echo "- Chan:   ${chan_address}"
+echo "- Dina:	${dina_address}"
+
+echo "======================================================"
+echo
+echo "Waiting for Lightning nodes to sync the blockchain"
+echo -n "- Waiting for Alice chain sync..."
+wait-for-node Alice "cli getinfo | jq -e \".synced_to_chain == true\""
+echo -n "- Waiting for Bob chain sync..."
+wait-for-node Bob "cli getinfo | jq -e \".blockheight > 100\""
+echo -n "- Waiting for Chan chain sync..."
+wait-for-node Chan "cli getinfo | jq -e \".blockHeight > 100\""
+echo -n "- Waiting for Dina chain sync..."
+wait-for-node Dina "cli getinfo | jq -e \".synced_to_chain == true\""
+echo "All nodes synched to chain"
+
+echo "======================================================"
+echo
+echo "Setting up connections and channels"
+echo "- Alice to Bob"
+
+# Connect only if not already connected
+run-in-node Alice "cli listpeers | jq -e '.peers[] | select(.pub_key == \"${bob_address}\")' > /dev/null" \
+&& {
+	echo "- Alice already connected to Bob"
+} || {
+	echo "- Open connection from Alice node to Bob's node"
+	wait-for-node Alice "cli connect ${bob_address}@Bob"
+}
+
+# Create channel only if not already created
+run-in-node Alice "cli listchannels | jq -e '.channels[] | select(.remote_pubkey == \"${bob_address}\")' > /dev/null" \
+&& {
+	echo "- Alice->Bob channel already exists"
+} || {
+	echo "- Create payment channel Alice->Bob"
+	wait-for-node Alice "cli openchannel ${bob_address} 1000000"
+}
+echo "Bob to Chan"
+run-in-node Bob "cli listpeers | jq -e '.peers[] | select(.id == \"${chan_address}\")' > /dev/null" \
+&& {
+	echo "- Bob already connected to Chan"
+} || {
+	echo "- Open connection from Bob's node to Chan's node"
+	wait-for-node Bob "cli connect ${chan_address}@Chan"
+}
+run-in-node Bob "cli listchannels | jq -e '.channels[] | select(.destination == \"${chan_address}\")' > /dev/null" \
+&& {
+	echo "- Bob->Chan channel already exists"
+} || {
+	echo "- Create payment channel Bob->Chan"
+	wait-for-node Bob "cli fundchannel ${chan_address} 1000000"
+}
+echo "Chan to Dina"
+run-in-node Chan "cli peers | jq -e '.[] | select(.nodeId == \"${dina_address}\" and .state == \"CONNECTED\")' > /dev/null" \
+&& {
+	echo "- Chan already connected to Dina"
+} || {
+	echo "- Open connection from Chan's node to Dina's node"
+	wait-for-node Chan "cli connect --uri=${dina_address}@Dina"
+}
+run-in-node Chan "cli channels | jq -e '.[] | select(.nodeId == \"${dina_address}\" and .state == \"NORMAL\")' > /dev/null" \
+&& {
+	echo "- Chan->Dina channel already exists"
+} || {
+	echo "- Create payment channel Chan->Dina"
+	wait-for-node Chan "cli open --nodeId=${dina_address} --fundingSatoshis=1000000"
+}
+echo "All channels created"
+echo "======================================================"
+echo
+echo "Waiting for channels to be confirmed on the blockchain"
+echo -n "- Waiting for Alice channel confirmation..."
+wait-for-node Alice "cli listchannels | jq -e '.channels[] | select(.remote_pubkey == \"${bob_address}\" and .active == true)'"
+echo "- Alice->Bob connected"
+echo -n "- Waiting for Bob channel confirmation..."
+wait-for-node Bob "cli listchannels | jq -e '.channels[] | select(.destination == \"${chan_address}\" and .active == true)'"
+echo "- Bob->Chan connected"
+echo -n "- Waiting for Chan channel confirmation..."
+wait-for-node Chan "cli channels | jq -e '.[] | select (.nodeId == \"${dina_address}\" and .state == \"NORMAL\")' > /dev/null"
+echo "- Chan->Dina connected"
+echo "All channels confirmed"
+
+
+echo "======================================================"
+echo -n "Check Alice's route to Dina: "
+run-in-node Alice "cli queryroutes --dest \"${dina_address}\" --amt 10000" > /dev/null 2>&1 \
+&& {
+	echo "Alice has a route to Dina"
+} || {
+	echo "Alice doesn't yet have a route to Dina"
+	echo "Waiting for Alice graph sync. This may take a while..."
+	wait-for-node Alice "cli describegraph | jq -e '.edges | select(length >= 1)'"
+	echo "- Alice knows about 1 channel"
+	wait-for-node Alice "cli describegraph | jq -e '.edges | select(length >= 2)'"
+	echo "- Alice knows about 2 channels"
+	wait-for-node Alice "cli describegraph | jq -e '.edges | select(length == 3)'"
+	echo "- Alice knows about all 3 channels!"
+	echo "Alice knows about all the channels"
+}
+
+echo "======================================================"
+echo
+echo "Get 10k sats invoice from Dina"
+dina_invoice=$(run-in-node Dina "cli addinvoice 10000 | jq -r .payment_request")
+echo "- Dina invoice: "
+echo ${dina_invoice}
+
+echo "======================================================"
+echo
+echo "Attempting payment from Alice to Dina"
+run-in-node Alice "cli payinvoice --json --force ${dina_invoice} | jq -e '.failure_reason == \"FAILURE_REASON_NONE\"'" > /dev/null && {
+	echo "Successful payment!"
+} ||
+{
+	echo "Payment failed"
+}

code/docker/setup-channels.sh

@@ -1,53 +0,0 @@
-#!/bin/bash
-
-echo Getting node IDs
-alice_address=$(docker-compose exec -T Alice bash -c "lncli -n regtest getinfo | jq .identity_pubkey")
-bob_address=$(docker-compose exec -T Bob bash -c "lightning-cli getinfo | jq .id")
-wei_address=$(docker-compose exec -T Wei bash -c "eclair-cli -s -j -p eclair getinfo| jq .nodeId")
-gloria_address=$(docker-compose exec -T Gloria bash -c "lncli -n regtest getinfo | jq .identity_pubkey")
-
-# The jq command returns JSON strings enclosed in double-quote characters
-# These will confuse the shell later, because double-quotes have special
-# meaning in a bash script.
-
-# We remove the double-quote character by using the shell string manipulation
-# expression: // removes the " character. Even here, we have to escape the "
-# character with a backslash because otherwise bash will interpret it as a
-# string closure.
-# A bit messy, but it works!
-alice_address=${alice_address//\"}
-bob_address=${bob_address//\"}
-wei_address=${wei_address//\"}
-gloria_address=${gloria_address//\"}
-
-# Let's tell everyone what we found!
-echo Alice: ${alice_address}
-echo Bob: ${bob_address}
-echo Wei: ${wei_address}
-echo Gloria: ${gloria_address}
-
-echo Setting up channels...
-echo Alice to Bob
-docker-compose exec -T Alice lncli -n regtest connect ${bob_address}@Bob
-docker-compose exec -T Alice lncli -n regtest openchannel ${bob_address} 1000000
-
-echo Bob to Wei
-docker-compose exec -T Bob lightning-cli connect ${wei_address}@Wei
-docker-compose exec -T Bob lightning-cli fundchannel ${wei_address} 1000000
-
-echo Wei to Gloria
-docker-compose exec -T Wei eclair-cli -p eclair connect --uri=${gloria_address}@Gloria
-docker-compose exec -T Wei eclair-cli -p eclair open --nodeId=${gloria_address} --fundingSatoshis=1000000
-
-echo Get 10k sats invoice from Gloria
-gloria_invoice=$(docker-compose exec -T Gloria bash -c "lncli -n regtest addinvoice 10000 | jq .payment_request")
-
-# Remove quotes
-gloria_invoice=${gloria_invoice//\"}
-echo Gloria invoice ${gloria_invoice}
-
-echo Wait for channel establishment - 60 seconds for 6 blocks
-sleep 60
-
-echo Alice pays Gloria 10k sats, routed around the network
-docker-compose exec -T Alice lncli -n regtest payinvoice --json --inflight_updates -f ${gloria_invoice}

code/lngraph/lngraph.py

@@ -0,0 +1,71 @@
+#!/usr/bin/env python3
+# coding: utf-8
+
+import networkx as nx
+import matplotlib.pyplot as plt
+import sys
+import ujson as json
+
+get_ipython().run_line_magic('matplotlib', 'inline')
+
+filename = 'lngraph0721'
+
+
+# Load channel graph exported from LND
+# To create the JSON file, use the following command
+# on an LND node that is fully synced:
+# lndcli describegraph > filename.json
+
+describegraph = open(filename+'.json', 'r')
+lngraph = json.load(describegraph)
+
+
+
+# Construct network graph
+graph = nx.Graph()
+
+# Add edges and nodes
+for edge in lngraph['edges']:
+    # Name the nodes by last 4 characters of node ID
+    node1 = edge['node1_pub'][-4:]
+    node2 = edge['node2_pub'][-4:]
+
+    graph.add_node(node1)
+    graph.add_node(node2)
+    graph.add_edge(node1, node2)
+
+# Show graph info before reduction
+print(nx.info(graph))
+
+# Remove nodes with fewer than 3 channels to make graph cleaner
+remove = [node for node,degree in dict(graph.degree()).items() if degree < 3]
+graph.remove_nodes_from(remove)
+
+# Show graph info after reduction
+print(nx.info(graph))
+
+
+
+
+
+# Set figure/diagram options (thin grey lines for channels, black dots for nodes)
+options = {
+    "node_color": "black",
+    "node_size": 2,
+    "edge_color" : "grey",
+    "linewidths": 0,
+    "width": 0.01,
+}
+
+# 16:9 image ratio
+fig = plt.figure(figsize=(16,9))
+
+# Spring layout arranges nodes automatically.
+# Channels cause "attraction" (spring), nodes cause "repulsion" (opposite)
+# k controls distance between nodes. Spread them out to make graph less dense
+# Seed for reproducible layout
+pos = nx.spring_layout(graph,k=0.4, iterations=10, seed=721)
+nx.draw(graph, pos, **options)
+
+# Save PNG image
+plt.savefig(filename+'.png', format="png", dpi=600)

code/lngraph/requirements.txt

@@ -0,0 +1,65 @@
+argon2-cffi
+async-generator
+attrs
+backcall
+bleach
+cffi
+cycler
+debugpy
+decorator
+defusedxml
+entrypoints
+graphviz
+ipykernel
+ipython
+ipython-genutils
+ipywidgets
+jedi
+Jinja2
+jsonschema
+jupyter
+jupyter-client
+jupyter-console
+jupyter-core
+jupyterlab-pygments
+jupyterlab-widgets
+kiwisolver
+MarkupSafe
+matplotlib
+matplotlib-inline
+mistune
+nbclient
+nbconvert
+nbformat
+nest-asyncio
+networkx
+notebook>=6.4.1
+numpy
+packaging
+pandocfilters
+parso
+pexpect
+pickleshare
+Pillow
+prometheus-client
+prompt-toolkit
+ptyprocess
+pycparser
+Pygments
+pyparsing
+pyrsistent
+python-dateutil
+pyzmq
+qtconsole
+QtPy
+scipy
+Send2Trash
+six
+terminado
+testpath
+tornado
+traitlets
+ujson
+wcwidth
+webencodings
+widgetsnbextension

colo.html

@@ -1,10 +1,15 @@
-<section id="colophon" data-type="colophon">
-  <h1>Colophon</h1>
+<section data-type="colophon" id="colophon" xmlns="http://www.w3.org/1999/xhtml">
+<h1>Colophon</h1>
 
-  <p>The animals on the cover of <em>Mastering the Lightning Network</em> are <em>Wood Ants (Formica Rufa)</em>.</p>
+<p>The animals on the cover of <em>Mastering the Lightning Network</em> are wood ants (<em>Formica rufa</em>). Commonly used to describe a broad group of ants, “wood ants” are those that either construct nests in forested areas or infest wood in a home. However, <em>Formica rufa</em> specifically refers to the mound-building red wood ants that are mainly found across southern Britain, northern-to-middle Europe, the Pyrenees mountain range, and Siberia. Sometimes, they are also found in North America in both coniferous and broad-leaf broken woodlands and parklands.</p>
 
-  <p>Many of the animals on O'Reilly covers are endangered; all of them are important to the world. To learn more about how you can help, go to <a href="http://animals.oreilly.com">animals.oreilly.com</a>.</p>
+<p>Also known as the southern wood ant, this subspecies of wood ants are aggressive, active, and large. The wood ant queens are typically 12–15 mm in size and can live up to 15 years. Worker ants, on the other hand, are slightly smaller at 8–10 mm and have a lifespan of anywhere between a few weeks to seven years depending on whether they’re male or female (males die soon after mating).</p>
 
-  <p>The cover illustration is by Karen Montgomery, based on a black and white engraving from <em>FILL IN CREDITS</em>. The cover fonts are Gilroy Semibold and Guardian Sans. The text font is Adobe Minion Pro; the heading font is Adobe Myriad Condensed; and the code font is Dalton Maag's Ubuntu Mono.</p>
+<p>Capable of producing formic acid in their abdomens, red wood ants can eject it up to a few feet away when threatened by predators. Their nests are usually conspicuous mounds of grass, twigs, or conifer needles, often built against a rotting tree stump in an area that the sunlight can easily reach. Wood ants live in large colonies that may have 100,000 to 400,000 workers and 100 queens. Red wood ants are very territorial, and known to attack and remove other ant species from the area.</p>
 
+<p>Red wood worker ants forage up to 50 meters from their nest to collect a natural resin found dripping from pine trees. In a behavior unique to wood ants, individual ants walk over the resin to disinfect themselves from bacteria and fungi. Additionally, they also eat aphid honeydew, small insects, and arachnids. Red wood ants are commonly used in forestry and often introduced into an area as a form of pest management.</p>
+
+<p>The red wood ants are currently a protected species and are categorized as “near threatened” by the IUCN. Many of the animals on O'Reilly covers are endangered; all of them are important to the world.</p>
+
+<p>The cover illustration is by Karen Montgomery, based on a black-and-white engraving from a loose plate, origin unknown. The cover fonts are Gilroy Semibold and Guardian Sans. The text font is Adobe Minion Pro; the heading font is Adobe Myriad Condensed; and the code font is Dalton Maag's Ubuntu Mono.</p>
 </section>

contrib/channel-construction.asciidoc

@@ -0,0 +1,383 @@
+After you have learnt about payment channels in the first half of the book and after we explained how they can be connected to a network we want to finally dig down and explain the construction of payment channels in detail.
+Of course, every detail of a technology exists for a reason and is important.
+But we believe that payment channels are the core and most fundamental building block of the Lightning Network around.
+It is literally built around the idea and concept of payment channels.
+Thus in order master the Lightning Network you should be able to understand how payment channels will be constructed.
+As several methods for channels exist we stress that it is not important to remember all the details of every method.
+As with most technologies it is important to understand the core concepts and building blocks which we will try to lay out for you as clearly as possible while still emphasizing on the technical details.
+
+In the previous chapters you have already learned that payment channels
+
+* allow two peers who created it to send and receive Bitcoin up to the amount specified by the capacity of the channel as often as they want to.
+* split the capacity of the channel into a balance between the two peers which - as long as the channel is open - is only known by the owners of the channel and increases privacy.
+* do not require peers to do any additional onchain transactions other than the one needed to open and - potentially at a later state - to close the channel.
+* can stay open for an arbitrary time. Potentially in the best case forever.
+* do not require peers to trust each other as any attempt by a peer to cheat would enable the other peer to take all the funds in the channel as a penalty.
+* can be connected to a network and allow peers to send money along a path of connected channels without the necessity to trust the intermediary nodes as they do not have the ability to steal the Bitcoin that are being forwarded.
+
+In this chapter we will dig deeper into the protocol details that are needed to open and close payment channels.
+Especially with the opening part of a payment channel the ideas for updating a channel should already become clear but we will defer to explain the details how HTLCs are being used in the channel operation chapter which comes directly after this chapter.
+Working through this rather technical chapter you will be able to understand how the protocol design achieves the main properties of payment channels.
+Where necessary some information from the first chapters of this book will be repeated.
+If you are new to the topic we highly encourage you to start there first.
+If you however already know a fair share about bitcoin script, OP_CODES and protocol design it might be sufficient to skip the previous chapter and start here with us.
+This books follows the construction of payment channels as described in BOLT 02 which is titled `peer protocol` and describes how two peers communicate to open, maintain and close a channel.
+As stated we will only discuss opening and closing a channel in this chapter and
+the operation and maintenance of a channel which means either attempting to make or forward a payment as well as failing or settling such attempts is the normal operation and will be discussed in the next chapter.
+
+=== Summary of what you (should) know about payment channels and Bitcoin
+
+Let's quickly summarize what you hopefully already know about payment channels on a technical level after reading the book thus far.
+Let us emphasize that knowing does not necessarily mean understanding yet.
+
+1. A payment channel is encoded as an unspent 2-of-2 multisignature transaction output.
+2. The capacity of the channel relates to the amount that is bound to the unspent 2-of-2 multisignature transaction output.
+3. It is opened with the help of a funding transaction that sends bitcoin to a 2-of-2 multisignature output.
+4. The funding tx can only be broadcasted to the Bitcoin network if a refund or settlement transaction exist.
+5. The channel also consists of a communication protocol that helps to initialize and maintain its state.
+6. The balance of the channel encodes how the capacity is split between the two peers who maintain the channel. Technically the balance is encoded by the most recent pair of a sequence of pairs of similar (but not equal) presigned commitment transactions.
+7. The commitment transaction will include scripts and contracts that allow owners of the channel to take all funds in case the other party tries breach the protocol
+8. There are three ways of closing a channel, the good, the bad and the ugly which refer to the mutual close, forced close and the penalty close respectively.
+
+.Bitcoin, Lightning and "Ownership" of funds
+****
+When someone says they 'own' bitcoin they typically mean that they know the private key of a bitcoin address that has some unspent transaction outputs (UTXOs).
+The private key allows the owner to produce a signature for a transaction that spends the bitcoin by sending it to a different address.
+Thus 'ownership' of bitcoin can be defined as the ability to spend that bitcoin.
+
+If you have an unpublished but signed transaction from a 2-of-2 multisignature address, where some outputs are sent to an address you own, and additionally you exclusively know one of the private keys of the multisignature address, then you effectively own the bitcoin of that output.
+Without your help no other transaction from the 2-of-2 multisignature address can be produced (as only you possesses one of the needed keys to sign a transaction that spends from this address)
+However, without the help of anybody else you can transfer the funds to an address which you control.
+You do that by broadcasting the transaction to the bitcoin network which will accept it as it has valid signatures.
+As the funds in this transaction go to a regular address for which you control the private key you can again move the funds and thus you effectively own them.
+On the Lightning Network ownership of your funds is almost always encoded with you having a pre-signed transaction spending from a 2-of-2 multisignature address.
+While your funds on the Lightning Network are called to be "off-chain" they are actually very much on chain and very much owned by you just like you might own other bitcoin.
+
+One thing about the Lightning Network is however messing with this understanding of ownership.
+On the Lightning Network there exist several presigned Transactions which allocate some of the bitcoin to you and some to your channel partner.
+Without the ability to sequence transaction and invalidate old ones via the penalty based revocation mechanism (or other techniques) we could not define clear ownership as the funds would be distributed among its two owners according to which ever transaction would first be broadcasted and successfully mined.
+
+If the last paragraph of this summary was confusing: No worries! We are getting there now!
+****
+
+## Opening a channel
+
+A payment channel needs to be established before it can be utilized to send, receive and route bitcoin.
+There is always liquidity tied to the payment channel.
+This liquidity is provided by the person who initiates the opening of the channel.
+We call that person the funder of the channel.
+At the time of writing this book the protocol only supports funding of a payment channel by the peer who initiates the opening of the channel.
+The funding of the payment channel happens by a regular on chain transaction.
+This funding transaction sends Bitcoin which the funder controlled to a 2-of-2 multisignature output that is controlled by both peers of the channel.
+In order to safely do so several things had to be prepared:
+
+. Both peers needed to have a secure communication channel established.
+. The funder needs to know the public key that is used by the other peer for their multisig address.
+. There needs to be a revokable refund transaction available that sends all the funds back to the funder in case the other peer becomes irresponsive.
+
+We will assume that the secure communication channel has already been established.
+You can learn more about this in the chapter about peer connection establishing.
+
+The second and third points are exactly why a channel opening protocol must exist and cannot be as easy as just sending bitcoin to a 2-of-2 multisig output.
+Especially the third point makes heavy use of the segwit upgrade but we will come to that.
+
+The entire channel opening protocol requires a 6 - way handshake and is thus considerably more complex than establishing a TCP connection.
+The Protocol goes in a sequential way in which every peer sends 3 messages.
+We can see an overview in this diagram (which was taken directly from BOLT2 of the Lightning-rfc):
+
+        +-------+                              +-------+
+        |       |--(1)---  open_channel  ----->|       |
+        |       |<-(2)--  accept_channel  -----|       |
+        |       |                              |       |
+        |   A   |--(3)--  funding_created  --->|   B   |
+        |       |<-(4)--  funding_signed  -----|       |
+        |       |                              |       |
+        |       |--(5)--- funding_locked  ---->|       |
+        |       |<-(6)--- funding_locked  -----|       |
+        +-------+                              +-------+
+
+
+### Step1: Signaling the intent to open a channel.
+
+When Alice wishes to open a channel with Bob she sends an `open_channel` message to Bob.
+This message tells Bob that Alice wishes to create a channel.
+While there is obviously not a unique way of designing a protocol we can think about what kind of information Alice might have to change with Bob, so that they can safely operate a payment channel together.
+In order to find a good answer we remind ourselves that in order for Alice and Bob to safely operate the channel each of them needs to control a presigned commitment transaction that spends from the output of the funding transaction.
+As the funding transaction will send the funds of the channel to a 2-of-2 multisig output it is very reasonable that Alice needs to tell Bob at some point in the protocol, what her key for that address looks like.
+Thus she can already put that information into the `open_channel` message via the `funding_pubkey` field.
+While the Lightning Network Protocol was created to scale Bitcoin the principles of the Protocol can be used on top of other blockchains as well.
+Thus Alice needs to inform Bob that she will use the Bitcoin Blockchain to secure this channel.
+She can do so by putting the hash of the bitcoin genesis block into the `chain_hash` field of the funding transaction.
+
+
+Obviously Alice needs to share some information with Bob about the channel that she wishes to create.
+Thus this message contains
+
+
+
+
+
+[NOTE]
+====
+These commitment transactions should never hit the blockchain and serve as a safety net for the participants in case the channel partner becomes unresponsive of disappears.
+They are also the reason why the Lightning Network is called an off-chain scaling solution.
+Each channel partner has both signatures for one of the commitment transactions from the sequence of pairs.
+====
+
+The split of the capacity is realized by a `to_local` and a `to_remote` output that is part of every commitment transaction.
+The `to_local` output goes to an address that is controlled by the peer that holds this fully signed commitment transaction.
+`to_local` outputs, which also exist in the second stage HTLC transactions - which we discuss in the routing chapter - have two spending conditions.
+The `to_local` output can be spent either at any time with the help of a revocation secret or after a timelock with the secret key that is controlled by the peer holding this commitment transaction.
+The revocation secret is necessary to economically disincentive peers to publish previous commitment transactions.
+Addresses and revocation secretes change with every new pair of commitment transactions that are being negotiated.
+The Lightning Network as a protocol defines the communication protocols that are necessary to achieve this.
+
+### Security of a Payment channel
+While the BOLTs introduce payment channels directly with the opening protocol we have decided to talk about the security model first.
+The security of payment channels come through a penalty based revocation system which help two parties to split the capacity of the payment channel into a balance sheet without the necessity to trust each other.
+In this chapter we start from an insecure approach of creating a payment channel and explain why it is insecure.
+We will then explain how time locks are being used to create revokable sequence maturity contracts that create the penalty based revocation system which economically incentivize people maintain the most recent state.
+After you understood these concepts we will quickly walk you through the technical details of opening and closing a channel.
+
+Any known payment channel construction uses a 2-of-2 multisgnature output as the basis of the payment channel.
+We call the amount that is attached to this output the capacity of the channel.
+In every case, both channel partners hold 1 secret key of the multisignature address which means that they can only collaboratively control the funds.
+
+#### An example for a highly insecure payment channel construction
+Let us assume Alice does not know the details about the Lightning Network and naively tries to open a payment channel in a way that will likely lead to the loss of her funds.
+Alice has heard that payment channel are 2-of-2 multisignature outputs.
+As she wants to have a channel with Bob and since she knows a public key from Bob she decides to open a channel by sending money to a 2-of-2 multisignature address that comes from Bob's and her key.
+We call the transaction that Alices used a **funding transaction** as it is supposed to fund the payment channel.
+However signing and broadcasting this funding transaction would be a huge mistake.
+As we have discussed the Bitcoins from the resulting UTXO can only be spent if Alice and Bob work together and both provide a signature for a transaction spending those coins.
+If Bob would not respond to Alice in future Alice would have lost her Bitcoins forever.
+That is because the coins would be stuck in the 2-of-2 multisignature address to which she has just sent them.
+
+Luckily Alice has previously read Mastering Bitcoin and she knows all the properties of Bitcoin script and is aware of the risks that are involved with sending Bitcoins to a 2-of-2 multisignature address to which she does not control both keys.
+She is also aware of the "Don't trust. Verify" principle that Bitcoiners follow and doesn't want to trust Bob to help her moving or accessing her coins.
+She would much more prefer to keep control over her coins even though they shall be stored in this 2-of-2 multisignature address.
+While this seems like an impossible problem, Alice has an idea:
+
+    What if she could already prepare a refund transaction (which we call commitment transaction in future) that sends all the bitcoin back to an address that she controls?
+
+Before broadcasting her funding transaction she already prepared and finishes it so that she knows the transaction id.
+She can now create the commitment transaction that spends the output of the funding transaction and ask Bob to provide a signature.
+At that time Bob has nothing to lose by signing the commitment transaction.
+He did not have Coins at the multisig address anyway.
+Even if he did Alice intends to spend from an output which Bob never was involved in.
+Thus at that point for Bob it is perfectly reasonable to sign the commitment transaction that spends the funding transaction.
+On the other side you as a reader might think:
+
+Why would Alice send money to a multisignature address just to prepare a transaction that sends the money back to her?
+
+We really hope you have wondered about this because this is really the point where the innovation begins.
+Just because in general people are expected to broadcast a transaction to the bitcoin network as soon as they have signed it no one forces you to do that.
+
+As Alice would loose access of her Bitcoins once she sends it to a 2-of-2 multisignature output for which she only controls one key, she needs to make sure that she will be able to regain access of her coins in case Bob becomes unresponsive.
+Thus before Alice publishes the funding transaction she will create another transaction that sends all the bitcoin from the 2-of-2 multisignature output back to an address which she controls.
+
+.The situation can be seen in the following picture
+image:images/channel-construction-opening-1.png["Channel construction..."]
+Of course for the commitment transaction Alice would need to get a signature from Bob before she can safely broadcast the funding transaction.
+After publishing the funding transaction instead of broadcasting the commitment transaction she will keep it in a safe place.
+For this to work Alice needs to be sure that the funding transaction could not be published with a different transaction id.
+This malleability was possible before the Segwit upgrade of Bitcoin.
+We will discuss the details later but didn't want to leave them out here.
+
+
+[NOTE]
+====
+This entire process might be surprising (... comparison with HTTP server push and AJAX...)
+
+====
+Having Segwit and this first commitment transaction is actually secure for Alice.
+We have seen the first of three main properties that commit transactions fulfill:
+
+    Commitment Transactions refund channel participants in case the other side becomes irresponsive.
+
+The second purpose was implicitly defined by the first purpose:
+
+    Commitment Transactions split the capacity of the channel into a balance which is owned by each partner.
+
+Initially this split means that all the capacity is naturally on the side of the partner who funded the channel.
+Of course during the lifetime of the channel the balance could change.
+For example Alice might want to send some funds to Bob.
+This could happen because she wants to pay Bob or because she wants Bob to forward the funds through a path of channels to another merchant that she wants to pay.
+Let us assume as an example that Alice wants to send 30k Satoshi to Bob.
+For now we can assume that through some communication protocol Alice and Bob would negotiate a double spent of the funding transaction output of 100k satoshi.
+The new commitment transaction for which Alice and Bob would exchange signatures would send 70k satoshi to Alice and 30k Satoshi to Bob.
+The situation can be seen in the following picture
+image:images/channel-construction-opening-2.png["A new balance in an insecure payment channel"]
+Whenever Alice and Bob want to change the balance of the payment channel they will negotiate a new commitment transaction.
+Effectively they double spend the funding transaction output.
+But as the commitment transactions are not broadcasted - as long as the channel stays open - they will be able to do that.
+
+At this point we want to emphasize that the section was labeled in a way that suggests that this construction is insecure.
+So the main question reads:
+
+    What can go wrong with the insecure payment channel?
+
+The thing that goes and makes this construction insecure lies within the mechanics of Bitcoin.
+The key innovation of Bitcoin was to prevent the double spending problem of electronic coins.
+After Alice and Bob have exchanged signatures for the second commitment transaction Bob cannot rely on the fact that he really owns 30k satoshi.
+Of course he could close the channel by publishing the second commitment transaction assigning 30k satoshi to an address that he controls.
+But similarly Alice could broadcast the first commitment transaction and transfer the entire capacity of the channel back to an address that she controls.
+As Bitcoin prevents double spending of the funding transaction miners will include only one of the two commitment transactions.
+Thus we need to adapt the idea with the commitment transactions to create the ability to revoke an old commitment transaction.
+Regarding the fact that Bob and Alice both have a copy of the transaction and that Bob cannot control the data that Alice has stored on her hardware, it seems pretty hopeless.
+Luckily, the scripting language in Bitcoin allows at least for changing commitment transactions in a way that economically disincentives channel partners from publish an outdated balances after they have negotiated a new balance.
+
+#### Secure Payment channels via Revokable Commitment transactions
+
+
+[NOTE]
+====
+In summary we can conclude that commitment transactions fulfill three purposes:
+1. Refund channel participants in case the other side becomes irresponsive
+2. Split the capacity of the channel into the current balance that peers have agreed upon.
+3. Allow revocation of old state through the means of a penalty via a revocable sequence maturity contract.
+====
+
+We have not yet explained how channel partners actually communicate to negotiate a new balance.
+Because it seems pretty amazing that we can make this swap revocation secret for signature atomic.
+In order to understand this we first need to understand the general communication of how a channel is opened.
+The actual negotiation of the new state is also done with HTLCs.
+That is why we only explain this in the routing chapter and ask you to stay patient.
+
+[NOTE]
+====
+*TODO: Move this note to routing chapter?*
+HTLCS fullfil the following purposes:
+1. Make a conditional payment.
+2. Help to update a new balance in a channel
+3. Make payments through a path of channel atomic, meaning that peers along the path cannot steal funds.
+====
+
+### Opening a payment channel
+We call the process of creating a new payment channel "opening a payment channel".
+Currently a payment channel can only exists between exactly two peers.
+Therefore you might be surprised to learn that even though two users are owning and maintaining the channel the current construction requires only one user to open the channel.
+This does not mean that only one peer is needed to open a channel.
+It does however mean that the user who opens the channel also has to provide the bitcoins to fund the channel.
+
+Let us stick to our example where Alice opens a channel with Bob with a capacity of 100k satoshi.
+This means that Alice provides 100k satoshi.
+Alice will do that by creating a so called funding transaction.
+This transaction sends 100k satoshi from an address that she - or her lightning node software controls - to a 2-of-2 multisig address for which she and Bob know 1 secret key each.
+The amount of Bitcoin that is sent to the multisig output by Alice is called the capacity of the payment channel.
+Thus for the reminder of the chapter in all examples we assume the payment channels that we use as examples already magically exist and the two peers Alice and Bob already have all the necessary data at hand.
+
+[NOTE]
+====
+Even though Alice and Bob both have a public node key to which they own the private secret opening a payment channel is not as easy as sending bitcoins to the 2 out of 2 multisignature output that belongs to the public keys of Alice and Bob.
+Let us assume for a moment that Alice would send 100k Satoshi to the Multisig address resulting from hers and Bob's public node id.
+In that case Alice will never be able to maintain her funds back without the help of Bob.
+Of course we want our payment channels to work in a way that Alice does not need to trust Bob.
+Bob could however refuse to sign a transaction that sends all those outputs back to an address that is controlled by Alice.
+He would be able to blackmail Alice to assign a significant amount of those Bitcoin to an output address that is controlled by him.
+Thus Bob can't steel the coins from Alice directly but he can threten Alice to have her coins lost forever.
+This example shows that unfortunately opening a channel will be a little bit more complex than just sending Bitcoins to a multisignature address.
+====
+
+
+
+[NOTE]
+====
+The importance of the segwit upgrade.
+
+
+====
+
+
+
+In order to avoid the reuse of addresses Alice and Bob will generate a new set of keys for the multisig address that they use to open the channel.
+Alice needs to inform Bob which key she intends to use for their channel and ask him which key he intends to use.
+She will do that by sending Bob and `open_channel` message signaling her interest to open a channel.
+This message contains a lot of additional data fields.
+Most of them specify meta data which is necessary for the channel operation and can be be safely ignored for now.
+We will only look at the following ones:
+
+* [chain_hash:chain_hash]
+* [32*byte:temporary_channel_id]
+* [u64:funding_satoshis]
+* [point:funding_pubkey]
+* [point:revocation_basepoint], [point:payment_basepoint], [point:delayed_payment_basepoint], [point:htlc_basepoint], [point:first_per_commitment_point]
+
+With the `chain_hash` Alice signals that she intends to open the channel on the Bitcoin blockchain.
+While the Lightning Network was certainly invented to scale the amount of payments that can be conducted on the Bitcoin Network it is interesting to note that the network is designed in a way that allows to build channels over various currencies.
+If a node has channels with more than one currency it is even possible to route payments through multi asset channels.
+However this turns out to be a little bit tricky in reality as the exchange rate between currencies might change which might lead the forwarding node to wait for a better exchange rate to settle or to abort the payment process.
+For the opening process the final channel id cannot be determined yet thus Alice needs to select a random channel id with Bob that she can use to identify the messages for this channel during the opening phase.
+This design decision allows multiple channels to exist between two nodes - though currently only LND supports this feature.
+Alice tells Bob for how many satoshis she wishes to open the channel.
+This information is necessary to construct the commitment transaction ...
+
+
+Once the channel is open Alice will be able to send 99k satoshi along this channel.
+Bob on the other side will be able to receive 99k satoshi along that channel.
+This means that initially Alice will not be able to receive Bitcoins on this channel and that Bob initially will not be able to send Bitcoin along that channel.
+
+
+== Other payment channel constuctions
+
+Other constructions of payment channels are known and being discussed by the developers.
+Historically speaking these are the Duplex Micropayment channels introduced by Christian Decker during his time as a PhD student at ETH Zurich and the eltoo channels which where also introduced by Christian Decker.
+The eltoo channels are certainly a more elgant and cleaner way of achieving payment channels with the afore mentioned properties.
+However they require the activation of BIP 118 and a softfork and are - at the time of writing - a potential future protocol change.
+Thus this chapter will only focus on the penalty based channels as described in the Lightning Network Whitepaper and specified in BOLT 02 which are currently supported by the protocol and the implementations.
+
+
+[NOTE]
+====
+The Lightning Network does not need consensus of features across it's participants.
+If the Bitcoin Softfork related to BIP 118 activates and people implement eltoo channels nodes that support eltoo can create payment channels and the onion routing of payments along a path of channels would work just fine even if some of the channels are the modern eltoo channels or some channels are the legacy channels.
+Actually when Lightning Network connections are established nodes signal feature bits of global and local features that they support.
+Thus having the ability to create eltoo channels would just be an additional feature bit.
+In this sense upgrading the Lightning Network is much easier than upgrading Bitcoin where consensus among all stakeholders is needed.
+====
+
+=== Multiparty channels and channel factories
+The current construction could be generalized to multiparty channels and channel factories.
+However the communication protocol would suffer from increased complexity.
+Especially the simplifications in the protocol that might result from eltoo will lead to such features.
+A channel factory is a...
+A multi party channel is a...
+
+Chapter overview:
+  * describes how channels are put together at the script+transaction level
+  * details how a channel if funded in the protocol
+  ** including Key derivation!
+  * details how a channel is updated in the protocol (moved to routing!)
+  * describes what needs to happen when a channel is force closed
+
+Relevant questions to answer:
+  * Channel construction:
+      * What's the difference between a replace-by-revocation based and a replace-by-versioning commitment format?
+      * What does the funding output script look like, what security guarantees does it give us?
+      * What's the difference between CSV and CLTV? How do both of these use the existing fields of the transaction to enforce new behavior?
+      * How do we implement revocation in our channel format?
+      * What does the script on the commitment to the broadcaster look like?
+      * What does the script on the commitment for the party that didn't broadcast look like?
+      * How are HTLCs constructed? What are second-level HTLCs?
+      * How has the commitment format in LN changed over time? What are some of the changes to the commitment format that've happened?
+  * Funding flow and messages:
+      * What are the messages exchanged to initiate a new channel with another peer?
+      * What do the parameters such as the max in flight do?
+      * How should the CSV values and the number of blocks until a channel is considered confirmed change with the size of the channel?
+      * What are wumbo channels? How are they enabled?
+      * What is an upfront shutdown address? What security does it offer?
+      * Is it possible to open multiple channels in a single transaction?
+  * Channel state machine:
+      * What does Medium Access Control mean in the context of network protocols?
+      * At a high level, how does the MAC protocol for 802.11 work?
+      * What steps need to happen for a new commitment state to be proposed and irrevocably committed for both parties?
+      * When is it safe for a party to forward a new HTLC to another peer? (may be out of scope for this chapter)
+      * Is it possible to commit a
+      * How does the current MAC protocol for the LN work?
+        * What does an htlc_add message contain?
+        * How are HTLCs cancelled or settled?
+        * Can both parties propose updates at the same time?
+        * Is it possible for a party to add a batch of HTLCs in a single go?
+        * What constraints exist that both parties need to adhere to?
+      * How are fees paid in a channel? Who pays which fees? Has this changed with newer commitment formats?
+      * How would the MAC protocol need to change if we used channels with symmetric state?

contrib/channel-graph.asciidoc

@@ -0,0 +1,86 @@
+Chapter overview:
+
+  * explains the channel graph, and how it's modified+verified
+
+Relevant questions to answer:
+
+  * Gossip announcements:
+      - How does a peer announce a new channel to the network?
+      - How do nodes verify a channel announcement? Why should they verify one in the first place?
+      - How does a node control _how_ a payment is routed through its channel?
+      - What knobs exist for a node to set in their channel updates?
+      - How often are channel updates sent?
+      - How does a node update its node in the channel graph? Do we we need to verify this?
+      - How quickly does an update propagate?
+      - What are "zombie" channels? Why do they matter?
+  * Channel graph syncing:
+      - What are the various ways a node can sync the channel graph?
+      - Which is the most efficient?
+      - What is the "gossip query" system?
+      - Does a node need to keep up with all gossip updates? Does this change if they're a routing node or mobile client?
+  * Protocol Extensions via Feature Bits and TLV:
+      * How can the channel graph be upgraded using feature bits and TLV fields?
+      * How does a receiver signal that they can accept MPP/AMP payments?
+      
+      
+### What are "Zombie" channels and why do they matter?
+
+A zombie channel can be defined as a channel in which your channel partner has been inactive for so long, that you do not expect them come back online in the future.
+This could be because your channel partner has lost access to their or has permanently shut down that node.
+As such, a zombie channel is technically still an open channel, but cannot be used to route payments.
+
+Zombie channels offer no benefit to the user but have several downsides:
+
+* Any capacity you have locked in the channel is useless for routing payments and could be allocated elsewhere
+* If you lose access to your own node and restore it with only your private key, you will lose access to funds in open zombie channels
+* If you lose access to your own node and restore it with your private key and channel backups, you will not be able to contact your channel partner to cooperatively close the channel and may also lose the funds
+* Public zombie channels are a burden on the network, as information about them is communicated to the rest of the network, but they cannot be used to route payments
+
+Identifying zombie channels is a challenge as it is not always clear if the channel partner is permanently offline.
+A partner node that is offline for a long period may eventually return online in the future.
+However, once a zombie channel is identified, it is recommended to close them and a force close is generally required.
+
+
+### Gossip Announcements
+
+#### How does a peer announce a new channel to the network?
+Let's assume that Alice and Bob have just opened a channel together.
+How do they let the rest of the network know, so that the channel can be used for forwarding payments?
+Well firstly, they don't need to.
+Alice and Bob can choose not to announce the channel and simply use it to transfer bitcoin between each other.
+In this case, they won't earn any fees for forwarding payments.
+However, assuming they do want the channel to be public, they will have marked the channel as public when they initially agreed to open it.
+
+First, they'll have to wait until the funding transaction is confirmed (usually six confirmations).
+Once it's confirmed, Alice's and Bob's nodes will now use the `channel_announcement` message to let the rest of the network know the good news.
+This announcement message contains some important information:
+
+* *Channel ID*: a short description of the channel that tells users which outputs in which transaction in which Bitcoin block were used to fund the channel
+* *Signatures from Alice and Bob*: Remember that the channel funds are locked in a 2-of-2 multisignature address, for which Alice and Bob each hold one of the two keys.
+Alice and Bob will each sign from their key, proving that their nodes control the funds in the channel.
+
+They will then send this `channel_announcement` message to each of their peers.
+
+Note that while the `channel_announcement` message makes their peers aware of their channel, their peers won't yet be able to use the channel for forwarding payments.
+First, Alice or Bob will have to communicate other information, such as their fee policy, which we will discuss below.
+But first, how do their peers verify that the channel announcement is legitimate?
+
+#### Verifying the channel
+
+Let's assume Chan receives this announcement from Bob.
+How does he know that this is a real channel, and why should he even bother to check it?
+
+Well, verifying a channel is pretty important.
+If Chan tries to forward any payments through a channel that doesn't exist, his payments are going to fail.
+Even if the channel does exist, Chan still needs assurances that Bob and Alice actually control the funds inside of it.
+Alice and Bob could potentially dupe him by claiming ownership of a channel that actually belongs to someone else.
+So Chan will definitely want to verify before he updates his channel graph.
+
+Firstly, Chan will check the channel ID to discover which Bitcoin transaction contains the channel funds.
+He will then look up this transaction on the Bitcoin blockchain and he should discover a P2WSH bitcoin UTXO that is signed by both Alice and Bob.
+Secondly, Chan will verify the signatures in the channel announcement message and confirm that the two nodes who signed the channel announcement are actually Alice and Bob.
+Once he's done so, he's convinced that the channel is funded and that Alice and Bob are the funders and owners of that channel.
+If any of these checks fail, he'll ignore the channel announcement.
+
+Once Chan is satisfied that the channel announcement checks out, he'll update his channel graph.
+He will also send information about this channel to his peers, and they will verify it for themselves.

contrib/e2e-presentation-layer.asciidoc

@@ -0,0 +1,128 @@
+Chapter overview:
+
+  * presentation layer, applications, how payment details are exchanged
+
+Relevant questions to answer:
+
+  * What is BOLT 11?
+  * What information does an invoice contain?
+  * How can invoices be extended to integrate new protocol features?
+  * What are some unique things that can be done with LN?
+    - recurring payments
+    - donation addrs
+    - keysend
+    - custom data
+
+
+=== What information does an invoice contain?
+
+A Lightning Network invoice is a request for payment issued by the receiver and contains all the information the sender needs to successfully execute the payment.
+Usually it will be in the form of a QR code or an alphanumeric string that looks something like this:
+
+_lnbc9150n1p05hx8upp5ug254f9nhymhu2kctm5j9qq28pvvfsqrdaj6fnxzhln023vyka6sdzz2pshjmt9de6zqen0wgsrjvf4ypcxj7r9d3ejqct5ypekzar0wd5xjuewwpkxzcm99cxqzjccqp2sp5k8nxp5jy26c00ny8asampc03z2edl3z784d80hz873g4jkkuqtvqrzjqgmkp5859l5tn0h6rlal5d44vlkl9r6hf03v6e3pnumr96rak85jqztsugqqkvcqqqqqqquyqqqqqqgq9q9qy9qsqwar8ak9hh4cu3evy6z0nzwpq7ax6mdums6utatejnzak78a9vfyq4ya9gnwsquaq5e257qc3fw2tdxqyk2k9fzgmldfd3urskyuzxmqpyy8tke_
+
+Invoice encoding and decoding is defined by BOLT #11
+footnote:[BOLT11 Github: https://github.com/lightningnetwork/lightning-rfc/blob/master/11-payment-encoding.md].
+The above string is composed of two sections, split by a seperator.
+
+The first part, _lnbc9150n_, is the human-readable part of the invoice.
+The _lnbc_ tells us that the invoice is for Lightning Network Bitcoin
+(it could be for Lightning Testnet or a different cryptocurrency).
+The _9150n_ tells us the invoice is for 915 satoshis (expressed here as 91500 millisatoshis).
+The last _1_ character in the string indicates the end of the human-readable section.
+
+Everything after the _1_ is the data part and contains the following information:
+
+* *Destination*: the ID of the node receiving the payment.
+* *Timestamp*: the date and time the invoice was created, measured in seconds past since 1970.
+* *Payment Hash*: the hash of the payment pre-image. Pre-images were discussed in the earlier chapter on Routing.
+* *Expiry Time*: the amount of time, in seconds, after which the invoice expires and can no longer be paid.
+* *CLTV Delta*: the delta to be used in the final HTLC in the path. Discussed in the earlier chapter on Routing.
+* *Signature*: a digital signature by the invoice issuer. If anything in the invoice is changed, the signature check will fail and the invoice will no longer be valid. This prevents attackers tampering with invoices.
+* _(Optional)_ *Description*: a human-readable explanation of what the payment is for.
+* _(Optional)_ *Backup Bitcoin Address*: an on-chain payment address in case payment of the invoice fails.
+* _(Optional)_ *Routing Hints*: to assist the payer in finding a path for the payment. Discussed in the earlier chapter on Path Finding.
+
+An invoice also contains other useful information.
+In the next section, we'll break down the above invoice and identify each individual part.
+
+
+==== Anatomy of a Lightning Invoice
+
+If we enter the above invoice into a invoice decoding tool, such as https://lndecode.com/, we get the following output:
+
+* *Network*: bitcoin mainnet
+* *Amount*: 0.00000915 BTC
+* *Date*: Sun, 30 Aug 2020 12:18:04 GMT
+* *Payment Hash*: e2154aa4b3b9377e2ad85ee922800a3858c4c0036f65a4ccc2bfe6f54584b775
+* *Description*: Payment for 915 pixels at satoshis.place.
+* *Expiration Time*: 600 seconds
+* *Min Final CLTV Expiry*: 10
+* *Payment Secret*: b1e660d24456b0f7cc87ec3bb0e1f112b2dfc45e3d5a77dc47f451595adc02d8
+* *Routing Info*:
+** _Public Key_: 023760d0f42fe8b9befa1ffbfa36b567edf28f574be2cd66219f3632e87db1e920
+** _Short Channel Id_: 0970e2000b330000
+** _Fee Base Msat_: 900
+** _Fee Proportional Millimonths_: 1
+** _CLTV Expiry Delta_: 40
+* *Feature Bits*: 00101000001000000000
+* *Signature*:
+** _R value_: 77467ed8b7bd71c8e584d09f313820f74dadb79b86b8beaf3298bb6f1fa56248
+** _S value_: 0a93a544dd0073a0a6554f03114b94b69804b2ac54891bfb52d8f070b138236c
+** _Recovery Flag_: 1
+* *Signing Data*: 6c6e6263393135306e0be9731f810d388552a92cee4ddf8ab617ba48a0028e16313000dbd9693330aff9bd51612ddd41a212830bcb6b2b73a103337b9101c989a903834bc32b6399030ba1039b0ba37b9b434b997383630b1b2970600a58c002a806963ccc1a488ad61ef990fd87761c3e22565bf88bc7ab4efb88fe8a2b2b5b805b00314808dd8343d0bfa2e6fbe87fefe8dad59fb7ca3d5d2f8b3598867cd8cba1f6c7a48025c388002ccc000000000e100000000400a02808504000
+* *Checksum*: yy8tke
+
+
+
+
+=== What are some unique things that can be done with LN?
+
+**Micropayments**: The currencies in most countries are divisible to a certain extent and not lower (e.g. $1 = 100c).
+However, it is usually not viable to send small amounts, e.g. $1 and less, due to transaction fees and other friction in the system.
+Bitcoin has similar issues due to transaction fees, and fees are likely to increase in the long-term.
+The Lightning Network can reasonably accommodate payments of the value of 1 satoshi, i.e. one hundred millionth of a Bitcoin.
+Even at an obscenely high Bitcoin value of $1m per Bitcoin, this would still allow the transfer of 1 US cent worth of value.
+As many Lightning implementations track values to the thousandth of a Satoshi (i.e. one milli-satoshi), payments could conceivably be even smaller than this.
+This would allow for micropayment business models such as "pay-per-article" or "pay-per-minute", which are not viable in the current financial system.
+
+**Anonymous Payments**: Bitcoin is pseudonymous at best and transactions are permanently stored on the public Bitcoin blockchain.
+Hence, there is always a risk that transactions can be linked back to users post-hoc.
+Technologies like CoinJoin and Pay-to-EndPoint can assist in giving Bitcoin users a greater degree of anonymity but cannot completely solve this problem.
+In contrast, users of the Lightning Network are not aware of other users' payments and, since channels can be private, they may not even be aware of other users' channels.
+Users are only aware of other users' payments insofar as they assist in routing payments; in this case they are unaware of both the source and the destination of the payment.
+As such, the Lightning Network provides for strong use cases for anonymous purchases.
+This would be of particular benefit to online stores and exchanges that accept Bitcoin as malicious attackers can monitor their addresses on the Bitcoin network to try and determine how much bitcoin the businesses owns
+footnote:[One variant of this is called a "dust attack", whereby an attacker can send a very small amount of Bitcoin (called a "dust output") to an address it knows is owned by a store or exchange.
+By monitoring where this small amount of bitcoin moves, it can determine which other addresses the exchange to store owns.
+This kind of attack is not possible on the Lightning Network].
+This is not possible on the Lightning Network.
+
+**Multiplayer Games**: Lightning Payments can be integrated into online and collaborative games.
+One example of this is _Satoshi's Place_, an online billboard where users can pay 1 satoshi to paint 1 pixel on a one-million-pixel canvas.
+What emerges is a constantly changing picture, where anyone can add, remove, or paint over pixels by paying.
+This example can be extended to many other kinds of collaborative games, where users can pay to participate.
+The Lightning Network can also be implemented directly into online games, such as MMORPGs, to facilitate in-game transactions.
+As Lightning wallets and Lightning invoices can be built directly into the games themselves, this completely bypasses the need for credit cards and the traditional financial system.
+While all of this is technically possible on Bitcoin, confirmation times and fees make this unfeasible.
+Bitcoin transactions are confirmed on average every ten minutes, although it could potentially take even longer.
+This exposes the merchant to the risk of accepting unconfirmed transactions.
+Lightning transactions, on the other hand, settle instantly and so are better from a user experience standpoint.
+
+**Earning "interest" on Bitcoin trustlessly**
+While Bitcoin may increase or decrease in value in terms of fiat currencies, it is an asset that does not offer a return in and off itself simply by holding it.
+The amount of Bitcoin one holds remains constant, and actually decreases as one moves it around due to transaction fees.
+
+Third-party services exist that provide interest on Bitcoin, but these services are in general not trustless. 
+Those wishing to earn a return on their Bitcoin holdings _trustlessly_ can do so by opening channels and routing payments in return for routing fees.
+This way, users can earn a return (i.e. "interest") by locking their Bitcoin into channels and offering liquidity to other users wishing to transact on the Lightning Network.
+Users doing so will need to pay the fees to open and close channels, as well as the cost of maintaining any hardware and network infrastructure to run a Lightning Node.
+However, as channels can be left open indefinitely, they _could_ earn a profit as long as there are sufficient users of the Lightning Network such that their routing fees are in excess of their channel fees and maintenance costs over the long term.
+This is trustless as users do not need to loan or send anyone their Bitcoin. Users only need to take on the efforts and risks of operating a Lightning node and storing Bitcoin in a hot wallet.
+
+**Use Cases outside of Finance**
+The Lightning Network's principal object is to move payments across a network. 
+Could it move data other than payments across its network too? It can. 
+Some projects such as _Whatsat_, _Sphinx Chat_, and _Juggernaut_ are implementing instant messaging chat apps on top of the Ligtning Network. 
+One can imagine other types of data such as electronic tickets, QR codes, or other forms of tokens flowing across the Lightning Network as well. 
+The use cases are limited only by our imagination and reach beyond finance.

contrib/fairness.asciidoc

@@ -0,0 +1,74 @@
+=== Trust, Fairness and Enforcement
+
+When people have competing interests, how can they establish enough trust to engage in some cooperative or transactional behavior? The answer to this question lies at the core of several scientific and humanistic disciplines, such as economics, sociology, behavioral psychology, and mathematics. Some of those disciplines give us "soft" answers, that depend on concepts such as reputation, fairness, morality, even religion. Other disciplines give us concrete answers that depend only on the assumption that the participants in these interactions will act rationally.
+
+In broad terms there are a handful of ways to ensure fair outcomes in interactions between individuals who may have competing interests:
+
+* Require trust - you only interact with people who you already trust, due to prior interactions, reputation, or familial relationships. This works well enough at small scale, especially within families and small groups, that it is the most common basis for cooperative behavior. Unfortunately, it doesn't scale and it suffers from tribalist (in-group) bias.
+
+* Rule of law - establish rules for interactions that are enforced by an institution. This scales better, but it cannot scale globally due to differences in customs and traditions, as well as the inability to scale the institutions of enforcement. Nasty side-effect: the institutions become more and more powerful as they get bigger and that leads to corruption.
+
+* Trusted third parties - put an intermediary in every interaction to enforce fairness. Combined with the "rule of law" to provide oversight of intermediaries, this scales better, but suffers from the same imbalance of power: the intermediaries get very powerful and attract corruption. Concentration of power leads to systemic risk and systemic failure ("Too big to fail").
+
+* Game theoretical fairness protocols - this last category emerges from the combination of the internet and cryptography and is the subject of this section. Let's see how it works and what its advantages and disadvantages are.
+
+==== Trusted protocols without intermediaries
+
+Cryptographic systems like Bitcoin and the Lightning Network are systems that allow you to transact with people (and computers) that you don't trust. This is often referred to as "trustless" operation, even though it is not actually trustless. You have to trust in the software that you run and you have to trust that the protocol implemented by that software will result in fair outcomes.
+
+The big distinction between a cryptographic system like this and a traditional financial system, is that in traditional finance you have a _trusted third party_, for example a bank, to ensure that outcomes are fair. A significant problem with such systems is that they give too much power to the third party and they are also vulnerable to a _single point of failure_. If the trusted third party itself violates trust or attempts to cheat, the basis of trust breaks.
+
+As you study cryptographic systems, you will notice a certain pattern: instead of relying on a trusted third party, these systems attempt to prevent unfair outcomes by using a system of incentives and disincentives. In cryptographic systems you place trust in the _protocol_, which is effectively a system with a set of rules that, if properly designed, will correctly apply the desired incentives and disincentives. The advantage of this approach is two fold. Not only do you avoid trusting a third party, you also reduce the need to enforce fair outcomes. So long as the participants follow the agreed protocol and stay within the system, the incentive mechanism in that protocol achieves fair outcomes without enforcement.
+
+The use of incentives and disincentives to achieve fair outcomes is one aspect of a branch of mathematics called _game theory_, which studies "models of strategic interaction among rational decision makers" footnote:[Wikipedia "Game Theory": https://en.wikipedia.org/wiki/Game_theory]. Cryptographic systems that control financial interactions between participants, such as Bitcoin and the Lightning Network rely heavily on game theory to prevent participants from cheating and allow participants who don't trust each other to achieve fair outcomes.
+
+While game theory and its use in cryptographic systems may appear confounding and unfamiliar at first, chances are you're already familiar with these systems in your everyday life, you just don't recognize them yet. Below we'll use a simple example from childhood to help us identify the basic pattern. Once you understand the basic pattern you will see it everywhere in the blockchain space and you will come to recognize it quickly and intuitively.
+
+In this book, we call this pattern a _Fairness Protocol_ defined as a process that uses a system of incentives and/or disincentives to ensure fair outcomes for participants who don't trust each other. Enforcement of a fairness protocol is only necessary to ensure that the participants can't escape the incentives or disincentives.
+
+==== A fairness protocol in action
+
+Let's look at an example of a fairness protocol, which may be familiar to any reader, perhaps as a memory from their childhood.
+
+Imagine a family lunch, with a parent and two children. The parent has prepared a bowl of fried potatoes ("french fries" or "chips" depending on which English dialect you use). The two siblings must share the plate of chips. The parent must ensure a fair distribution of chips to each child, otherwise the parent will have to hear constant complaining (maybe all day) and there's always a possibility of the unfair situation escalating to violence. What is a parent to do?
+
+[NOTE]
+====
+Any similarity between the scenario above and Andreas' childhood experiences with his two cousins is entirely coincidental and should not be mentioned again. The battles of the french fries created enough drama and should be left in the past.
+====
+
+There are a few different ways that fairness can be achieved in this strategic interaction between two siblings that do not trust each other and have competing interests. The naive but commonly used method is for the parent to use their authority as a trusted third party: they split the bowl of chips into two servings. This is similar to traditional finance, where a bank, accountant or lawyer acts as a trusted third party to prevent any cheating between two parties who want to transact.
+
+The problem with this scenario is that this puts a lot of power in the hands of the trusted third party. The parent is accused of playing favorites and not sharing the chips equally. The siblings may fight over the chips, dragging the parent into their fight. Eventually the parent threatens to never again cook french fries if it always results in fights. It is an empty threat, and so the cycle repeats daily.
+
+But a much better solution exists: the siblings are taught to play a game called "split and choose". At each lunch one sibling splits the bowl of chips into two servings and the *other* sibling gets to choose which serving they want. Almost immediately, the siblings figure out the dynamic of this game. If the one splitting makes a mistake or tries to cheat, the other sibling can "punish" them by choosing the bigger bowl. It is in the best interest of both siblings, but especially the one splitting the bowl, to play fair. Only the cheater loses in this scenario. The parent doesn't even have to use their authority or enforce fairness. All the parent has to do is _enforce the protocol_; as long as the siblings cannot escape their assigned roles of "splitter" and "chooser", the protocol itself ensures a fair outcome without the need for any intervention. The parent can't play favorites or distort the outcome.
+
+==== Security primitives as building blocks
+
+In order for a fairness protocol like this to work, there need to be certain guarantees, or _security primitives_ that can be combined to ensure enforcement. The first security primitive is _strict time ordering/sequencing_: the "splitting" action must happen before the "choosing" action. It's not immediately obvious, but unless you can guarantee that action A happens before action B, then the protocol falls apart. The second security primitive is _commitment with non-repudiation_. Each sibling must commit to their choice of role: either splitter or chooser. Also, once the splitting has been completed, the splitter is committed to the split they created - they cannot repudiate that choice and go try again.
+
+Cryptographic systems offer a number of security primitives that can be combined in different ways to construct a fairness protocol. In addition to sequencing and commitment, we can also use many other tools:
+
+- Hash functions to fingerprint data, as a form of commitment, or as the basis for a digital signature.
+- Digital signatures for authentication, non-repudiation, and proof of ownership of a secret.
+- Encryption/decryption to restrict access to information to authorized participants only.
+
+This is only a small list of a whole "menagerie" of security and cryptographic primitives that are in use. More basic primitives and combinations are invented all the time.
+
+In our real-life example, we saw one form of fairness protocol called "split and choose". This is just one of a myriad different fairness protocols that can be built by combining the building blocks of security primitives in different ways. But the basic pattern is always the same: two or more participants interact without trusting each other, by engaging in a series of steps that are part of an agreed protocol. The protocol's steps arrange incentives and disincentives to ensure that if the participants are rational, cheating is counter-productive and fairness is the automatic outcome. Enforcement is not necessary to get fair outcomes - it is only necessary to keep the participants from breaking out of the agreed protocol.
+
+Now that you understand this basic pattern, you will start seeing it everywhere in Bitcoin, the Lightning Network and many other systems. Let's look at some specific examples, next.
+
+==== Example of the fairness protocol
+
+The most prominent example of a "fairness protocol", is Bitcoin's consensus algorithm _Proof of Work_ (PoW). In Bitcoin, miners compete to verify transactions and aggregate them in blocks. To ensure that the miners do not cheat, without entrusting them with authority, Bitcoin uses a system of incentives and disincentives. Miners have to use a lot of electricity doing "work", that is embedded as a "proof" inside every block. This is achieved because of a property of hash functions where the output value is randomly distributed across the entire range of possible outputs. If miners succeed in producing a valid block fast enough, they are rewarded by earning the block reward for that block.  Forcing miners to use a lot of electricity before the network considers their blocks means that they have an incentive to correctly validate the transactions in the block. If they cheat or make any kind of mistake, their block is rejected and the electricity they used to "prove" it is wasted. No one needs to force miners to produce valid blocks, the reward and punishment incentivize them to do so. All the protocol needs to do is ensure that only valid blocks with proof of work are accepted.
+
+The "fairness protocol" pattern can also be found in many different aspects of the Lightning Network:
+
+* Those who fund channels make sure that they have a refund transaction signed before they publish the funding transaction.
+
+* Whenever a channel is moved to a new state, the old state is "revoked" by ensuring that if anyone tries to broadcast it, they lose the entire balance and get punished.
+
+* Those who forward payments know that if they commit funds forward, they can either get a refund or they get paid by the node preceding them.
+
+Again and again, we see this pattern. Fair outcomes are not enforced by any authority. They emerge as the natural consequence of a protocol that rewards fairness and punishes cheating. A fairness protocol that harnesses self-interest by directing it towards fair outcomes.

contrib/history.asciidoc

@@ -17,7 +17,7 @@ A key requirement for a second layer protocol such as Lightning (which will be d
 The `nSequence` field was intended to allow users to transmit updated versions of a transaction to the network, changing the outputs of a transaction, effectively creating a payment channel.
 Such a payment channel would then be valid as long as the transaction was not mined.
 
-According to a mailing list post in 2013, by early Bitcoin developer Mike Hearn, Satoshi Nakamoto envisioned this construct for high frequency trading.footnote:HearnBitcoinDev[Mike Hearn on Bitcoin-dev - April 16th 2013 - Anti DoS for tx replacement http://web.archive.org/web/20190501234813/https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2013-April/002417.html.]
+According to a mailing list post in 2013 by early Bitcoin developer Mike Hearn, Satoshi Nakamoto envisioned this construct for high frequency trading.footnote:HearnBitcoinDev[Mike Hearn on Bitcoin-dev - April 16th 2013 - Anti DoS for tx replacement http://web.archive.org/web/20190501234813/https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2013-April/002417.html.]
 
 However, there were some weaknesses in this initial formulation that limited its potential. Firstly, the payment channel would only be open until the transaction was mined in a block, either limiting the duration of the payment channel or handing control of the payment channel to the miners. Secondly, there was no economic incentive for miners to respect the `nSequence` number (to replace the state of the channel in the mempool), reducing the utility of this mechanism.
 
@@ -25,9 +25,9 @@ The Revocable Sequence Maturity Contracts (RSMC), which formed payment channels
 
 Using this new feature of the `nSequence` number, the RSMC was able to enforce a novel penalty mechanism: each time a party goes to claim a state on chain (which may or may not be the latest), the other party is given a period of time to challenge that claim (enforced by the `nSequence` number of `OP_CHECKSEQUENCEVERIFY`).
 
-If a participant of the channel is able to present evidence to the blockchain of an intent to claim a revoked state, then the honest party s rewarded with all the funds of the cheating party.
+If a participant of the channel is able to present evidence to the blockchain of an intent to claim a revoked state, then the honest party is rewarded with all the funds of the cheating party.
 
-This enforcement of prior revoked states (only the latest channel state is valid) created a very strong economic incentive via the penalty mechanism: if a party attempts to renegade a signed contract, they lose all their funds!
+This enforcement of prior revoked states (only the latest channel state is valid) created a very strong economic incentive via the penalty mechanism: if a party attempts to renege a signed contract, they lose all their funds!
 
 This "challenge period" enforced by the `nSequence` was one of the novel contributions of the RSMC to the payment channel design space: the blockchain was used as a court wherein disputes could be handled objectively.
 
@@ -50,13 +50,13 @@ Alice cannot effectively broadcast the refund transaction to be mined in a block
 Therefore, Bob can securely receive more updates to the channel balance as long as it remains `open`.
 
 This mechanism would allow two users to engage in several smaller transactions which all happened outside of the Bitcoin network.
-While this construction of the unidirectional payment channel would have solved the custody problem of exchanges it has never been widely implemented.
+While this construction of the unidirectional payment channel would have solved the custody problem of exchanges, it has never been widely implemented.
 
 A fundamental blocker to deploying payment channels in production was the issue of transaction malleability.
 
 In the worst case, this could be exploited by an attacker to prevent a party from closing the channel on-chain as their settlement transaction had a chance to become invalidated if the funding transaction has it's transaction ID inadvertently changed.
 
-Also as a payment channel this system was not too useful as the channel could only at total send the total amount of provided bitcoin in the funding transaction.
+Also, as a payment channel this system was not too useful because the channel could only at total send the total amount of provided bitcoin in the funding transaction.
 Once the timelock was over or all bitcoin were sent to B the channel would have to be closed.
 The obvious idea of opening two channels one from A to B and one from B to A would not have helped as each of those channels would have to be closed and reestablished once it ran dry.
 The core breakthrough for the Lightning Network to become a reality was the ability to create payment channels which technically can live forever and can send money back and forth as often as the peers wish to in combination with routing payments among several channels.
@@ -66,8 +66,8 @@ The core breakthrough for the Lightning Network to become a reality was the abil
 You can watch a video explaining the construction and operation of unidirectional payment channels online at: https://youtu.be/AcP3czefanM
 ====
 
-Surprisingly both properties took quite some while until the community figured them out.
-Technically speaking the unidirectional payment channel has all the important ingredients (funding transaction to a 2-2 multisignature wallet, a transaction spending from the wallet encoding the balance, a timelock to allow refunding if the other side becomes unresponsive, off chain communication and the fact that no additional trust other than the one in the bitcoin network) of modern payment channels which are used in the Lightning Network.
+Surprisingly, both properties took quite some while until the community figured them out.
+Technically speaking, the unidirectional payment channel has all the important ingredients (funding transaction to a 2-2 multisignature wallet, a transaction spending from the wallet encoding the balance, a timelock to allow refunding if the other side becomes unresponsive, off chain communication and the fact that no additional trust, other than trust in the bitcoin network, is required) of modern payment channels which are used in the Lightning Network.
 Despite being widely used in today's world, we will study the unidirectional payment channel in more depth in this book as it is an easy to understand educational example to approach the construction of today's payment channels.
 This setup has one safety issue as transactions have been malleable without the segwit upgrade.
 A problem that needed to be solved for any payment channel construction that we know up till today and which has been fixed in August 2017.
@@ -85,13 +85,13 @@ The key difference was that a trusted party would have co-signed the spend of th
 The Ultra Server was not able to steal bitcoin.
 
 The next day, probably in response to Gavin's blogpost, Meni Rosenfeld started a discussion related to how these ideas could be combined.footnote:[Meni Rosenfeld on Bitcointalk - July 5th 2012 - Trustless, instant, off-the-chain Bitcoin payments http://web.archive.org/web/20190419103457/https://bitcointalk.org/index.php?topic=91732.0]
-As Hashed Timelocked Contracts have neither been invented nor seen to solve the issue of trustless routing, Rosenfeld imagined trusted routing nodes.
-Without mentioning the term network or routing of payments the idea of connecting payment channels and being able to send funds from anyone to anyone else even if there was no direct channel was born.
-In Rosenfelds solution payment providers would be the ultraservers and they would settle the transactions among themselves based on trust.
-It took us another 3 years until the Lightning Network whitepaper emerged which had solved all the bits and bolts necessary to get rid of the trust in Rosenfelds solution.
+As Hashed Timelocked Contracts had neither been invented nor seen to solve the issue of trustless routing, Rosenfeld imagined trusted routing nodes.
+Without mentioning the term network or routing of payments, the idea of connecting payment channels and being able to send funds from anyone to anyone else, even if there was no direct channel between them, was born.
+In Rosenfeld's solution, payment providers would be the ultraservers and they would settle the transactions among themselves based on trust.
+It took us another 3 years until the Lightning Network whitepaper emerged which had solved all the bits and bolts necessary to get rid of the trust in Rosenfeld's solution.
 
-It was 2013 that Bitcoin developer Mike Hearn referred to Meni Rosenfelds proposal and suggesting to reactivate the `nSequence` field which Satoshi previously had deactivated.footnote:HearnBitcoinDev[]
-Also Hearn referred to a section on the contracts article talking about the case of micropayment channels with the help of `nSequence`.
+It was 2013 that Bitcoin developer Mike Hearn referred to Meni Rosenfeld's proposal and suggesting to reactivate the `nSequence` field which Satoshi previously had deactivated.footnote:HearnBitcoinDev[]
+Also, Hearn referred to a section on the contracts article talking about the case of micropayment channels with the help of `nSequence`.
 
 Links:
 * https://en.bitcoin.it/w/index.php?title=Contract&oldid=36712#Example_7:_Rapidly-adjusted_.28micro.29payments_to_a_pre-determined_party

contrib/old_onion_routing_draft.asciidoc

@@ -0,0 +1,318 @@
+
+
+
+At a high level, each node in the route is only _explicitly_ told: how to validate the incoming HTLC packet (remember all details need to be correct for a payment to flow!), who the next hop in the route is, and how to modify the incoming HTLC packet into a valid outgoing HTLC packet to forward to the next node.
+Combined with the fact that intermediate forwarding nodes aren't explicitly given the sender and receiver of a payment, nodes are given the _least_ amount of information they need to successfully forward a payment.
+In addition to these privacy enhancing attributes, intermediate nodes aren't able to arbitrarily modify an HTLC packet, as all information is encrypted and cryptically authenticated with integrity checks carried out at each hop to ensure contents haven't been modified.
+Readers familiar with onion routing may have realized that we'll be using some clever cryptographic technique application to achieve all thees traits.
+We call this series of clever application of cryptographic techniques: sourced based onion routing!
+
+
+
+[NOTE]
+====
+While the Lightning Network also uses an onion routing scheme it is actually very different to the onion routing scheme that is used in the TOR network.
+Aside from the distinct cryptographic techniques they use, the biggest difference is that TOR is being used for arbitrary data to be exchanged between two participants where on the Lightning Network the main use case is to pay people and transfer data that encodes monetary value.
+In the Lightning Network, we're only concerned with transmitting the details that are needed for a successful payment.
+On the Lightning Network there is no analogy to the exit nodes of the Tor Network as there's no need to "exit" the network: all payments flow within the network.
+Although, in an idea model only a precise amount of information is leaked by a route, in practice several "side channels' exist, that may allow an adversary to deduce more information about a route.
+As an example, information about CTLV deltas, or the set of possible routes in the network may give away additional information about a given route.
+Similar to Tor, onion routing in the Lightning Network isn't secure against a global passive adversary (one that can monitor all links and information flows in the network).
+Today in the network, every node in the route sees the same payment hash, meaning that if two nodes are "compromised" more details of the route are leaked.
+On the TOR network nodes can theoretically be connected via a full graph as every node could create an encrypted connection with every other node on top of the Internet Protocol almost instantaneously and at no cost.
+On the Lightning Network payments can only flow along existing payment channels.
+Removing and adding of those channels is a slow and expensive process as it requires on-chain bitcoin transactions.
+On the Lightning Network nodes might not be able to forward a payment package because they do not own enough funds on their side of the payment channel.
+On the other hand there are hardly any plausible reasons other then its wish to act maliciously why a TOR node might not be able to forward an onion.
+Last but not least the Lightning Network can actually run on Tor to use it as a message transport layer.
+This means that all connections of a node with its peers and the resulting communication will by obfuscated once more through the TOR network.
+====
+
+Lets stick to our example in which Alice still wants to tip Dina and has decided to use the path via Bob and Chan.
+We note that there might have been alternative paths from Alice to Dina but for now we will just assume it is this path that Alice has decided to use.
+In order to kick off the transfer, Alice needs to send a special message to Bob to kick off the multi-hop transfer.
+You'll learn about the specific structure of this message in later chapters, but for now we'll call it an "HTLC Add" message.
+Aside from the amount, the payment hash, and the time-lock, this message also contains an opaque field use to store encrypted forwarding information.
+Today in the network, this field is 1366 bytes, as that's the _fixed_ size length of the onion packet. #TODO(roasbeef): explain security properties earlier
+This onion contains all the information about the path that Alice intends to use to send the payment to Dina.
+However Bob who receives the onion cannot read all the information about the path as most of the onion is hidden from him through a sequence of encryptions.
+The name onion comes from the analogy to an onion that consists of several layers. In our case every layer corresponds to one round of encryption.
+Each round of encryption uses different encryption keys.
+They are chosen by Alice in a way that only the rightful recipient of an onion can peel of (decrypt) the top layer of the onion.
+
+For example after Bob received the onion from Alice he will be able to decrypt the first layer and he will only see the information that he is supposed to forward the onion to Chan by setting up an HTLC with Chan.
+The HTLC with Chan should use the same Payment Hash as the receiving HTLC from Alice.
+The amount of the forwarded HTLC was specified in Bob decrypted layer of the onion.
+It will be slightly smaller than the amount of his incoming HTLC from Alice.
+The difference of these two amounts has to be at least as big as to cover the routing fees that Bob's node announced earlier on the gossip protocol.
+
+In order to set up the HTLC Bob will modify the onion a little bit in a deterministic manner.
+He removes the information that he could read from it and passes it along to Chan.
+
+Chan in turn is only able to see that he is supposed to forward the package to Dina.
+Chan knows he received the onion from Bob but has no clue that it was actually Alice who initiated the onion in the first place.
+In this way every participant is only able to peel of one layer of the onion by decrypting it.
+Each participant will only learn the information it has to learn to fulfill the routing request.
+For example Bob will explicitly be told that Alice offered him an HTLC and sent him an onion and that he is supposed to offer an HTLC to Chan and forward a slightly modified onion.
+Bob isn't explicitly told if Alice is the originator of this payment as she could also just have forwarded the payment to him.
+Due to the layered encryption he cannot see the inside of Chan's, and Dina's layer.
+The only thing Bob is told explicitly is that he was involved in a path that involved Alice, him and Chan.
+
+While the Onion is decrypted layer by layer while it travels along the path from Alice via Bob and Chan to Dina it is created from the inside layer to the outside layers via several rounds of encryption.
+Being created from the inside means that the construction starts with the Onion Package that Dina is supposed to receive in plain text.
+Let us now look at the construction of the Onion that Alice has to follow and at the exact information that is being put inside each layer of the onion.
+
+The onions are a data structure that at every hop consists of four parts:
+
+1. The version byte
+2. The header consisting of a public key that can be used by the recipient to produce the shared secret for decrypting the outer layer and to derive the public key that has to be put in the header of the modified onion for the next recipient.
+3. The payload
+4. an authentication via an HMAC.
+
+For now we will ignore how the public keys are derived and exchanged and focus on the payload of the onion.
+Only the payload is actually encrypted and will be peeled of layer by layer.
+The payload consists of a sequence of a sequence of per hop data.
+This data can come in two formats the legacy one and the Type Length Value (TLV) Format.
+While the TLV format offers more flexibility in both cases the routing information that is encoded into the onion is the same for every but the last hop.
+For example, with the new TLV format, the sender can actually included the preimage in the payload for the last hop.
+This is nice as it allow a payer to initiate a payment without the necessity to ask the payer for an invoice and payment hash first.
+We will this feature called key send in a different chapter.
+
+A node needs three pieces of information to forward the package:
+
+1. The short channel id of the next channel along which it is supposed to forward the onion by setting up an HTLC with the same payment hash.
+2. The amount that it is supposed to be forwarded and thus being used in the HTLC.
+3. Timelock information encoded to a `cltv_delta` is the last piece of information that is needed as HTLCs are hashed time locked contracts.
+
+For easier readability we have used just a small integer as `short_channel_ids` in the following example and graphics.
+
+[[routing-onion-1]]
+.`per_hop` payload of Dinas onion and the encrypted
+image:images/routing-onion-1.png[]
+
+We can see that Alice has created some per hop data for Dina.
+The short channel id is set to 0 signaling Dina that this payment is intended to be for her.
+Note that this example is slightly simplified, in that Dina can also use attributes of the onion packet format itself to be able to know when she's the final hop.
+The amount to forward is set to 3000.
+On the incoming HTLCs Dina should have seen that exact amount.
+Usually this amount is intended to say how many satoshis should be forwarded.
+Since the short channel id was set to zero in this particular case it is interpreted as the payment amount.
+Finally the CLTV delta which Dina should use to forward the payment is also set to block height 800 (the current height minus Dina's CLTV grace delta) as Dina is the final hop.
+These data fields consist of 20 Bytes.
+The Lightning Network protocol permits usage of up to 65 bytes to signal routing information in the Onion for every hop.
+
+- 1 Byte Realm which signals nodes how to decode the following 32 Bytes.
+- 32 Byte for routing payload information (20 of which we have already used).
+- 32 Byte of a Hashed Message Authentication code.
+
+As we'll see in later sections, the more modern onion payloads used in Lightning today are much more flexible in that they allow a series of arbitrary key-values pairs.
+These arbitrary key-value pairs can be used to extend the protocol in an end-to-end manner, as it many cases, only the sender and receiver need to know how to interpret the data.
+In the next diagram we can see how the per hop payload for Dina looks like.
+
+[[routing-onion-2]]
+.`per_hop` payload of Dinas onion and the encrypted
+image:images/routing-onion-2.png[]
+
+On important feature to protect the privacy is to make sure that onions are always of equal length independent of their position along the payment path.
+Thus onions are always expected to contain 20 entries of 65 Bytes with per hop data.
+If this wasn't the case, and the onion packet shrank as it was being processed, then this would leak information about the true path length to nodes in the route as the packet would be smaller the further down the route we went.
+Since Dina is the final recipient of the payment, we only have 65 bytes worth of data to fill with actual content.
+The remaining bytes are filled with random bytes to pad out the packet in an unpredictable manner.
+
+Taking a step back, before Alice is able to prepare the remainder of the packet, we needs to generate an ephemeral key (a key only used once).
+This ephemeral key is then used to generate a series of additional keys, which are themselves used for encryption, authentication, and also as input to a CSPRNG to deterministically generate the set of random filler bytes.
+In the spirit of onion encryption, Alice will begin encrypting the payload from the last hop, adding a new layer of encryption with each new hop.
+During processing, each node will authenticate the contents of the payload, then process the packet (decrypting it and shifting around some bytes) to prepare it for processing by the next node in the route.
+As we want each node to use a new shared secret to authenticate and encrypt its portion of the packet, the Sphinx onion packet format uses a _re-randomization_ scheme to allow Alice to generate a single ephemeral Diffie-Hellman key for the entire route.
+Rather than occupying space in the routing payload for N public keys, with this little trick, we're able to only include a single public key, which is used for ECDH at each step, and randomized in a deterministic manner for the next hop.
+
+[[routing-onion-3]]
+.`per_hop` payload of Dinas onion and the encrypted
+image:images/routing-onion-3.png[]
+
+You can see that Alice put the encrypted payload inside the full Onion Package which contains the public keys from the secret key that she used to derive the shared secret.
+The full onion packet also has a version byte in the beginning (for future extensibility) and an HMAC for the entire Onion.
+When Dina receives the Onion packet she will extract the public key from the unencrypted part of the onion package.
+Dina then uses ECDH to derive the shared secret using that ephemeral public key which she'll use to process the packet in full.
+The properties of ECDH make is such that only Alice and Dina are able to derive the corresponding shared secret.
+
+After the encrypted Onion for Dina is created Alice will create the next outer layer by creating the onion for Chan.
+
+She truncates 65 Bytes from the end of the encrypted onion and prepends the truncated onion with 65 Byte per Hop data for Chan.
+The per hop data follows the same structure as the per hop data for Dina.
+Thus she starts with the Realm Byte that she will set to 0 again.
+Then comes the short channel id.
+This is set to 452 as Chan is meant to use the channel with this channel ID as the next outgoing channel.
+She sets the amount to 3000 satoshi as this is the amount that Dina is supposed to receive.
+Finally she uses the CLTV delta added to the current height that was announced for this channel on the gossip protocol and that Chan should use for the HTLC when he forwards the Onion.
+Notice how this CTLV expiry (the expiry is the current height plus the delta) increase as we travel forwards (towards the sender) in the route.
+As we'll see later, this series of decrementing time-locks must carefully be set in order to avoid time-based race conditions in the created contracts.
+Again 12 Bytes of zeros are padded and an HMAC is computed.
+Note that she did not have to compute filler this time as she already has too much data with the encrypted inner onion.
+That is why the inner onion had to be truncated at the end.
+This is the plain text version of Chans Onion payload and can be seen in the following diagram:
+
+[[routing-onion-4]]
+.`per_hop` payload of Dinas onion and the encrypted
+image:images/routing-onion-4.png[]
+
+We emphasize that Chan cannot decrypt the inner part of the onion (that's still encrypted from his PoV), as he cannot derive the encryption keys.
+However the information for Chan should also be protected from others.
+Thus Alice conducts another ECDH.
+This time with Chan's public key and the randomized ephemeral key pair.
+She uses the shared secret to encrypt the onion payload.
+She would be able to construct the entire onion for Chan - which actually Bob does while he forwards the onion.
+The Onion that Chan would receive can be seen in the following diagram:
+
+[[routing-onion-5]]
+.`per_hop` payload of Dinas onion and the encrypted
+image:images/routing-onion-5.png[]
+
+Note that in the entire onion there will be Chan's ephemeral public key.
+We've omitted the details here for brevity, but notice how only a single ephemeral key is communicated.
+During processing each node will re-randomize the ephemeral key for the following node.
+Luckily the ephemeral keys that Alice used for the ECDH with Dina can be derived from the ephemeral key that she used for Chan.
+Thus after Chan decrypts his layer he can use the shared secret and his ephemeral public key to derive the ephemeral public key that Dina is supposed to use and store it in the header of the Onion that he forwards to Dina.
+The exact process to generate the ephemeral keys for every hop will be explained at the very end of the chapter.
+Similarly it is important to recognize that Alice removed data from the end of Davids onion payload to create space for the per hop data in Chan's onion.
+Thus when Chan has received his onion and removed his routing hints and per hop data the onion would be too short and he somehow needs to be able to append the 65 Bytes of filled junk data in a way that the HMACs will still be valid.
+This process of filler generation as well as the process of deriving the ephemeral keys is described in the end of this chapter.
+What is important to know is that every hop can derive the Ephemeral Public key that is necessary for the next hop and that the onions save space by always storing only one ephemeral key instead of all the keys for all the hops.
+
+
+Finally after Alice has computed the encrypted version for Chan she will use the exact same process to compute the encrypted version for Bob.
+For Bobs onion she actually computes the header and provides the ephemeral public key herself.
+Note how Chan was still supposed to forward 3000 satoshis but how Bob was supposed to forward a different amount.
+The difference is the routing fee for Chan.
+Bob on the other hand will only forward the onion if the difference between the amount to forward and the HTLC that Alice sets up while transferring the Onion to him is large enough to cover for the fees that he would like to earn.
+
+[NOTE]
+====
+We have not discussed the exact cryptographic algorithms and schemes that are being used to compute the ciphertext from the plain text.
+Also we have not discussed how the HMACs are being computed at every step and how everything fits together while the Onions are always being truncated and modified on the outer layer.
+For readers seeking more details with respect to the cryptographic algorithms used, we invite them to review BOLT 04 itself in full.
+====
+
+[[routing-onion-6]]
+.`per_hop` payload of Dinas onion and the encrypted
+image:images/routing-onion-6.png[]
+
+Since we use the network itself for transport of these onion packets, Alice is able to construct the entire onion without needing to communicate directly with each node in the route.
+She only needs a public key from each participant which is the public `node_id` of the Lightning node and known to Alice.
+In the network today, Alice learns about the public key via the gossip network, which is described in Chapter N.
+
+
+===== CLTV expiry and deltas
+
+==== Pitfalls with source based Routing and HTLCs
+
+In the first part of the routing chapter you have learnt that payments securely flow through the network via a path of HTLCs.
+You saw how a single HTLC is negotiated between two peers and added to the commitment transaction of each peer.
+In the second part you have seen how the necessary information for setting up HTLCs along a path of hops are being transferred via onion packets from the source to the sender.
+However, in the above scenarios, we only discussed flows where everything goes as expected (the optimistic path).
+In this section, we'll now turn out attention into the various scenarios where the payment flow across the route breaks down.
+
+First, it's important to know that once a node sends a fully valid onion packet out to the first hop, they cannot directly influence the course of the route.
+In other words:
+
+* You cannot force nodes to forward the onion immediately.
+* You cannot force nodes to send back an error if they cannot forward the onion because of missing liquidity or other reasons.
+* You cannot be sure that the recipient has the preimage to the payment hash or releases it as soon as the HTLCs of the correct amount arrived.
+
+When sending out an HTLC and its corresponding onion packet, you as the sender must be prepared to wait the worst-case CTLV timelock period before funds are returned back to the sender (if the route fails).
+This explicit awareness of the worst-case delay when sending a payment may be difficult to explain properly from a user experience perspective for end user wallets.
+You want to quickly pay a person but the payment path that your node has chosen has CLTV deltas that quickly add up to several 100 blocks which is a couple of days on the Bitcoin network.
+This means now that if nodes on the path misbehave - on purpose or maybe just because they have a downtime which your node didn't know about - you will have to wait even though you don't see a preimage.
+You must not send out another onion along a different path which uses the same payment hash because there is a risk that both payments will eventually settle.
+While our own experience is that most payments find a path and settle in far less than 10 seconds, the Lightning Network protocol cannot and does not give any service level agreement that within this time payments will settle or fail.
+
+[NOTE]
+====
+There are some ideas that might solve this issue to some degree by allowing the payer to abort a payment. You can find more about this idea under the terms `cancelable payments` or `stuckless payments`. However the proposals that exist only reverse the problem as now the sender can misbehave and the recipient loses control. Another solution is to use many paths in a multipath payment and include some redundancy and ignore the problem that a path takes longer to complete.
+====
+
+Despite these principle problems there are plausible situations in which the routing process fails and in which honest nodes can and should react.
+This is why the onion protocol has the ability to send back errors in a fail-fast manner that allows nodes to remove the HTLC *off-chain*, without the need to close the channels.
+Some - but not all - of the reasons for errors could be:
+
+* A node does not have enough liquidity to set up the next HTLC
+* The next payment channel does not exist anymore as it might have been closed while the onion was being routed to the node which was supposed to forward the onion along its channel.
+* While the channel might still be open - as the funding transaction was never spent - it might happen that the other peer is offline. This of course prevents the node to forward the onion.
+* The key exchanges of the sender might have been wrong so the decryption of the onion failed or the HMACs do not match. (also in case someone tried to tamper with the onion)
+* The recipient might not have issued an invoice and does not know the payment details.
+* The amount of the final HTLC is too low and the recipient does not want to release the preimage.
+
+If any of the above errors are encountered, a node will send back an encrypted error reply onion back to the sender.
+The reply onion will be encrypted at each hop with the same shared secrets that have been used to construct the onion or decrypt a layer.
+These shared keys are all known to the originator of the payment.
+The innermost onion contains the error message and an HMAC for the error message.
+The process makes sure that the sender of the onion and recipient of the reply can be sure that the error really originated from the node that the error message says it's from.
+Another important step in the process of handling errors is to abort the routing process.
+We discussed that the sender of a payment cannot just remove the HTLC on the channel along which the sender sent the payment.
+Recall for example the situation in which Alice sent an onion to Bob who set up an HTLC with Chan.
+If Alice wanted to remove the HTLC with Bob this would put a financial risk on Bob.
+He fears that his HTLC with Chan might still be fulfilled meaning that he could not claim the reimbursement from Alice.
+Thus Bob would never agree to remove the HTLC with Alice unless he has already removed his HTLC with Chan.
+If however the HTLC between Alice and Bob are set up and the HTLC between Bob and Chan are set up but Chan encounters problems with forwarding the onion it is perfectly Chan has more options than Alice.
+While sending back the error Onion to Bob, Chan could ask him to remove the HTLC.
+Bob has no risk in removing the HTLC with Chan and Chan also has no risk as there is no downstream HTLC.
+Removing an HTLC is the reverse of adding one in the first place from the PoV of the commitment transaction.
+In case of errors, peers signal that they wish to remove the HTLC by sending an `update_fail_htlc` or `update_fail_malformed_htlc` message.
+These messages contain the id of an HTLC that should be removed in the next version of the commitment transaction.
+In the same handshake-like process that was used to exchange `commitment_signed` and `revoke_and_ack` messages, the new state and thus pair of commitment signatures has to be negotiated and agreed upon.
+This also means while the balance of a channel that was involved in a failed routing process will not have changed at the end it will have negotiated two new commitment transactions.
+Despite having the same balance it must not got back to the previous commitment transaction which did not include the HTLC as this commitment transaction was revoked.
+If it was used to force close the channel the channel partner would have the ability to create a penalty transaction and get all the funds.
+
+==== Settling HTLCs
+In the last section you learned about the error cases that can happen with onion routing via the chain of HTLCs.
+You've learned how HTLCs are removed if there is an error.
+Of course HTLCs also need to be removed and the balance needs to be updated if the chain of HTLCs was successfully set up to the destination and the preimage is being released.
+Not surprisingly this process is initiated with another lightning message called `update_fulfill_htlc`.
+You will remember that HTLCs are set up and supposed to be removed with a new balance for the recipient in exchange for a secret `preimage`.
+Recalling the full-duplex protocol with `commitment_signed` and `revoke_and_ack` messages you might wonder how to make this exchange `preimage` for the new state atomic.
+The cool thing is it doesn't have to be.
+Once a channel partner with an accepted incoming HTLC knows the preimage, they can just safely pass it to the channel partner.
+That is why the `update_fulfill_htlc` message contains only the `channel_id`, the `id` of the HTLC, and the `preimage`.
+You might think that the channel partner could now refuse to sign a new channel state by sending `commitment_siged` and `revoke_and_ack` messages.
+This is not a problem though.
+In that case the recipient of the offered HTLC can just go on chain by force closing the channel.
+Once this has happened the preimage can be used to claim the HTLC output.
+
+==== Some Considerations for routing nodes
+Accepting an HTLC removes funds from a peer that the peer cannot utilize unless the HTLC is removed due to success or failure.
+Similarly forwarding an HTLC binds some of the funds from your nodes payment channel until the HTLC has been removed again.
+As we've explained at the very beginning of the chapter, engaging into the forwarding process of HTLCs does neither yield a direct risk to lose any funds nor does it gain the chance to gain any funds.
+However the funds in jeopardy could be locked for some time.
+In the worst case the routing process needs to be resolved on-chain as the payment channel was forced closed due to some other circumstances.
+In that case outstanding HTLCs produce additional on-chain food print and costs.
+Thus there are two small economic risks involved with the participation in the routing process:
+
+. Higher on-chain fees in case of forced channel closures due to the higher footprint of HTLCs.
+. Opportunity costs of locked funds. While the HTLC is active the funds cannot be used for any other purpose.
+
+Owners of routing nodes might want to monitor the routing behavior and opportunities and compare them to the on-chain costs and the opportunity costs in order to compute their own routing fees that they wish to charge in order to accept and forward HTLCs.
+
+Additionally, one should notice that HTLCs are outputs in the commitment transaction.
+The Lightning network protocol allows users to pay a single satoshi.
+However it is impossible to set up HTLCs for this small amount.
+The reason is that the corresponding outputs in the commitment transaction would be below the dust limit.
+Such cases are solved in practice with the following trick:
+Instead of setting up an HTLC the amount is taken from the output of the sender but not added to the output of the recipient.
+Thus the HTLCs which are below the dust limit can be understood to be additional fees in the commitment transaction.
+Most Lightning Nodes support the configuration of minimum accepted HTLC values.
+Operators have to consider if they want to risk overpaying fees or losing funds in the forced channel closure cases because the commitment transactions have been added to the fees.
+
+Explain fee and time-lock considerations
+The “HTLC Switch” analogy compared to regular network switch
+Circuit map concept, how to handle forwarding
+Pipeline styles for HTLCs
+Error handling and encryption for HTLCs
+
+Explain “one little trick” of DH re-randomization
+Explain how we keep the packet size fixed, what’s MAC’d, etc
+Introduce the new modern payload format which uses TLV
+
+=== Routing failures
+
+// Failure to route, stuck payments.

contrib/p2p.asciidoc

@@ -0,0 +1,124 @@
+Chapter overview:
+
+  * high level description of p2p interaction for the LN
+
+Relevant questions to answer:
+
+  * Encrypted P2P Transport:
+      - What is the noise protocol? How does it differ from TLS? Who created it
+
+      - and what are some of primary design principles?
+
+      - What is an authenticated key exchange?
+
+      - Why does Noise offer various handshakes? What are some of unique properties certain handshakes offer?
+
+      - What is key rotation in the context of a complete handshake? Why is it important?
+
+      - What is "brontide"? How is it used in LN today? How can it be upgraded in the future?
+  * LN Message Format:
+
+      - What kind of framing is used in the LN message format? What's the max message size and why is it in place?
+
+      - What is a varint? Why is it used in the protocol?
+
+      - What are the message types of some of the popular messages in the protocol?
+
+      - How can new messages be added in the future?
+  * Feature bits:
+
+      - What are feature bits in the network, and how+where are they advertised?
+
+      - How can feature bits be used to phase in new features to the protocol?
+
+      - Today, what are some of the major feature bits used in the system?
+
+      - What's the difference between and end-to-end network upgrade and an internal network upgrade? How's the analogous to the evolution of routers and protocols in the existing internet?
+
+  * TLV Message Extensions:
+
+      - What does TLV stand for?
+
+      - How is this related to the existing protobuf message format?
+
+      - Where are TLV fields used in the protocol today?
+
+      - How can TLV fields be used to extend the protocol, existing messages, and the onion itself?
+      
+      - Sidenote that TLV can be used by upcoming Instant Messaging chat apps like `Whatsat`, `Sphinx Chat` or `Juggernaut`
+
+
+### Feature Bits
+
+#### What is are Feature Bits and why do we need them?
+ 
+Feature bits, or feature flags, are a string of 1s and 0s that Lightning nodes use to communicate to each other which features and upgrades they have enabled.
+But why would they need to do this?
+
+As discussed in earlier chapters, the Lightning Network protocol does not require all nodes to be in agreement on a common ruleset and list of features, which is quite different to the Bitcoin protocol.
+If someone wants to introduce a new feature into Bitcoin (such as Replace-by-Fee) then either all nodes will need to accept this feature or the feature needs to be backwards-compatible.
+However, if a new feature is introduced into the Lightning Network, Lightning users don't need to wait for the rest of the network to upgrade.
+They can start using the new feature immedietely. 
+Even if that feature is still in development, they can implement it as long as they can find other users willing to implement the feature along with them
+footnote:[The Lightning Network itself was famously put into use before it's official "launch". Despite LN developers warning users that the software was still in beta and had bugs, users around the world set up their own nodes and used the software recklessly].
+
+One example of a feature would be multi-part payments, discussed in the chapter on Path-Finding.
+Despite being a feature still in development at the time of writing, it was already deployed by some nodes.
+Given the inherent freedom in the Lightning Network's design there will never be a global consensus on the Lightning Network as to which features are supported and which aren't.
+Some nodes may support multi-part payments, some may not, and some may never decide to support them.
+As such, nodes need of a way of signalling to each other which features they support and which they do not.
+They do so using pairs of 1s and 0s called Feature Bits.
+
+#### How do Feature Bits work?
+
+Whenever nodes communicate with each other, whether through invoices or other methods, a part of that message is reserved to signal which features the node has enabled.
+For example, the `channel_announcement` and `node_announcement` messages described in BOLT #7 have a predefined `features` field reserved for this information.
+The `features` field will take the form of a string of paired bits that will look something like this:
+
+[feature-bits-example]
+----
+00101000001000000000
+----
+
+Flags are numbered from the least-significant bit (i.e. from right from left), starting from 0.
+Flags are also paired, so that bits 0 and 1 form a pair, 2 and 3 form a pair, and so on.
+BOLT #9 contains the full list of which digits stand for which features
+footnote:[https://github.com/lightningnetwork/lightning-rfc/blob/master/09-features.md].
+An odd-numbered bit pair (01) communicates that the feature is backwards-compatible.
+An even-numbered bit pair (10) communicates that the feature is mandatory.
+The rule of thumb is: "it's ok to be odd".
+
+Using BOLT #9 we can break down the above string as follows:
+
+[[feature-bits-breakdown]]
+.Breakdown of a feature bit string
+[options="header"]
+|===
+| Bit Number | Feature                          | Feature Bits | Status                         
+| 0/1        | `option_data_loss_protect`       | 00     | Not enabled                    
+| 2/3        | `initial_routing_sync`           | 00     | Not enabled                    
+| 4/5        | `option_upfront_shutdown_script` | 00     | Not enabled                    
+| 6/7        | `gossip_queries`                 | 00     | Not enabled                    
+| 8/9        | `var_onion_optin`                | 01     | Enabled (Backwards-Compatible) 
+| 10/11      | `gossip_queries_ex`              | 00     | Not enabled                    
+| 12/13      | `option_static_remotekey`        | 00     | Not enabled                    
+| 14/15      | `payment_secret`                 | 01     | Enabled (Backwards-Compatible) 
+| 16/17      | `basic_mpp`                      | 01     | Enabled (Backwards-Compatible) 
+| 18/19      | `option_support_large_channel`   | 00     | Not enabled                    
+|===
+
+
+If Alice sees this string in a node announcement message from Bob's node, then she knows from bits 16 and 17 that Bob's node supports multi-part payments.
+And because the pair is odd, Alice knows that the feature is backwards-compatible.
+If Alice also has multi-part payments enabled, then Alice and Bob can make use of this feature.
+If Alice does not have multi-part payments enabled, she can simply ignore this and get on with her life.
+She could still open a channel with Bob without having the feature enabled herself.
+However, if the feature bits were even then the feature would be mandatory.
+If Alice did not also have this feature enabled, she would have to find another node to open a channel with.
+Alice also knows from bits 18 and 19 that Bob does not have large channels enabled.
+If she would like to open a large channel, then she would have to find someone else who has large channels enabled to open it with.
+
+In this way, upgrades and new features can be rolled out on the Lightning Network in an asynchronous bottom-up way.
+Users can enable the features they want when they are ready to do so.
+Power users can use and test features without waiting for them to be formally released, and more conservative users are not pressured into using a new feature before they are comfortable to do so.
+Users can simply signal to each other which features they support, and even if they do not agree on the feature set, they can still connect and transact as long they agree on all mandatory features.

contrib/path-finding.asciidoc

@@ -0,0 +1,588 @@
+.Chapter overview:
+
+# newly suggested structure
+
+## Many paths lead to Rome. Which one to choose?
+* some with enough liquidity
+* some that are cheap
+* some that don't lock liquidity for too long
+--> optimization problem (tend to be hard in general)
+
+## What are good features
+* fees should be chep
+* success probabilities
+** Why is liquidity a problem? Why do channel balances have to be private and what ist the problem with that? 
+*** if balance was not private nodes would have to tell everyone else about every channel update (that happens with payments) --> Scales as poorly as bitcoin
+*** it is also nice for privacy
+** How to obtain them? 
+** without quantifying the uncertainty this is like being blind without help and trying to walk around.
+* other features
+** CLTV to minimize locked capital in bad cases
+** provenance scores
+
+## Generating candidate pahts by solving / estimating the optimizaion problem
+* Dijkstra for single paths
+* Backward computation because of fees
+
+## Trial and Error Loop
+* the loop
+** Generate candidate paths
+** try to send
+** update konwledge of the `uncertainty network`
+* question: how long to remember the knowledge
+* predict expected number of attempts
+* issues: success rates drop with larger amounts --> Expected number of attempts rises
+
+## Can we do better than single paths
+* Pros and cons 
+++ smaller payments
+-- more channels involved
+* math says it is still an optimization problem or a min cost flow problem. 
+* depending on the goal algorithms do exist. 
+* Well known analogy from logistcs: How do you transport x goods from A to B with least cost of transport
+
+## Perspective of Routing nodes
+* provide a service to earn a fee
+* how to improve the service? 
+** increase liquidity / provide more liquidity
+*** more channels --> better liquidity
+*** larger channels ---> higher reliability
+*** abvoe goals (more channels, larger channels) are in conflict for fixed amount of liquidity, how to solve? ---> Depends on goals.
+** increase updatime
+** increase reliability via rebalancing
+*** pro actively
+*** lazy (JIT - Routing)
+** offer outsourcing of routing for light clients via trampoline payments
+
+
+# old stuff
+
+Relevant questions to answer:
+
+  * What is packet switching? What is circuit switching? Which one does LN use today?
+  * In the abstract what is path finding?
+  * What is dijkstra's? What modifications need to be made to apply it to this domain?
+  * Why must path finding happen backwards (receiver to sender)?
+  * How is the information contained in a channel update used in path finding?
+  * How can errors sent during payment routing help the sender to narrow their search space?
+  * What is payment splitting? How does it work? What alternatives exist?
+  * What information can be sent to intermediate and the final node aside from the critical routing data?
+  * What are multi-hop locks? What addition privacy and security guarantees to they offer?
+  * How can the flexible onion space be used to enabled packet switching in the network?
+
+==== Finding a path
+
+Payments on the Lightning Network are forwarded along a path of channels from one participant to another.
+Thus, a path of payment channels has to be selected.
+If we knew the exact channel balances of every channel, we could easily compute one or more payment paths using any of the standard path finding algorithms taught in good computer science programs.
+Actually, when we consider multipath payments, it is rather a flow problem than a path finding problem.
+Since flows consist of several paths we conveniently talk about path finding.
+If exact information about channel balances were available, we could solve those problems in a way as to minimize the fees that would have to be paid by the payer to the nodes forwarding the payment.
+However, as discussed, the balance information of all channels cannot be available to all participants of the network.
+Thus, we need to have one or more innovative path finding strategies.
+These strategies must relate closely to the routing algorithm that is used.
+As we will see in the next section, the Lightning Network uses a source-based onion-routing protocol for routing payments.
+In such a protocol it is the responsibility of the sender, i.e. payer, to find a path through the network.
+With only partial information about the network topology available this is a real challenge and active research is still being conducted into optimizing this part of the Lightning Network implementations.
+The fact that the path finding problem in the Lightning Network is not fully solved is a major point of criticism towards the technology.
+The path finding strategy currently implemented in Lightning nodes is to probe paths until one is found that has enough liquidity to forward the payment.
+While this is not optimal and leaves ample room for improvements, it should be noted that even this simplistic strategy works well.
+This probing is done by the Lightning node or wallet and is not directly seen by the user of the software.
+The user might suspect that probing is taking place if the payment is not going through instantly.
+The current algorithm also does not necessarily result in the path with the lowest fees.
+
+
+=== What is "Source-Based" routing and why does the Lightning Network use it?
+
+_Source-based routing_ is a method of path-finding where the sender, i.e. the source, plans the path from itself, through the intermediary nodes, to the final destination.
+Once a path has been found and selected, the sender sends the payment to the first intermediary node, who sends it to the second intermediary node, and so on until it reaches the destination.
+While a payment is traveling along a path, the path typically does not get changed by any of the intermediary nodes, even if a shorter path or a cheaper path (in terms of routing fees) exists.
+
+One of the reasons the Lightning Network uses source-based routing is to protect user privacy.
+As discussed in the chapter on _Onion Routing_, the intermediary nodes transmitting the payment are not aware of the full path of the payment. They only know the node they received it from and the node they are sending it to.
+
+The destination, i.e. the payment recipient, is less able to find a good path.
+Even if it specifies a path in the invoice, that path may no longer be viable by the time the invoice is paid, which could be several minutes or several days later.
+The recipient can, however, specify "routing hints" in the invoice to assist the sender in finding a possible path.
+
+On the other hand, source-based routing comes with some inherent drawbacks.
+The sender chooses the path based on its current understanding of the topological map of the Lightning network.
+As discussed in previous chapters, this map is necessarily incomplete. The sender cannot be aware of all the channels. And even if it is aware of them, it  will not always know their latest balances.
+The balances of channels change with every payment. Consequently, any topological knowledge becomes obsolete in a short space of time.
+The standard path finding mechanism in source-based onion-routing that is implemented in all Lightning Network implementations is the following:
+
+. Given the limited local topological knowledge the sender tries to find one or more routing paths.
+. Select an arbitrary path of payment channels which satisfies 3 conditions: 
+  * path connects sender and receiver of the payment, 
+  * all channels on path have a presumed capacity of at least the payment amount, 
+  * all channels on path accept HTLCs of the payment amount.
+. Construct the "onion" from destination to sender according to the meta data of the channels (base fee, fee rate, CLTV delta).
+. Send out the "onion" and expect one of two possible results returned:
+  * Preimages are returned by nodes if the payment settles successfully
+  * Error is returned if the payment fails.
+. If the payment settles, the sender updates its topological knowledge based on this new information for future payments. The algorithm terminates.
+. If the payment fails, the sender updates its topological knowledge based on this new information. It then selects a different path and starts the process again from the beginning.
+
+This means that with every attempted payment nodes actually probe the network and also learn some information about how balances are distributed.
+Implementations will usually prioritise cheaper paths or exclude channels which have recently failed.
+In that sense the selection is not completely arbitrary.
+Even with such primitive heuristics in place it could still be considered a random process or a random walk through the channel graph. 
+There can be several reasons why a payment may fail along the way.
+Reasons for failure include: a routing node became unreachable, a routing channel no longer has the required balance, a routing node doesn't accept new HTLCs, the owner of a channel increased the channel fees, or the channel was closed in the interim.
+Furthermore, there is no guarantee that the route chosen was the cheapest in terms of fees or the shortest in terms of channels involved.
+At the time of writing this book, this is a design trade-off made to protect user privacy.
+
+=== Paths are constructed from destination to source
+Let us use our standard example in which Alice wants to send a payment of 100k satoshi on a path via Bob and Chan to Dina.
+The _path_ obviously looks like (Alice)-->(Bob)-->(Chan)-->Dina.
+Bob and Chan will charge routing fees to forward the _onion_.
+As you already know, nodes can charge two types of fees.
+First, the _base fee_ will be charged for any successful forwarding and settlement of an HTLC.
+This fee is constant and independent of the amount that the node is forwarding.
+Secondly, nodes might charge a _fee rate_ which is proportional to the forwarded amount.
+For simplicity assume that the fee rate of Bob and Chan is expensive with 1% for Bob and 2% for Chan.
+For simplicity furthermore assume neither Bob nor Chan take a base fee.
+When Alice constructs the onion she has to include the routing fees as the difference of the incoming HTLC and the outgoing HTLC.
+Let us assume she computes the routing fees for the onion incorrectly.
+Alice knows that 1% of 100k satoshi is 1k satoshi which she belives she should include in Bob's onion.
+Similarly she knows that 2% of 100k satoshi is 2k satoshi which she belives she should include in Chan's onion.
+An inexperienced Alice would incorrectly believe her total fee to be 3k satoshi. But she is wrong.
+Look at the incorrect onion from our naive Alice. Bob would reject this onion. 
+
+----
+"route": [
+      {
+         "id": "Bob",
+         "channel": "357",
+         "direction": 1,
+         "satoshi": 103000,
+         "forward": 102000,
+         "dealy": 187,
+      },
+      {
+         "id": "Chan", 
+         "channel": "74",
+         "direction": 1,
+         "satoshi": 102000,
+         "forward": 100000,
+         "dealy": 183,
+      },
+      {
+         "id": "Dina",
+         "channel": "452",
+         "direction": 0,
+         "satoshi": 100000,
+         "dealy": 153,
+      }
+   ]
+}
+----
+
+The reason for Bob to not forward the onion is that he expects the incoming amount to be 1% larger then the amount he is supposed to forward.
+Thus he would like to receive an incoming ammount of `103020` satoshi (102000 + 1%) which is 20 satoshi more than our uninformed Alice actually sent him.
+According to Bob's fee schedule Bob will reject this onion.
+If Alice constructed the onion from the destinatin towards the source, she would have started with 100k satoshi for Dina. 
+In the next step she would have added Chan's 2% fee to compute 102k for Chan's input. 
+In the last step she would have applied Bob's fee (1%) to 102k to derive 102k + 1020 satoshi. 
+That makes a total of 103,020 satoshi that she needs to send to Bob.
+As the routing fees can increase the amount that is being forwarded even beyond the capacity of small channels, it makes sense to start the construction of the onion and the path finding at the destination and work from the destination back towards the sender.
+
+[NOTE]
+====
+Onions are constructed from the inside to the outside. Hence, onions are built starting with the destination. 
+However, this is not the reason why path finding has to start with the destination node. 
+====
+
+=== Fundamentals about path finding
+Finding a path through a graph is a problem modern computers can solve rather efficiently.
+Developers mainly choose breadth-first search if the edges are all of equal weight. 
+In cases where the edges are not of equal weight the Dijkstra Algorithm is used.
+In our case the weights of the edges could represent the routing fees.
+Only edges with a capacity larger than the amount to be sent will be included in the search.
+In this basic form pathfinding in the Lightning network is very simple and straight forward.
+However, as we have already discussed in the introduction, channel balances cannot be shared with every participant every time a payment takes place as this  would prevent scaling the network.
+This turns our easy theoretical computer science problem into a rather complex real-world problem.
+We now have to solve a pathfinding problem with only partial knowledge.
+For example, we suspect which edges might be able to forward a payment because their capacity seems big enough.
+But we can't be certain unless we try it out or ask the channel owners directly.
+Even if we were able to ask the channel owners directly, their balance might change by the time we have asked others, computed a path, constructed an onion and send it along.
+Not only do we have soley limited information but the information we have is highly dynamic and might change at any point in time without our knowledge.
+
+One general observation that everyone can easily make is that if every node along a path is able to forward a certain amount of satoshis, these nodes will also be able to forward a lower amount of satoshis.
+This is why many people intuitively believe that multipath payments might be a good strategy.
+Instead of finding one path where every node has a large amount of liquidity the task is split into smaller ones.
+Another reason is of course that the sender of a payment might just not have the amount they wish to send available in one single channel but distributed over several of his channels.
+We leave it to later sections of this chapter to discuss the strengths and weaknesses of multipath payments.
+We simply note that multipath payments are equivalent to finding a flow between the source and the destination.
+Finding flows in a static graph with full knowledge is computationally marginally more expensive than computing a shortest path.
+On the other hand, given the dynamic reality of the Lightning Network and the fact that we do not need to compute a maximum flow, it is currently not known if the flow problem is more or less difficult than finding a path.
+Both problems seem to have about the same difficulty and the problems are partially related as we will see in the following sections.
+
+=== Probing-based pathfinding algorithm on the Lightning Network
+In order to deterministically find a path nodes would need to know the balances of remote payment channels and these balances would have to be static.
+As this is not the case in the Lightning Network, nodes use a probing-based algorithm.
+In its most basic form the algorithm works as follows:
+
+. Select a random path to the destination node
+. Construct and send the onion
+. wait for the response of the onion
+. If response is a valid preimage, then routing was successful and the algorithm terminates.
+. If response is a failure notification, then start over from step 1.
+
+Nodes will use various sources of information to improve the selection of a random path.
+The main source of information is the gossip protocol.
+From the gossip protocol a node learns which other nodes exist and which channels have been opened.
+This will basically provide a network view that can be used to run graph algorithms that generate plausible paths.
+One fitting algorithm is the breadth-first seach traversal.
+The graph algorithm will usually be constrained to channels whose capacity exceeds the payment amount.
+In practice, due to channel reserve and the assumption that the capacity in the channel will not be sitting completely on one side, it is smarter to prefer larger channels.
+
+The second source of information is the blockchain itself.
+Channel closings are not announced via the gossip protocol.
+However, as the funding transaction is encoded by the short channel id of the channel and as it will be spent on closing the channel, nodes can use this on-chain information to update their knowledge about the network of channels.
+
+Past payments form a third source of information.
+Onions can return with errors.
+Knowing for example that the third hop along a path returns an error of _insufficient balance_ means that the first two channels had enough balance and that the third channel did not have enough balance.
+In general, edges with errors can be removed from the set of edges similarly to the edges with insufficient capacity.
+Nodes can accumulate knowledge and update their knowledge with every failed or successful payment attempt.
+It is important that nodes are careful with this data.
+As the capacity information of channels from the gossip protocol and the blockchain data are verifiably correct, the data returned in failed onions can be incorrect.
+Nodes might simply send an error back because they do not want to reveal balance information.
+Besides, channel data continuously changes over time as the Lightning Network is very dynamic.
+This implies that nodes should only use such data if it is not too old or use it only with limited confidence.
+As time advances this information becomes stale and outdated and the confidence in this data diminishes.
+
+The fourth source of information that the node can use are the routing hints in the BOLT 11 invoices.
+Remember that a regular payment process starts with the person who wants to receive money producing a random secret and hashing it to derive the payment hash.
+This hash is usually transported to the sender via an invoice.
+Invoices typically contain some meta data including some routing hints.
+This is imperative if the person who wants to be paid does not have announced channels. In that case some unannounced channels will be specified within the invoice.
+Otherwise the payer would not even be able to find a path to the "hidden" destination node.
+Routing hints might also be used by the receiving node to indicate which public channels have enough inbound capacity to forward the payment.
+In general, the longer a payment path is, the more likely it becomes that a channel with insufficient balance is selected.
+Thus, receiving hints from the receiver indicating on which channels it wishes to receive funds is definitely helpful for the sender.
+
+
+=== Improvements on source-based onion-routing
+
+The probing-based approach that is used in the Lightning Network has several shortcomings.
+Sending out an onion takes a certain amount of time.
+The time depends on how many hops the onion is supposed to be forwarded, on the speed of nodes processing the onion, and on the topology on the network.
+In the following diagram you can see how the round-trip time for onions in general increases with the amount of hops that the onion has encoded.
+
+[[pathfinding-probing]]
+.Research shows that the onion round-trip time depends on the distance (CC-BY-SA Tikhomirov, Sergei & Pickhardt, Rene & Biryukov, Alex & Nowostawski, Mariusz. (2020). Probing Channel Balances in the Lightning Network.)
+image:images/probingtimes.ppm[]
+This diagram is just a snapshot from an experiment in early 2020 and results might change.
+We learn from the diagram that payments can take several seconds while the node probes several paths.
+This is due to the fact that a single onion can easily take a few seconds to return and a sender might have to send several onions sequentially while probing for a successful path.
+In comparison, this will still be much faster than waiting for confirmations on a Bitcoin block; but it is not performant enough in an environment where payments need to settle fast.
+People standing in a line at the grocery store cash register prefer not to wait several seconds.
+Thus, Lightning developers have come up and implemented the following improvements to the probing algorithms. 
+We are also hopeful that additional improvements and optimizations can be discovered in the future.
+
+==== Improvements to probing
+Nodes ordinarily probe the network when making a payment. But nothing prevents them from probing the network periodically.
+Instead of making a real payment, nodes could send out one or multiple _fake_ payments. 
+A fake payment is nothing but an onions with a random payment hash.
+Given the properties of the hash function, it is save to assume that nobody knows the preimage.
+If the payment amount is small enough, a fake payment will fail at the destination and this allows the sending node to learn about the balances on the path.
+There are clear downsides to this approach.
+It produces spam and heavy network load and therefore this behaviour is discouraged.
+However, participants cannot easily be stopped from doing this.
+Channel partners can detect this type of abuse by observing frequent payments that always fail.
+As punishment channel partners can decide to produce errors right away without providing balance information
+or they can decide to close the abused channel. 
+
+[Note]
+====
+We want you to understand that Lightning Network by design does not have perfect privacy.
+While a lot of information is not easily accessible, every time a path is probed the node learns something about the state of the network at that point in time. 
+====
+
+Please note that one should never send two onions at the same time with the same payment hash for which the recipient knows the preimage.
+As long as the onion is being processed and routed the payment is out of control of the sender.
+In case two onions are sent at the same time, the recipient could very well release the preimage twice and get paid twice.
+This is the reason why arbitrary probing should be conducted with a fake, i.e. purely random, payment hash.
+With fake payment hashes the sender can probe concurrently as long as the sender has enough funds to pay for all the HTLCs.
+Successful probing does not guarantee a following successful payment.
+Assume a fake onion returns indicating that the payment hash was unknown to the recipient but otherwise the path has been possible.
+The sender now uses the same path to send the payment with the corrent payment hash.
+In the interim, the balance of a channel along the path changes rendering the path unworkable.
+In this case the sender has to start all over again.
+Admittedly the risk for this to happen is rather small but the possibility exists.
+
+A potential improvement has been outlined by a suggested mechanism labelled as _stuckless payments_.
+The proposal of _stuckless payments_ received positive feedback from developers.
+It is unlikely that the mechanism is implemented before the Lightning Network switches from _Hashed Timelock Contracts_ (HTLCs) to _Point Timelock Contracts_ (PTLCs). PTLCs in turn will only be implemented after _Schnorr Signatures_ are activated on the Bitcoin Network.
+Stuckless payments give control back to the sender of an onion.
+We don't explain the details here, but stuckless payments empower the sender to cancel an onion.
+This is great for redundant and concurrent pathfinding.
+The sender can now send out several _real_ onions without fear of being charged multiple times.
+The first onion that arrives at the recipient will be settled.
+All others will be canclled.
+This increases the usuability of the Lightning Network on several levels.
+One advantage is that the sender can try several paths at the same time.
+The second advantage is that the path is locked, i.e. reserved, after it is found until it is settled.
+This means that the sender can either cancel the onion or bring the onion to a successful conclusion.
+In particular, the probed path once locked cannot change or be used by other routing requests in the interim between probing and setting up the HTLCs that are used to fulfill the request. The found path remains reserved until cancelled or the payment is successfully completed.
+Using stuckless payments the time for a successful payment will reduce drastically.
+The distadvantage is that the sender has to lock more bitcoin during the pathfinding process.
+Due to timeouts these bitcoin can remain locked for several days before being released again.
+Although this should not happen too frequently.
+Another drawback is that the execution of this mechanism utilizes more resources of routing nodes.
+
+
+==== Multipath payments
+Everyone can easily make the following observation:
+
+----
+Let's say your node has discovered a path along which a certain amount of Satoshis can be routed.
+If so, then any onion with an smaller amount of Satoshis can also be routed successfully along that path at the given time.
+One can conclude that a smaller amount has a higher likelihood to be routed successfully to the destination than a larger amount.
+----
+
+This supposition ignores some edge cases which we ignore for this discussion. The above observation might not hold true
+for small amounts of Satoshis. Certain node operators might not be interested in routing small amounts because they might 
+consider them as "not profitable enough". Node operators might weigh other node resources against the tiny profit of a
+small payment and simply reject payments below a given threshold or minimum. What is "small" and what to reject will 
+be defined by each operator on its personal preferences. 
+
+But for the general case, researchers and developers have already tested this postulate and confirmed it multiple times emperically.
+
+With this assumption in mind it seems natural to split a payment amount and send several smaller payments along various paths.
+When one of the smaller payments fails it will be retried and probed just as one would do with a single larger payment.
+While the main idea is easy to understand, we want to discuss the details, advantages, and disadvantages of this mechanism further.
+
+A receiving node will see an incoming HTLC for a certain payment hash.
+If the onion signals that the node is the final recipient and if the amount of the HTLC is less than the one specified in the invoice, the node would normally not accept the HTLC and send back an error notification.
+However, using the _Total Value Locked_ (TLV) format of onions a sender can specify a total amount of the payment which is bigger than the HTLC.
+In the TLV case, the recipient can safely accept the HTLC and wait for more HTLCs to arrive.
+All parts of the payment will use the same payment hash.
+The recipient will only release the preimage if the sum of all incoming HTLCs is at least the specified payment amount.
+
+[Note]
+====
+**Multipath or multipart payments?** You might have noticed that we named the chapter "multipath" payments but mentioned in the last paragraph that such a payment consists of several parts.
+The protocol specification uses the abbrivation _MPP_ for _multipart payments_.
+Multipath is just a special case of multipart. 
+Multipart covers all the cases of multipath plus the unusual case where multiple parts use the same path. 
+For simplicity we take the liberty to also abbriviate multipath payments with MPP.
+====
+
+It is important to recognize that a node that forwards HTLCs does not have to distinguish a single full payment from a partial multipart payment.
+Only the receiving node needs to distinguish the two cases. Only the receiver needs to be ready to accept multipart payments.
+In the BOLT 11 invoice specification there is a field for _feature bits_.
+If a node wishes to accept multipart payments it must signal this by setting the corresponding feature bit (bit 16 of 17).
+If a node wishes to send a multipart payment it can do so if the receiving node has signaled their willingess to accept such payments.
+Currently there is no mechanism for routing nodes to split the payment amount and onion into several parts or merge several incoming HTLCs into a single onion.
+
+Besides the potentially better chances to find smaller routes the sender might want to use a multipart payment because it does not have enough balance in a single payment channel.
+If the channel had enough capacity this could be resolved with a circular rebalancing - which we will discuss in the next section.
+However if the payment amount is bigger than the largest capacity of a channel that the sender has the sender can only pay the invoice if the recipient allows and supports multipart payments.
+Similarly a recipient might not be able to receive a single payment of the requested amount and would have the interest of signaling multi part payments.
+Luckily nodes will do this automatically and practially always signal the support for multi part payments if the implementation supports this feature.
+The standard Lightning Network implementations which follow BOLT 1.1 all support this feature.
+
+Multipart payments will almost always be more expensive than a single payment.
+You will remember that the fees that routing nodes charge consist of a fee rate and of a base fee.
+The total fee rate of a multipart payment stays roughly the same as a single payment.
+However the base fee is added independent of the amount making multipart payments in most cases more expensive.
+As the sender pays the fees the sender will not necessarily have the interest of splitting the payment in too many parts.
+Thus implementations usually integrate multi part payments into the probing based approach.
+For example after a single payment would not got through the node might split the amount into two payments and try a multipart payment with smaller amounts.
+Those mulitpart payments could again be split down if they are not successfull along a route.
+
+The advantages of multi part payments are quite obvious:
+
+. bigger payment sizes
+. higher success rates
+
+On the other side we have a couple of downsides:
+
+. Higher fees
+. More HTLCs locked / more load on the network
+. Potentially longer times. If only a single part gets stuck all the other HTLCs in flight have to wait locking liquidity of many nodes for a potentially longer time
+. Leaks more information as the network is practically probed more heavily.
+
+
+==== Rebalancing
+
+In this chapter you have already learnt that the path finding problem on the lightning network is actually rather a problem of finding a flow - which consists of several paths.
+Very early research about pathfinding in payment channel networks suggests \footnote{FIND LINK} that rebalancing channels does not change the flow properties between nodes.
+With rebalancing we mean shifting liquidity from one channel to another channel for example via a circular payment.
+There is also the notion of offchain / onchain swaps with swapping services.
+This form of rebalancing certainly changes also the topological properties like the flow of the network.
+As rebalancing via circular self payments would not change the overall amount that an arbitrary node can send to any other node people thought that rebalancing is not very useful.
+However in practice a node hardly wants to find the perfect flow or multipath to be able to send the absolute maximum amount to another node.
+Nodes are rather interested in quickly finding a sufficient large flow so that they can make a reasonable payment.
+Research conducted by Rene Pickhardt (one of the authors of this book) indicated that circular rebalancing operations improve the overall successrate in the network for arbitrary payments.
+It turns out that there is various ways how rebalancing can be used and in some form it even resembles the functionality of a multi path payment.
+Thus we decided to devote a section here on basics about rebalancing and how it can be used to improve the pathfinding abilities of the network.
+
+We made the experience that most people call their payment channel balanced if they own the same amount of bitcoin in that channel as their channel partner.
+While this seems intuitive we want to show that this intuition does not seem to be the best intuition for our goals.
+In order to see this let us assume the Lightning Network at some point in time looks exactly like that.
+All channels split the capacity 50 - 50 dividing it into half between the channel partners.
+
+[[rebalancing-1]]
+.A part of the Lightning Network where all the channel balances are distributed 50/50.
+image:images/rebalancing-1.png[]
+
+It is quite clear that after already one single payment such a 50 - 50 state would be destroyed.
+You can see this in the following graph.
+
+[[rebalancing-2]]
+.The Bob - Chan channel becomes now imbalanced
+image:images/rebalancing-2.png[]
+
+you can see that after Bob made a payment of 1 million satoshi to Chan the channel balance was shifted.
+Bob now has 1.5 million satoshi on the channel and chan has 3.5 million satoshi on the channel.
+The balance ratio went from 50/50 to 30/70.
+The other 2 channels however styed with 50/50.
+
+Chan decides that he wants to have a 50/50 channel with Bob.
+There are 3 ways of how he can achieve this.
+
+. He can send back 1 milion satoshi to Bob
+. He can use an onchain swapping service
+. He can send a circular onion
+
+Sending back the money would be quite expensive and does not seem to be a realistic option.
+Using an onchain swapping service after every payment to rebalance channels seems also problematic.
+The entire idea of creating the Lightning Network was to have less on chain transaction and be able to send money between people without the necessity to do on chain transactions.
+Thus there is only the last option which means that Chan could move the money from the Bob-Chan channel via the Bob-Erica channel to hhis Erica-Chan channel. 
+
+[[rebalancing-4]]
+.Chan tries to rebalance the Bob-Chan channel in the unbalanced network via a circular onion of 1 mio Satoshi.
+image:images/rebalancing-4.png[]
+
+The problem in the new network can easily be seen on the next picture.
+While the Bob-Chan channel now becomes 50/50 again all the other channel turned into a 30/70 split ratio. 
+
+[[rebalancing-5]]
+.Rebalancing one channel produces imbalanced other channels
+image:images/rebalancing-5.png[]
+
+An interesting oversvation about this rebalancing can be made though!
+After the payment and the rebalancing it looked like Bob initially had sent Money not via the Bob-Chan channel but via the path along Erica. 
+
+[[rebalancing-6]]
+.Rebalancing is equivalent to having selected a different payment path to begin with.
+image:images/rebalancing-6.png[]
+
+This observation is actually quite interesting.
+While the math theory tells us that rebalancing channels does not change the max flow between two nodes we see that it has changed the selected path of a payment.
+Due to the onion routing and the privacy goals that are implemented in it we have a source based routing and thus assume the sender always has to select and thus find the path.
+However this is not true!
+When rebalancing comes into place we can use the local knowledge of the distribution of balances that nodes might have to help with selection of paths and finding a total payment path / multi path or flow.
+We will explore this idea a little bit more in the upcoming section about JIT routing.
+
+Remember in our example after Bob has paid Chan Bob had a total amount of 4 million satoshi, Chan had a total of 6 million satoshi and Erica still had 5 million satoshi as before.
+Of course it would be possible to have payment channels between these three people with that distribution of funds so that everyone has 50% of the capacity on their side of the payment channel.
+
+[[rebalancing-7]]
+.50/50 balances with upteded capacities.
+image:images/rebalancing-7.png[]
+
+While the above picture shows that it is possible to have 50/50 channls after the payment this could only be achieved if the capacities would have been changed.
+Changing the capacity of channels is only possible by closing and opening the channel or with the help of a technique called splicing.
+The later is not widely deployed yet and would also depend on onchain transactions.
+
+We hope that you have seen from this example a few things:
+
+. Off-chain rebalancing does not change the fact how much money can flow from sender to receiver.
+. Making payments changes how much money sender and receiver can send or receive. This is similar to the physical world where you also can only spend the cash that you have received first.
+. The goal to have channels in a 50/50 state is not possible for all the nodes all the time and thus probably not a good one.
+. Rebalancing in combination with payments changes the way money flew from the sender to the recipient. In particular it shifts can shift the responsability to find a path from the sender to several nodes on the network - even they don't know which path they are trying to find.
+. Thus rebalancing can be a nice tool to support path finding.
+
+With these conclusings let us look more precisely what would be good rebalancing strategies for nodes.
+
+The main problem with Lightning network channels from a routing and pathfinding perspective is that the liquidity is not known.
+From that perspective the 50/50 approach which is not achievable makes sense.
+If nodes could assume that other nodes always have a certain amount of the capacity on their side they could use that fraction of the capacity to make path finding decisions.
+Initially all the channel balance of newly opened channels is on one side.
+Thus if there is a new node which has opened some channels and received some channels all the channels are unbalanced and routing is always only possible in one direction.
+
+Nodes and node operators could look at the channel balance coefficient which is defined as the ratio between the balance they hold on that channel divided by the capacity of that channel.
+As the balance can never be below zero and never exceed the capacity this channel balance coefficient will always be between 0 and 1.
+A node can easily compute the channel balance coefficient for all its channels.
+By the way in the case of the 50/50 rebalancing the coefficients would all have the value of 0.5.
+
+Researchers demonstrated that the overall likelihood to find a path increases if nodes aim to rebalance their channels in a way that their local channel balance coefficients all take the same value.
+This target value can easily be computed as the amount of total funds that a node owns on the network devided by the sum of all capacities of channels that the node maintains.
+We call this target value the node balance coefficient \nu.
+Nodes can check wich channels have channel balance coefficient that is bigger than \nu and which have a channel balance coeffcient that is smaller than \nu.
+after identifying such channels it makes sense to make circular self payments from the channels with too mcuh liquidity to the channels with too little liquidity.
+
+This approach has an economical drawback.
+Doing a circular self payment is not for free.
+The nodes along the circular path will charge routing fees which always have to be paid by the initiator of the payment.
+This would be your node if you wanted to rebalance your channels.
+It might be justified for you to pay those fees upfront because you might earn them back with the routing fees that you charge if you can successfully forward payments.
+However you do not really know in which direction you will have to route payments later.
+In the worst cast you moved liquidity from a channel which you could have used perfectly to fulfill routing requests along that edge in this direction.
+Not only would you have paid routing fees for a rebalancing operation you would also have depleeted your channel more quickly and might face the need to rebalance again.
+
+We hope that you are not discouraged at this moment.
+Rebalancing is still a viable thing.
+While proactive rebalancing increases the reliablity of the network it is currently economically not viable.
+However you could rebalance reactively or Just in Time at the moment when necessary.
+Imagine you have a an incoming HTLCs and the onion says you are supposed to forward the payment along a channel where you lack sufficient balance.
+The standard case of the protocol would be to return the onion with an onion and remove the incoming HTLC.
+However noone stops your node from shortly interrupting the routing process and conduct a rebalancing operation to provide yourself with sufficient liquidity on the channel in question.
+This method is called JIT-Routing as it helps nodes to reactively provide themselves with enough liquidity just in time.
+
+The just in time Routing scheme has 2 major advantages over source based routing.
+
+. It increases the privacy of channels. If nodes that do not have sufficient liquidity return the onions an attacker can use that behavior to probe for the channel balance. However if nodes rebalance their channels they will always be able to forward the payment and protect themselves from probing attacks.
+. More importantly it resembles multipart payments in which the splitting of the payment is not been decided by the sender who would not know how balances remotely are distributed but the splitting would be achieved by the routing node that knows its local topology.
+
+Let us elaborate on the second point and take the example in which Bob was supposed to forward the onion from Alice to Chan but does have enough liquidity on the channel with Chan.
+If Bob now does a cebalancing operation through Erica and is able to afterwards forward the payment along to Bob he has effectively split the payment at his node to flow along two paths.
+One part flows directly to Chan and the other part takes the path over Erica to Chan.
+It is obvious that splitting a payment at the node that can't forward the entire payment is much more reliable and effective than letting the sender decide how to split a payment and into which amounts.
+
+We thus can see that with the help of JIT-Routing rebalancing and multipart payments are actually not so different concepts and ideas.
+There is another way how mutlipart payments and rebalancing can be combined.
+Let us recall that nodes should always aim to have similar channel balance coefficients.
+So if a node wants to make a multipart payment it could split the payment in such a way that it rebalances its channels.
+Meaning it would only pay from channels on which it currently has too much liquidity.
+Also it would use larger parts for the channels that have way too much liquidity and smaller amount for the channels that have just a little bit too much liquidity.
+The optimal amounts can easily be computed with the following formulars.
+
+
+
+TODO: somehow describe this better without being too scientific. Tool and code can be found at: https://github.com/lightningd/plugins/pull/83
+
+```
+new_funds = sum(b) - a
+
+# assuming all channels have capacity of 1 btc
+cap = len(b)
+nu = float(new_funds) / cap
+ris = [1*(float(x)/1 - nu) for x in b]
+
+real_ris = [x for x in ris if x > 0]
+s = sum(real_ris)
+payments = [a*x/s for x in real_ris]
+```
+
+In fact this multipath rebalancing could also be utilized in the process of JIT routing.
+Instead of shifting all the funds from one channel to the destination channel a node could use a circular multipart payment.
+
+
+
+* (proactive / reactive) Rebalancing
+* Imbalance measures
+* goals for rebalancing (low Gini coefficient and not 50 / 50)
+* optimization problem / game theory
+* JIT Routing
+
+==== Optimizations for Multi path payments
+The rebalancing goal with local channel balance coefficients could actually be integrated into multi path payments.
+Thus if a node decides to send a payment along several paths it could very well use this opportunity to split the payment in a way that it improves the imbalance of its own channels.
+So instead of splitting payments by 2 in a divide and conquorer strategy the node could use the following formula ...
+

copyright.html

@@ -1,115 +1,55 @@
-<section data-type='copyright-page' xmlns='http://www.w3.org/1999/xhtml'>
-  <h1>Mastering the Lightning Network</h1>
-  <p class='author'>
-    by
-    <span class='firstname'>Andreas </span>
-    <span class='othername mi'>M. </span>
-    <span class='surname'>Antonopoulos</span>
-  </p>
-    <p class='author'>
-      <span class='firstname'>Olaoluwa </span>
-      <span class='surname'>Osuntokun</span>
-    </p>
-	  <p class='author'>
-	    <span class='firstname'>René </span>
-	    <span class='surname'>Pickhardt</span>
-	  </p>
-  <p class='copyright'>Copyright © 2020 O&apos;Reilly Media. All rights reserved.</p>
-  <p class='printlocation'>Printed in the United States of America.</p>
-  <p class='publisher'>
-    Published by
-    <span class='publishername'>O&apos;Reilly Media, Inc.</span>
-    , 1005 Gravenstein Highway North, Sebastopol, CA 95472.
-  </p>
-  <p>
-    O&apos;Reilly books may be purchased for educational, business, or sales
-    promotional use. Online editions are also available for most titles (
-    <a href='http://oreilly.com/safari'>http://oreilly.com/safari</a>
-    ). For more information, contact our corporate/institutional sales
-    department: 800-998-9938 or
-    <span data-type='email'>corporate@oreilly.com</span>
-    .
-  </p>
-  <ul class='stafflist'>
-    <li>
-      <span class='staffrole'>Editors:</span>
-       Michele Cronin and Michelle Smith
-    </li>
-    <li>
-      <span class='staffrole'>Production Editor:</span>
-       FILL IN PRODUCTION EDITOR
-    </li>
-    <li>
-      <span class='staffrole'>Copyeditor:</span>
-       FILL IN COPYEDITOR
-    </li>
-    <li>
-      <span class='staffrole'>Proofreader:</span>
-       FILL IN PROOFREADER
-    </li>
-    <li>
-      <span class='staffrole'>Indexer:</span>
-       FILL IN INDEXER
-    </li>
-    <li>
-      <span class='staffrole'>Interior Designer:</span>
-       David Futato
-    </li>
-    <li>
-      <span class='staffrole'>Cover Designer:</span>
-       Karen Montgomery
-    </li>
-    <li>
-      <span class='staffrole'>Illustrator:</span>
-       Rebecca Demarest
-    </li>
-  </ul>
-  <ul class='printings'>
-    <li>
-      <span class='printedition'>December 2020:</span>
-       First Edition
-    </li>
-  </ul>
-  <!--Add additional revdate spans below as needed.-->
-  <div>
-    <h1 class='revisions'>Revision History for the First Edition</h1>
-    <ul class='releases'>
-      <li>
-        <span class='revdate'>YYYY-MM-DD:</span>
-         First Release
-      </li>
-    </ul>
-  </div>
-  <p class='errata'>
-    See
-    <a href='http://oreilly.com/catalog/errata.csp?isbn=9781492054863'>http://oreilly.com/catalog/errata.csp?isbn=9781492054863</a>
-     for release details.
-  </p>
-  <div class='legal'>
-    <p>
-      The O’Reilly logo is a registered trademark of O’Reilly Media, Inc.
-      Mastering the Lightning Network, the cover image, and related trade dress
-      are trademarks of O’Reilly Media, Inc.
-    </p>
-    <p>
-      The views expressed in this work are those of the author(s), and do not
-      represent the publisher&apos;s views. While the publisher and the
-      author(s) have used good faith efforts to ensure that the information and
-      instructions contained in this work are accurate, the publisher and the
-      author(s) disclaim all responsibility for errors or omissions, including
-      without limitation responsibility for damages resulting from the use of or
-      reliance on this work. Use of the information and instructions contained
-      in this work is at your own risk. If any code samples or other technology
-      this work contains or describes is subject to open source licenses or the
-      intellectual property rights of others, it is your responsibility to
-      ensure that your use thereof complies with such licenses and/or rights.
-	  This book is not intended as legal or financial advice; use the
-	  appropriate advice. Please consult a qualified professional if you
-        require legal/financial advice.
-    </p>
-  </div>
-  <div class='copyright-bottom'>
-    <p class='isbn'>978-1-492-05486-3</p>
-    <p class='printer'>[FILL IN]</p>
-  </div>
+<section data-type="copyright-page" xmlns="http://www.w3.org/1999/xhtml">
+<h1>Mastering the Lightning Network</h1>
+
+<p class="author">by <span class="firstname">Andreas </span> <span class="othername mi">M. </span> <span class="surname">Antonopoulos</span>, <span class="firstname">Olaoluwa </span> <span class="surname">Osuntokun</span>, and <span class="firstname">René </span> <span class="surname">Pickhardt</span></p>
+
+<p class="copyright">Copyright © 2022 aantonop Books LLC, René Pickhardt, and uuddlrlrbas LLC. All rights reserved.</p>
+
+<p class="printlocation">Printed in the United States of America.</p>
+
+<p class="publisher">Published by <span class="publishername">O'Reilly Media, Inc.</span>, 1005 Gravenstein Highway North, Sebastopol, CA 95472.</p>
+
+<p>O'Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (<a href="http://oreilly.com">http://oreilly.com</a>). For more information, contact our corporate/institutional sales department: 800-998-9938 or <span data-type="email"><em>corporate@oreilly.com</em></span>.</p>
+
+<ul class="stafflist">
+	<li><span class="staffrole">Acquisitions Editor:</span> Michelle Smith</li>
+		<li><span class="staffrole">Development Editor:</span> Michele Cronin</li>
+	<li><span class="staffrole">Production Editor:</span> Kristen Brown</li>
+	<li><span class="staffrole">Copyeditor:</span> Tom Sullivan</li>
+	<li><span class="staffrole">Proofreader:</span> Sonia Saruba</li>
+	<li><span class="staffrole">Indexer:</span> WordCo Indexing Services, Inc.</li>
+	<li><span class="staffrole">Interior Designer:</span> David Futato</li>
+	<li><span class="staffrole">Cover Designer:</span> Karen Montgomery</li>
+	<li><span class="staffrole">Illustrator:</span> Kate Dullea</li>
+</ul>
+
+<ul class="printings">
+	<li><span class="printedition">December 2021:</span> First Edition</li>
+</ul>
+<!--Add additional revdate spans below as needed.-->
+
+<div>
+<h1 class="revisions">Revision History for the First Edition</h1>
+
+<ul class="releases">
+	<li><span class="revdate">2021-11-22:</span> First Release</li>
+
+</ul>
+</div>
+
+<p class="errata">See <a href="http://oreilly.com/catalog/errata.csp?isbn=9781492054863">http://oreilly.com/catalog/errata.csp?isbn=9781492054863</a> for release details.</p>
+
+<div class="legal">
+<p>The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. <em>Mastering the Lightning Network</em>, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc.</p>
+
+<p>The views expressed in this work are those of the authors, and do not represent the publisher's views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. This book is not intended as legal or financial advice; use the appropriate advice. Please consult a qualified professional if you require legal/financial advice.</p>
+
+<p><em>Mastering the Lightning Network</em> is offered under the Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 International License (CC BY-NC-ND 4.0).</p>
+</div>
+
+<div class="copyright-bottom">
+<p class="isbn">978-1-492-05486-3</p>
+
+<p class="printer">[LSI]</p>
+</div>
 </section>

cover.html

@@ -0,0 +1,3 @@
+<figure data-type="cover">
+<img src="images/cover.png"/>
+</figure>

e2e-presentation-layer.asciidoc

@@ -1,53 +0,0 @@
-Chapter overview:
-  * presentation layer, applications, how payment details are exchanged
-
-Relevant questions to answer:
-  * What is BOLT 11?
-  * What information does an invoice contain?
-  * How can invoices be extended to integrate new protocol features?
-  * What are some unique things that can be done with LN?
-    * recurring payments
-    * donation addrs
-    * keysend
-    * custom data
-
-=== What are some unique things that can be done with LN?
-
-**Micropayments**: The current financial system in most countries is divisible to a certain extent and not lower (E.g. $1 = 100c).
-However it is usually not viable to send small amounts, e.g. $1 and less, due to transaction fees and other friction in the system.
-Bitcoin has similar issues due to transaction fees, and fees are likely to increase in the long-term.
-The Lightning Network can reasonably accommodate payments of the value of 1 satoshi i.e. one hundred millionth of a Bitcoin.
-Even at an obscenely high Bitcoin value of $1m per Bitcoin, this would still allow the transfer of 1 US cent worth of value.
-As many Lightning implementations track values to the thousandth of a Satoshi (i.e. one milli-satoshi), payments could conceivably be even smaller than this.
-This would allow for micropayment business models such as "pay-per-article", which are not viable in the current system.
-
-**Anonymous Payments**: Bitcoin is pseudonymous at best and transactions are permanently stored on the public Bitcoin blockchain.
-Hence there is always a risk that transactions can be linked back to users post-hoc.
-Technologies like CoinJoin and Pay-to-EndPoint can assist in giving Bitcoin users a greater degree of anonymity but cannot completely solve this problem.
-In contrast, users of the Lightning Network are not aware of other users' payments and, since channels can be private, they may not even be aware of other users' channels.
-Users are only aware of other users' payments insofar as they assist in routing payments; in this case they are unaware of both the source and the destination of the payment.
-As such, the Lightning Network has a strong use case for anonymous purchases.
-This would be of particular benefit to online stores and exchanges that accept Bitcoin as malicious attackers can monitor their addresses on the Bitcoin network to try and determine how much bitcoin the businesses owns; something that is not possible on the Lightning Network
-footnote:[One variant of this is called a "dust attack", whereby an attacker can send a very small amount of Bitcoin (called a "dust output") to an address it knows is owned by a store or exchange.
-By monitoring where this small amount of bitcoin moves, it can determine which other addresses the exchange to store owns.
-This kind of attack is not possible on the Lightning Network].
-
-**Multiplayer Games**: Lightning Payments can be integrated into online and collaborative games.
-One example of this is Satoshi's Place, an online billboard where users can pay 1 satoshi to paint 1 pixel on a million pixel canvas.
-What emerges is a constantly changing picture where anyone add, remove, or paint over by paying.
-This example can be extended to many other kinds of collaborative games where users can pay to participate.
-The Lightning Network can also be implemented directly into online games, such as MMORPGs, to facilitate in-game transactions.
-As Lightning wallets and Lightning invoices can be built directly into the games themselves, this completely bypasses the need for the credit cards and the traditional financial system.
-While all of this is technically possible on Bitcoin, confirmation times and fees make this unfeasible.
-Transactions are confirmed on average every ten minutes, although it could potentially take even longer.
-This exposes the merchant to the risk of accepting unconfirmed transactions.
-Lightning transactions, on the other hand, settle instantly and so are better from a user experience standpoint for.
-
-**Earning "interest" on Bitcoin trustlessly**
-While Bitcoin may increase or decrease in value in terms of fiat currencies, it is an asset that does not offer a return in and off itself simply by holding it.
-The amount of Bitcoin one holds remains constant, and actually decreases as one moves it around due to transaction fees.
-Those wishing to earn a return on their holdings in Bitcoin terms can do so by opening channels and routing payments in return for routing fees.
-In this way, users can earn a return (i.e. "interest") by locking their Bitcoin into channels and offering liquidity to other users wishing to transact on the Lightning Network.
-Users doing so will need to pay the fees to open and close channels, as well as the cost of maintaining any hardware to run a Lightning Node.
-However, as channels can be left open indefinitely, they could earn a profit as long as there are sufficient users of the Lightning Network such that their routing fees are in excess of their channel fees and maintenance costs over the long term.
-This is trustless as users do not need to loan or send anyone their Bitcoin; they only need to take the risks of operating a Lightning node and storing Bitcoin in a hot wallet.

failure_types_table.asciidoc

@@ -0,0 +1,40 @@
+[[failure_types_table]]
+.Onion error failure types
+[options="header"]
+|===
+| Type | Symbolic name | Meaning
+| `PERM\|1` | +invalid_realm+ | The `realm` byte was not understood by the processing node
+| `NODE\|2` | +temporary_node_failure+ | General temporary failure of the processing node
+| `PERM\|NODE\|2` | +permanent_node_failure+ | General permanent failure of the processing node
+| `PERM\|NODE\|3` | +required_node_fea⁠ture_​missing+ | The processing node has a required feature which was not in this onion
+| `BADONION\|PERM\|4` | +invalid_onion_version+ | The `version` byte was not understood by the processing node
+| `BADONION\|PERM\|5` | +invalid_onion_hmac+ | The HMAC of the onion was incorrect when it reached the processing node
+| `BADONION\|PERM\|6` | +invalid_onion_key+ | The ephemeral key was unparsable by the processing node
+| `UPDATE\|7` | +temporary_channel_​fail⁠ure+ | The channel from the processing node was unable to handle this HTLC,
+but may be able to handle it, or others, later
+| `PERM\|8` | +permanent_channel_​fail⁠ure+ | The channel from the processing node is unable to handle any HTLCs
+| `PERM\|9` | +required_channel_​fea⁠ture_missing+ | The channel from the processing node requires features not present in
+the onion
+| `PERM\|10` | +unknown_next_peer+ | The onion specified a `short_channel_id` which doesn't match any
+leading from the processing node
+| `UPDATE\|11` | +amount_below_minimum+ | The HTLC amount was below the `htlc_minimum_msat` of the channel from
+the processing node
+| `UPDATE\|12` | +fee_insufficient+ | The fee amount was below that required by the channel from the
+processing node
+| `UPDATE\|13` | +incorrect_cltv_expiry+ | The `cltv_expiry` does not comply with the `cltv_expiry_delta` required by
+the channel from the processing node
+| `UPDATE\|14` | +expiry_too_soon+ | The CLTV expiry is too close to the current block height for safe
+handling by the processing node
+| `PERM\|15` | +incor⁠rect_or_unknown_​pay⁠ment_details+ | The `payment_hash` is unknown to the final node, the `payment_secret` doesn't
+match the `payment_hash`, the amount for that `payment_hash` is incorrect, or
+the CLTV expiry of the HTLC is too close to the current block height for safe
+handling
+| `18` | +final_incor⁠rect_​cltv_expiry+ | The CLTV expiry in the HTLC doesn't match the value in the onion
+| `19` | +final_incor⁠rect_​htlc_amount+ | The amount in the HTLC doesn't match the value in the onion
+| `UPDATE\|20` | +channel_disabled+ | The channel from the processing node has been disabled
+| `21` | +expiry_too_far+ | The CLTV expiry in the HTLC is too far in the future
+| `PERM\|22` | +invalid_onion_payload+ | The decrypted onion per-hop payload was not understood by the processing node
+or is incomplete
+| `23` | +mpp_timeout+ | The complete amount of the multipart payment was not received within a
+reasonable time
+|===

github_contributors.asciidoc

@@ -0,0 +1,47 @@
+* 8go (@8go)
+* Aaqil Aziz (@batmanscode)
+* Alexander Gnip (@quantumcthulhu)
+* Alpha Q. Smith (@alpha_github_id)
+* Ben Skee (@benskee)
+* Brian L. McMichael (@brianmcmichael)
+* CandleHater (@CandleHater)
+* Daniel Gockel (@dancodery)
+* Dapeng Li (@luislee818)
+* Darius E. Parvin (@DariusParvin)
+* Doru Muntean (@chriton)
+* Eduardo Lima III (@elima-iii)
+* Emilio Norrmann (@enorrmann)
+* Francisco Calderón (@grunch)
+* Francisco Requena (@FrankyFFV)
+* François Degros (@fdegros)
+* Giovanni Zotta (@GiovanniZotta)
+* Gustavo Silva (@GustavoRSSilva)
+* Guy Thayakorn (@saguywalker)
+* Haoyu Lin (@HAOYUatHZ)
+* Hatim Boufnichel (@boufni95)
+* Imran Lorgat (@ImranLorgat)
+* Jeffrey McLarty (@jnmclarty)
+* John Davies (@tigeryant)
+* Julien Wendling (@trigger67)
+* Jussi Tiira (@juhi24)
+* Kory Newton (@korynewton)
+* Lawrence Webber (@lwebbz)
+* Luigi (@gin)
+* Maximilian Karasz (@mknoszlig)
+* Omega X. Last (@omega_github_id)
+* Owen Gunden (@ogunden)
+* Patrick Lemke (@PatrickLemke)
+* Paul Wackerow (@wackerow)
+* Randy McMillan (@RandyMcMillan)
+* René Köhnke (@rene78)
+* Ricardo Marques (@RicardoM17)
+* Sebastian Falbesoner (@theStack)
+* Sergei Tikhomirov (@s-tikhomirov)
+* Severin Alexander Bühler (@SeverinAlexB)
+* Simone Bovi (@SimoneBovi)
+* Srijan Bhushan (@srijanb)
+* Taylor Masterson (@tjmasterson)
+* Umar Bolatov (@bolatovumar)
+* Warren Wan (@wlwanpan)
+* Yibin Zhang (@z4y1b2)
+* Zachary Haddenham (@senf42)

glossary.asciidoc

@@ -1,293 +1,269 @@
-[preface]
+[glossary]
 [[glossary]]
-== Quick Glossary
+== Glossary
 
-This quick glossary contains many of the terms used in relation to Bitcoin. These terms are used throughout the book, so bookmark this for a quick reference.
-
-++++
-TODO:
-
-Some additional definitions, to be cleaned up and moved into alphabetic order are in the commented-out area below
-++++
-
-////
-
-* blockchain: a single distributed ledger agreed upon by a network of participating nodes. The Lightning Network does not use a blockchain to transact, but requires transactions recorded in a blockchain in order for bitcoin to enter and leave the network.
-* channel: a channel is a financial relationship between two nodes on the Lightning Network. Two users can open a channel with each other using a Bitcoin transaction, and transact with each other by moving bitcoin from one side of the channel to the other.
-* capacity: channels require bitcoin to be pre-loaded into them before they can be used. This becomes the maximum amount of bitcoin that can be transacted using this channel i.e. it's capacity.
-** in-bound capacity: the maximum amount of bitcoin that can be received using a channel. Your in-bound capacity is increased when a user opens a channel with you, or you make a payment to another user.
-** out-bound capacity: the maximum amount of bitcoin that can be sent using a channel. Your out-bound capacity is increased when you open a channel with another user, or you receive a payment from another user.
-* invoice: a request for payment from another user that can take the form of a text string or a QR code. Lightning Invoices can be specified with a description and an amount the invoicer is requesting.
-* node: a node is a participant on the Lightning Network. Nodes can open and close channels with each other, route payments from other nodes, and manage their own wallets. Typically a Lightning Network node user will also run a Bitcoin Node to keep track of the status of on-chain payments
-* on-chain/off-chain: a payment is considered "on-chain" if it is included in the Bitcoin (or other underlying) blockchain where it is publicly visible to all nodes. Payments that are not visible in the underlying blockchain are "off-chain"
-* route: when making a payment from one user to another, the payment will move along many intermediary nodes before reaching the receiver. This path from the sender to the receiver forms a route on the network.
-** routing fees: each intermediary node will request a fee for transmitting the payment. The sum of these are the routing fees paid by the sender
-* transaction: a payment from one user to another. Lightning Network transactions are Bitcoin transactions not yet recorded on the Bitcoin blockchain.
-** funding transaction: a transaction that locks bitcoin into a smart contract to open a channel.
-** settlement transaction: a transaction that closes a channel, and allocates the locked bitcoin to the channel owners according to the final balance of the channel.
-** penalty transaction: if one user tries to "cheat" by claiming a prior state of the channel, the other user can publish a penalty transaction to the Bitcoin blockchain, which allocates all bitcoin in that channel to them.
-* wallet: an application that manages private keys in order to send and receive bitcoin. Lightning Wallets have additional features over and above Bitcoin Wallets in that they can open and close channels, and send and receive Lightning payments.
-
-////
+This quick glossary contains many of the terms used in relation to Bitcoin and the Lightning Network. These terms are used throughout the book, so bookmark this for a quick reference.
 
 address::
-    A Bitcoin address looks like +1DSrfJdB2AnWaFNgSbv3MZC2m74996JafV+. It consists of a string of letters and numbers. It's really an encoded base58check version of a public key 160-bit hash. Just as you ask others to send an email to your email address, you would ask others to send you bitcoin to one of your Bitcoin addresses.
+    Bitcoin addresses compactly encode the information necessary to pay a receiver. A modern address consists of a string of letters and numbers that starts with bc1 and looks like +bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4+. An address is shorthand for a receiver's locking script, which can be used by a sender to sign over funds to the receiver. Most addresses either represent the receiver's public key or some form of script that defines more complex spending conditions. The preceding example is a bech32 address encoding a witness program locking funds to the hash of a public key (See _Pay-to-Witness-Public-Key-Hash_). There are also older address formats that start with 1 or 3 that use the Base58Check address encoding to represent public key hashes or script hashes.
 
-Asymmetric Cryptographic System::
+
+asymmetric cryptographic system::
     Asymmetric cryptography, or public-key cryptography, is a cryptographic system that uses pairs of keys: public keys which may be disseminated widely, and private keys which are known only to the owner.
-    The generation of such keys depends on cryptographic algorithms based on mathematical problems to produce one-way functions.
+    The generation of such keys depends on cryptographic algorithms based on mathematical problems to produce functions that are easy to solve one way, but very difficult to solve in reverse.
     Effective security only requires keeping the private key private; the public key can be openly distributed without compromising security.
-    https://en.wikipedia.org/w/index.php?title=Public-key_cryptography&oldid=877579180
 
-Autopilot::
-    Autopilot is a recommendation engine for Lightning Network nodes that uses statistics of the known topology to suggest which nodes they should open channels with.
+autopilot::
+    An autopilot is a recommendation engine for Lightning nodes that uses statistics of the Lightning Network topology to suggest which nodes they should open channels with.
     Depending on the implementation of the autopilot, the channel capacity may also be recommended.
-    Autopilots are not part of the Lightning Network Protocol.
+    An autopilot is not part of the LN Protocol.
 
 balance::
-    The balance of a payment channel is encoded by the most recent commitment transaction.
-    The balance states the amount of bitcoin that belongs to each channel partner and the amount that are in flying HTLCs (HTLCs which are currently in the routing process and have not been settled yet).
-    The sum of the balance sheet equals the capacity.
-    The channel balance is only known by the channel partners.
-    If the channel balance is completely on one side of the channel, i.e. one channel partner has all of the bitcoin in the channel, this particular partner cannot receive any payments through this channel. This partner can, however, send payments and forward HTLCs.
-    The balance that a node has on its own side of the channel (and is thus able to spend) is called the outbound capacity.
-    The node's channel partner would refer to that balance as its inbound capacity, i.e. the amount that it is able to receive.
-    Nodes should aim to have balanced channels with similar inbound and outbound capacities.
+    The balance of a payment channel is the amount of bitcoin that belongs to each channel partner.
+    For example, Alice could open a channel with Bob for the value of 1 BTC.
+    The channel balance is then 1 BTC to Alice and 0 BTC to Bob.
+    As the users transact, the channel balance will update.
+    For example, if Alice sends 0.2 BTC to Bob, then the balance is now 0.8 BTC to Alice and 0.2 to Bob.
+    When the channel is closed, the bitcoin in the channel will be divided between the two channel partners according to the latest balance encoded in the commitment transaction.
+    In the Lightning Network, the ability to send and receive payments is limited by channel balances.
+    See _capacity_.
 
 bech32::
-    A checksummed base32 address format, at most 90 characters long, and capable of error correction. It is native to segregated witness (BIP173). Also referred to as "bc1" because of the current starting values of each address. Transactions made using bech32 are smaller in most cases, and therefore, may only require a lower fee.
+    bech32 refers to a generic check-summed base32-encoded format featuring strong error-detection guarantees. While bech32 was originally developed to be used as the address format for native SegWit outputs (BIP-173), it is also used to encode lightning invoices (BOLT #11). While native SegWit version 0 outputs (P2WPKH and P2WSH) use bech32, higher native SegWit output versions (e.g., Pay-to-Taproot or P2TR) use the improved variant bech32m (BIP-350). bech32m addresses are sometimes referred to as "bc1" addresses,  reflecting the prefix of such addresses. Native SegWit outputs are more blockspace-efficient than older addresses and therefore may reduce transaction fees for the owner of such an address.
 
-bip, BIP::
-    Bitcoin Improvement Proposals. A set of proposals that members of the Bitcoin community have submitted to improve Bitcoin. For example, BIP-21 is a proposal to improve the Bitcoin uniform resource identifier (URI) scheme. BIPs can be found at https://github.com/bitcoin/bips.
+Bitcoin Improvement Proposal (BIP)::
+    A proposal that members of the Bitcoin community have submitted to improve Bitcoin. For example, BIP-21 is a proposal to improve the Bitcoin uniform resource identifier (URI) scheme. BIPs can be found at https://github.com/bitcoin/bips[GitHub].
 
 bitcoin, Bitcoin::
-    Depending on context, could refer to the name of the currency unit (the coin), the network or the underlying enabling protocol. Written as bitcoin with a lowercase "b" usually refers to the currency unit. Bitcoin with an uppercase "B" usually refers to the protocol. See https://www.bitcoin.org for general information. The source code can be found at https://github.com/bitcoin/bitcoin.
+    Depending on the context, could refer to the name of the currency unit (the coin), the network, or the underlying enabling protocol. Written as bitcoin with a lowercase "b" usually refers to the currency unit. Bitcoin with an uppercase "B" usually refers to the protocol or system.
 
 Bitcoin mining::
-    Bitcoin mining serves two purposes.
-    Firstly, Bitcoin transactions are validated and fixed in the sense that they cannot be double-spent.
-    Secondly, new bitcoin are being created within the coinbase of each block.
+    Bitcoin mining is the process of constructing a block from recent Bitcoin transactions and then solving a computational problem required as proof of work.
+    It is the process by which the shared bitcoin ledger (i.e., the Bitcoin blockchain) is updated and by which new transactions are included in the ledger.
+    It is also the process by which new bitcoin is issued.
+    Every time a new block is created, the mining node will receive new bitcoin created within the coinbase transaction of that block.
 
 block::
-    A grouping of transactions, marked with a timestamp, and a fingerprint of the previous block. The block header is hashed to produce a proof of work, thereby validating the transactions. Valid blocks are added to the main blockchain by network consensus.
+    A block is a data structure in the Bitcoin blockchain that consists of a header and a body of Bitcoin transactions.
+    The block is marked with a timestamp and commits to a specific predecessor (parent) block.
+    When hashed, the block header provides the proof of work that makes the blockchain probabilistically immutable.
+    Blocks must adhere to the rules enforced by network consensus to extend the blockchain.
+    When a block is appended to the blockchain, the included transactions are considered to have their first confirmation.
 
 blockchain::
-    The blockchain is an irreversible distributed database storing all Bitcoin transactions.
-    The irreversibility comes from the fact that each group of transactions, referred to as a block, is validated by solving a Proof of Work riddle and including the hash of the previous block.
-    Thus the data can only be changed by an entity providing more than 51% of the computational power of the Bitcoin network.
-    Blocks currently have a size limit of 1 MB.
-    New blocks have a statistical probability of being produced every ten minutes.
+    The blockchain is a distributed log, or database, of all Bitcoin transactions.
+    Transactions are grouped in discrete updates called blocks, limited up to 4 million weight units.
+    Blocks are produced approximately every 10 minutes via a stochastic process called mining.
+    Each block includes a computationally intensive "proof of work."
+    The proof of work requirement is used to regulate the block intervals and protect the blockchain against attacks to rewrite history:
+    an attacker would need to outdo existing proof of work to replace already published blocks, making each block probabilistically immutable as it is buried under subsequent blocks.
 
 BOLT::
-    BOLT, or Basics Of Lightning Technology, is the formal specification of the Lightning Network Protocol. It serves only as such without delving into implementation, unlike Bitcoin, in which both are one and the same. It is available in https://github.com/lightningnetwork/lightning-rfc.
-
-Breach Remedy Transaction::
-    A transaction claiming the outputs of a Revocable Sequence Maturity Contract with the help of the revocation key.
-    This can only happen if a channel partner was not following the protocol and tried to publish (willingly or due to a software bug) an old channel state.
+    BOLT, or Basis of Lightning Technology, is the formal specification of the Lightning Network. Unlike Bitcoin, which has a reference implementation that also serves as the protocol's specification, the various LN implementations follow BOLT so they can work with one another to form the same network. It is available at https://github.com/lightningnetwork/lightning-rfc[GitHub].
 
 capacity::
     The capacity of a payment channel is equivalent to the amount of bitcoin provided by the funding transaction.
-    As the funding transaction is publicly visible on the blockchain, and the channel is announced via the gossip protocol, the capacity is public information.
-    It does not reveal any information about which of the channel partners owns how much bitcoin in the channel.
-    A high capacity does not guarantee that the channel can be used for routing in both directions.
-    The capacity does not change if some of the capacity is locked up in HTLCs.
-    This means that even a high capacity channel could at any time be unable to route a payment of a certain amount in both directions.
-    It is not to be confused with the balance.
+    Because the funding transaction is publicly visible on the blockchain, and the channel is announced via the gossip protocol, the capacity is public information.
+    It does not reveal any information about how much bitcoin each of the channel partners owns in the channel, i.e., the balance.
+    A high capacity does not guarantee that the channel can be used for routing in both pass:[<span class="keep-together">directions</span>].
 
 c-lightning::
-    Implementation of the Lightning Network Protocol by the Victoria based company https://blockstream.com[Blockstream]. It is written in C. Source code is at https://github.com/ElementsProject/lightning.
+    Implementation of the LN Protocol by the Victoria-based company https://blockstream.com[Blockstream]. It is written in C. Source code is at https://github.com/ElementsProject/lightning[GitHub].
 
 closing transaction::
-    If both channel partners agree to close a channel they will create an exercise settlement transaction that reflects the most recent commitment transaction.
-    It does not include any Hashed Time Lock Contracts or Revocable Sequence Maturity Contracts.
-    After exchanging signatures for a closing transaction no further channel updates should be made, as this one allows one side to enforce the closing transaction on the blockchain.
-    Mutually closing a channel with the help of a closing transaction has the advantage that fewer blockchain transactions are required to claim all funds, in comparison to unilaterally forcing a channel close by publishing a commitment transaction. Additionally, funds are for both parties immediately spendable from a closing transaction.
+    If both channel partners agree to close a channel, they will create a settlement transaction that reflects the most recent commitment transaction.
+    After exchanging signatures for a closing transaction, no further channel updates should be made.
+    Mutually closing a channel with the help of a closing transaction has the advantage that fewer blockchain transactions are required to claim all funds, in comparison to unilaterally forcing a channel close by publishing a commitment transaction. Additionally, funds for both parties are immediately spendable from a closing transaction.
+
+CLTV::
+    CLTV is an acronym/abbreviation for the Bitcoin Script operator OP_CHECKLOCKTIMEVERIFY. This defines an absolute blockheight before an output can be spent. The atomicity of the routing process heavily depends on CLTV values in HTLCs. Routing nodes announce, via the gossip protocol, their expected CLTV expiry deltas that they wish for any incoming and outgoing HTLCs.
 
 coinbase::
-    A special field used as the sole input for coinbase transactions. The coinbase allows claiming the block reward and provides up to 100 bytes for arbitrary data.
-    The block reward consists of two things.
-    First, newly generated coins. The amount of allowed coins to be generated is part of the consensus rules and decreases over time based on the current block height.
-    Second, the miner is also allowed to add all the fees of the transactions from the current block to the coinbase.
+    The coinbase is a special field only permitted in the sole input of coinbase transactions.
+    The coinbase allows up 100 bytes of arbitrary data, but since BIP-34, it must first feature the current block height to ensure that coinbase transactions are unique.
     Not to be confused with coinbase transaction.
 
 coinbase transaction::
-    The first transaction in a block. Always created by a miner, it includes a single coinbase.
-    Not to be confused with coinbase.
+    The first transaction in a block which is always created by a miner and which includes a single coinbase.
+    The coinbase transaction may claim the block reward and assign it to one or more outputs.
+    The block reward consists of the block subsidy (newly created bitcoin) and the sum of all transaction fees from transactions included in the block.
+    Coinbase outputs can only be spent after maturing for 100 blocks.
+    If the block includes any SegWit transactions, the coinbase transaction must include a commitment to the witness transaction identifiers in an additional output.
 
 cold storage::
-    Refers to keeping a reserve of bitcoin offline. Cold storage is achieved when Bitcoin private keys are created and stored in a secure offline environment. Cold storage is important for anyone with bitcoin holdings. Online computers are vulnerable to hackers and should not be used to store a significant amount of bitcoin.
+    Refers to keeping an amount of bitcoin offline. Cold storage is achieved when Bitcoin private keys are created and stored in a secure offline environment. Cold storage is important to protect bitcoin holdings. Online computers are vulnerable to hackers and should not be used to store a significant amount of bitcoin.
 
-Commitment Transaction::
-    Commitment Transactions encode the balance of the payment channel with the help of one output for each channel partner by spending the funding transaction.
-    When payments are being made or forwarded through the channel, a double-spend of the commitment transactions is made by creating a new pair of commitment transactions.
-    One output also holds a Revocable Sequence Maturity Contract which is made to disincentivize a channel partner to broadcast an old commitment transaction to the Bitcoin network.
-    This effectively invalidates old commitment transactions.
-    Broadcasting a commitment transaction forces an unilateral channel close.
-    Up to 483 Hashed Time Lock Contracts can be stored as additional outputs in the commitment transaction to allow the routing of payments.
-    In order to be able to ascribe blame in the case of unilateral channel closes, each channel partner has a slightly different commitment transaction.
-    // TODO probably don't explain the difference with the RSMC here
-
-computationally easy::
-    A problem is considered to be computationally easy if there exists an algorithm that is able to compute the solution to the problem rather quickly.
-
-computationally hard::
-    A problem is considered to be computationally hard if no algorithm exists or is known that is able to compute the solution to the problem rather quickly.
+commitment transaction::
+     A commitment transaction is a Bitcoin transaction, signed by both channel partners, that encodes the latest balance of a channel.
+     Every time a new payment is made or forwarded using the channel, the channel balance will update, and a new commitment transaction will be signed by both parties.
+     Importantly, in a channel between Alice and Bob, both Alice and Bob keep their own version of the commitment transaction, which is also signed by the other party.
+     At any point, the pass:[<span class="keep-together">channel</span>] can be closed by either Alice or Bob if they submit their commitment transaction to the Bitcoin blockchain.
+     Submitting an older (outdated) commitment transaction is considered _cheating_ (i.e., a protocol breach) in the Lightning Network and can be penalized by the other party, claiming all the funds in the channel for themselves, via a penalty transaction.
 
 confirmations::
-    Once a transaction is included in a block, it has one confirmation. As soon as _another_ block is mined on the same blockchain, the transaction has two confirmations, and so on. Six or more confirmations are considered sufficient proof that a transaction cannot be reversed.
+    Once a transaction is included in a block, it has one confirmation. As soon as _another_ block is mined on the blockchain, the transaction has two confirmations, and so on. Six or more confirmations are considered sufficient proof that a transaction cannot be reversed.
 
 contract::
-    A contract is a set of Bitcoin transactions which result together in a certain desired behavior.
-    Examples are RSMCs to create a trustless, bi-directional payment channel or HTLCs to create a mechanism which allows trustless forwarding of payments through third parties.
+    A contract is a set of Bitcoin transactions that together result in a certain desired behavior.
+    Examples are RSMCs to create a trustless, bidirectional payment channel, or HTLCs to create a mechanism that allows trustless forwarding of payments through third parties.
 
-Diffie Hellman Key Exchange::
-    On the Lightning Network, the Elliptic Curve Diffie-Hellman method is used.
-    It is an anonymous key agreement protocol that allows two parties, each having an elliptic-curve public-private key pair, to establish a shared secret over an insecure communication channel.
+Diffie–Hellman Key Exchange (DHKE)::
+    On the Lightning Network, the Elliptic Curve Diffie–Hellman (ECDH) method is used.
+    It is an anonymous key agreement protocol that allows two parties, each having an elliptic curve public-private key pair, to establish a shared secret over an insecure communication channel.
     This shared secret may be directly used as a key, or to derive another key.
     The key, or the derived key, can then be used to encrypt subsequent communications using a symmetric-key cipher.
-    An example of the derived key would be the shared secrete between the ephemeral session key of a sender of an onion with the nodes public key of a hop of the onion as described and used by the SPHINX Mix Format.
-    Via https://en.wikipedia.org/w/index.php?title=Elliptic-curve_Diffie%E2%80%93Hellman&oldid=836070673
+    An example of the derived key would be the shared secret between the ephemeral session key of a sender of an onion with the node's public key of a hop of the onion, as described and used by the SPHINX Mix Format.
 
 digital signature::
-    A digital signature is a mathematical scheme for verifying the authenticity of digital messages or documents.
-    A valid digital signature gives a recipient reason to believe that the message was created by a known sender, that the sender cannot deny having sent the message, and that the message was not altered in transit.
-    They can be seen as cryptographic commitments in which the message is not hidden.
-    https://en.wikipedia.org/w/index.php?title=Digital_signature&oldid=876680165
+    A digital signature is a mathematical scheme for verifying the authenticity and integrity of digital messages or documents.
+    It can be seen as a cryptographic commitment in which the message is not hidden.
 
 double-spending::
     Double-spending is the result of successfully spending some money more than once.
-    Bitcoin protects against double-spending by verifying each transaction added to the blockchain plays by the rules that the inputs for the transaction have not previously already been spent.
-    The Revocable Sequence Maturity Contracts used to construct payment channels heavily attempt to double-spend bitcoin.
+    Bitcoin protects against double-spending by verifying that each transaction added to the blockchain adheres to the rules of consensus; this means checking that the inputs for the transaction have not been previously spent.
 
-downstream payment::
-    TBD.
-
-ECDSA::
-    Elliptic Curve Digital Signature Algorithm or ECDSA is a cryptographic algorithm used by Bitcoin to ensure that funds can only be spent by their rightful owners.
+Elliptic Curve Digital Signature Algorithm (ECDSA)::
+    Elliptic Curve Digital Signature Algorithm or ECDSA is a cryptographic algorithm used by Bitcoin to ensure that funds can only be spent by the holder of the correct private key.
 
 Eclair::
-    Implementation of the Lightning Network Protocol by the Paris based company https://acinq.co[ACINQ]. It is written in Scala. Source code is at https://github.com/ACINQ/eclair.
+    Implementation of the LN Protocol by the Paris-based company https://acinq.co[ACINQ]. It is written in Scala. Source code is at https://github.com/ACINQ/eclair[GitHub].
 
 encoding::
-    Encoding is the process of converting a message into a different form.
-    For example, converting a human-readable form to a digitally space-efficient form.
+    Encoding is the process of converting a message into a different form. For example, converting a number from decimal to a hexadecimal.
 
 Electrum server::
-    An Electrum server is a bitcoin node with an additional interface (API) is often required by bitcoin wallets that do not run a full node. For example, these wallets check the status of specific transactions or broadcasts transactions to the mempool using Electrum server APIs. Some Lightning wallets also use Electrum servers, so even if they are non-custodial, they may compromise user sovereignty in that users trust the Electrum server to provide accurate information and privacy in that calls made to the Electrum server may reveal private information.
+    An Electrum server is a Bitcoin node with an additional interface (API). It is often required by bitcoin wallets that do not run a full node. For example, these wallets check the status of specific transactions or broadcast transactions to the mempool using Electrum server APIs. Some Lightning wallets also use Electrum servers.
 
 ephemeral key::
-    Ephemeral keys are mainly within the SPHINX Mix Format and Onion Routing on the Lightning Network.
-    They are generated for each execution of the routing process.
+    Ephemeral keys are keys that are only used for a short time and not retained after use. They are often derived for use in one session from another key that is held long-term. Ephemeral keys are mainly used within the SPHINX Mix Format and onion routing on the Lightning Network.
     This increases the security of transported messages or payments.
-    Even if an ephemeral key leaks, only information about a single payment becomes public.
+    Even if an ephemeral key leaks, only information about a single session becomes public.
+
+feature bits::
+    A binary string that Lightning nodes use to communicate to each other which features they support.
+    Feature bits are included in many Lightning messages as well as BOLT #11.
+    They can be decoded using BOLT #9, and will tell nodes which features the node has enabled, and whether these are backward compatible.
+    Also known as feature flags.
 
 fees::
-    The sender of a transaction often includes a fee to the network for processing the requested transaction.
-    Not to be confused with a routing fee for payments on the Lightning Network.
-    Nodes on the Lightning Network are allowed to take a routing fee for forwarding payments.
-    The routing fee is the sum of a fixed _base_fee_ and a _fee_rate_ which depends on the payment amount.
+    In the context of the Lightning Network, nodes will charge routing fees for forwarding other users' payments.
+    Individual nodes can set their own fee policies which will be calculated as the sum of a fixed +base_fee+ and a +fee_rate+ that depends on the payment amount.
+    In the context of Bitcoin, the sender of a transaction pays a transaction fee to miners for including the transaction in a block.
+    Bitcoin transaction fees do not include a base fee and depend linearly on the weight of the transaction, but not on the amount.
 
 funding transaction::
-    The funding transaction is used to open a payment channel.
-    From the perspective of the Bitcoin network, the process of opening a channel by creating a RSMC is started by creating the funding transaction and finished by broadcasting it to the Bitcoin network and have it included in the blockchain.
-    The value of the funding transaction is exactly the capacity of the payment channel.
-    The output of the funding transaction is a 2-out-of-2 multisignature script (multisig) where each channel partner controls one key.
-    It is supposed to be spent by the commitment transactions or by the closing transaction.
-    Due to its multisig nature, it can only be spent mutually.
-    It is part of the RSMC to ensure that either side of the channel can withdraw their funds without the necessity to trust the channel partner.
+    The funding transaction is used to open a payment channel. The value (in bitcoin) of the funding transaction is exactly the capacity of the payment channel.
+    The output of the funding transaction is a 2-of-2 multisignature script (multisig) where each channel partner controls one key. Due to its multisig nature, it can only be spent by mutual agreement between the channel partners.
+    It will eventually be spent by one of the commitment transactions or by the closing transaction.
 
-globalfeatures::
-    Globalfeatures of a Lightning Network node are the features of interest for all other nodes.
+global features (+globalfeatures+ field)::
+    Global features of a Lightning node are the features of interest for all other nodes.
     Most commonly they are related to supported routing formats.
-    They are announced in the `_init_` message of the peer protocol as well as the `_channel_announcement_` and `_node_announcement_` messages of the gossip protocol.
+    They are announced in the `init` message of the peer protocol as well as the `channel_announcement` and `node_announcement` messages of the gossip protocol.
 
-Gossip Protocol::
-    Lightning Network nodes send and receive information about the topology of the Lightning Network through gossip messages which are exchanged with their peers.
-    The gossip protocol is mainly defined in BOLT 7 and defines the format of the _node_announcement_, _channel_announcement_ and _channel_update messages_.
-    In order to prevent SPAM, node announcement messages will only be forwarded if the node already has a channel and channel announcement messages will only be forwarded if the funding transaction of the channel has been confirmed by the Bitcoin network.
-    Usually Lightning nodes connect with their channel partners, but it is fine to connect with any other Lightning node in order to process gossip messages.
+gossip protocol::
+    LN nodes send and receive information about the topology of the Lightning Network through gossip messages which are exchanged with their peers.
+    The gossip protocol is mainly defined in BOLT #7 and defines the format of the `node_announcement`, `channel_announcement`, and `channel_update` messages.
+    To prevent spam, node announcement messages will only be forwarded if the node already has a channel, and channel announcement messages will only be forwarded if the funding transaction of the channel has been confirmed by the Bitcoin network.
+    Usually, Lightning nodes connect with their channel partners, but it is fine to connect with any other Lightning node to process gossip messages.
 
 hardware wallet::
     A hardware wallet is a special type of Bitcoin wallet which stores the user's private keys in a secure hardware device.
-    Currently, hardware wallets are not available for Lightning Network nodes as they need to be online to participate in the protocol.
-    Several groups are currently working on solutions.
+    As of writing the book, hardware wallets are not available for LN nodes because the keys used by Lightning need to be online to participate in the protocol.
 
 hash::
-    A digital fingerprint of some binary input.
+    A fixed-size digital fingerprint of some arbitrary-length binary input. Also known as a _digest_.
+
+hash-based message authentication code (HMAC)::
+    HMAC is an algorithm for verifying the integrity and authenticity of a message based on a hash function and a cryptographic key.
+    It is used in onion routing to ensure the integrity of a packet at each hop, as well as within the Noise Protocol variant used for message encryption.
 
 hash function::
-    A cryptographic hash function is a mathematical algorithm that maps data of arbitrary size to a bit string of a fixed size (a hash) and is designed to be a one-way function, that is, a function which is infeasible to invert.
-    The only way to recreate the input data from an ideal cryptographic hash function's output is to attempt a brute-force search of possible inputs to see if they produce a match, or use a rainbow table of matched hashes.
-    The ideal cryptographic hash function has five main properties: It is deterministic so the same message always results in the same hash.
-    It is quick to compute the hash value for any given message.
-    It is infeasible to generate a message from its hash value except by trying all possible messages.
-    A small change to a message should change the hash value so extensively that the new hash value appears uncorrelated with the old hash value.
-    It is infeasible to find two different messages with the same hash value.
-    https://en.wikipedia.org/w/index.php?title=Cryptographic_hash_function&oldid=868055371
+    A cryptographic hash function is a mathematical algorithm that maps data of arbitrary size to a bit string of a fixed size (a hash) and is designed to be a one-way function, that is, a function that is infeasible to invert.
+    The only way to recreate the input data from an ideal cryptographic hash function's output is to attempt a brute-force search of possible inputs to see if they produce a match.
 
-hashlocks::
-    A hashlock is a type of encumbrance that restricts the spending of an output until a specified piece of data is publicly revealed. Hashlocks have the useful property that once any hashlock is opened publicly, any other hashlock secured using the same key can also be opened. This makes it possible to create multiple outputs that are all encumbered by the same hashlock and which all become spendable at the same time.
+hashlock::
+    A hashlock is a Bitcoin Script spending condition that restricts the spending of an output until a specified piece of data is revealed. Hashlocks have the useful property that once any hashlock is revealed through spending, any other hashlocks secured using the same key can also be spent. This makes it possible to create multiple outputs that are all encumbered by the same hashlock and which all become spendable at the same time.
 
-HODL/Hold Invoices::
-    HODL/Hold invoices are effectively standard HTLC LN invoices with the exception that the recipient can “hold” the funds, deferring to settle the transaction until some condition has been met. The sender remains committed unless the recipient opts to cancel the transaction.
-
-HTLC::
-    A Hashed TimeLock Contract or HTLC is a class of payments that use hashlocks and timelocks to require that the receiver of a payment either acknowledges receiving the payment prior to a deadline by generating cryptographic proof of payment (usually called the preimage of the payment hash) or forfeits the ability to claim the payment, returning it to the payer.
-    On the Lightning Network HTLCs are outputs in the commitment transaction of a payment channel and are used to enable the trustless routing of payments.
+hash time-locked contract (HTLC)::
+    A hash time-locked contract (HTLC) is a Bitcoin Script that consists of hashlocks and timelocks to require that the recipient of a payment either spends the payment prior to a deadline by presenting the hash preimage or the sender can claim a refund after the timelock expires.
+    On the Lightning Network, HTLCs are outputs in the commitment transaction of a payment channel and are used to enable the trustless routing of payments.
 
 invoice::
-    The payment process on the Lightning Network is initiated by the payee who issues an invoice.
-    Invoices include the payment hash, the amount, a description and the expiry time.
+    The payment process on the Lightning Network is initiated by the recipient (payee) who issues an invoice, also known as a _payment request_.
+    Invoices include the payment hash, the amount, a description, and the expiry time. Lightning invoices are defined in BOLT #11.
     Invoices can also include a fallback Bitcoin address to which the payment can be made in case no route can be found, as well as hints for routing a payment through a private channel.
 
+just-in-time (JIT) routing::
+   Just-in-time (JIT) routing is an alternative to source-based routing that was first pass:[<span class="keep-together">proposed</span>] by coauthor René Pickhardt.
+   With JIT routing, intermediary nodes along a path can pause an in-flight payment to rebalance their channels before proceeding with the payment.
+   This might allow them to successfully forward payments that might otherwise have failed due to a lack of outgoing capacity.
+
 Lightning message::
-   A Lightning message is an encrypted data string that can be sent between two peers on the Lightning Network. Similar to other communication protocols Lightning messages consist of a header and a body. The header and the body have their own HMAC. This ensures that the headers of fixed length will also be encrypted and adversaries won't be able to figure out what messages are being sent by inspecting the length.
+   A Lightning message is an encrypted data string that can be sent between two peers on the Lightning Network. Similar to other communication protocols, Lightning messages consist of a header and a body. The header and the body have their own HMAC. Lightning messages are the main building block of the messaging layer.
 
-Lightning Network, Lightning Network Protocol, Lightning Protocol::
+Lightning Network, Lightning Network Protocol, pass:[<span class="keep-together">Lightning Protocol</span>]::
    The Lightning Network is a protocol on top of Bitcoin (or other cryptocurrencies).
-   It creates a network of payment channels which enables the trustless forwarding of payments through the network with the help of HTLCs and Onion Routing.
+   It creates a network of payment channels which enables the trustless forwarding of payments through the network with the help of HTLCs and onion routing.
    Other components of the Lightning Network are the gossip protocol, the transport layer, and payment requests.
-   The source code is available at https://github.com/lightningnetwork.
 
-Lightning Network Node, Lightning Node, node::
-    TBD.
+Lightning Network protocol suite::
+   The Lightning Network protocol suite consists of five layers that are responsible for various parts of the protocol.
+   From bottom (the first layer) to the top (the fifth layer), these layers are called the network communication layer, the messaging layer, the peer-to-peer layer, the routing layer, and the payment layer.
+   Various BOLTs define parts of one or several layers.
+
+Lightning Network node, Lightning node::
+    A computer participating in the Lightning Network, via the Lightning peer-to-peer protocol.
+    Lightning nodes have the ability to open channels with other nodes, send and receive payments, and route payments from other users.
+    Typically, a Lightning node user will also run a Bitcoin node.
 
 lnd::
-    Implementation of the Lightning Network Protocol by the San Francisco based company https://lightning.engineering[Lightning Labs].
-    It is written in Go. Source code is at https://github.com/lightningnetwork/lnd.
+    Implementation of the LN Protocol by the San Francisco-based company https://lightning.engineering[Lightning Labs].
+    It is written in Go. Source code is at https://github.com/lightningnetwork/lnd[GitHub].
 
-localfeatures::
-    Localfeatures of a Lightning Network node are the configurable features of direct interest of the peer.
-    They are announced in the `_init_` message of the peer protocol as well as the `_channel_announcement_` and `_node_announcement_` messages of the gossip protocol.
+local features (field: +localfeatures+)::
+    Local features of an LN node are the configurable features of direct interest to its peers.
+    They are announced in the `init` message of the peer protocol as well as in the `channel_announcement` and `node_announcement` messages of the gossip protocol.
+
+locktime::
+    Locktime, or more technically nLockTime, is the part of a Bitcoin transaction that indicates the earliest time or earliest block when that transaction may be added to the blockchain.
+
+messaging layer::
+    The messaging layer builds on top of the network connection layer of the Lightning Network protocol suite.
+    It is responsible for ensuring an encrypted and secure communication and exchange of information via the chosen network connection layer protocol.
+    The messaging layer defines the framing and format of Lightning Messages as defined in BOLT #1.
+    The feature bits defined in BOLT #9 are also part of this layer.
 
-Locktime::
-    Locktime, or more technically nLockTime, is the part of a transaction which indicates the earliest time or earliest block when that transaction may be added to the blockchain.
 
 millisatoshi::
-    The smallest unit of account on the Lightning Network. A millisatoshi is one hundred billionth of a single bitcoin. A millisatoshi is one thousandth of one Satoshi. Millisatoshis do not exist, nor can they be settled on the Bitcoin network.
+    The smallest unit of account on the Lightning Network. A millisatoshi is one hundred billionth of a single bitcoin. A millisatoshi is one thousandth of one satoshi. Millisatoshis do not exist on, nor can they be settled on, the Bitcoin network.
+
+multipart payments (MPP)::
+ 	Multipart payments (MPP), often also referred to as multipath payments, are a method for splitting the payment amount into multiple smaller parts and delivering them along one or more paths. Since MPP can send many or all parts over a single path, the term multipart payment is more accurate than multipath payment. In computer science, multipart payments are modeled as network flows.
 
 multisignature::
-    Multisignature (multisig) refers to requiring more than one key to authorize a Bitcoin transaction.
-    Payment channels are always encoded as multisignature addresses requiring one signature from each peer of the payment channel.
-    In the standard case of a two-party payment channel a 2-2 multisignature address is used.
-
-Neutrino::
-    Neutrino is a later alternative to SPV that also verifies whether certain transactions are contained in a block without downloading the entire block. However, it offers a number of improvements over SPV: Neutrino does not transmit any information that would allow a third party to determine users’ identities, it facilitates the use of non-custodial apps, and it reduces the computational load on full nodes. The trade-off for these improvements is that Neutrino requires more data from the full node than SPV.
+    Multisignature (multisig) refers to a script that requires more than one signature to authorize spending.
+    Payment channels are always encoded as multisig addresses requiring one signature from each partner of the payment channel.
+    In the standard case of a two-party payment channel, a 2-of-2 multisig address is used.
 
 node::
-    See Lightning Network Node
+    See _Lightning Network node_.
 
 network capacity::
-    Lightning network capacity is the value of bitcoin locked and circulated inside Lightning Network and is the sum of capacities of each channel.
-    It is a mesurement of the maximum value a user can transfer in Lightning Network because routing nodes will need to have sufficient balances. It also reflects the usage of Lightning Network to some extent, because the higher value is circulated inside Lightning Network the more likely that more people are using it.
+    LN capacity is the total amount of bitcoin locked and circulated inside the Lightning Network.
+    It is the sum of capacities of each public channel.
+    It reflects the usage of the Lightning Network to some extent because we expect that people put bitcoin into Lightning channels to spend it or forward other users' payments.
+    Hence the higher the amount of bitcoin in Lightning channels, the higher the expected usage of the Lightning Network.
+    Note that since only public channel capacity can be observed, the true network capacity is unknown. Also, since a channel's capacity can enable an unlimited number of payments back and forth, network capacity does not imply a limit of value transferred on the Lightning Network.
+
+network connection layer::
+    The lowest layer of the Lightning Network protocol suite.
+    Its responsibility is to support internet protocols like IPv4, IPv6, TOR2, and TOR3, and use them to establish a secure cryptographic communication channel as defined in BOLT #8, or to speak DNS for the bootstrapping of the network as defined in BOLT #10.
 
 Noise_XK::
-    The template of the Noise protocol framework to establish an authenticated and encrypted communication channel between two peers of the Lightning Network.
+    The template of the Noise Protocol Framework to establish an authenticated and encrypted communication channel between two peers of the Lightning Network.
     X means that no public key needs to be known from the initiator of the connection.
     K means that the public key of the receiver needs to be known.
-    More particular (from: http://www.noiseprotocol.org/noise.html) the protocol enables.
-    Encryption to a known recipient, strong forward secrecy. This payload is encrypted based on an ephemeral-ephemeral DH as well as an ephemeral-static DH with the recipient's static key pair. Assuming the ephemeral private keys are secure, and the recipient is not being actively impersonated by an attacker that has stolen its static private key, this payload cannot be decrypted. Sender authentication resistant to key-compromise impersonation (KCI). The sender authentication is based on an ephemeral-static DH ("es" or "se") between the sender's static key pair and the recipient's ephemeral key pair. Assuming the corresponding private keys are secure, this authentication cannot be forged.
-    // the noise protocol documentation is according to their IPR section public domain. The author is Trevor Perrin (noise@trevp.net)
 
 onion routing::
     Onion routing is a technique for anonymous communication over a computer network.
@@ -295,183 +271,191 @@ onion routing::
     The encrypted data is transmitted through a series of network nodes called onion routers, each of which peels away a single layer, uncovering the data's next destination.
     When the final layer is decrypted, the message arrives at its destination.
     The sender remains anonymous because each intermediary knows only the location of the immediately preceding and following nodes.
-    With the SPHINX Mix Format, the final destination also remains anonymous as only the previous router could see it but does not know if they are routing it to the final node or just the next hop.
-    https://en.wikipedia.org/w/index.php?title=Onion_routing&oldid=870849217
 
 output::
-    Output, transaction output, or TxOut is an output in a transaction which contains two fields: a value field for transferring zero or more satoshis and a pubkey script for indicating what conditions must be fulfilled for those satoshis to be further spent.
+    The output of a Bitcoin transaction, also called an unspent transaction output (UTXO).
+    An output is an indivisible amount of bitcoin that can be spent, as well as a script that defines what conditions need to be fulfilled for that bitcoin to be spent.
+    Every bitcoin transaction consumes some outputs of previously recorded transactions and creates new outputs that can be spent later by subsequent transactions.
+    A typical bitcoin output will require a signature to be spent, but outputs can require the fulfillment of more complex scripts.
+    For example, a multisignature script requires two or more key holders sign before the output can be spent, which is a fundamental building block of the Lightning Network.
 
-P2PKH::
-    P2PKH or Pay-to-PubKey-Hash is a type of transaction that pays a Bitcoin address that contain P2PKH scripts.
-    An output locked by a P2PKH script can be unlocked (spent) by presenting a public key and a digital signature created by the corresponding private key.
+Pay-to-Public-Key-Hash (P2PKH)::
+    P2PKH is a type of output that locks bitcoin to the hash of a public key. An output locked by a P2PKH script can be unlocked (spent) by presenting the public key matching the hash and a digital signature created by the corresponding private key.
 
-P2SH::
-    P2SH or Pay-to-Script-Hash is a powerful type of transaction that greatly simplifies the use of complex transaction scripts. With P2SH the complex script that details the conditions for spending the output (redeem script) is not presented in the locking script. Instead, only a hash of it is in the locking script.
+Pay-to-Script-Hash (P2SH)::
+    P2SH is a versatile type of output that  allows the use of complex Bitcoin Scripts. With P2SH, the complex script that details the conditions for spending the output (redeem script) is not presented in the locking script. Instead, value is locked to the hash of a script, which must be presented and fulfilled to spend the output.
 
 P2SH address::
-    P2SH addresses are Base58Check encodings of the 20-byte hash of a script, P2SH addresses use the version prefix "5", which results in Base58Check-encoded addresses that start with a "3". P2SH addresses hide all of the complexity, so that the person making a payment does not see the script.
+    P2SH addresses are Base58Check encodings of the 20-byte hash of a script. P2SH addresses start with a "3." P2SH addresses hide all of the complexity, so that the sender of a payment does not see the script.
 
-P2WPKH::
-    The signature of a P2WPKH (Pay-to-Witness-Public-Key-Hash) contains the same information as a P2PKH spending, but is located in the witness field instead of the scriptSig field. The scriptPubKey is also modified.
+Pay-to-Witness-Public-Key-Hash (P2WPKH)::
+	P2WPKH is the SegWit equivalent of P2PKH, using a segregated witness. The signature to spend a P2WPKH output is put in the witness tree instead of the ScriptSig field. See _SegWit_.
 
-P2WSH::
-    The difference between P2SH and P2WSH (Pay-to-Witness-Script-Hash) is about the cryptographic proof location change from the scriptSig field to the witness field and the scriptPubKey that is also modified.
+P2WPKH address::
+	The "native SegWit v0" address format, P2WPKH addresses are bech32-encoded and start with "bc1q".
+
+Pay-to-Witness-Script-Hash (P2WSH)::
+    P2WSH is the SegWit equivalent of P2SH, using a segregated witness. The signature and script to spend a P2WSH output is put in the witness tree instead of the ScriptSig field. See _SegWit_.
+
+P2WSH address::
+	The "native Segwi v0" script address format, P2WSH addresses are bech32-encoded and start with "bc1q".
+
+Pay-to-Taproot (P2TR)::
+	Activating in November 2021, Taproot is a new output type that locks bitcoin to a tree of spending conditions, or a single root condition.
+
+P2TR address::
+	The Taproot address format, representing SegWit v1, is a bech32m-encoded address and starts with "bc1p".
 
 payment::
-    A payment occurs if we transfer bitcoin within the Lightning Network.
-    Payments are generally not seen on the blockchain.
-    The recipient initiates a payment by creating an invoice.
-    The invoice includes a payment hash which is the hash of a secret preimage.
-    This payment hash is used by the Hashed Time Lock Contracts during the routing process.
+    A Lightning payment occurs when bitcoin is transferred within the Lightning Network. Payments are generally not seen on the Bitcoin blockchain.
 
-payment channels::
-    A micropayment channel or payment channel is a class of techniques designed to allow users to make multiple Bitcoin transactions without committing all of the transactions to the Bitcoin blockchain. In a typical payment channel, only two transactions are added to the blockchain, but an unlimited or nearly unlimited number of payments can be made between the participants.
 
 payment channel::
-    Payment channels are the core building blocks of the Lightning Network.
-    They can be used to send a predefined amount of bitcoin back and forth between two parties.
-    Sending bitcoin over a payment channel happens off chain, so only the funding transaction and either the commitment or closing transaction is stored in the blockchain.
-    There are currently three methods known to construct a fully duplex bidirectional payment channel.
-    Christian Decker proposed a method in his PhD thesis based on invalidation trees.
-    In the Lightning Network whitepaper, Joseph Poon and Tadge Dryja describe the Revocable Sequence Maturity Contract based method that is currently being implemented on the Lightning Network.
-    Recently Christian Decker et al. came up with the Eltoo mechanism which would require a Bitcoin softfork.
+    A payment channel is a financial relationship between two nodes on the Lightning Network, created using a bitcoin transaction paying a multisignature address.
+    The channel partners can use the channel to send bitcoin back and forth between each other without committing all of the transactions to the Bitcoin blockchain.
+    In a typical payment channel only two transactions, the funding transaction and the commitment transaction, are added to the blockchain.
+    Payments sent across the channel are not recorded in the blockchain and are said to occur "off-chain."
+
+payment layer::
+    The top and fifth layer of the Lightning Network protocol suite operates on top of the routing layer.
+    Its responsibility is to enable the payment process via BOLT #11 invoices.
+    While it heavily uses the channel graph from the gossip protocol as defined in BOLT #7, the actual strategies to deliver a payment are not part of the specification of the protocol and are left to the implementations.
+    As this topic is very important to ensure reliability of the payment delivery process, we included it in this book.
 
 peer::
-    Two parties which form a payment channel are called peers.
-    In particular, they are connected via an encrypted, authenticated communication over a TCP Socket.
+    The participants in a peer-to-peer network. In the Lightning Network, peers connect to each other via encrypted, authenticated communication through a TCP socket, over IP or Tor.
 
-Penalty Transaction::
-    Look at the Breach Remedy Transaction.
+peer-to-peer layer::
+    The peer-to-peer layer is the third layer of the Lightning Network protocol suite and works on top of the messaging layer.
+    It is responsible for defining the syntax and semantics of information exchanged between peers via Lightning messages.
+    This consists of control messages as defined in BOLT #9; channel establishment, operation, and closing messages as defined in BOLT #2; as well as gossip and routing messages as defined in BOLT #7.
+
+private channel::
+    A channel not announced to the rest of the network.
+    Technically, "private" is a misnomer because these channels can still be identified through routing hints and commitment transactions.
+    They are better described as "unannounced" channels.
+	With an unannounced channel, the channel partners can send and receive payments between each other as normal.
+    However, the rest of the network will not be aware of the channel and so cannot typically use it to route payments.
+    Because the number and capacity of unannounced channels is unknown, the total public channel count and capacity only accounts for a portion of the total Lightning Network.
 
 preimage::
-    In mathematics, given a function $f$ and a value $h$ the preimage of $h$ with respect to $f$ is the set of values $R = \{r_1,r_2,...\}$ such that $f(r_i) = h$ for all $\r_i \in R$.
-    In layman's terms, it is the set of values which is mapped to $h$ by the function $f$.
-    This preimage set can be empty, finite or infinite.
-    In cryptography, the function $f$ is usually taken to be a hash function.
-    Cryptographers use the term preimage for an arbitrary element of $R$.
-    In particular, when using SHA-256 we should state that each element has an infinite amount of preimages.
-    Yet it is still believed to be computationally hard to find such a preimage.
-proof of work::
-    A piece of data that requires significant computation to find. In Bitcoin, miners must find a numeric solution to the SHA256 algorithm that meets a network-wide target, the difficulty target.
+	In the context of cryptography and specifically in the Lightning Network, the preimage refers to the input of a hash function that produces a specific hash. It is not feasible to compute the preimage from the hash (hash functions only go one way). By selecting a secret random value as a preimage and calculating its hash, we can commit to that preimage and later reveal it. Anyone can confirm that the revealed preimage correctly produces the hash.
 
-Relative Timelock::
-    Relative Timelock is a kind of timelock that allows an input to specify the earliest time it can be added to a block based on how long ago (which is relative) the output referred by that input was included in a block. Such a feature is jointly achieved by nSequence field and CheckSequenceVerify opcode, which are introduced by BIP68/112/113.
+Proof of Work (PoW)::
+    Data that requires significant computation to find, and can be easily verified by anyone to prove the amount of work that was required to produce it.
+    In Bitcoin, miners must find a numeric solution to the SHA-256 algorithm that meets a network-wide target, called the difficulty target.
+    See _Bitcoin mining_ for more information.
 
-Revocable Sequence Maturity Contract::
-    This contract is used to construct a payment channel between two Bitcoin or Lightning Network users who do not need to trust each other.
-    The name comes from a sequence of states which are encoded as commitment transactions and can be revoked if wrongfully published and mined by the Bitcoin network.
-    These contracts are commonly referred to as RSMCs.
-    Unlike a HTLC, whose timeout is to make a HTLC temporary, and therefore should be absolute; a RSMC timeout is meant to only start when a commitment transaction is mined, and therefore should be using a Relative Timelock.
+Point Time-Locked Contract (PTLC)::
+    A Point Time-Locked Contract (PTLC) is a Bitcoin script that allows a conditional spend either on the presentation of a secret or after a certain blockheight has passed, similar to an HTLC. Unlike HTLCs, PTLCs do not depend on a preimage of a hash function but rather on the private key from an elliptic curve point. The security assumption is thus based on the discrete logarithm. PTLCs are not yet implemented on the Lightning Network.
+
+relative timelock::
+    A relative timelock is a type of timelock that allows an input to specify the earliest time the input can be added to a block. The time is relative and is based on when the output referenced by that input was recorded in a block. Relative timelocks are set by the +nSequence+ transaction field and +CHECKSEQUENCEVERIFY+ (CSV) Bitcoin Script opcode, which was introduced by BIP-68/112/113.
+
+Revocable Sequence Maturity Contract (RSMC)::
+    This contract is used to construct a payment channel between two Bitcoin or LN users who do not need to trust each other.
+    The name comes from a sequence of states that are encoded as commitment transactions and can be revoked if wrongfully published and mined by the Bitcoin network.
 
 revocation key::
-    Each Revocable Sequence Maturity Contract contains two revocation keys.
-    Each channel partner knows one revocation key.
-    Knowing both revocation keys, the output of the Revocable Sequence Maturity Contract can be spent within the predefined timelock.
-    Revocation keys are used to disincentivize channel partners from broadcasting an old channel state.
-    While negotiating a new channel state the old revocation keys are being shared.
-    Revocation keys are used instead of signatures since they can be derived with an HD key derivation scheme.
-    This makes it less cumbersome to store all revocation keys of old states.
+    Every RSMC contains two revocation keys. Each channel partner knows one revocation key. Knowing both revocation keys, the output of the RSMC can be spent within the predefined timelock.  While negotiating a new channel state, the old revocation keys are shared, thereby "revoking" the old state.  Revocation keys are used to discourage channel partners from broadcasting an old channel state.
 
 RIPEMD-160::
-    RIPEMD-160 is a 160-bit cryptographic hash function. RIPEMD-160 is a strengthened version of RIPEMD with a 160-bit hash result, and is expected to be secure for the next ten years or more.
+    RIPEMD-160 is a cryptographic hash function that produces a 160-bit (20-byte) hash.
+
+routing layer::
+    The fourth layer of the Lightning Network protocol suite operates on top of the peer-to-peer layer.
+    Its responsibility is to define the cryptographic primitives and necessary communication protocol to allow the secure and atomic transport of bitcoin from a sending node to a recipient node.
+    While BOLT #4 defines the onion format that is used to communicate transport information to remote peers with whom no direct connections exist, the actual transport of the onions and cryptographic primitives are defined in BOLT #2.
 
 topology::
-    The topology of the Lightning Network describes the shape of the Lightning Network as a mathematical graph.
-    Nodes of the graph are the Lightning Network nodes or participants.
-    The edges of the graph are the payment channels.
-    The topology of the Lightning Network is publicly broadcast with the help of the gossip protocol unless nodes decide to act privately.
-    This means that the Lightning Network is probably larger than the announced number of nodes.
+    The topology of the Lightning Network describes the shape of the Lightning Network as a mathematical graph. Nodes of the graph are the Lightning nodes (network participants/peers). The edges of the graph are the payment channels.
+    The topology of the Lightning Network is publicly broadcast with the help of the gossip protocol, with the exception of unannounced channels.
+    This means that the Lightning Network may be significantly larger than the announced number of channels and nodes.
     Knowing the topology is of particular interest in the source-based routing process of payments in which the sender discovers a route.
-    Also, the topology is important for features like the autopilot.
 
 satoshi::
-    A satoshi is the smallest denomination of bitcoin that can be recorded on the blockchain. It is the equivalent of 0.00000001 bitcoin and is named after the creator of Bitcoin, Satoshi Nakamoto. ((("satoshi")))
+    A satoshi is the smallest unit (denomination) of bitcoin that can be recorded on the blockchain. One satoshi is 1/100 millionth (0.00000001) of a bitcoin and is named after the creator of Bitcoin, Satoshi Nakamoto.
 
 Satoshi Nakamoto::
-    Satoshi Nakamoto is the name used by the person or group of people who designed Bitcoin and created its original reference implementation, Bitcoin Core. As a part of the implementation, they also devised the first blockchain database. In the process, they were the first to solve the double-spending problem for digital currency. Their real identity remains unknown.
+    Satoshi Nakamoto is the name used by the person or group of people who designed Bitcoin and created its original reference implementation. As part of the implementation, they also devised the first blockchain database. In the process, they were the first to solve the double-spending problem for digital currency. Their real identity remains unknown.
 
-Script::
-    Bitcoin uses a scripting system for transactions. Forth-like, Script is simple, stack-based, and processed from left to right. It is purposefully not Turing-complete, with no loops.
+Schnorr signature::
+    A new digital signature scheme that will be activated in Bitcoin in November 2021. It enables innovations on the Lightning Network, such as efficient PTLCs (an improvement on HTLCs).
+
+script, Bitcoin Script::
+    Bitcoin uses a scripting system for transactions called Bitcoin Script. Resembling the Forth programming language, it is simple, stack-based, and processed from left to right. It is purposefully Turing-incomplete, without loops or recursion.
 
 ScriptPubKey (aka pubkey script)::
-    ScriptPubKey or pubkey script, is a script included in outputs which sets the conditions that must be fulfilled for those satoshis to be spent. Data for fulfilling the conditions can be provided in a signature script.
+    ScriptPubKey or pubkey script, is a script included in outputs which sets the conditions that must be fulfilled for those outputs to be spent. Data for fulfilling the conditions can be provided in a signature script. See also _ScriptSig_.
 
 ScriptSig (aka signature script)::
-    ScriptSig or signature script, is the data generated by a spender which is almost always used as variables to satisfy a pubkey script.
-
-Second stage HTLC::
-    TBD.
+    ScriptSig or signature script is the data generated by a spender, which are almost always used as variables to satisfy a pubkey script.
 
 secret key (aka private key)::
-    The secret number that unlocks bitcoin sent to the corresponding address. pass:[<span class="keep-together">A secret</span>] key looks like the following:
-+
-----
-5J76sF8L5jTtzE96r66Sf8cka9y44wdpJjMwCxR3tzLh3ibVPxh
-----
+    The secret number that unlocks bitcoin sent to the corresponding address. pass:[<span class="keep-together">A secret</span>] key looks like this: +5J76sF8L5j&#x200b;TtzE96r66Sf8cka9y44wdpJjMwCxR3tzLh3i&#x200b;bVPxh+.
 
-Segregated Witness::
-    Segregated Witness is an upgrade to the Bitcoin protocol, which technological innovation separates signature data from Bitcoin transactions. Segregated Witness was deployed as a soft fork; a change that technically makes Bitcoin’s protocol rules more restrictive.
+Segregated Witness (SegWit)::
+    Segregated Witness (SegWit) is an upgrade to the Bitcoin protocol introduced in 2017 that adds a new witness for signatures and other transaction authorization proofs. This new witness field is exempt from the calculation of the transaction ID, which solves most classes of third-party transaction malleability. Segregated Witness was deployed as a soft fork and is a change that technically makes Bitcoin’s protocol rules more restrictive.
 
-SHA::
-    The Secure Hash Algorithm or SHA is a family of cryptographic hash functions published by the National Institute of Standards and Technology (NIST).
+Secure Hash Algorithm (SHA)::
+    The Secure Hash Algorithm or SHA is a family of cryptographic hash functions published by the National Institute of Standards and Technology (NIST). The Bitcoin protocol currently uses SHA-256, which produces a 256-bit hash.
 
-short channel id (scid)::
-    Once a channel is established, the index of the funding transaction on the blockchain is used as the short channel id to uniquely identify the channel.
-    The short channel id consists of 8 bytes referring to 3 numbers.
-    In its serialized form it depicts these 3 numbers as decimal values separated by the letter **x**.
+short channel ID (scid)::
+    Once a channel is established, the index of the funding transaction on the blockchain is used as the short channel ID to uniquely identify the channel.
+    The short channel ID consists of eight bytes referring to three numbers.
+    In its serialized form, it depicts these three numbers as decimal values separated by the letter "x" (e.g., +600123x01x00+)
     The first number (4 bytes) is the block height.
     The second number (2 bytes) is the index of the funding transaction with the blocks.
     The last number (2 bytes) is the transaction output.
 
 simplified payment verification (SPV)::
-    SPV or simplified payment verification is a method for verifying particular transactions were included in a block without downloading the entire block. The method is used by some lightweight Bitcoin clients.
+    SPV or simplified payment verification is a method for verifying that particular transactions were included in a block without downloading the entire block. The method is used by some lightweight Bitcoin and Lightning wallets.
 
 source-based routing::
     On the Lightning Network, the sender of a payment decides the route of the payment.
     While this decreases the success rate of the routing process, it increases the privacy of payments.
     Due to the SPHINX Mix Format used by onion routing, all routing nodes do not know the originator of a payment or the final recipient.
-    Source-based routing is fundamentally different to how routing works on the Internet Protocol.
+    Source-based routing is fundamentally different from how routing works on the Internet Protocol.
 
 soft fork::
-    Soft fork, or Soft-Forking Change, is a temporary fork in the blockchain which commonly occurs when miners using non-upgraded nodes don't follow a new consensus rule their nodes don’t know about.
-    Not to be confused with fork, hard fork, software fork or Git fork.
+    Soft fork, or soft-forking change, is a protocol upgrade that's forward and backward compatible, so it allows both old nodes and new nodes to continue using the same chain.
 
 SPHINX Mix Format::
-    A particular technique for Onion Routing used in the Lightning Network and invented by George Danezis and Ian Goldberg in 2009.
-    With the SPHINX Mix Format, each message of the onion package is padded with some random data so that no single hop can estimate how far on the route they are.
+    A particular technique for onion routing used in the Lightning Network and invented by https://cypherpunks.ca/~iang/pubs/Sphinx_Oakland09.pdf[George Danezis and Ian Goldberg in 2009].
+    With the SPHINX Mix Format, each message of the onion package is padded with some random data so that no single hop can estimate how far along the route it has traveled.
     While the privacy of the sender and receiver of the payment is protected, each node is still able to return an error message along the path to the originator of the message.
-    The paper can be found at https://cypherpunks.ca/~iang/pubs/Sphinx_Oakland09.pdf
 
-Submarine Swaps::
-    Submarine Swaps enable transfers between on-chain addresses and off-chain locations, like the Lightning Network. Just as standard LN transfers chain payments by means of HTLCs that make the final claim on funds conditional on the recipient revealing a secret to all links in the chain, Submarine Swaps use the same logic and procedure to transfer funds across the on-chain/off-chain barrier with minimal trust. They can also be used to enable transfers from another chain, say Litecoin, to an off-chain LN address. Reverse Submarine Swaps allow bitcoin transfers in the opposite direction that is, from an off-chain LN location to an on-chain address.
+submarine swap::
+    A submarine swap is a trustless atomic swap between on-chain Bitcoin addresses and off-chain Lightning Network payments. Just as LN payments use HTLCs that make the final claim on funds conditional on the recipient revealing a secret (hash preimage), submarine swaps use the same mechanism to transfer funds across the on-chain/off-chain barrier with minimal trust. Reverse submarine swaps allow swaps in the opposite direction, from an off-chain LN payment to an on-chain Bitcoin address.
 
-timelocks::
-    A timelock is a type of encumbrance that restricts the spending of some bitcoin until a specified future time or block height. Timelocks feature prominently in many Bitcoin contracts, including payment channels and hashed timelock contracts.
+timelock::
+    A timelock is a type of encumbrance that restricts the spending of some bitcoin until a specified future time or block height. Timelocks feature prominently in many Bitcoin contracts, including payment channels and HTLCs.
 
 transaction::
-    Transactions are a binary format used by the Bitcoin protocol to transfer bitcoin from one address to another.
-    Several transactions are built into a block which has to be confirmed by the Bitcoin network through the process of mining.
-    Transactions can only be included in a block if they contain a valid signature (more precisely a valid input script) matching the output script defined by the previous owner.
-    The first transaction in each block is called the coinbase and generates new bitcoin.
-    Transactions can also contain contracts and should not be confused with payments.
+    Transactions are data structures used by Bitcoin to transfer bitcoin from one address to another.
+    Several thousand transactions are aggregated in a block, which is then recorded (mined) on the blockchain.
+    The first transaction in each block, called the coinbase transaction, generates new bitcoin.
 
 transaction malleability::
-    Transaction malleability is a property that hash of a transaction can change without changing the semantic of the transaction (the UTXOs it is spending, the destinations and the corresponding amounts).
-    For example, altering the signature can change the hash of a transaction, because of the non-deterministism of ECDSA signing.
-    A commitment transaction needs the hash of a funding transaction, if the hash of the funding transaction changes, transactions depending on it will become invalid. This will make users unable to claim the refunds if there is.
-    Segregated Witness soft fork addresses this issue and therefore is an important upgrade to support Lightning Network.
+    Transaction malleability is a property that the hash of a transaction can change without changing the semantics of the transaction.
+    For example, altering the signature can change the hash of a transaction.
+    A commitment transaction needs the hash of a funding transaction, and if the hash of the funding transaction changes, transactions depending on it will become invalid. This will make users unable to claim the refunds if there are any.
+    The Segregated Witness soft fork addresses this issue and was therefore an important upgrade to support the Lightning Network.
 
 transport layer::
-    In computer networking, the transport layer is a conceptual division of methods of a model of how computers talk to each other.
-    The transport layer provides communication services between computers such as flow control, verification, and multiplexing (to allow multiple applications to work on a computer at the same time).
+    In computer networking, the transport layer is a conceptual division of the methods used by computers (and ultimately applications) to talk to each other.
+    The transport layer provides communication services between computers, such as flow control, verification, and multiplexing (to allow multiple applications to work on a computer at the same time).
 
 unspent transaction output (UTXO)::
-    UTXO is an unspent transaction output that can be spent as an input to a new transaction.
-
-upstream payment::
-    TBD.
+    See _output_.
 
 wallet::
-    Software that holds all your Bitcoin addresses and secret keys. Use it to send, receive, and store your bitcoin.
+    A wallet is a piece of software that holds Bitcoin private keys. It is used to create and sign Bitcoin transactions. In the context of the Lightning Network, it also holds revocation secrets of old channel state and the latest presigned commitment transactions.
 
-Some contributed definitions have been sourced under a CC-BY license from the https://en.bitcoin.it/wiki/Main_Page[Bitcoin Wiki], https://en.wikipedia.org[Wikipedia], https://github.com/bitcoinbook/bitconbook[Mastering Bitcoin] or from other open source documentation sources.
+watchtower::
+    Watchtowers are a security service on the Lightning Network that monitor payment channels for potential protocol breaches.
+    If one of the channel partners goes offline or loses their backup, a watchtower keeps backups and can restore their channel information.
++
+Watchtowers also monitor the Bitcoin blockchain and can submit a penalty transaction if one of the partners tries to "cheat" by broadcasting an outdated state. Watchtowers can be run by the channel partners themselves, or as a paid service offered by a third party. Watchtowers have no control over the funds in the channels themselves.
+
+Some contributed definitions have been sourced under a CC-BY license from the https://en.bitcoin.it/wiki/Main_Page[Bitcoin Wiki], https://en.wikipedia.org[Wikipedia], https://github.com/bitcoinbook/bitcoinbook[_Mastering Bitcoin_], or from other open source publications.