When notifying the invoice registry for an exit hop htlc we also want to
include its custom records. The channelLink, the other caller of this
method, already populates this field. So we make sure the contest
resolver does so too.
When reporting an error or a success case of a payment to a
blinded paths, the amounts to forward for intermediate hops
are set to 0 so we need to use the receiver amount instead.
Fixes a bug and makes the function more robust. Before
we would always return the encrypted data size of last hop
of the last path. Now we return the greatest last hop payload
not always the one of the last path.
To be able to do MPP payment to multiple blinded routes we need
to add a constant dummy hop as a final hop to every blined path.
This is used when sending or querying a blinded path, to let the
pathfinder be able to send MPP payments over different blinded
routes. For this dummy final hop we use a NUMS key so that we
are sure no other node can use this blinded pubkey either in a
normal or blinded route.
Moreover this helps us handling the mission control data for
blinded paths correctly because we always consider the blinded
pubkey pairs which are registered with mission control when
a payment to a blinded path fails.
For calculating the available auxiliary bandwidth of a channel, we need
access to the inbound custom wire records of the HTLC packet, which
might contain auxiliary information about the worth of the HTLC packet
apart from the BTC value being transported.
With this commit we move the traffic shaper definition from the routing
package to the HTLC switch package as a preparation for being able to
use it there as well.
At the same time we rename it to AuxTrafficShaper to be more in line
with the other auxiliary components.
In this commit, we complete a recently added feature by ensuring that
even if we go through the RBF loop to create a txn, that we still
populate the `outpointToIndex` map. Unit tests have been updated to
ensure this is always set as expected.
In this commit, we update the `Budget()` call to factor in the
`extraBudget` value. Otherwise, when we go to intialize the fee
function, we won't factor in the extra budget, and will determine that
we can't broadcast/bump.
In this commit, we expand the `NotifyBroadcast` to include an outpoint
index. This is useful as it indicates the index of a given required tx
out input.
With this commit, we update all the resolvers to pass in the new htlc
resolution blobs. Along the way, we remove the old blocking guard on
this resolution logic for HTLCs with blobs.
In this commit, we add the set of HtlcBlobs to the taprootBriefcase
struct. This new field will store all the resolution blobs for a given
HTLC. We also add some new property based tests along the way for
adequate test coverage.
In this commit, we add a new method to obtain an option of a preimage to
the input.Input struct. This is useful for callers that have an Input,
and want to optionally obtain the preimage.
In this commit, we populate the resolution blobs for the incoming and
outgoing HTLCs. We take care to populate the AuxSigDesc with the correct
information, as we need to pass along the second-level aux signature and
also sign desc along with it.
Similar to the other blobs we have for the commitment output force close
resolution, these blobs will be used to ensure that we have everything
needed to sweep aux HTLCs.
In this commit, we add some additional attributes to the ResolutionReq
struct. These will be used to make sure that we can properly handle all
the HTLC variants, on chain.
The `AuxSigDesc` will be used to communicate if an HTLC needs to go to
the second level or not. It contains the second-level sig information
needed to finalize a broadcast to the second level.
Currently if an incorrect number of timestamps is given, we fail later
on in the GossipSyncer. It makes more sense to fail right away, since we
already do that for incorrect SCID formats (e.g., unsorted or duplicate
SCIDs). There is already a matching check in Encode for incorrect number
of timestamps, so adding this check to Decode makes things symmetric.
In this commit, we bring the timeout resolver more in line with the
success resolver by using the sweeper to handle the HTLC offered remote
timeout outputs. These are outputs that we can sweep directly from the
remote party's commitment transaction when they broadcast their version
of the commitment transaction.
With this change, we slim down the scope slightly by only doing this for
anchor channels. Non-anchor channels will continue to use the
utxonursery for this output type for now.
If the delivery address is P2TR, function InternalKeyForAddr checks its
existance in the wallet to return internal key for it in case it is a custom
taproot channel. It used to return the error returned by wallet.AddressInfo.
The error is now ignored if it is ErrAddressNotFound error. This fixes
"lncli closechannel --delivery_addr <external p2tr address" case.
In this commit, we make sig job handling when singing a next commitment
non-blocking by allowing the shutdown of a channel link to prevent
further waiting on sig jobs by the channel state machine. This addresses
possible cases where the aux signer may be shut down via a separate quit
signal, so the state machine could block indefinitely on receiving an
update on a sig job.
This is a requirement for replacing the quit channel with a Context.
The Done() channel of a Context is always recv-only, so all users of
that channel must not expect a bidirectional channel.
Similar to the sweeper, when we're about to make a new breach
transaction, we ask the sweeper for a new change address, if it has one.
Then when we go to publish, we notify broadcast.
In this commit, we start to use the AuxSweeper (if present) to obtain a
new extra change addr we should add to the sweeping transaction. With
this, we'll take the set of inputs and our change addr, and then maybe
gain a new change addr to add to the sweep transaction.
The extra change addr will be treated as an extra required tx out,
shared across all the relevant inputs. This'll also be used in
NeedWalletInput to make sure that we add an extra input if needed to be
able to pay for the change addr.
In this commit, we add a new AuxSweeper interface. This'll take a set of
inputs, and a change addr for the sweep transaction, then optionally
return a new sweep output to be added to the sweep transaction.
We also add a new NotifyBroadcast method. This'll be used to notify
that we're _about_ to broadcast a sweeping transaction. The set of
inputs is passed in, which allows the caller to prepare for the ultimate
broadcast of the sweeping transaction.
We also add ExtraTxOut to BumpRequest pass fees to NotifyBroadcast. This
allows the callee to know the total fee of the sweeping transaction.
In this commit, we refactor all the other constructors for the input to
use MakeBaseInput. We also add a new set of functional options as well.
This'll be useful later on to ensure that new options are properly
applied to all the input types.
This will be used by external callers to modify the way we resolve
contracts on chain. For a given contract, we'll store an extra "blob",
that will later be presented during the sweeping phase.
This commit was added to the 0-19-staging branch recently and therefore
didn't make it into a previous part yet. So it's unrelated to the
changes in this part but is required for the whole custom channel saga.
With this commit we populate additional information about the close
outputs (including potential custom channel data) in the close update
RPC message.
This will allow custom channels to find out how the additional close
outputs look like on chain and what data they might commit to.
We also hook up the aux custom data formatter, so it can format the
custom channel data to JSON.
In this commit, we move to add the internal key to the delivery addr. This way, we give the aux chan closer the extra information it may need to properly augment the normal co-op close process.
This commit introduces a basic integration test for the invoice
HTLC modifier. The test covers scenarios where an invoice is settled with a
payment that is less than the invoice amount, facilitated by the invoice
HTLC modifier.
With this commit we encode the custom records as a TLV stream into the
custom channel data field of the invoice HTLC.
This allows the custom data parser to parse those records and replace it
with human-readable JSON on the RPC interface.
This commit enhances the itest LND node harness to include support for
the new `HtlcModifier` RPC endpoint.
At the same time we move another method to the correct file.
This commit introduces a singleton invoice HTLC modifier RPC server and
an endpoint to activate it. The server interfaces with the internal
invoice HTLC modifier interpreter, handling the marshalling between RPC
types and internal formats.
This commit updates the invoice registry to utilize the settlement
interceptor during the invoice settlement routine. It allows the
interceptor to capture the invoice, providing interception clients an
opportunity to determine the settlement outcome.
This commit introduces a new invoice htlc interceptor service
that intercepts invoice HTLCs during their settlement phase. It forwards
HTLCs to a subscribed client to determine their settlement outcomes.
This commit also introduces an interface to facilitate integrating the
interceptor with other packages.
This commit adds an optional data parser that can inspect and in-place
format custom data of certain RPC messages.
We don't add an implementation of the interface itself, as that will be
provided by external components when packaging up lnd as a bundle with
other software.
To make sure we attempt to read the results of the sig batches in the
same order they're processed, we sort them _before_ submitting them to
the batch processor.
Otherwise it might happen that we try to read on a result channel that
was never sent on because we aborted due to an error.
We also use slices.SortFunc now which doesn't use reflection and might
be slightly faster.
In this commit, we start to use the new AuxSigner to obtain+verify aux sigs for all second level HTLCs. This is similar to the existing SigPool, but we'll only attempt to do this if the AuxSigner is present (won't be for most channels).
Due to a recent refactor, the HTLCs are no longer an exported type.
Custom channels need access to those updates, so we provide them in a
read-only manner.
In this commit, we add a new aux signer interface that's meant to mirror the SigPool. If present, this'll be used to (maybe) obtain signatures for second level HTLCs for certain classes of custom channels.
For the initiator, once we get the signal that the PSBT has been
finalized, we'll call into the aux funder to get the funding desc. For
the responder, once we receive the funding_created message, we'll do the
same.
We now also have local+remote aux leaves for the commitment transaction.
Some old TODO comments that in retrospect aren't required anymore are
removed as well.
In this commit, we modify the aux funding work flow slightly. We won't
be able to generate the full AuxFundingDesc until both sides has
sent+received funding params. So we'll now only attempt to bind the
tapscript root as soon as we send+recv the open_channel message.
We'll now also make sure that we pass the tapscript root all the way
down into the musig2 session creation.
In this commit, we make a new `AuxFundingController` interface capable of processing messages off the wire. In addition, we can use it to abstract away details w.r.t how we obtain a `AuxFundingDesc` for a given channel.
We'll now use this whenever we get a channel funding request, to make sure we pass along the custom state that a channel may require.
This struct will house all the information we'll need to do a class of custom channels that relies primarily on adding additional items to the tapscript root of the HTLC/commitment/funding outputs.
The objective of this commit is to make paymentDescriptor a private
data structure so we can quarantine it to the lnwallet package.
To accomplish this we had to prevent it from leaking out via the
arguments or return values of the public functions in lnwallet.
This naturally had consequences for the htlcswitch package as we
choose other mechanisms for tracking the data that paymentDescriptor
was responsible for.
Astoundingly, this was highly successful and allowed us to remove
a ton of redundant code. The diff for this commit represents a
substantial reduction in total lines of code as well as extraneous
arguments and return values from key functions.
This also sets the stage for future commits where we actually will
be attempting to rid lnwallet of paymentDescriptor completely.
This is part of a systematic removal of PaymentDescriptor from the public
API of the lnwallet package. This marks the last change needed before we
make the PaymentDescriptor structure private.
Here we add a function that is capable of recovering LogUpdates from
paymentDescriptors and we refactor the lnwallet code to use this
rather than doing JIT inline construction of the LogUpdates.
In this commit we track the ChannelID on the PaymentDescriptor.
This will be useful in upcoming commits that need to be able to
reconstruct lnwire.Message values from PaymentDescriptors as the
Messages that are exchanged to update channel state all include
the ChannelID.
In this commit we introduce a convenience method to LightningChannel
to allow us to quickly grab the channel id. This will be important
in upcoming commits where we need to remember the ChannelID to
reconstruct update messages.
We might be trying to send an invoice amount that's greater than the size of the channel, but once you factor in the custom channel logic, an actual HTLC can be sent over the channel to pay that larger payment.
As a result, we'll skip over this check if a have a custom HTLC.
With this commit we make sure the first hop custom records aren't lost
on restart/resume of a payment, so we persist it as part of the
PaymentCreationInfo struct.
This commit extends the forward HTLC intercept response with fields that
can be used in conjunction with a `ResumeModified` action to modify the
intercepted HTLC p2p message.
Introduce `ResumeModified` action to resume standard behavior of a p2p
message with optional modifications as specified by the client during
interception.
- Introduce the field `CustomRecords` to the type `UpdateFulfillHtlc`.
- Encode and decode the new field into the `ExtraData` field of the
`update_fulfill_htlc` wire message.
- Empty `ExtraData` field is set to `nil`.
- Introduce the field `CustomRecords` to the type `UpdateAddHtlc`.
- Encode and decode the new field into the `ExtraData` field of
the `update_add_htlc` wire message.
Because we restrict custom SCID aliases to be in a specific range, we
export the range start and end values so a user of the RPCs we're going
to add in the next commits can adjust their values to fit within the
range.
With the new RPC calls that we are going to add in the next commits, it
will be possible for users to add (local only, non-gossipped) SCID
aliases for channels. Since those will be in the same range as the ones
given out by RequestAlias, we need to make sure that when we generate a
new one that it doesn't collide with an already existing one.
This commit squashes the below operations for a net result where
we have an expanded capability of assessing pending updates. This
is made possible by packing the components into Duals in the prior
commits. We squash the operations to simplify review.
htlcswitch+lnwallet: rename PendingLocalUpdateCount
lnwallet: complete pending update queries API for LightningChannel
lnwallet+htlcswitch: consolidate NumPendingUpdates using ChannelParty
This commit makes the observation that we can cleanly define the
NumPendingUpdates function using a single expression by taking
advantage of the relevant fields being properly packed into Duals.
This is yet another commit that packs a symmetric structure into
a Dual. This is the last one needed for the time being to consolidate
Num{X}UpdatesPendingOn{Y} functions into a single one.
This commit, like the last one packs the update logs into a symmetric
Dual structure. This will allow us to index into them more concisely
in higher order logic.
This commit packs the LightningChannel's localCommitmentChain and
remoteCommitmentChain into a Dual structure for better symmetric
access. This will be leveraged by an upcoming commit where we want
to more concisely express how we compute the number of pending
updates.
In this commit, we start to thread thru the new aux tap leaf structures to all relevant areas. This includes: commitment outputs, resolution creation, breach handling, and also HTLC scripts.
This may be useful for custom channel types that base everything off the index (a global value) rather than the output index (can change with each state).
In this commit, we add a TLV blob to the PaymentDescriptor struct. We also now thread through this value from the UpdateAddHTLC message to the PaymentDescriptor mapping, and the other way around.
In this commit, we add a new AuxLeafStore which can be used to dynamically fetch the latest aux leaves for a given state. This is useful for custom channel types that will store some extra information in the form of a custom blob, then will use that information to derive the new leaf tapscript leaves that may be attached to reach state.
In this commit, we also add the custom TLV blob to the internal commitment struct that we use within the in-memory commitment linked list.
This'll be useful to ensure that we're tracking the current blob for our in memory commitment for when we need to write it to disk.
In this commit, for each channel, we'll now start to store an optional custom blob. This can be used to store extra information for custom channels in an opauqe manner.
In this commit, we update all the taproot scripts to also accept an
optional aux leaf. This aux leaf can be used to add more redemption
paths for advanced channels, or just as an extra commitment space.
With this commit, the channel is now aware of if it's a musig2 channel, that also has a tapscript root. We'll need to always pass in the tapscript root each time we: make the funding output, sign a new state, and also verify a new state.
This isn't hooked up yet to the funding manager, but with this commit, we can now start to write internal unit tests that handle musig2 channels with a tapscript root.
In this commit, we consolidate the root bucket TLVs into a new struct.
This makes it easier to see all the new TLV fields at a glance. We also
convert TLV usage to use the new type param based APis.
We want to export some of our CLI code to re-use in other projects. But
in Golang you cannot import code from a `main` package.
So we need to move the actual code into its own package and only have
the `func main()` in the `main` package.
This lets us get rid of the mutex usage there. We also shift the algo slightly to increment by 1, then use that as the next value, which plays nicer with the atomics.
This commit introduces the `CustomRecords` type in the `lnwire` package,
designed to hold arbitrary byte slices. Each entry in this map can
associate with TLV type values that are greater than or equal to 65536.
In this commit, we rename the files as assembler.go houses the primary
interfaces/abstractions of the package. In the rest of the codebase,
this file is near uniformly called interface.go, so we rename the file
to make the repo more digestible at a scan.
In this commit, we fix a bug that would cause a global message router to
be stopped anytime a peer disconnected. The global msg router only
allows `Start` to be called once, so afterwards, no messages would
properly be routed.
With this commit, we allow the `MsgRouter` to be available in the
`ImplementationCfg`. With this, programs outside of lnd itself are able
to now hook into the message processing flow to direct handle custom
messages, and even normal wire messages.
Over time with this, we should be able to significantly reduce the size
of the peer.Brontide struct as we only need all those deps as the peer
needs to recognize and handle each incoming wire message itself.
In this commit, we add a new abstract message router. Over time, the
goal is that this message router replaces the logic we currently have in
the readHandler (the giant switch for each message).
With this new abstraction, can reduce the responsibilities of the
readHandler to *just* reading messages off the wire and handing them off
to the msg router. The readHandler no longer needs to know *where* the
messages should go, or how they should be dispatched.
This will be used in tandem with the new `protofsm` module in an
upcoming PR implementing the new rbf-coop close.
In this commit, we expand some of the existing chan sync tests to cover
taproot channels (the others already did). Along the way, we always
assert that the `PartialSig` is populated on retransmission. In
addition, we now send the new commit sig rather than the existing
in-memory one to test the new logic that re-signs the commitment.
Previously, the SQL implementation of the invoice query simply
converted the start and end timestamps to time and used them
in SQL queries to check for inclusivity. However, this logic
failed when the start and end timestamps were equal.
This commit addresses and corrects this issue.
Previously SQL invoice updater ignored the set ID hint when updating an
AMP invoice resulting in update subscriptions returning all of the AMP
state as well as all AMP HTLCs. This commit synchornizes behavior with
the KV implementation such that we now only return relevant AMP state
and HTLCs when updating an AMP invoice.
When fetching an AMP invoice we sometimes filter HTLCs to selected set
IDs, however we always kept the full AMP state which is irrelevant as it
contains state for all AMP payments. This was a side effect of
UpdateInvoice needing to serialize the whole invoice when storing after
an update but it is an unwanted "feature" as users will need to filter
to relevant set when listing an AMP payment or subsribing to an update.
In this commit, we fix an existing bug with the taproot channel type
that can cause force closes if a peer disconnects while attempting to
send the commitment signature.
Before this commit, since the `PartialSig` we send is never committed to
disk, the version read wouldn't contain the musig2 partial sig. We never
write these signatures to disk, as each time we make a new session, we
need to generate fresh nonces to avoid nonce-reuse.
Due to the above interaction, if we went to re-send a signature after a
disconnection, the `CommitSig` message we sent wouldn't actually contain
a `PartialSigWithNonce`, causing a protocol error.
This commit hooks up the banman to the gossiper:
- peers that are banned and don't have a channel with us will get
disconnected until they are unbanned.
- peers that are banned and have a channel with us won't get
disconnected, but we will ignore their channel announcements until
they are no longer banned. Note that this only disables gossip of
announcements to us and still allows us to open channels to them.
This commit introduces a ban manager that marks peers as banned if
they send too many invalid channel announcements to us. Expired
entries are purged after a certain period of time (currently 48 hours).
This commit adds the ability to store closed channels by scid in
the database. This will allow the gossiper to ignore channel
announcements for closed channels without having to do any
expensive validation.
We will not add a buffer to the chan policy for blinded paths in case
the sender amount violates the minHTLC restriction in the first place.
Moreover we disgard a route fast if the payment amount is smaller than
the minHTLC along the route.
Removes a check where we would NOT allow to create a blinded invoice
with an expiry (invoice expiry in seconds considered as block time)
lower than the min_final_ctlv_delta.
ChanUpdate timestamps are now restircted so that they cannot be
more than two weeks into the future. Moreover channels with both
timestamps in the ReplyChannelRange msg either too far in the past
or too far in the future are not queried.
Moreover fix unitests.
Initially in lnd, we didn't store the extra TLV data that could be
dangling off of gossip messages. This was fixed initially in lnd v0.5
with this PR: https://github.com/lightningnetwork/lnd/pull/1825.
Within the PR, we incorrect set the `ExtraOpaqueData` (extra TLV blob)
of the `ChannelAnnouncement` to the value stored in `edge`, which is
actually our channel update. As 6-ish years ago we didn't yet have
anything that used the TLV gossip fields, this went unnoticed.
Fast forward to 2024, we shipped an experimental version of inbounbd
fees. This starts to store additional data in the `ExtraOpaqueData`
field, the TLV for the inbound fee. Initially, everything is valid when
the first `ChannelAnnouncement` is sent, but as soon as a user attempts
to set an inbound fee policy, we'd incorrectly swap in that new
serialized TLV for the _channel announcement_:
841e24399c (diff-1eda595bbebe495bd74a6a0431c46b66cb4e8b53beb311067c010feac2665dcbR2560).
Since we're just trying to generate a new `channel_update`, we don't
also regenerate the signature for the `channel_announcement` message. As
a result, we end up storing a `channel_announcement` with an invalid sig
on disk, continuing to broadcast that to peers.
This commit fixes the instantiation of the BlindingPoint member of
PaymentDescriptor during the conversion from persisted LogUpdates.
Previously, the blinding point was not set correctly. The test from the
previous commit is also updated to now assert that this behaviour is now
correct.
This commit adds a new route blinding itest that demonstrates that the
reloading and re-forwarding of an UpdateAddHTLC message on restart
currently is done incorrectly for a blinded path payment. This is due to
the fact that the blinding point member is not currently set correctly.
This is fixed in the next commit which will also change the test to
assert that the behaviour is now correct.
The BlindedPathConfig struct is nice for invoice creation but when we
use the Invoice message for viewing an invoice, it would be nicer to see
an "is_blinded" field.
We have existing logic to attempt to reliably send a channel update to
the remote peer. In the wild, we've seen this fail, as it's possible
right when we send the update the peer disconnects.
In this commit, we implement a simple fix which is just to send the chan
update each time we connect to the remote party.
Fixes https://github.com/lightningnetwork/lnd/issues/6870.
This commit adds two functional options to the zpay32.Decode function.
`WithKnownFeatureBits` allows the caller to overwrite the default set of
known feature bits used by the function.
`WithErrorOnUnknownFeatureBit` allows the caller to instruct the
function to error out if the invoice that is decoded contaijns unknown
feature bits. We then use this new error-out option from the
`rpcServer`'s `extractPaymentIntent` method.
In this commit, we adjust BuildBlindedPaymentPaths to only error out
completely if none of the paths it received from FindRoutes resulted in
a usable blinded path.
since `AddInvoiceData` is config _per invoice_ where as `AddInvoiceConfig`
is config for the invoice server itself and so pretty much should stay
the same for the lifetime of LND. This change sets us up for moving some
of the blinded path config options to be changeable per AddInvoice call
rather that having fixed config values in the config file.
To reflect the new sweeping behavior, also makes it easier to be used as
we need to method to quickly cleanup force closes without concerning the
details when we are not testing the force close behavior.
The method `FetchClosedChannels` sometimes prematurely mark a pending
force closing channel as finalized, therefore we need to furthur check
`FetchPendingChannels` to make sure the channel is indeed finalized.
This commit adds a check during router's startup and fails the inflight
HTLCs if they are routing using channels unknown to us. The channels are
unknown because they are already closed, usually long time ago.
Adds a utility function to be able to compute the outgoing routing
amount from the incoming amount by taking inbound and outbound fees into
account. The discussion was contributed by user feelancer21, see
f6f05fa930.
Because the Go version used to run the `go list` commands is below the
minimum version specified in the main go.mod file, every time the `make
rpc` command is executed, the Golang runtime is downloaded twice, which
looks like this and takes a couple of seconds at least:
go: downloading go1.21.4 (linux/amd64)
go: downloading go1.21.4 (linux/amd64)
We fix this by using the correct minimum version.
This commit introduces a new API to return information on which party opened
the channel using the new ChannelParty type. It does not change the underlying
structure of how we store this information.
To support the new comma-separated list of etcd hosts in db.etcd.host,
we need to bump the `kvdb` submodule version.
This also fixes a leader election bug in the etcd code.
This commit breaks the ChannelConstraints structure into two
sub-structures that reflect the fundamental differences in how
these parameters are used. On its face it may not seem necessary,
however the distinction introduced here is relevant for how we
will be implementing the Dynamic Commitments proposal.
This commit extends our healtcheck with an optional leader check. This
is to ensure that given network partition or other cluster wide failure
we act as soon as possible to avoid a split-brain situation where a new
leader is elected but we still hold onto our etcd client.
Previously our RPC calls to etcd would hang even in the case of properly
set dial timeouts and even if there was a network partition. To ensure
liveness we need to make sure that calls fail correctly in case of
system failure. To fix this we add a default timeout of 30 seconds to
each etcd RPC call.
Implemented linter scripts to ensure consistency of the Go version
specified in Dockerfiles and YAML files. These scripts verify that the
Go version used across these files is uniform, enhancing maintainability
and reducing configuration errors.
This commit also introduces a `GO_VERSION` Makefile variable to control
the Go version used throughout the project.
We shift the duty of determining the policies to the backward pass as
the forward pass will only be responsible for finding the corrected
receiver amount.
Note that this is not a pure refactor as demonstrated in the test, as
the forward pass doesn't select new policies anymore, which is less
flexible and doesn't lead to the highest possible receiver amount. This
is however neccessary as we otherwise won't be able to compute
forwarding amounts involving inbound fees and this edge case is unlikely
to occur, because we search for a min amount for a route that was most
likely constructed for a larger amount.
This commit introduces a ChannelParty type to LND. It is useful for
consolidating all references to the duality between the local and
remote nodes. This is currently handled by having named struct rows
or named boolean parameters, named either "local" or "remote". This
change alleviates the programmer from having to decide which node
should be bound to `true` or `false`. In an upcoming commit we will
change callsites to use this.
In this commit we opt to make the internal response channel fully
private and instead expose methods for doing resolution. This
prevents internal implementation details from leaking a little bit
better than the previous iteration.
It is common throughout the codebase to send data to a remote
goroutine for processing. Typically, along with the data we are
processing, we also send a one-shot channel where we intend to
listen for the response. This type encapsulates that pattern.
Renamed the Makefile variable `GO_VERSION` to `ACTIVE_GO_VERSION` for
improved clarity. This change also frees up the name `GO_VERSION` to be
used for defining the global Go version for reproducible binaries in a
subsequent commit.
With this PR we might call the stop method even when the start
method of a subsystem did not successfully finish therefore we
need to make sure we guard the stop methods for potential panics
if some variables are not initialized in the contructors of the
subsystems.
This commit does two things. It starts up the server in a way that
it can be interrupted and shutdown gracefully.
Moreover it makes sure that subsystems clean themselves up when
they fail to start. This makes sure that depending subsytems can
shutdown gracefully as well and the shutdown process is not stuck.
Continue adding some complexity behind the BlindedPaymentPathSet. What
we do here is add a new IntroNodeOnlyPath method. The assumption we
make here is: If multiple blinded paths are provided to us in an invoice
but one of those paths only includes an intro node, then there is no
point in looking at any other path since we know that the intro node is
the destination node. So in such a case, we would have discarded any
other path in the `NewBlindedPaymentPathSet` constructor. So then we
would only have a single blinded path made up of an introduction node
only. In this specific case, in the `newRoute` function, no edge passed
to the function would have a blindedPayment associated with it (since
there are no blinded hops in this case). So we will have a case where
`blindedPathSet` passed to `newRoute` is not nil but `blindedPayment` is
nil since nonce was extacted from any edge. If this happens then we can
assume that this is the Intro-Node-Only situation described above. And
so we grabe the associated payment from the path set.
Instead of needing to remember how to handle the FinalCLTV value of a
blinded payment path at various points in the code base, we hide the
logic behind a unified FinalCLTVDelta method on the blinded path.
If multiple blinded paths are provided, they will each have a different
pub key for the destination node. This makes using our existing
pathfinding logic tricky since it depends on having a single destination
node (characterised by a single pub key). We want to re-use this logic.
So what we do is swap out the pub keys of the destinaion hop with a
pseudo target pub key. This will then be used during pathfinding. Later
on once a path is found, we will swap the real destination keys back in
so that onion creation can be done.
This commit introduces a new type, `BlindedPaymentPathSet`. For now, it
holds only a single `BlindedPayment` but eventually it will hold and
manage a set of blinded payments provided for a specific payment. To
make the PR easier to follow though, we start off just letting it hold a
single one and do some basic replacements.
Even if no HTLCs are at stake we are going to register the anchor
outputs with the sweeper subsystem with a default high deadline.
We need to do this, because otherwise we are not able to bump the
fee of the closing transaction manually.
We split up the functionality in getRouteUnifiers into checking that all
edges exist via getEdgeUnifiers and then add a backward pass that will
be responsible for determining the sender amount.
We remove the node pub key from the error string, as in route building
this is duplicate info, which can be determined from the input keys,
further it's not available in the backward pass anymore.
We refactor the BuildRoute test to use the require library and add a
test case for a max HTLC violation on the last hop.
The time lock weight for a hop is supposed to be proportional to the
amount that is sent/locked, but in a previous change we switched to the
net amount, where inbound fees aren't yet applied. This is corrected in
this commit.
When iterating edges, pathfinding checks early whether using an edge
would violate the requested total fee limit for a route. This check is
done on the net amount (an amount the inbound fee is calculated with).
However, a possible next hop's fee discount leads to a reduction in fees
and as such using the net amount leads to assuming a higher cumulative
fee than the route really has, excluding the path erroneously. We
perform the fee limit check on the amount to send, which includes both
inbound and outbound fees. This should be possible as the first hop's
outbound fee is zero and therefore doesn't have to be checked in the
end.
Add the missing SERVER_ACTIVE state to the readiness probe. Without
this, a node that is ready to accept RPC calls would be incorrectly
considered not ready.
This commit expands the definition of the dust limit to take into
account commitment fees as well as dust HTLCs. The dust limit is now
known as a fee exposure threshold. Dust HTLCs are fees anyways so it
makes sense to account for commitment fees as well. The link has
been modified slightly to calculate dust. In the future, the switch
dust calculations can be removed.
This commit introduces more sophisticated code for selecting dummy hop
policy values for dummy hops in blinded paths.
For the case where the path does contain real hops, the dummy hop policy
values are derived by taking the average of those hop polices. For the
case where there are no real hops (in other words, we are the
introduction node), we use the default policy values used for normal
ChannelUpdates but then for the MaxHTLC value, we take the average of
all our open channel capacities.
Setting default values for the channel opening fee rate is already
done elsewhere therefore we remove on of those checks and return
an error if no fee rate is specified.
Add an itest that tests the addition of dummy hops to a blinded path. By
testing that invoices containing such a path can be paid, it also tests
the peeling of dummy hops by the receiver.
Make various sender side adjustments so that a sender is able to send an
MP payment to a single blinded path without actually including an MPP
record in the payment.
Update one of the route blinding itests to do a full end-to-end test
where the recipient generates and invoice with a blinded path and the
sender just provides that invoice to SendPayment.
The tests also covers the edge case where the recipient is the
introduction node.
The route blinding itests are now updated so that recipient logic is
tested. The creation of a blinded route is also now done through the
AddInvoice API instead of manually.
If a blinded path payload contains a signal that the following hop on
the path is a dummy hop, then we iteratively peel the dummy hops until
the final payload is reached.
We've covered all the logic for building a blinded path to ourselves and
putting that into an invoice - so now we start preparing to actually be
able to recognise the incoming payment as one from a blinded path we
created.
The incoming update_add_htlc will have an `encrypted_recipient_data`
blob for us that we would have put in the original invoice. From this we
extract the PathID which we wrote. We consider this the payment address
and we use this to derive the associated invoice location.
Blinded path payments will not include MPP records, so the payment
address and total payment amount must be gleaned from the pathID and new
totalAmtMsat onion field respectively.
This commit only covers the final hop payload of a hop in a blinded
path. Dummy hops will be handled in the following commit.
We further break up the extracTLVPayload into more modular pieces. The
pieces are structured in such a way as to prepare for extracTLVPayload
being called in a recursive manner from within
`deriveBlindedRouteForwardingInfo` when we add the logic for handling
dummy hops in a later commit. With this refactor, we completey remove
the BlindingKit's DecryptAndValidateFwdInfo method.
In this refactor commit, we extract all the steps from extractTLVPayload
that have to do with parsing the payload from the sender and verifying
the presence of various fields from the sender.
In preparation for calling the TLV payload parsing logic recursively for
when we need to peel dummy hops from an onion, this commit creates a new
extractTLVPayload function. This is a pure refactor.
Expose the ability to add blinded paths to an invoice. Also expose
various configuration values.
We also let the lncfg.Invoices struct satisfy the Validator interface so
that we can verify all its config values in one place.
Add a `FindBlindedPaths` method to the `ChannelRouter` which will use
the new `findBlindedPaths` function to get a set of candidate blinded
path routes. It then uses mission control to select the best of these
paths.
Note that as of this commit, the MC data we get from these queries won't
mean much since we wont have data about a channel in the direction
towards us. But we do this now in preparation for a future PR which will
start writing mission control success pairs for successful receives from
blinded route paths.
This commit adds a new function, `findBlindedPaths`, that does a depth
first search from the target node to find a set of blinded paths to the
target node given the set of restrictions. This function will select and
return any candidate path. A candidate path is a path to the target node
with a size determined by the given hop number constraints where all the
nodes on the path signal the route blinding feature _and_ the
introduction node for the path has more than one public channel. Any
filtering of paths based on payment value or success probabilities is
left to the caller.
Here we add a new `Blind` option to the `AddInvoiceData` which will
signal that the new invoice should encode a blinded route.
Certain other changes are also made in the case that this invoice
contains a blinded route:
1) the payment address/secret no longer needs to be in the invoice
itself since it will be put in the `PathID` recored of the encrypted
recipient record for our hop.
2) When we sign the invoice, we now use an ephemeral key since we dont
want the sender to be able to derive our real node pub key from the
invoice signature.
3) The invoice's FinalCLTV field should be zero for blinded invoices
since the CLTV delta info will be communicated in the accumulated
route policy values.
This commit adds all the logic for building a blinded path (from a given
route) and packaging it up in a zpay32.BlindedPaymentPath struct so that
it is ready for adding to an invoice. It also includes logic for padding
a path with dummy hops.
Note that in this commit, the logic for choosing an actual path to us
that can then be used in a blinded path is abstracted away. This logic
will be fleshed out in a future commit.
This commit adds a helper function that will be used to adjust a hops
policy values by certain given increase and decrease multipliers. This
will be used in blinded paths to give policy values some buffer to avoid
easy probing of blinded paths.
This commit adds a function that can be used to compute the accumulated
path policy for a blinded path as defined in the spec:
db278ab9b2/04-onion-routing.md (L255)
This commit adds a helper function called `padHopInfo` along with a test
for it. This function will be used later on when building a blinded
path. It is used to ensure that all encrypted blobs of a blinded path
that we construct are padded to the same size.
We need a new feature bit for BOLT11 invoices in order to indicate that
they contain the new blinded path tagged field. Tagged fields pre-date
TLV and so nodes who dont understand them will simply skip them.
Therefore the feature bit helps them to fail fast.
Previously we may get a floor feerate when calling `EstimateFeePerKW`
due to the fee estimator not finishing its startup process, which gives
us an empty fee map.
This is now fixed to return an error if the estimator is not started.
This commit adds a new expected field, `min_relay_feerate`, in the
response body returned from the API source, allowing the API to specify
a min relay feerate to be used instead of the FeePerKwFloor.
This change is backwards compatible as for an old API source which
doesn't specify the `min_relay_feerate`, it will be interpreted as zero.
This test was previously working because we'd mine an extra block to
confirm the coins inside `FundCoinsUnconfirmed` when it's a neutrino
backend, as shown in
fdd28c8d88/lntest/harness.go (L1431)
Since neutrino has trouble seeing unconfirmed balance, we now send some
coins to the wallet, confirm those, then do a self-transfer so the node
will have unconfirmed outputs to perform the test.
This commit makes sure the sweep requests are received before mining
blocks to trigger the actual sweeping.
In addition, `testFundingExpiryBlocksOnPending` is updated to deal with
the old `channel link not found` issue.
This commit fixes the heuristic we use for identifying the party
that broadcast a Simple Taproot Channel commitment transaction.
Prior to this change we checked if the last script element was an
OP_DROP. However, both the local and remote commitment outputs
have an OP_DROP at the end.
The new approach checks the resolver's SignDescriptor and compares
that key to the keys in the channel's local ChannelConfig. If the
key is the delay key, we know that it is our commitment transaction.
This commit fixes an issue where we did not properly detect and
therefore record the coop close transaction if it used the newer
RBF coop close v2 scheme. This only affects coop closes of
taproot channels today.
This commit is a large refactor that moves over various responsibilities
from the ChannelRouter to the graph.Builder. These include all graph
related tasks such as:
- graph pruning
- validation of new network updates & persisting new updates
- notifying topology update clients of any changes.
This is a large commit but:
- many of the files are purely moved from `routing` to `graph`
- the business logic put in the graph Builder is copied exactly as is
from the ChannelRouter with one exception:
- The ChannelRouter just needs to be able to call the Builder's
`ApplyChannelUpdate` method. So this is now exported and provided to
the ChannelRouter as a config option.
- The trickiest part was just moving over the test code since quite a
bit had to be duplicated.
This is preparation for an upcoming commit that will move over various
responsibilities from the ChannelRouter to the graph Builder. So that
that commit can be a pure code-move commit, the template for the new
sub-system is added up front here.
In preparation for adding a clean Graph DB interface, we create a
version of FetchLightningNode that doesnt allow a caller to provide in a
transaction.
In this commit, we completely remove the Router's dependence on a Graph
source that requires a `kvdb.RTx`. In so doing, we are more prepared for
a future where the Graph source is backed by different DB structure such
as pure SQL.
The two areas affected here are: the ChannelRouter's graph access that
it uses for pathfinding. And the SessionSource's graph access that it
uses for payments.
The ChannelRouter gets given a Graph and the SessionSource is given a
GraphSessionFactory which it can use to create a new session. Behind the
scenes, this will acquire a kvdb.RTx that will be used for calls to the
Graph's `ForEachNodeChannel` method.
In preparation for structs outside of the `routing` package implementing
this interface, export `routingGraph` and rename it to `Graph` so as to
avoid stuttering.
In this commit, we further reduce the routingGraph interface and this
time we make it more node-agnostic so that it can be backed by any graph
and not one with a concept of "sourceNode".
Only include the final hop's cltv delta in the total timelock
calculation if the route does not include a blinded path. This is
because in a blinded path, the final hops final cltv delta will be
included in the blinded path's accumlated cltv delta value.
With this commit, we remove the responsibility of remembering not to set
the `finalHop.cltvDelta` from the caller of `newRoute`. The relevant
test is updated accordingly.
This commit adds a blinded_paths field to the PayReq proto message. A
new helper called `CreateRPCBlindedPayments` is then added to convert
the zpay32 type to the existing `lnrpc.BlindedPaymentPath` type and add
this to the `PayReq` in the `DecodePayReq` rpc method.
When we start creating blinded paths to ourselves, we will want to be
able to pad the data for each hop so that the `encrypted_recipient_data`
for each hop is the same. We add a `PadBy` method that allows a caller
to add a certain number of bytes to the padding field. Note that adding
n bytes won't always mean that the encoded payload will increase by size
n since there will be overhead for the type and lenght fields for the new
TLV field. This will also be the case when the number of bytes added
results in a BigSize bucket jump for TLV length field. The
responsibility of ensuring that the final payloads are the same size is
left to the caller who may need to call PadBy iteratively to achieve the
goal. I decided to leave this to the caller since doing this at the
actual TLV level will be quite intrusive & I think it is uneccessary to
touch that code for this unique use case.
Add the PathID (tlv type 6) field to BlindedRouteData. This will be used
for the final hop of a blinded route. A new constructor is also added
for BlindedRouteData which can specifically be used for the final hop.
For the final hop in a blinded route, the SCID and RelayInfo fields will
_not_ be set. So these fields need to be converted to optional records.
The existing BlindedRouteData constructor is also renamed to
`NewNonFinalBlindedRouteData` in preparation for a
`NewFinalBlindedRouteData` constructor which will be used to construct
the blinded data for the final hop which will contain a much smaller set
of data. The SCID and RelayInfo parameters of the constructor are left
as non-pointers in order to force the caller to set them in the case
that the constructor is called for non-final nodes. The other option
would be to create a single constructor where all parameters are
optional but I think this makes it easier for the caller to make a
mistake.
Later on in this series, we will need to know during path finding if an
edge we are traversing was derived from a blinded payment path. In
preparation for that, we add a BlindedPayment member to the
`unifiedEdge` struct.
The reason we will need this later on is because: In the case where we
receive multiple blinded paths from the receipient, we will first swap
out the final hop node of each path with a single unified target node so
that path finding can work as normal. Once we have selected a route
though, we will want to know which path an edge belongs to so that we
can swap the correct destination node back in.
Add a constructor for unified edge. In upcoming commits, we will add a
new member to unifiedEdge and a constructor forces us to not forget to
populate a required member.
Expand the AdditionalEdges interface with a BlindedPayment method. In
upcoming commits, we will want to know if an AdditionalEdge was derived
from a blinded payment or not and we will also need some information
from the blinded payment it was derived from. So we expand the interface
here to avoid needing to do type casts later on. The new method may
return nil if the edge was not derived from a blinded payment.
This commit is purely a refactor. In it, we let the `BlindedEdge` struct
carry a pointer to the `BlindedPayment` that it was derived from. This
is done now because later on in the PR series, we will need more
information about the `BlindedPayment` that an edge was derived from.
Since we now pass in the whole BlindedPayment, we swap out the
`cipherText` member for a `hopIndex` member so that we dont carry around
two sources of truth in the same struct.
Previously, when using the native schema, invoice expiries were incorrectly
stored as 64-bit values (expiry in nanoseconds instead of seconds), causing
overflow issues. Since we cannot determine the original values, we will set
the expiries for existing invoices to 1 hour with this migration.
Previously paginated queries offseted the add_index_get, add_index_let,
settle_index_get and settle_index_let parameters with the paginators
current page offset, however this was incorrect as we can just use
SQL's LIMIT/OFFSET to paginate. This commit fixes this issue and adds an
optional parameter to the constructor of the invoice SQL store to set
page size. This is useful when testing as we can now cover pagination
correctly with our existing unit tests.
This commit removes another check for TLV payload support of the
destination node. We assume TLV payloads as the default everywhere else,
so we just remove two checks that were previously forgotten.
Building current lnd `0.18` fails with older go (`1.19.7`).
* Updated go download path to 1.22.4
* Updated hashes
* Added `rm -rf` instructions as per [go.dev instructions](https://go.dev/doc/install)
`NewResult` makes it easy to wrap a normal function call in a result
value.
`Err` can be used to check the error case of the result without
unpacking the entire thing.
`AndThen2` allows a caller to compose a function on two results values,
with the closure only executing if both values are non-error.
This removes duplication of in-memory data during the periodic flushing
stage of the mission control store.
The existing code entirely duplicates the in-memory cache of the store,
which is very wasteful when only a few additional results are being
rotated into the store.
This has a significant performance penalty specially for wallets that
remain online for a long time with a low volume of payments. The worst
case scenario are wallets that see at most 1 new payment a second, where
the entire in-memory cache is recreated every second.
This commit improves the situation by determining what will be the
actual changes that need to be committed before initiating the db
transaction and only keeping track of these to update the in-memory
cache if the db tx is successful.
This modifies the mission control store to avoid running the one second
ticker for flushing data when there is no work to be done.
This improves performance of a quiscent LN node by avoiding a one second
interval busy loop that does nothing when there are no payments flowing
through the node.
This modifies the mission control store to avoid doing any work when no
new payment result entries are in the queue to be processed.
The mission control store maintains keeps the latest N (in production:
1000) entries in its DB, evicting older entries when new ones are added.
Currently, its implementation is somewhat less performant than it could
be.
This commit adds an early return to the storeResults function to avoid
doing any DB or memory operations when its outstanding queue is empty,
improving the performance during quiescent periods of the LN node's
execution.
In this commit we set up the payment loop context
according to user-provided parameters. The
`cancelable` parameter indicates whether the user
is able to interrupt the payment loop by cancelling
the server stream context. We'll additionally wrap
the context in a deadline if the user provided a
payment timeout.
We remove the timeout channel of the payment_lifecycle.go
and in favor of the deadline context.
index
When abandoning a channel, we remove it from the graph and then add it
to the zombie channel index. However, if we then process a ChannelUpdate
for this channel it will result in us calling `processZombieUpdate`
which will delete the edge from the zombie index. The abandon channel
itest currently asserts that a channel is no longer in the graph by
asserting that when querying for the channel we get a "marked as zombie"
error but to account for the above series of operations, we now also
allow it to just not be found at all ("edge not found").
We may get a flake like the following,
```
lnd_route_blinding_test.go:468:
Error Trace: /Users/runner/work/lnd/lnd/itest/lnd_route_blinding_test.go:468
/Users/runner/hostedtoolcache/go/1.22.3/arm64/src/runtime/asm_arm64.s:1222
Error: Received unexpected error:
rpc error: code = Canceled desc = context canceled
Test: TestLightningNetworkDaemon/tranche15/144-of-156/bitcoind/disable_introduction_node
```
This happens when the test successfully finishes, the parent context is
canceled, causing the child context to return an error. We fix it by
ignoring it in the goroutine.
This commit fixes#8535 by changing how we assess toSelfAmount inside
the chainWatcher.
In certain cases users may wish to close out channel funds to external
delivery addresses set either during open or close.
Prior to this change we only consider addresses that our wallet is
aware of.
This change now identifies outputs as to_self outputs if the delivery
script matches OR if our wallet is aware of the address. In certain
edge cases it can be possible for there to be more than one output
that matches these criteria and in that case we will return the sum
of those values.
Previously the error message produced when `CltvExpiry` is less
than the minimum final cltv (18 at present) set by
`routing.MinCLTVDelta` inserted the values into the wrong spots of the
formatted string.
Allows users of the RPC CLI to set the `min_final_cltv_expiry_delta`
described in BOLT's 11, 7, and 2 by setting the `cltv_expiry` flag when
calling either of the `addinvoice` or `addholdinvoice` RPC's from
`lncli`.
In this commit, we add a new result type, which is just Either[A, error]. This is useful as we can always pass around/accept a single value. We add an Unpack() method which is the most idiomatic way of handling errors in Go, as we revert back to (A, error) when necessary.
Fixes the problem that inbound base fee and fee rate are overwritten
with 0 if they are not specified in PolicyUpdateRequest. This ensures
backward compatibility with older rpc clients that do not yet support
the inbound feature.
This commit changes how the deadline is calculated for CPFP anchor
sweeping. In order to sweep the second-level HTLCs, we need to first
get the FC tx confirmed. If we use a larger conf target for CPFP, we'd
end up having few blocks to sweep the HTLCs, as these two sweeping txns
share the deadline of the HTLC, as shown below,
```
More aggressive on the CPFP part.
|-CPFP-|-----HTLC-----|
Share the deadlines evenly.
|---CPFP---|---HTLC---|
More aggressive on the HTLC part.
|-----CPFP-----|-HTLC-|
```
In this commit, we decide to share the deadlines evenly as a starting
point so neither side will have a short of deadlines.
The `createTestPeerWithChannel` function is made to return this newly
created struct and error.
This is useful because upcoming commits would require us returning more
objects from the function.
Signed-off-by: Ononiwu Maureen <maureen.ononiwu@outlook.com>
In this function `createTestPeer` is changed to
`createTestPeerWithChannel`. This is useful in coming commits where we
decouple the process of creating test peer from the function.
Signed-off-by: Ononiwu Maureen <maureen.ononiwu@outlook.com>
Add a new `make install-all` command that will perform all `make
install` actions along with generating the manpages. The `manpages`
command is then removed from the existing `make install` command. The
docker build is then updated to use the new `make install-all` command.
This is done because some users running `make install` may be doing so
in environments where they do not have write access to the directory
where the man pages need to be written to.
Add a temporary mutex around the calls to sqldb.NewTestSqliteDB to
ensure that only a single thread can access it at a time. This is a
temporary work around that can be removed once this race condition in
the sqlite repo has been resolved:
https://gitlab.com/cznic/sqlite/-/issues/180
This commit adds contributors that didn't add themselves to the release
notes by extracting their GitHub username (or, if available their
name and surname from GitHub) from the git log manually.
We mine quite a few blocks in this test to trigger a htlc timeout,
so it can be pretty slow. This fix adds assertions for additional
"state steps" that happen in between the failing of the htlc on-chain
and asserting that we're fully cleared out so that slower running
machines won't timeout.
In this commit, we update the project and relevant sub-modules to sqldb
v1.0.2. The next step is to tag a new version of kvdb, then update the
main module to use that.
In this commit, we fix a race in the `TestPackager` series on channeldb.
A few tests were sharing the same global variable of the set of log
updates, which includes a pointer to an HTLC. The `ExtraData` value of
the HTLC would then be mutated once we go to encode the message on disk.
To fix this, we the global with a function that returns a new instance
of all the test data.
```
==================
WARNING: DATA RACE
Write at 0x0000021b0a48 by goroutine 2896:
github.com/lightningnetwork/lnd/lnwire.(*ExtraOpaqueData).PackRecords()
/home/runner/work/lnd/lnd/lnwire/extra_bytes.go:74 +0x546
github.com/lightningnetwork/lnd/lnwire.EncodeMessageExtraData()
/home/runner/work/lnd/lnd/lnwire/extra_bytes.go:121 +0x4d
github.com/lightningnetwork/lnd/lnwire.(*UpdateAddHTLC).Encode()
/home/runner/work/lnd/lnd/lnwire/update_add_htlc.go:164 +0x5af
github.com/lightningnetwork/lnd/lnwire.WriteMessage()
/home/runner/work/lnd/lnd/lnwire/message.go:330 +0x351
github.com/lightningnetwork/lnd/channeldb.WriteElement()
/home/runner/work/lnd/lnd/channeldb/codec.go:186 +0x1975
github.com/lightningnetwork/lnd/channeldb.WriteElements()
/home/runner/work/lnd/lnd/channeldb/codec.go:247 +0x14f
github.com/lightningnetwork/lnd/channeldb.serializeLogUpdate()
/home/runner/work/lnd/lnd/channeldb/channel.go:2529 +0x3c
github.com/lightningnetwork/lnd/channeldb.putLogUpdate()
/home/runner/work/lnd/lnd/channeldb/forwarding_package.go:525 +0xae
github.com/lightningnetwork/lnd/channeldb.(*ChannelPackager).AddFwdPkg()
/home/runner/work/lnd/lnd/channeldb/forwarding_package.go:489 +0x684
github.com/lightningnetwork/lnd/channeldb_test.TestPackagerOnlyAdds.func1()
/home/runner/work/lnd/lnd/channeldb/forwarding_package_test.go:283 +0x4c
github.com/btcsuite/btcwallet/walletdb/bdb.(*db).Update()
/home/runner/go/pkg/mod/github.com/btcsuite/btcwallet/walletdb@v1.4.2/bdb/db.go:429 +0xe5
github.com/lightningnetwork/lnd/kvdb.Update()
/home/runner/go/pkg/mod/github.com/lightningnetwork/lnd/kvdb@v1.4.6/interface.go:16 +0x258
github.com/lightningnetwork/lnd/channeldb_test.TestPackagerOnlyAdds()
/home/runner/work/lnd/lnd/channeldb/forwarding_package_test.go:282 +0x17b
testing.tRunner()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1595 +0x238
testing.(*T).Run.func1()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1648 +0x44
Previous write at 0x0000021b0a48 by goroutine 2898:
github.com/lightningnetwork/lnd/lnwire.(*ExtraOpaqueData).PackRecords()
/home/runner/work/lnd/lnd/lnwire/extra_bytes.go:74 +0x546
github.com/lightningnetwork/lnd/lnwire.EncodeMessageExtraData()
/home/runner/work/lnd/lnd/lnwire/extra_bytes.go:121 +0x4d
github.com/lightningnetwork/lnd/lnwire.(*UpdateAddHTLC).Encode()
/home/runner/work/lnd/lnd/lnwire/update_add_htlc.go:164 +0x5af
github.com/lightningnetwork/lnd/lnwire.WriteMessage()
/home/runner/work/lnd/lnd/lnwire/message.go:330 +0x351
github.com/lightningnetwork/lnd/channeldb.WriteElement()
/home/runner/work/lnd/lnd/channeldb/codec.go:186 +0x1975
github.com/lightningnetwork/lnd/channeldb.WriteElements()
/home/runner/work/lnd/lnd/channeldb/codec.go:247 +0x14f
github.com/lightningnetwork/lnd/channeldb.serializeLogUpdate()
/home/runner/work/lnd/lnd/channeldb/channel.go:2529 +0x3c
github.com/lightningnetwork/lnd/channeldb.putLogUpdate()
/home/runner/work/lnd/lnd/channeldb/forwarding_package.go:525 +0xae
github.com/lightningnetwork/lnd/channeldb.(*ChannelPackager).AddFwdPkg()
/home/runner/work/lnd/lnd/channeldb/forwarding_package.go:489 +0x684
github.com/lightningnetwork/lnd/channeldb_test.TestPackagerAddsThenSettleFails.func1()
/home/runner/work/lnd/lnd/channeldb/forwarding_package_test.go:490 +0x4c
github.com/btcsuite/btcwallet/walletdb/bdb.(*db).Update()
/home/runner/go/pkg/mod/github.com/btcsuite/btcwallet/walletdb@v1.4.2/bdb/db.go:429 +0xe5
github.com/lightningnetwork/lnd/kvdb.Update()
/home/runner/go/pkg/mod/github.com/lightningnetwork/lnd/kvdb@v1.4.6/interface.go:16 +0x2cd
github.com/lightningnetwork/lnd/channeldb_test.TestPackagerAddsThenSettleFails()
/home/runner/work/lnd/lnd/channeldb/forwarding_package_test.go:489 +0x1e7
testing.tRunner()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1595 +0x238
testing.(*T).Run.func1()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1648 +0x44
Goroutine 2896 (running) created at:
testing.(*T).Run()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1648 +0x82a
testing.runTests.func1()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:2054 +0x84
testing.tRunner()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1595 +0x238
testing.runTests()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:2052 +0x896
testing.(*M).Run()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1925 +0xb57
github.com/lightningnetwork/lnd/kvdb.RunTests()
/home/runner/go/pkg/mod/github.com/lightningnetwork/lnd/kvdb@v1.4.6/test_utils.go:23 +0x26
github.com/lightningnetwork/lnd/channeldb.TestMain()
/home/runner/work/lnd/lnd/channeldb/setup_test.go:10 +0x308
main.main()
_testmain.go:321 +0x303
Goroutine 2898 (running) created at:
testing.(*T).Run()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1648 +0x82a
testing.runTests.func1()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:2054 +0x84
testing.tRunner()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1595 +0x238
testing.runTests()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:2052 +0x896
testing.(*M).Run()
/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.21.4.linux-amd64/src/testing/testing.go:1925 +0xb57
github.com/lightningnetwork/lnd/kvdb.RunTests()
/home/runner/go/pkg/mod/github.com/lightningnetwork/lnd/kvdb@v1.4.6/test_utils.go:23 +0x26
github.com/lightningnetwork/lnd/channeldb.TestMain()
/home/runner/work/lnd/lnd/channeldb/setup_test.go:10 +0x308
main.main()
_testmain.go:321 +0x303
==================
```
This itest adds a test that we still propagate blinded errors back
properly after a restart with an on-chain resolution. The test also
updates our sendpayment timeout to longer so that there's time to
resolve the on chain claim.
In this commit, we fix an inconsistent in the API related to AMP
payments. When a payment request isn't specified, we require the `--amp`
flag on the CLI to make an AMP payment. However, for payment requests,
we don't require this flag. To fix this inconsistency, we now require
the `--amp` flag to _also_ be set for payment requests.
This type is useful when one wants to encode an integer as an underlying BigSize record. It wraps any integer, then handles the transformation into and out of the BigSize encoding on disk.
In this commit, we add a new type alias for a blob type. This type can be used in areas where a byte slice is used to store a TLV value, which may be a fully opaque nested TLV.
Create our error encrypter with a wrapped type if we have a blinding
point present. Doing this in the iterator allows us to track this
information when we have both pieces of information available to us,
compared to trying to handle this later down the line:
- Downstream link on failure: we know that we've set a blinding point
for out outgoing HTLC, but not whether we're introduction or not
- Upstream link on failure: once the failure packet has been sent
through the switch, we no longer know whether we were the introduction
point (without looking it up / examining our payload again /
propagating this information through the switch).
Introduce two wrapper types for our existing SphinxErrorEncrypter
that are used to represent error encrypters where we're a part of a
blinded route. These encrypters are functionally the same as a sphinx
encrypter, and are just used as "markers" so that we know that we
need to handle our error differently due to our different role.
We need to persist this information to account for restart cases where
we've resovled the outgoing HTLC, then restart and need to handle the
error for the incoming link. Specifically, this is relevant for:
- On chain resolution messages received after restart
- Forwarding packages that are re-forwarded after restart
This is also generally helpful, because we can store this information
in one place (the circuit) rather than trying to reconstruct it in
various places when forwarding the failure back over the switch.
We need to know what role we're playing to be able to handle errors
correctly, but the information that we need for this is held by our
iterator:
- Whether we had a blinding point in update add (blinding kit)
- Whether we had a blinding point in payload
As we're now going to use the route role return value even when our
err!=nil, we rename the error to signal that we're using less
canonical golang here.
An alternative to this approach is to attach a RouteRole to our
ErrInvalidPayload. The downside of that approach is:
- Propagate context through parsing (whether we had updateAddHtlc)
- Clumsy handling for errors that are not of type ErrInvalidPayload
When handling blinded errors, we need to know whether there was a
blinding key in our payload when we successfully parsed our payload
but then found an invalid set of fields. The combination of
parsing and validation in NewPayloadFromReader means that we don't know
whether a blinding point was available to us by the time the error is
returned.
This commit splits parsing and validation into two functions so that
we can take a look at what we actually pulled of the payload in between
parsing and TLV validation.
This commit moves all our validation related to the presence of fields
into ValidateParsedPayloadTypes so that we can handle them in a single
place. We draw the distinction between:
- Validation of the payload (and the context within it's being parsed,
final hop / blinded hop etc)
- Processing and validation of encrypted data, where we perform
additional cryptographic operations and validate that the fields
contained in the blob are valid.
This helps draw the line more clearly between the two validation types,
rather than splitting some payload-releated blinded hop processing
into the encrypted data processing part. The downside of this approach
(vs doing the blinded path payload check _after_ payload validation)
is that we have to pass additional context into payload validation
(ie, whether we got a blinding point in our UpdateAddHtlc - as we
already do for isFinalHop).
This commit adds handling for malformed HTLC errors related to blinded
paths. We expect to receive these errors _within_ a blinded path,
because all non-introduction nodes are instructed to return malformed
errors for failures.
Note that we may actually switch back to a malformed error later on if
we too are a relaying node in the route, but we handle that case the
incoming link.
We only want to run the coverage on some select tests, since we don't
want to make tests with slow backends even slower.
And because we use "parallel: true" when uploading the coverage results,
those will only be assembled once we send "parallel-finished: true"
using the goveralls action.
So we do that with a new "finish" job.
This commit changes how we transform from a deadline option to a
concrete deadline value - previously this is done when we decide to
cluster inputs, and we now move it to a step earlier - once an input is
received via `SweeperInput`, we will immediately transform its optional
deadline into a real value. For inputs that come with a deadline option,
since the Some will be used, it makes no difference. For inputs with
None as their deadlines, we need this change to make sure the default
deadlines are assigned accurately.
This commit adds a test case to check the no deadline sweeping behavior.
The sweeper is updated to make sure it can handle the case for neutrino
backend.
This commit adds a new check for neutrino backend - when the inputs in
the sweeping tx are spent by a third party, we will send back a
`TxFailed` event to free the rest of the inputs for re-grouping.
This commit moves the offering of second-level outputs one block
earlier. The sweeper will check the required locktime and wait until it
matures. This is needed so the second-level outputs can be aggregated
properly.
This commit adds a new config method `QueryIncomingCircuit` that can be
used to query the payment's incoming circuit for giving its outgoing
circuit key.
Previously we don't allow confTarget to be 0, which ended up the final
position being never reached. We fix it here by allowing confTarget to
be 0 in case the deadline has already been passed for a given input.
This commit removes the method `CreateSweepTx` and makes sure when
sweeping the htlc output via the direct-preimage spend, it's offered via
the `SweepInput` interface.
`IncubateOutputs` never takes more than one HTLC, so we change the
params to be optional, which helps with the following commit where we
pass the deadline height when incubating outgoing HTLCs.
This commit changes `findCommitmentDeadline` to
`findCommitmentDeadlineAndValue` to calculate the value left from all
the time-sensitive HTLCs after subtracting their budgets. This value is
then used to calculate the budget to be used when sweeping the anchor
output.
This commit adds a new group config `BudgetConfig` to allow users
specifying their own preference when sweeping outputs. And a new config
option `NoDeadlineConfTarget` is added in case the user wants to use a
different "lazy" conf target.
This commit changes the method `ClusterInputs` to also take a default
deadline height. Previously, when calculating the default deadline
height for a non-time sensitive input, we would first cluster it with
other non-time sensitive inputs, then give it a deadline before we are
about to `sweep`. This is now moved to the step where we decide to
cluster inputs, allowing time-sensitive and non-sensitive inputs to be
grouped together, if they happen to share the same deadline heights.
When `handleNewInput` fails, it means there's something terribly wrong
with the chain backend, which means we need to stop the current process
and let user handle it.
This takes the old `createSweepTx` and refactors it to be
sweep-specific. A sweeping txns differs from a normal tx as it doesn't
need to take outputs as params.
This commit fixes an edge case that the sweeper's best known block
height is behind arbitrator's, which may cause an issue when creating
sweeping tx, as we may end up using an old block height from
arbitrator's view.
This commit exports and renames the following variable names:
- `PendingInput` is now `PendingInputResponse` as it's responding to a
request.
- `pendingInput` is now renamed and exported as `SweeperInput`.
- `pendingInputs` is now renamed and exported as `InputsMap`.
This commit is first made from running:
```
gofmt -d -w -r 'PendingInput -> PendingInputResponse' .
gofmt -d -w -r 'pendingInput -> SweeperInput' .
gofmt -d -w -r 'pendingInputs -> InputsMap' .
```
And followed by some docs and variable names fixes.
There's no need use the prefix `pending` as the inputs in the sweeper
can only be pending, so it's renamed, also to avoid the confusion with
the type `pendingInputs`.
This commit adds a private type `mSatPerKWeight` that expresses a given
fee rate in millisatoshi per kw. This is needed to increase the
precision of the fee function. When sweeping anchor inputs, if using a
deadline delta of over 1000, it's likely the delta will be 0 sat/kw due
to precision.
This commit finishes the implementation of `TxPublisher` by adding the
monitor process. Whenever a new block arrives, the publisher will check
all its monitored records and attempt fee bumping them if necessary.
This commit adds `TxPublisher` which implements `Bumper` interface. This
is part one of the implementation that focuses on implementing the
`Broadcast` method which guarantees a tx can be published with
RBF-compliant. It does so by leveraging the `testmempoolaccept` API,
keep increasing the fee rate until an RBF-compliant tx is made and
broadcasts it.
This tx will then be monitored by the `TxPublisher` and in the following
commit, the monitoring process will be added.
This commit adds the method `MaxFeeRateAllowed` to calculate the max fee
rate. The caller may specify a large MaxFeeRate value, which cannot be
cover by the budget. In that case, we default to use the max feerate
calculated using `budget/weight`.
This commit adds a new interface, `FeeFunction`, to deal with
calculating fee rates. In addition, a simple linear function is
implemented, hence `LinearFeeFunction`, which will be used to calculate
fee rates when bumping fees. Check #4215 for other type of fee functions
that can be implemented.
As shown in the following commit, fee rate calculation will now be
handled by the fee bumper, hence there's no need to expose this on
`InputSet` interface.
This commit adds a new interface, `Bumper`, to handle RBF for a given
input set. It's responsible for creating the sweeping tx using the input
set, and monitors its confirmation status to decide whether a RBF should
be attempted or not.
We leave implementation details to future commits, and focus on mounting
this `Bumper` interface to our sweeper in this commit.
This commit adds `BudgetAggregator` as a new implementation of
`UtxoAggregator`. This aggregator will group inputs by their deadline
heights and create input sets that can be used directly by the fee
bumper for fee calculations.
This commit adds `BudgetInputSet` which implements `InputSet`. It
handles the pending inputs based on the supplied budgets and will be
used in the following commit.
This commit changes `markInputsPendingPublish` to take `InputSet` only.
This is needed for the following commits as we won't be able to know the
tx being created beforehand, yet we still want to make sure these inputs
won't be grouped to another input set as it complicates our RBF process.
This commit changes how `WithCoinSelectLock` is used - previously the
lock is held when creating the input sets, now it's only be held after
the input sets have been created and explicitly signal they need wallet
inputs.
This commit makes the `ClusterInputs` directly returning the `InputSet`
so the sweeper doesn't know about the existence of `Cluster` interface.
This way we can have a deeper interface as the sweeper only needs to
interact with `Aggregator` only to get the final input sets, leaving the
implementation details being managed by `SimpleAggregator` and future
aggregators.
Previously the fee rate is tracked at cluster level, which may not be
accurate as each cluster is then divided into input sets. And these sets
are what's actually included in the final sweeping tx. To properly
reflect the final fee rate used in the sweeping tx, `InputSet` is added
so more customized clustering logic can be implemented in the future.
For intance, atm it's clustered by fee rates, while we may also cluster
by deadlines, urgencies, etc.
This commit adds a new interface `Cluster` to manage cluster-level
inputs grouping. This new interface replaces the `inputCluster` and will
be futher refactored so the sweeper can use a much smaller coin
selection lock.
This commit changes the source that drives the state changes in the
sweeper. Previously we used a ticker with default interval of 30s to
trigger sweepings periodically. The assumption is, within this 30s we'd
batch multiple inputs into one transaction to maximize profits. However,
the efficacy of this batch is questionable.
At a high level, we can put our inputs into two categories - one that's
forced, and one that's not. For forced inputs, we should sweep them
immediately as the force flag indicates they are very urgent, eg,
CPFPing the force closing tx. For non-forced inputs, such as anchors
or HTLCs with CLTV that's far away, we can wait to sweep them till a new
block comes in and triggers the sweeping process.
Eventually, all inputs will be deadline-aware, and the sweeper will
consult our fee bumper about the most economical fee rate to be used for
a given deadline. Since the deadlines here are blockstamp, it's also
easier to manage them if the sweeper is also using blockstamp instead of
timestamp.
sweeper
This commit implements a new method, `LookupInputMempoolSpend` to do
lookups in the mempool. This method is useful in the case when we only
want to know whether an input is already been spent in the mempool by
the time we call.
This commit removes the logic where we remove an input when it's been
published more than 10 times. This is needed as in our future fee
bumper, we might start with a low fee and rebroadcast the same input for
hundred of blocks.
This commit changes how a new input sweep request is handled - now we
will query the mempool and see if it's already been spent. If so, we'll
update its state as we may need to RBF this input.
This commit refactors the grouping logic into a new interface
`UtxoAggregator`, which makes it easier to write tests and opens
possibility for future customized clustering strategies.
The old clustering logic is kept as and moved into `SimpleAggregator`.
This commit adds a new interface `FeePreference` which makes it easier
to write unit tests and allows more customized implementation in
following commits.
This commit moves `DetermineFeePerKw` into the `Estimate` method on
`FeePreference`. A few callsites previously calling `DetermineFeePerKw`
without the max fee rate is now also temporarily fixed by forcing them
to use `Estimate` with the default sweeper max fee rate.
This commit refactors the sweeper so the method `feeRateForPreference`
is now moved to `FeePreference`, which makes our following refactor
easier to handle.
This commit modifies the sweeper store to save a `TxRecord` in db
instead of an empty byte slice. This record will later be used to bring
RBF-awareness to our sweeper.
This commit introduces a RecordProducer implementation for Record
that serves as the identity function. This makes it easier for us
to mix Primitive and Dynamic records in the same RecordProducer
collections.
Fixes#8626.
This commit updates btcwallet to the latest version that correctly
includes the MasterKeyFingerprint for all generated addresses, including
change addresses derived from imported watch-only accounts.
Previously if the `reverse` named arg was unset (value of NULL), then
SQL would order by NULL instead of ID causing undifined ordering of the
returned rows. To fix that we check for NULL and also make sure to set
the `reverse` arg in the code explicitly as it in the generated code it
is an `interface{}` instead of `bool`.
For SQL transactions, we often accumulate results in variables declared
outside the closure's scope. To eliminate the need for manually clearing
these containers, we introduce a reset function to ExecTx, mirroring the
approach already adopted in kvdb.
In this commit, we fix a bug that would cause the entire db to shutdown
if hit a panic (since db operations in the main buckets exit with a
panic) while executing a txn call back. This might be a postgres error
we need to check, so we don't want to bail out, and instead want to pass
up the error to the caller so we can retry if needed.
With the new postgres concurrency control, an error may come from a
bucket function that's actually a postgres error. In this case, we need
to return early so we can retry the txn. Otherwise, we'll be working
with an aborted tx, and never actually return the error so we don't auto
retry.
Some sub-systems like btcwallet will return an error from the database,
but they won't properly wrap it. As a result, we were unable to actually
catch the serialization errors in the first place. To work around this,
we'll now attempt to parse the error string directly.
This commit turns off route blinding for the daemon while we're waiting
on full handling for blinded errors. The feature remains on for itests
so that tests covering blinding can run as usual.
We don't support receiving blinded in this PR - just intercept and
settle instead. The HTLC's arrival on the interceptor indicates that
it was successfully forwarded on a blinded hop.
Reject any HTLCs that use us as an introduction point in a blinded
route if we have disabled route blinding. We have to do this after
we've processed the payload, because we only know we're an introduction
point once we've processed the payload itself.
If we received a payload with a encrypted data point set, our forwarding
information should be set from the information in our encrypted blob.
This behavior is the same for introduction and relying nodes in a
blinded route.
To separate blinded route parsing from payload parsing, we need to
return the parsed types map so that we can properly validate blinded
data payloads against what we saw in the onion.
When we have a HTLC that is part of a blinded route, we need to include
the next ephemeral blinding point in UpdateAddHtlc for the next hop. The
way that we handle the addition of this key is the same for introduction
nodes and relaying nodes within the route.
This commit introduces a blinding kits which abstracts over the
operations required to decrypt, deserialize and reconstruct forwarding
data from an encrypted blob of data included for nodes in blinded
routes.
Add an option to disable route blinding, failing back any HTLC with
a blinding point set when we haven't got the feature enabled.
Note that this commit only handles the case where we're chosen as the
relaying node (where the blinding point is in update_add_htlc), we'll
add handling for the introduction node case once we get to handling of
blinded payloads).
This reverts commit 717facc202.
It turns out that we can't do this since it would result in
incompatibility with LDK. The spec has been updated to reallow
optional gossip queries so we revert this commit.
In this commit, we add the coin selection strategy option to the following
on-chain RPCs `fundpsbt`, `batchopenchannel`, `estimatefee`, `sendcoins`,
`sendmany`, and `sendoutputs`.
In this commit we add coin selection strategy option to the
following on-chain rpc commands `fundpsbt`, `fundtemplatepsbt`,
`batchopenchannel`, `estimatefee`, `sendcoins`, and `sendmany`.
In this commit, we add the coin selection strategy option
to all on-chain RPCs `FundPsbt`, `BatchOpenChannel`, `EstimateFee`,
`SendMany`, `SendCoins`, `SendOutputs`.
In this commit, the tlv extension of a channel update message is parsed.
If an inbound fee schedule is encountered, it is reported in the
graph rpc calls.
As SQL serializaton errors can lead to transaction retries we need to
ensure that we reset any out of scope containers where we accumulate
results. Otherwise partial data from a previously executed transacton
retry may be returned along with the latest result.
This field is incorrectly suffixed as "msat", when it is actually
interpreted as the proportional fee rate. This is the value that we
should be using because the sender will calculate proportional fees
accordingly. This is a breaking change to the RPC, but on an
experimental and unreleased API.
When we have payments inside of a blinded route, we need to know
the incoming amount to be able to back-calculate the amount that
we need to forward using the forwarding parameters provided in the
blinded route encrypted data. This commit adds the payment amount
to our DecodeHopIteratorRequest so that it can be threaded down to
payment forwarding information creation in later commits.
This commit adds an optional blinding point to payment descriptors and
persists them in our HTLC's extra data. A get/set pattern is used to
populate the ExtraData on our disk representation of the HTLC so that
callers do not need to worry about the underlying storage detail.
Add blinding points to update_add_htlc. This TLV will be set for
nodes that are relaying payments in blinded routes that are _not_
the introduction node.
This commit adds encoding and decoding for blinded route data blobs.
TLV fields such as path_id (which are only used for the final hop)
are omitted to minimize the change size.
We'll need to pack feature vectors for route blinding, so we pull
the encoding/decoding out into separate functions (currently
contained in ChannelType). Though it's more lines of code, we keep
most of the ChannelType assertions so that we strictly enforce
use of the alias.
In this commit we add the LND node name to the temporary directory's
name. This change makes it easier to identify a particular test node's
temporary directory.
This commit also replaces the depreciated `ioutil.TempDir` function call
with `os.MkdirTemp`.
This commit introduces a filterManager that uses our outbound peers'
feefilter values to determine an acceptable minimum feerate that
ensures successful transaction propagation. Under the hood, a moving
median is used as it is more resistant to shocks than a moving average.
This commit adds a check to ensure that we don't start LND with native
SQL invoice DB if we already have any invoices in our KV channeldb. This
is needed as native SQL invoices is an experimental feature and we do
not yet support migration.
This commit removes a merge artifact introduced by a rebase of a
previous PR.
While we're at it, we streamline the grammar, formatting and general
look-and-feel of the release notes.
When determining the max fee rate of a channel we used to scale
the fee rate depending on our available local balance on this channel.
This lead to a special case that if a channel would be drained we
could especially decrease the fee rate even down to the fee floor.
Now we make sure that our max fee rate will not be lower than the
old fee rate to make sure in case our channel is locally drained
we do not continue to decrease fees too low.
Bitcoind will not report any fee estimation in case it has not
enough data available. We used to just set the min mempool fee
in such cases but this might not represent the current fee situation
of the bitcoin network. We return an error now so that we will use
the fallback fee instead.
In the tower session itest, we make sure to mine blocks on session count
assertion failure. This required because the session count is expected
to change only when 2 things both happen: 1) a closable session is
queued and 2) a new block notification comes through _after_ the session
has been queued. Only then will it be deleted. So we need to do this to
prevent the case where all the block notifications are consumed _before_
the session is queued for deletion. This flake tended to happen in the
windows itest.
Because we pass in the same config into multiple notifiers, we sometimes
get a data race in the unit tests. By creating a copy of the config we
avoid that.
Since we fixed a number of issues in chainntnfs.NewBitcoindBackend that
makes it compatible with bitcoind v26.0, we now want to use that
function in all our unit tests.
With the chainntnfs.NewMiner now being optimized for not creating
nodes with colliding ports, we use it in all unit tests that spin up
temporary miners.
This commit fixes a number of issues with our temporary bitcoind nodes
that we spin up for unit tests:
- We didn't remove the node's temporary data dir after tests finished.
- We used random ports which lead to unwanted port collisions.
- We used ipc:// unix sockets for ZMQ which currently isn't supported
in bitcoind v26.0 due to a regression. Since we can reliably create
new non-colliding ports now, we should use TCP for ZMQ anyway.
With this commit we create a new function that returns system wide
unique ports by using a single file to keep track of previously used
ports. We'll want to use this everywhere whenever we need to listen on a
new, random port during unit or integration tests.
Because we now have a unique source, we don't need to apply the port
offset that was used for the different tranches of parallel running
integration tests before.
To make it less confusing what version of bitcoind is actually
installed, we now require the version to be specified as a command line
argument.
Because we tag the version in the docker image tag as bitcoin-core:XX
but the binary internally is located under /opt/bitcoin-XX.Y/, we
manually set Y to 0.
In this commit, we support utilizing the usage of enviroment variables in `lnd.conf` file
for `rpcuser` and `rpcpass` fields by adding `supplyEnvValue` function in `config.go`
that returns the value of the specified environment variable, the fall-back value
if provided, or the original input string if no matching environment variables are found or set.
To avoid build caches getting larger and larger, causing our builds to
fail, we separate the caches of the different job types.
If we don't separate them, then the caches can end up being a
combination of the different jobs, which isn't useful (for example the
build cache of the cross compilation build isn't useful in an
integration test and vice versa).
This commit moves the constants LndInternalLockID and
DefaultLockDuration from the walletrpc package to the chanfunding
package, moves DefaultReservationTimeout from lncfg to chanfunding,
and also updates the lncli package with the new location.
Allow package users to provide custom callback which will
execute whenever a healthcheck fails.
Allow package users to provide custom callback which will
execute whenever a healthcheck succeeds.
In this commit, rather than attempt to reduce the scope of the current
`CODEOWNERS` file, we instead rename it as a way to stop Github from
automatically assigning reviewers based on it. Instead, we put things in
a "hint" mode as a way for PR OPs to figure out who they should manually
assign PRs today.
With the CODEOWNERS file reduced to a code review assignment hint, we
aim to instead promote usage of the `@code-review` team which will use a
load balancing algorithm for automatic code review assignment.
The purpose of this interface is to distinguish messages that are
bound to a particular channel_id from those that are not. There is
no reason this ought to be a property of the peer package as it is
entirely determined by the contents of the message itself. Therefore
we move it to the lnwire package so it can be used without having
import cycles elsewhere in the codebase.
Here we notice that the only use of the Peer call on the link is
to find out what the peer's pubkey is. To avoid leaking handles to
IO actions outside the interface we reduce the surface area to
just return the peer's public key.
This commit gives the linter container access to the local machine's
build and module cache, drastically decreasing the run time of
subsequent linter runs (no difference on first run with this change).
In this commit the mission control based fee estimation
is supplemented with a payment probe estimation which can
lead to more accurate estimation results. The probing
utilizes a hop-hint heurisic to detect routes through LSPs
in order to manually estimate fees to destinations behind
an LSP that would otherwise block the payment probe.
In the year of our lord 2024 we should not be writing for loops
for standard operations. Here we introduce named slice operations
not found in the golang slices package. Note all these functions
are pure.
In this commit, we bump the version of the master branch to v0.17.99.
This reflects the fact that master will be a super set of all the minor
releases until v0.18, and will also include changes towards v0.18.
This commit adds an optional separate "native" SQL backend that will
host all tables supporting native SQL. For now since we don't have many
such tables we'll keep it in one file for SQLite, but it may be split up
if desired.
This commit is part of a refactor that unifies configuration of the
sqldb and kvdb packages for SQL backends.
In order to unify the SQLite and Postgres configuration under sqldb we
first need to ensure that the final config types are compatible with
the alreay deployed versions.
This change will enable us to use a single PostgreSQL container instead of
spawning new a one for each (parallel) unit test reducing overall test
runtime.
This commit attempts to fix some issues with the invoice store's schema that we
couldn't foresee before the implementation was finished. This is safe as the
schema has not been instantiated yet outside of unit tests. Furthermore the
commit updates invoice store SQL queries according to fixes in the schema as
well as to prepare the higher level implementation in the upcoming commits.
In this commit, we update new Taproot related TLVs (nonces, partial sig,
sig with nonce, etc). Along the way we were able to get rid of some
boiler plate, but most importantly, we're able to better protect against
API misuse (using a nonce that isn't initialized, etc) with the new
options API. In some areas this introduces a bit of extra boiler plate,
and where applicable I used some new helper functions to help cut down
on the noise.
Note to reviewers: this is done as a single commit, as changing the API
breaks all callers, so if we want things to compile it needs to be in a
wumbo commit.
This commit adds a new sub command to the wallet that allows using the
new funding option from a template.
Creating a new command is way easier for the user to understand than
adding multiple flags that are only valid in certain combinations.
For the itest in the next commit we'll need to be able to fetch the
input information for an address over RPC. The only piece missing is the
address' index, which we add in this commit. Everything else should be
derivable from the ListAddresses and ListAccounts calls.
In some situations (for example in Taproot Assets), we need to be able
to prove that an address is a bare BIP-0086 address that doesn't commit
to any script. We can do that by providing the BIP-0032 derivation info
and internal key.
In this commit, we add two new methods that simplify usage of an
handling a value wrapped in an Option. Thes methods allow a caller to
force the nil/None check up front, and either obtain the value, or fail
the current execution or test.
This should replace most usages of `UnsafeFromSome` where ever used
today.
In the following commits we'll add a new, third, funding option for
funding PSBTs: It will allow users to specify pre-selected inputs but
still request the wallet to perform coin selection up to the total
output sum amount.
This is a helper function that we will need to accurately determine the
weight of inputs specified in a PSBT.
Due to the nature of P2WSH and script-spend P2TR inputs, we can only
accurately estimate their weights if the full witness is already known.
So this helper function rejects inputs that use a script spend path but
don't fully specify the complete witness stack.
We want to re-use the logic that determines what change amount is left
over depending on whether we add or don't add a change output to a
transaction, respecting the change output's dust limit.
We'll want to be able to tell the coin selection algorithm that we
intend to add any change to an existing output instead of assuming that
a change output is always created.
If we add any left over change to an existing output, we can skip the
dust amount check as we assume the selected existing output already has
a non-dust amount requested (responsibility of the caller to assert).
Before this commit the coin selection logic in the chanfunding package
would always assume that there is a P2WSH funding output and potentially
a P2TR change output. But because we want to re-use the coin selection
for things other than just channel funding, we make the logic more
generic by allowing us to specify both the existing weight of the
transaction (the already known, static parts of the TX) as well as the
type of the potential change output we would use.
With this commit we prepare for the lnwallet channel funding logic to be
aware of the config-level coin selection strategy by adding it to the
wallet config.
In this commit, we start persisting shutdown info when we send the
Shutdown message. When starting up a link, we also first check if we
have previously persisted Shutdown info and if we have, we start the
link in shutdown mode meaning that it will not accept any new outgoing
HTLC additions and it will queue the shutdown message after any pending
CommitSig has been sent.
In this commit, we add the ability to start a link in shutdown mode.
This means that we immediately disable any new HTLC adds in the outgoing
direction and that we queue up a Shutdown message after the next
CommitSig message is sent (or immediately if no CommitSig message is
owed).
ShutdownInfo contains any info about a previous Shutdown message that we
have sent. This commit adds this type along with read and write methods
for it in the channel db. The existence of the ShutdownInfo on disk
represents the fact that we have previously sent the Shutdown message
and hence that we should resend it on re-establish.
This commit adds an itest to demonstrate that the following bug exists:
If channel Shutdown is initiated but then a re-connect is done before
the shutdown is complete, then the initiating node currently does not
properly resend the Shutdown message as required by the spec. This will
be fixed in an upcoming commit.
This commit moves calls to link.DisableAdds to outside link.OnCommitOnce
call backs. This is done so that if a user requests shutdown, then we
can immediately block any new outgoing HTLCs. It's only the sending of
the Shutdown message that needs to wait until after any pending
CommitSig message is sent.
In this commit, the `ChannelUpdateHandler`'s `EnableAdds` and
`DisableAdds` methods are adjusted to return booleans instead of errors.
This is done becuase currently, any error returned by these methods is
treated by just logging the error since today all it means is that the
proposed update has already been done. And so all we do today is log the
error. But in future, if these methods are updated to return actual
errors that need to be handled, then we might forget to handle them
correctly at the various call sights. So we instead change the signature
of the function to just return a boolean. In future, if we do need to
return any error, we will have to go inspect every call sight in any
case to fix compliation & then we can be sure we are handling the errors
correctly.
In this commit, we adjust the DeleteCommitmentUpdate method so that it
marks a session as Terminal (if there are updates to delete) since once
we have deleted a commitment update from a session - the session is no
longer useable.
This commit adds the DeactiateTower method to the wtclient.ClientManager
interface along with its implementation. A test is also added for the
new method.
This new method sets the tower's status to inactive so that it is not
loaded at startup as a candidate tower. We also ensure that a tower's
status is set to active if the CreateTower is called when the tower
already exists.
This commit updates the DeleteCommittedUpdate DB method to delete all of
a given session's committed updates instead of just one at a time. The
reason for this is that in an upcoming commit, we will introduce a
"Terminal" session state - once we have deleted a committed update for a
session it should be considered "Terminal" and there is never a case
where we would only want to delete one committed update and not the
rest. So we want these two actions (deleting committed updates of a
session and setting it's status to terminal) to be atomic.
This commit extracts the InvoiceDB construction from all invoice and
registry tests such that we can later on run subtests with multiple
backends without needing to use tags.
With the introducation of the `InvoiceUpdater` interface we are now
able to move the non-kv parts of `UpdateInvoice` completely under
the invoices package. This is a preprequisite for being able to use
the same code-base for the sql InvoiceDB implementation of
UpdateInvoice.
This commit introduces the InvoiceUpdater interface which is meant
to abstract and assist the in-memory invoice update procedure with
the accompanying database updates. These abstract updater steps will
enable further refactoring later while also ensuring that a full
SQL implementation of the InvoiceDB interface will be possible.
With this commit updateInvoiceAmpState becomes getUpdatedInvoiceAmpState
which will only return the new AMP state but that needs to be applied at
the call site. This is a part of a larger refactor to gather all
mutations of an invoice update to be later applied by the invoice
updater.
This change moves the HTLC state change out of the cancelSingleHtlc
function. This is part of the larger refactor of collecting all changes
to be later applied by the invoice updater.
With this refactor updateHtlc is renamed to getUpdatedHtlcState and
changed such that it won't change the HTLC's state and resolve time but
instead returns whether the change is needed. This change is part of a
multi-commit refactor to ensure that all changes to the invoice will be
tracked individually.
This commit is a small refactor to move all actual DB updates after
an invoice state is update to separate methods. This is a small
preliminary change before we completely decouple DB updates from
in-memory invocie update.
Export MINING_ADDRESS before docker-compose up to ensure it's retained for
subsequent operations, fixing block generation errors due to missing mining
addresses.
Since we have two other examples of XArbitrator, we rename
BreachArbiter to BreachArbitrator to keep things consistent.
The aim is to reduce the amount of lore you need to know to
intuit where things are or what they do.
These two messages will be used to implement the new and improved co-op
closing protocol. This PR also show cases how to use the new
`tlv.OptionalRecord` type to define and handle TLV level parsing.
I think we can make one additional helper function to clean up some of
the boiler plate for the encode/decode.
The final hop size is calculated differently therefore we extract
the logic in its own function and also account for the case where
the final hop might be a blinded hop.
In the previous commit the AdditionalEdge interface was introduced
with both of its implementations `BlindedEdge` and `PrivateEdge`.
In both cases where we append a route either by a blinded route
section or private route hints we now use these new types. In
addition the `PayloadSizeFunc` type is introduced in the
`unifiedEdge` struct. This is necessary to have the payload size
function at hand when searching for a route hence not overshooting
the max sphinx package size of 1300 bytes.
In this commit, we add a new `Zero` method for the `RecordT` type. This
method allows a caller to create the zero record for a type without
needing to reference the actual TLV type.
With this we go from this:
```go
sig1 := tlv.ZeroRecordT[tlv.TlvType1, Sig]()
```
To this:
```
sig1 := c.CloserNoClosee.Zero()
```
By default ListSweeps will list all sweeps known to the wallet. Over
time this may become expensive to call and callers may not be interested
in the full history. This commit adds a `startheight` parameter to the
`ListSweeps` RPC call. This parameter can be used to fetch sweeps only
from a specified start block height or only unconfirmed ones if it is
set to -1.
Over the last few commits we have systematically eliminated all but
two states. This allows us to replace it with a boolean to encode
the two remaining states. We would like to be able to eliminate this
field entirely, but doing so requires being able to prove that the
concurrent request block is necessary. This is more difficult and
will be left to future commits.
lnwallet: remove unused channelPendingPayment channelState
Since this state is never set nor read, we remove it completely.
lnwallet: remove redundant channelDispute channelState
In this case, even though we do set this value, it is never read.
Further, the times we read the field at all from LightningChannel
we want the situation of force-closure to block any other
concurrent closure attempts, so we change the sites where we set
channelDispute to channelClosed.
lnwallet: remove redundant channelClosing channelStatus
This value is never used to impact control flow so we need not set
it. We also need not have it.
By default ListSweeps will list all sweeps known to the wallet. Over
time this may become expensive to call and callers may not be interested
in the full history. This commit adds a `startheight` parameter to the
`ListSweeps` RPC call. This parameter can be used to fetch sweeps only
from a specified start block height or only unconfirmed ones if it is
set to -1.
A new interface AdditionalEdge is introduced which allows us to
enhance the CachedEdgePolicy with additional information. We
currently only need a PayloadSize function which is then used to
determine the exact payload size when building a route to adhere
to the sphinx package size limit (BOLT04).
Also fix a case where this deprecated flag is used. We will always
bypass the active channels check when `DisconnectPeer` because
`!r.cfg.UnsafeDisconnect` is always false.
This commit fixes a heap escape found in `GetBlock`. This happens
because the `msgBlock` is a pointer returned from `getBlockImpl`, and
the whole `getBlockImpl` escapes to the heap because it's referenced in
two places,
- in the `Cache`, it's used as a reference type to create the new block.
- this pointer is also returned and now needs to stay on the heap.
The fix is simple, we now make a copy of the block and use the copy
instead, freeing `getBlockImpl` from the heap.
bitcoind v25.0 updated the `sendrawtransaction` RPC to have an optional
argument `maxburnamount` with a default value of 0. This means our
existing test that uses burning output cannot be published, hence, we
remove the usage of it in our tests and replace it with a normal tx.
We should not fail to start the node if a republish attempt failed for a
channel's closing tx. Instead, we log an error to continue the startup
and let other channels continue their operations.
Previously, `PublishTransaction` in `btcwallet` will first mark the tx
label in db first before broadcasting it to the network. Even though the
broadcast may fail with the error already in mempool or already
confirmed, this tx label updating is still performed. To maintain the
old behavior, we now catch the errors from `TestMempoolAccept`, and make
sure to call the `PublishTransaction` to mark the tx label even there
are errors from mempool acceptance check.
This commit adds a mempool acceptance check before broadcasting a given
transaction. To maintain the current behavior from
`BtcWallet.PublishTransaction`, the two errors, `ErrInMempool` and
`ErrAlreadyConfirmed` returned from `TestMempoolAccept` are ignored.
This builds on the concurrent queue to create a generic way to allow
goroutines to pub/sub information. An example includes being notified
each time a state machine is able to carry out a new state transition.
We do things this way to keep behavior consistent across REST, gRPC
and CLI consistent. This was done to not alter the way we handle
Recv calls from the streams.
This commit fixes a heap escape found in `GetBlock`. This happens
because the `msgBlock` is a pointer returned from `getBlockImpl`, and
the whole `getBlockImpl` escapes to the heap because it's referenced in
two places,
- in the `Cache`, it's used as a reference type to create the new block.
- this pointer is also returned and now needs to stay on the heap.
The fix is simple, we now make a copy of the block and use the copy
instead, freeing `getBlockImpl` from the heap.
The error was never used as the init couldn't return an error, so we do
away with that. We also modify the main event loop dispatch to more
closely match other areas of the codebase.
In this commit, we make all calls to disconnect after a ping/pong
violation is detected in the `PingManager` async. We do this to avoid
circular waiting that may occur if the disconnect call back ends up
waiting on the peer goroutine to be torn down. If this happens, then the
peer goroutine will be blocked on the ping manager fully tearing down,
which is blocked on the peer disconnect succeeding.
This is a similar class of issue we've delt with recently as pertains to
the peer and the server: sync all back execution must not lead to
a circular waiting loop.
Fixes#8379
When the link is flushing in the incoming direction, it means
adds are invalid. The best chance we have at dealing with this
is to drop the connection. This should roll back the channel
state to the last CommitSig. If the remote has already sent a
CommitSig we haven't received yet, channel state will be
re-synchronized with a ChannelReestablish message upon
reconnection and the protocol state that caused us to flush
the link will be rolled back. In the event that there was some
non-deterministic behavior in the remote that caused them to
violate the protocol, we have a decent shot at correcting it
this way, since reconnecting will put us in the cleanest
possible state to try again.
We don't need to try a link shutdown when the chan closer is fetched
since, by this commit, the only callsite manages the shutdown semantics.
After removing the call to tryLinkShutdown, we no longer need the
function at all.
In order to handle shutdown requests when there are still HTLCs on
the link, we have to manage the shutdown process via the link's
lifecycle hooks. This means we can't use the simple `tryLinkShutdown`
anymore and instead queue a `Shutdown` message at the next opportunity
to do so -- when we send our next `CommitSig`
This PR addresses the following:
- Install and Configure protolint to enforce the protobuf style guide rules in the CI.
- Fix the protolinting issues (package and import ordering) while maintaining the comaptibility.
In this commit, the FetchChanInfos ChannelGraph method is updated to
take in an optional read transaction for the case where it is called
from within another transaction.
In order to emphasise the fact that the ChannelGraph's cacheMu should be
acquired before calling the `delChannelEdge` method, we add the `Unsafe`
postfix and add a comment to alert readers.
In this commit, we ensure that the channeldb cacheMu mutex is only ever
aquired _before_ acquiring the main DB lock in the cases where the two
locks need to be held simultaneously.
With this commit, the deadlock demonstrated in the previous commit is
now fixed.
This commit adds a test that calls many of the ChannelGraph methods
concurrently and in a random order. This test demonstrates that a
deadlock currently exists in the ChannelGraph since the test does not
complete. This is fixed in the next commit.
In this commit, we let the explicit wtclient.Manager struct be passed in
to PopulateDependencies instead of the tower client interface. We need
to do this since we do allow the tower client interface to be nil if the
client is not active. So to avoid the golang gotcha where the interface
value will be seen as not nil even though the underlying value is nil,
we pass in the explicit pointer instead.
This commit adds a new FlagTaprootChannel Flag which is then used to
construct a new blob Type: TypeAltruistTaprootCommit. New watchtower
feature bits are also added (4/5).
Before this commit, in the watchtower client's DiskOverflowQueue, there
could be a situation _if no consumer was reading from the queue_ that
could lead to an in-memory build up of backup tasks instead of the
expected disk overflow. This commit fixes this by ensuring that if a
task is read from disk, then the mode is set to disk mode to ensure that
any new items are persisted to disk in this scenario.
The unit tests added in the previous commit is updated here in order to
show that the fix does in-fact fix the test.
In this commit, some temporary variables and logic is added to the
DiskOverflowQueue for easy stop/go control from unit tests. This is then
used to write a temporary unit tests that demonstrates a race condition
that can cause the queue to be in disk mode when it should be in memory
mode. This new code & test will be removed after the issue has been
fixed.
In this commit, we use the newly added session listing options to ensure
that we only see a session as exhausted if it does not have any un-acked
updates on disk. This fixes the bug previously demonstrated.
In this commit, we adjust the PostEvaluateFilterFn to also take in a
count representing the number of committed updates (ie, persisted
un-acked updates) that the session has. This will be made use of in an
upcoming commit.
This commit adds a test to demonstrate an edge case that can result in
the "tower has un-acked updates" error being thrown when a user is
attempting to remove a tower. This will be fixed in an upcoming commit.
The AMP struct in a hop was never being set when deserizlied. Also,
the AMP TLV record was not being added when the hop was serialized.
This sets the TLV record when serializing and correctly sets the
AMP struct on the hop when that record is present.
Co-authored-by: BitcoinCoderBob <90647227+BitcoinCoderBob@users.noreply.github.com>
Co-authored-by: Tee8z <tee8z@protonmail.com>
This commit adds three optional command line flags to the
encryptdebugpackage: --peers, --onchain and --channels.
Each of them adds the output of extra commands to the encrypted debug
package.
With this commit we add a new way to encrypt and decrypt a sensitive
payload: By using Diffie-Hellman over a local and remote key to derive
at a shared secret key.
With this commit we add a new helper function that recursively turns the
runtime configuration into a flat key/value map that is human-readable,
using the dot notation for nested values that is also used in the config
file or command line flags.
We take into account a fee buffer of twice the current fee rate
of the commitment transaction plus an additional htlc output
when we are the opener of the channel hence pay when publishing the
commitment transaction. This buffer is not consensus critical
because we only consider it when we are in control of adding a
new htlc to the state. The goal is to prevent situations
where we push our local balance below our channel reserve due to
parallel adding of htlcs to the state. Its not a panacea for these
situations but until we have __option_simplified_update__ deployed
widely on the network its a good precaution to protect against
fee spikes and parallel adding of htlcs to the update log.
Moreover the way the available balance for a channel changed.
We now need to account for a fee buffer when we are the channel
opener. Therefore all the tests had to be adopted.
In this commit, we convert the `JusticeKit` struct to an interface.
Then, we add two implementations of that interface:
1) The `legacyJusticeKit` which implements all the methods of
`JusticeKit`
2) The `anchorJusticKit` which wraps the `legacyJusticeKit` and just
re-implements the `ToRemoteOutputSpendInfo` method since.
In preparation for the next commit which will introduce the `JusticeKit`
interface, here we just move the code related to building the actual
justice kit packet into a separate file.
In this commit a new enum, CommitmentType, is introduced and initially
there are 3 CommitmentTypes: Legacy, LegacyTweakless and Anchor.
Then, various methods are added to `CommitmentType`. This allows us to
remove a bunch of "if-else" chains from the `wtclient` and `lookout`
code. This will also make things easier to extend when a new commitment
type (like Taproot) is added.
We add a WhenSome that'll pass in the actual underlying value, and not
just the record.
We also add `SomeRecord` to make it easier to create TLV records w/
`Some` set.
This will be useful when decoding optional TLV records, as we can use
this to look up in the `typeMap` for a given field to see if we decoded
it or not.
With this change, callers can now examine a struct instance of the
`TlvType` instance to figure out programmatically which integer type is
maps to. This'll be useful when decoding optional TLV types into a wire
struct (knowing when to set it to Some vs None).
In this commit, we modify the RecordT type to allow callers to re-use
the Record definition of a different type, but use the new type param to
override the integer type used on the wire.
This will let use do things like encode a signature using the same
RecordProducer instance, but with a diff type in another context.
The upcoming use for this is allowing our `lnwire.Sig` type to be
encoded in the same message using distinct TLV integer types (new co-op
close protocol).
We do not expect blinding errors from the final node:
1. If the introduction is the recipient, they should use regular errors.
2. Otherwise, nodes have no business sending this error when they are
not part of a blinded route.
Note: this refactor updates the inequality used from >= 2 to > 1 to
align with the rest of this file so that we express this concept
consistently throughout the code.
This commit adds handling for route blinding errors that are reported
by the introduction node in a multi-hop blinded route. As the
introduction node is always responsible for handling blinded errors,
it is not penalized - only the final hop is penalized to discourage the
blinded route without filling up mission control with ephemeral
results.
If this error code is reported by a node that is not an introduction
node, we penalize the node because it is returning an error code that
it should not be using.
This commit adds handling for errors that originate after the
introduction node when making payment to a blinded route. This
indicates that the introduction node is not obeying the spec, so
it is punished for the violation.
Fix our existing test to have a valid intermediate hop that will pass
stricter validation. Previously, we did not specify a next channel for
an intermediate hop (which violates bolt4).
Previously, we'd use the value of nextChanID to infer whether a payload
was for the final hop in a route. This commit updates our packing logic
to explicitly signal to account for blinded routes, which allow zero
value nextChanID in intermediate hops. This is a preparatory commit
that allows us to more thoroughly validate payloads.
Previously, we were using nextChanID to determine whether a hop
payload is for the final recipient. This is no longer suitable in a
route-blinding world where intermediate hops are allowed to have zero
nextChanID TLVs (as this information is provided to forwarding nodes
in their encrypted data). This commit updates payload reading to use
the signal provided by sphinx that we are on the last packet, rather
than implying it from the contents of a hop.
Update test to include the sphinx action to more closely represent
reality. This will be required when we add more validation to the
presence of a nextChanID field. A MoreHops action is chose because
we're testing the case with a payload that contains forwarding info.
In this commit, we add a new type, `RecordT[T, V]` to reduce some of the
common boiler plate for TLV types. This type lets you take either a
primitive type, or an existing Record, and gain common methods used to
create tlv streams.
It also serves as extra type annotation as well, since wire structs can
use this to wrap any existing type and gain the relevant record methods.
This implementation ensures that the very definition of the field also
binds the TLV type value. It does this by using the generated code to
map a struct like TlvType1 to an actually Type like Type(1).
This shouldn't need to be run again, as this implementation restricts
things to just values 0-99, due to a hard upper limit with the way Go
unions work under the hood.
In this commit, we add some new code generation to the codebase. As
we'll see in a future commit, this'll allow us to create a new Record[T,
V] type, where T is actually a concrete _struct_ that implements a
special interface that deems it as a valid TLV type.
Add the missing channel field to the final hop in our clear text
route test case. Note that this is the channel of the hop. With the
addition of stricter validation, we'll need this so that the
penultimate hop has a non-zero next channel ID.
The RemoveTransaction endpoint removes the transaction with the
provided txid including all its descendants from the internal wallet.
We still keep watching for the address of the transation in case
the transcation is confirmed nonetheless. This command is particular
useful for neutrino backends because new bitcoind versions do not
reply with an invalid transaction error code when the tx published
fails to be included into the mempool (fullnodes do).
For anchor channels and neutrino backends we need to make sure
that sweeps of the same exclusive group are removed when one of
them is confirmed. Otherwise for neutrino backends those sweep
transaction are always rebroadcasted and are blocking funds in
the worst case scenario.
This commit is a set-up commit. It extracts the logic from
`MarkEdgeLive` to a helper `markEdgeLive` method so that the logic can
be called from within other kvdb.Update blocks. This will be used in the
next commit.
In this commit, we make a new sub-module from the `fn` package. This
lets other projects use it w/o pulling in all of lnd, and also other
sub-packages within lnd.
We also add a temp replace directive for new module. We'll remove this
once we tag the final version.
This commit introduces a wrapper function fetchFundingTxWrapper
which calls fetchFundingTx in a goroutine. This is to avoid an issue
with pruned nodes where the router is attempting to stop, but the
prunedBlockDispatcher is waiting to connect to peers that can serve
the block. This can cause the shutdown process to hang until we
connect to a peer that can send us the block.
This commit introduces a wrapper function fetchFundingTxWrapper
which calls fetchFundingTx in a goroutine. This is to avoid an issue
with pruned nodes where the router is attempting to stop, but the
prunedBlockDispatcher is waiting to connect to peers that can serve
the block. This can cause the shutdown process to hang until we
connect to a peer that can send us the block.
This commit moves over the last two methods, `RegisterChannel` and
`BackupState` from the `Client` to the `Manager` interface. With this
change, we no longer need to pass around the individual clients around
and now only need to pass the manager around.
To do this change, all the goroutines that handle channel closes,
closable sessions needed to be moved to the Manager and so a large part
of this commit is just moving this code from the TowerClient to the
Manager.
This commit removes the mutex from ClientStats and instead puts that in
clientStats which wraps ClientStats with a mutex. This is so that the
tower client interface can return a ClientStats struct without worrying
about copying a mutex.
Simiarly to the previous commit, this commit moves the RemoveTower
method from the Client to the TowerClientManager interface. The manager
handles any DB related handling. The manager will first attempt to
remove the tower from the in-memory state of each client and then will
attempt to remove the tower from the DB. If the removal from the DB
fails, the manager will re-add the tower to the in-memory state of each
client.
In this commit we move the AddTower method from the Client interface to
the TowerClientManager interface. The wtclientrpc is updated to call the
`AddTower` method of the Manager instead of calling the `AddTower`
method of each individual client. The TowerClient now is also only
concerned with adding a new tower (or new tower address) to its
in-memory state; the tower Manager will handle adding the tower to the
DB.
In this commit, the `Stop` and `Start` methods are removed from the
`Client` interface and instead added to the new `Manager`. Callers now
only need to call the Manager to start or stop the clients instead of
needing to call stop/start on each individual client.
Introduce a wtclient `Manager` which handles tower clients. It indexes
clients by the policy used. The policy field is thus removed from the
`Config` struct which configures the Manager and is instead added to a
new `towerClientCfg` which configures a specific client managed by the
manager. For now, only the `NewClient` method is added to the Manager.
It can be used to construct a new `TowerClient`. The Manager currently
does notthing with the clients added to it.
This commit also adds tests for the DB changes made in the previous
commit since we can now read the new field with the FetchChanInfos
method.
The commit following this one does the backfill migration.
In this commit, a new key, cChanMaxCommitmentHeight, is added to the
channel details bucket. This key will hold the highest commitment number
that the tower has been handed for this channel. In this commit, we
start writing to it in the two places where a backup is first persisted
in the tower client db: 1) CommitUpdate and 2) in the Queue's `addItem`
method. The logic for both 1 & 2 is tested in the next commit which adds
a DB helper that allows us to read the new field.
A follow up commit will do a migration to back-fill the new field.
This is to avoid a potential race on WriteMessage and Flush internals.
Because there is no locking on WriteMessage and Flush, if we allow
writeMessage calls in Start after the writeHandler has started,
the writeMessage calls may call WriteMessage/Flush at the same time
that writeMessage calls from the writeHandler does. Since there is
no locking, internals like b.nextHeaderSend can race and cause
panics.
This is to avoid a potential race on WriteMessage and Flush internals.
Because there is no locking on WriteMessage and Flush, if we allow
writeMessage calls in Start after the writeHandler has started,
the writeMessage calls may call WriteMessage/Flush at the same time
that writeMessage calls from the writeHandler does. Since there is
no locking, internals like b.nextHeaderSend can race and cause
panics.
Running test cases in parallel with shared state means that they
can intermingle if they're run at the same time and set up incorrect
values when we assert on the number of channels we have. Moving the
peer setup into the parallel test run fixes this because no single
test case can interfere with the other.
In processZombieUpdate, the SCID passed to MarkEdgeLive should _not_ be
derived from the ChannelEdgeInfo ChannelID field since this field will
not be populated when GetChannelByID returns a ChannelEdgeInfo along
with an ErrZombieEdge error. So this commit ensures that a usable
SCID is provided to processZombieUpdate.
Let MarkEdgLive return a new ErrNotZombieEdge error if an entry with the
given channel ID cannot be found. In processZombieUpdate, we then
check for this error and log accordingly.
This commit makes sure a testing payment is created via
`createDummyLightningPayment` to ensure the payment hash is unique to
avoid collision of the same payment hash being used in uint tests. Since
the tests are running in parallel and accessing db, if two difference
tests are using the same payment hash, no clean test state can be
guaranteed.
This commit adds unit tests for `resumePayment`. In addition, the
`resumePayment` has been split into two parts so it's easier to be
tested, 1) sending the htlc, and 2) collecting results. As seen in the
new tests, this split largely reduces the complexity involved and makes
the unit test flow sequential.
This commit also makes full use of `mock.Mock` in the unit tests to
provide a more clear testing flow.
The old payment lifecycle is removed due to it's not "unit" -
maintaining these tests probably takes as much work as the actual
methods being tested, if not more so. Moreover, the usage of the old
mockers in current payment lifecycle test is removed as it re-implements
other interfaces and sometimes implements it uniquely just for the
tests. This is bad as, not only we need to work on the actual interface
implementations and test them , but also re-implement them again in the
test without testing them!
This commit adds a new interface method `TerminalInfo` and changes its
implementation to return an `*HTLCAttempt` so it includes the route for
a successful payment. Method `GetFailureReason` is now removed as its
returned value can be found in the above method.
This commit adds a new struct, `stateStep`, to decide the workflow
inside `resumePayment`.
It also refactors `collectResultAsync` introducing a new channel
`resultCollected`. This channel is used to signal the payment
lifecycle that an HTLC attempt result is ready to be processed.
This commit makes sure we only fail attempt inside `handleSwitchErr` to
ensure the orders in failing payment and attempts. It refactors
`collectResult` to return `attemptResult`, and expands `handleSwitchErr`
to also handle the case where the attemptID is not found.
This commit refactors the `resumePayment` method by adding the methods
`checkTimeout` and `requestRoute` so it's easier to understand the flow
and reason about the error handling.
This commit removes the method `launchShard` and splits its original
functionality into two steps - first create the attempt, second send the
attempt. This enables us to have finer control over "which error is
returned from which system and how to handle it".
This commit starts handling switch error inside `sendAttempt` when an
error is returned from sending the HTLC. To make sure the updated
`HTLCAttempt` is always returned to the callsite, `handleSwitchErr` now
also returns a `attemptResult`.
This commit removes the `launchOutcome` and `shardResult` and uses
`attemptResult` instead. This struct is also used in `failAttempt` so we
can future distinguish critical vs non-critical errors when handling
HTLC attempts.
`handleSwitchErr` is now responsible for failing the given HTLC attempt
after deciding to fail the payment or not. This is crucial as
previously, we might enter into a state where the payment's HTLC has
already been marked as failed, and while we are marking the payment as
failed, another HTLC attempt can be made at the same time, leading to
potential stuck payments.
This commit removes the unclear abstraction `shardHandler` that's used
in our payment lifecycle. As we'll see in the following commits,
`shardHandler` is an unnecessary layer and everything can be cleanly
managed inside `paymentLifecycle`.
Without this the following could happen:
* InboundPeerConnected is called while we already have an inbound
connection with the peer. This calls removePeer which calls Disconnect.
* If the peer is starting up in Start, it may be sending messages
synchronously via SendMessage(true, ...). This eventually calls the
writeMessage function which will exit if disconnect is set to 1.
* Since Disconnect was called, disconnect will be 1 and writeMessage
will exit, causing writeHandler to exit.
* If there is more than 1 message being sent, later messages will
queue in queueHandler but be unable to get into sendQueue as the
writeHandler goroutine has exited.
* The synchronous sends will be waiting on the errChan indefinitely
and startReady will never get closed meaning Disconnect will never
proceed.
The end result is that the server's mutex will be held until shutdown.
Avoid this by using writeMessage to bypass the writeHandler goroutine.
Without this the following could happen:
* InboundPeerConnected is called while we already have an inbound
connection with the peer. This calls removePeer which calls Disconnect.
* If the peer is starting up in Start, it may be sending messages
synchronously via SendMessage(true, ...). This eventually calls the
writeMessage function which will exit if disconnect is set to 1.
* Since Disconnect was called, disconnect will be 1 and writeMessage
will exit, causing writeHandler to exit.
* If there is more than 1 message being sent, later messages will
queue in queueHandler but be unable to get into sendQueue as the
writeHandler goroutine has exited.
* The synchronous sends will be waiting on the errChan indefinitely
and startReady will never get closed meaning Disconnect will never
proceed.
The end result is that the server's mutex will be held until shutdown.
Avoid this by using writeMessage to bypass the writeHandler goroutine.
Finally, The LightningNode object is removed from ChannelEdgePolicy.
This is a step towards letting ChannelEdgePolicy reflect exactly the
schema that is on disk.
This is also nice because the `Node` object is not necessarily always
required when the ChannelEdgePolicy is loaded from the DB, so now it
only get's loaded when needed.
In preparation for the next commit which will remove the
`*LightningNode` from the `ChannelEdgePolicy` struct,
`FetchLightningNode` is modified to take in an optional transaction so
that it can be utilised in places where a transaction exists.
To prepare for the `kvdb.Backend` member of `ChannelEdgeInfo` being
removed, the `FetchOtherNode` method is moved from the `ChannelEdgeInfo`
to the `ChannelGraph` struct.
Having a `ForEachChannel` method on the `LightningNode` struct itself
results in a `kvdb.Backend` object needing to be stored within the
LightningNode struct. In this commit, this method is replaced with a
`ForEachNodeChannel` method on the `ChannelGraph` struct will perform
the same function without needing the db pointer to be stored within the
LightningNode. This change, the LightningNode struct more closely
represents the schema on disk.
The existing `ForEachNodeChannel` method on `ChannelGraph` is renamed to
`ForEachNodeDirectedChannel`. It performs a slightly different function
since it's call-back operates on Cached policies.
We've only ever made macaroons with the v2 versions, so we should
explicitly reject those that aren't actually v2. We add a basic test
along the way, and also add a similar check for the version encoded in
the macaroon ID.
Prior to this commit, taproot channels had a bug:
- If a disconnect happened before peer.AddNewChannel was called,
then the subsequent reconnect would call peer.AddNewChannel and
attempt the ChannelReestablish dance.
- peer.AddNewChannel would call NewLightningChannel with
populated nonce ChannelOpts. This in turn would call
InitRemoteMusigNonces which would create a new musig pair session
and set the channel's pendingVerificationNonce to nil.
- During the reestablish dance, ProcessChanSyncMsg would be called.
This would also call InitRemoteMusigNonces, except it would fail
since pendingVerificationNonce was set to nil in the previous
invocation.
To fix this, we add a new functional option to signal to the init logic
that it doesn't need to call InitRemoteMusigNonces in in
ProcessChanSyncMsg.
This commit updates all related tests to reflect the latest anchor
sweeping behavior. Previously, anchor sweeping is always attempted as
CPFP when a force close is broadcast, while now it only happens when the
deadline is less than 144. For non-CPFP purpose sweeping, it will happen
after one block is mined after the force close transaction is confirmed
as the anchor will be resent to the sweeper with a floor fee rate, hence
making it economical to sweep.
Since we now only perform CPFP when both the fee rate is higher and the
deadline is less than 144, we need to update the test to reflect that
Bob will not CPFP the force close tx for the channle Alice->Bob.
This commit changes from always sweeping anchor for a local force close
to only do so when there is an actual time pressure. After this change,
a forced anchor sweeping will only be attempted when the deadline is
less than 144 blocks.
This commit sorts wallet UTXOs by their values when using them for
sweeping inputs. This way we'd avoid locking large UTXOs when sweeping
inputs and also provide an opportunity to aggregate wallet UTXOs.
The link will send an update_fail_malformed_htlc, so we need to set
the BADONION bit. Since there isn't a replay-specific error, we
set the failure code to InvalidOnionVersion which has the BADONION bit.
We've only ever made macaroons with the v2 versions, so we should
explicitly reject those that aren't actually v2. We add a basic test
along the way, and also add a similar check for the version encoded in
the macaroon ID.
Prior to this commit, taproot channels had a bug:
- If a disconnect happened before peer.AddNewChannel was called,
then the subsequent reconnect would call peer.AddNewChannel and
attempt the ChannelReestablish dance.
- peer.AddNewChannel would call NewLightningChannel with
populated nonce ChannelOpts. This in turn would call
InitRemoteMusigNonces which would create a new musig pair session
and set the channel's pendingVerificationNonce to nil.
- During the reestablish dance, ProcessChanSyncMsg would be called.
This would also call InitRemoteMusigNonces, except it would fail
since pendingVerificationNonce was set to nil in the previous
invocation.
To fix this, we add a new functional option to signal to the init logic
that it doesn't need to call InitRemoteMusigNonces in in
ProcessChanSyncMsg.
This commit removes the interface methods `GenQueryURL` and
`ParseResponse` so the `WebAPIEstimator` can care less about the
implementation details of this interface.
This commit adds a new method `GetFeeMap` to handle querying the web API
and parse the response into a map that's used by the fee estimator. In
doing so we can provide a deeper abstraction over the `WebAPIFeeSource`
interface.
The test is identical to other onion failure fuzz tests, except that it
uses a custom equality function to get around the nil != []byte{} issue
with reflect.DeepEqual.
Fuzz tests for decoding and encoding of onion failure messages, based on
the fuzz harness for other lnwire messages. The onion failure messages
were uncovered by existing fuzz tests.
When we first go to boot up the syncer, when we're in the active phase,
after we do the historical sync, we'll send a timestamp message that we
want everything, then transition to the passive mode. The test didn't
account for this extra message before, as the last test was being
re-used here (ran in parallel).
We fix this by asserting that the first expected message is sent, then
also the follow up messages as well.
In this commit, [we prep for some upcoming changes in
Go](https://go.dev/blog/loopvar-preview) by running our tests with
`GOEXPERIMENT=loopvar`. This changes the loop semantics to fix a common
bug where a scoping issue causes a variable to be re-used, which can
cause unintended bugs down the line.
If everything passes with this flag on, then we can keep it on, and also
rest a bit easier knowing that the compiler will fix a class of bugs
that would previously pop up for us.
This commit changes how we create the input sets which are used to
construct the sweeping transactions. Assume the sweeper has two inputs,
one is new and one is retried, we'd end up having two transactions,
- tx1: which spends both the new and old inputs.
- tx2: which spends the new inputs only.
When publishing these txes, depending on which one gets into the mempool
first, the other one will be viewed as an RBF for the first one since
they both spending the same input(the new input).
This is now fixed by only attempt to publish the second tx when there
isn't a first tx - when there is a tx1, it means the new inputs are
already used in this tx along with retried inputs, hence there's no need
to publish tx2 which spends the new inputs only.
This commit attempts to make the polling logic in sweeper more linear.
Previously, the sweep's timer is reset/restarted in multiple places,
such as when a new input comes in, or a new block comes in, or a
previous input being spent, making it difficult to follow. We now remove
the old timer and replaces it with a simple polling logic - we will
schedule sweeps every 5s(default), and if there's no input to be swept,
we'd skip, just like the previous `scheduleSweep` does.
It's also worthy noting that, although `scheduleSweep` triggers the
timer to tick, by the time we do the actual sweep in `sweepCluster`,
conditions may have changed. This is now also fixed because we only have
one place to create the clusters and sweeps.
This commit refactors some of the bookkeeping around the ping logic
inside of the Brontide. If the pong response is noncompliant with
the spec or if it times out, we disconnect from the peer.
This change makes the generation of the ping payload a no-arg
closure parameter, relieving the pingHandler of having to
directly monitor the chain state. This makes use of the
BestBlockView that was introduced in earlier commits.
In this commit we change how we select invoices to follow or delete when
starting the InvoiceRegistry to instead of using the deperecated scan
func from channeldb, use specific functions to gather pending and delete
canceled invoices.
This commit makes sure an input is only added to the cluster when it has
successfully estimated its fee rate. Previously, when an error is
returned from `feeRateForPreference`, we'd still add this input to the
cluster, resulting a **lower** fee rates being used because when
averaging the fee rates, we'd think this input has zero fee rate
specified.
An unit test is patched to make the method `clusterByLockTime` more
robust.
* sweep: use longer variable name for clarity in `addToState`
* sweeper: add more docs and debug logs
* sweep: prioritize smaller inputs when adding wallet UTXOs
This commit sorts wallet UTXOs by their values when using them for
sweeping inputs. This way we'd avoid locking large UTXOs when sweeping
inputs and also provide an opportunity to aggregate wallet UTXOs.
* contractcourt+itest: relax anchor sweeping for CPFP purpose
This commit changes from always sweeping anchor for a local force close
to only do so when there is an actual time pressure. After this change,
a forced anchor sweeping will only be attempted when the deadline is
less than 144 blocks.
* docs: update release notes
* itest: update test `testMultiHopHtlcLocalChainClaim` to skip CPFP
Since we now only perform CPFP when both the fee rate is higher and the
deadline is less than 144, we need to update the test to reflect that
Bob will not CPFP the force close tx for the channle Alice->Bob.
* itest: fix `testMultiHopRemoteForceCloseOnChainHtlcTimeout`
* itest: update related tests to reflect anchor sweeping
This commit updates all related tests to reflect the latest anchor
sweeping behavior. Previously, anchor sweeping is always attempted as
CPFP when a force close is broadcast, while now it only happens when the
deadline is less than 144. For non-CPFP purpose sweeping, it will happen
after one block is mined after the force close transaction is confirmed
as the anchor will be resent to the sweeper with a floor fee rate, hence
making it economical to sweep.
* multi: extend InvoiceDB methods with a context argument
This commit adds a context to InvoiceDB's methods. Along this refactor
we also extend InvoiceRegistry methods with contexts where it makes
sense. This change is essential to be able to provide kvdb and sqldb
implementations for InvoiceDB.
* channeldb: restrict invoice tests to only use an InvoiceDB instance
* docs: update release notes for 0.18.0
* htlcswitch/hop: use InvalidOnionVersion for replayed packets
The link will send an update_fail_malformed_htlc, so we need to set
the BADONION bit. Since there isn't a replay-specific error, we
set the failure code to InvalidOnionVersion which has the BADONION bit.
* release-notes: update for 0.17.1
For some reason lnd doesn't properly shut down on some windows itests,
and the process still running makes it impossible to access its log
files.
So for the Windows integration test, we attempt to kill the whole
process tree once the process indicates it is done.
* tlv: fuzz tests for primitives
* tlv: fuzz tests for BigSize
We use a new harness to compare decoded values instead of encoded
values, since there may be some unparsed bytes in the original data.
* tlv: fuzz tests for truncated integers
These fuzz tests are identical to non-truncated integers, except that we
allow the fuzzer to choose decode lengths shorter than the length of
normal integers.
* tlv: fuzz tests for streams
* fixup! tlv: fuzz tests for truncated integers
loop over decode length
* fixup! tlv: fuzz tests for streams
better documentation
* funding: remove unused field `newChanBarriers`
This commit removes the occurance of `newChanBarriers` as it's not used
anywhere.
* funding: rename method names to clear the funding flow
Slightly refactored the names so it's easier to see which side is
processing what messages.
* lnwallet: sanity check empty pending channel ID
This commit adds a sanity check to make sure an empty pending channel ID
will not be accepted.
Note: This commit can be dropped before merge, it's mostly added
to make the PR easier to manually test against other
implementations that have bolt 12 invoices implemented already!
This commit updates route construction to backfill the fields
required for payment to blinded paths and set amount to forward
and expiry fields to zero for intermediate hops (as is instructed
in the route blinding specification).
We could attempt to do this in the first pass, but that loop
relies on fields like amount to forward and expiry to calculate
each hop backwards, so we keep it simple (stupid) and post
processes the blinded portion, since it's computationally cheap
and more readable.
Add the option to include a blinded route in a route request (exclusive
to including hop hints, because it's incongruous to include both), and
express the route as a chain of hop hints.
Using a chain of hints over a single hint to represent the whole path
allows us to re-use our route construction to fill in a lot of the
path on our behalf.
This commit introduces a single struct to hold all of the parameters
that are passed to FindRoute. This cleans up an already overloaded
function signature and prepares us for handling requests with blinded
routes where we need to perform some additional processing on our
para (such as extracting the target node from the blinded path).
When we introduce blinded routes, some of our hops are expected
to have zero amounts to forward in their hop payload. This commit
updates our hop fee logic to attribute the full blinded route's
fees to the introduction node. We can't actually know where/how
these fees are distributed, so we collect them all at the
introduction node.
With the addition of blinded routes, we now need to account for the
possibility that intermediate nodes payloads will not have an amount
and expiry set because that information is provided by the recipient
encrypted data blob. This commit updates our payload packing to only
optionally include those fields.
This commit adds the encrypted_data, blinding_point and total_amt_msat
tlvs to the known set of even tlvs for the onion payload. These TLVs
are added in two places (the onion payload and hop struct) because
lnd uses the same set of TLV types for both structs (and they
inherently represent the same thing).
Note: in some places, unit tests intentionally mimic the style
of older tests, so as to be more consistently readable.
This commit adds a representation of blinded payments, which include a
blinded path and aggregate routing parameters to be used in payment to
the path.
This commit refactors the params used in lifecycle to prefer
`HTLCAttempt` over `HTLCAttemptInfo`. This change is needed as
`HTLCAttempt` also wraps settled and failure info, which is useful in
the following commits.
This commit turns `MPPayment` into an interface inside `routing`. Having
this interface gives us the benefit to write more granular unit tests
inside payment lifecycle. As seen from the modified unit tests, several
hacky ways of testing the `SendPayment` method is now replaced by a mock
over `MPPayment`.
This commit adds a new method, `NeedWaitAttempts`, to properly decide
whether we need to wait for the outcome of htlc attempts based on the
payment's current state.
This commit moves the struct `paymentState` used in `routing` into
`channeldb` and replaces it with `MPPaymentState`. In the following
commit we'd see the benefit, that we don't need to pass variables back
and forth between the two packages. More importantly, this state is put
closer to its origin, and is strictly updated whenever a payment is read
from disk. This approach is less error-prone comparing to the previous
one, which both the `payment` and `paymentState` need to be updated at
the same time to make sure the data stay consistant in a parallel
environment.
This commit moves the creations of hop and htlcAdd message from
`createNewPaymentAttempt` to `sendPaymentAttempt` to clean up the code
and further pave the way to decomposite the lifecycle.
This commit updates the `fee()` method in `weightEstimator` to make sure
when doing CPFP we are not exceeding the max allowed fee rate. In order
to use the max fee rate, we need to modify several methods to pass the
configured value to the estimator.
When one of the itest tranches fails, because we are running tests in
parallel, other tranches will still be running, which caused 7z to fail
at zipping the logs because race reads. This commit fixes it by making
sure we are waiting other tranches to finish before moving to zipping
and uploading the logs.
For about a year [1], the spec has prescribed encoding channel_updates
with their type prefix (0x0102) in onion failure messages. LND can
decode correctly with or without the prefix but hasn't been writing the
prefix during encoding. This commit starts writing the prefix.
[1] https://github.com/lightning/bolts/pull/979
This commit removes the `Litecoin`, `LtcMode` and `LitecoindMode`
members from the main LND `Config` struct. Since only the bitcoin
blockchain is now supported, this commit also deprecates the
`cfg.Bitcoin.Active` config option.
This commit applies the new method `Terminated`. A side effect from
using this method is, we can now save one query `fetchPayment` inside
`FetchInFlightPayments`.
This commit introduces more granular statuses to better determine a
payment's current state. Based on whether there are inflight HTLCs, the
state of each of the HTLCs, and whether the payment is failed, a total
of 5 states are derived, which can give a finer control over what action
to take based on a given state.
Also, `fetchPayment` now uses `decidePaymentStatus`. After applying the
new function, the returned payment status would be different,
```
| inflight | settled | failed | reason | previous -> now |
|:--------:|:-------:|:------:|:------:|:---------------------------------:|
| true | true | true | yes | StatusInFlight(unchanged) |
| true | true | true | no | StatusInFlight(unchanged) |
| true | true | false | yes | StatusInFlight(unchanged) |
| true | true | false | no | StatusInFlight(unchanged) |
| true | false | true | yes | StatusInFlight(unchanged) |
| true | false | true | no | StatusInFlight(unchanged) |
| true | false | false | yes | StatusInFlight(unchanged) |
| true | false | false | no | StatusInFlight(unchanged) |
| false | true | true | yes | StatusSucceeded(unchanged) |
| false | true | true | no | StatusSucceeded(unchanged) |
| false | true | false | yes | StatusSucceeded(unchanged) |
| false | true | false | no | StatusSucceeded(unchanged) |
| false | false | true | yes | StatusFailed(unchanged) |
| false | false | true | no | StatusInFlight(unchanged) |
| false | false | false | yes | StatusFailed(unchanged) |
| false | false | false | no | StatusInFlight -> StatusInitiated|
```
This commit adds a new method, `Registrable`, to help decide whether
adding new HTLCs to a given payment is allowed. A new unit test,
`TestRegistrable` is also added to test it.
This commit adds a new payment status, `StatusInitiated`, to properly
represent the state where a payment is newly created without attempting
any HTLCs. Since the `PaymentStatus` is a memory representation of a
given payment's status, the enum used for the statuses are directly
updated.
This commit changes `fetchPaymentStatus` to return an error when the
payment creation info bucket cannot be found. Instead of using the
ambiguous status `StatusUnknown`, we tell the callsite explictly that
there's no such payment. This also means the vague `StatusUnknown` can
now be removed as it has multiple meanings.
* Add instructions to generate type hints
Use mypy to generate .pyi files as well. These files are useful for type hinting in IDEs.
* Update python.md
fix lines that got swapped in copy-paste
* remove mypy
mypy is not a dependency
This PR is a follow up, to a [follow
up](https://github.com/lightningnetwork/lnd/pull/7938) of an [initial
concurrency issue](https://github.com/lightningnetwork/lnd/pull/7856)
fixed in the peer goroutine.
In #7938, we noticed that the introduction of `p.startReady` can cause
`Disconnect` to block. This happens as `Disconnect` cannot be called
until `p.startReady` has been closed. `Disconnect` is also called from
`InboundPeerConnected` (the case of concurrent peers, so we need to
remove one of the connections) while the main server mutex is held. If
`p.Start` blocks for any reason, then this leads to the deadlock as: we
can't disconnect until we've finished starting, and we can't finish
starting as we need the disconnect caller to exit as it has the mutex.
In this commit, we now make the call to `prunePersistentPeerConnection`
async. The call to `prunePersistentPeerConnection` eventually wants to
grab the server mutex, which triggers the circular waiting scenario
above.
The main learning here is that no calls to the main server mutex path
can block from `p.Start`. This is more or less a stop gap to resolve the
issue initially introduced in v0.16.4. Assuming we want to move forward
with this fix, we should reexamine `p.startReady` all together, and also
revisit attempt to refactor this section of the code to eliminate the
mega mutex in the server in favor of a dedicated event loop.
In this commit, we modify the incoming contest resolver to use a
concurrent queue. This is meant to ensure that the invoice registry
subscription loop never blocks. This change is meant to be minimal and
implements option `5` as outlined here:
https://github.com/lightningnetwork/lnd/issues/8023.
With this change, the inner loop of the subscription dispatch method in
the invoice registry will no longer block, as the concurrent queue uses
a fixed buffer of a queue, then overflows into another queue when that
gets full.
Fixes https://github.com/lightningnetwork/lnd/issues/7917
This tests that the funding manager doesn't immediately consider
a coinbase transaction that is also a funding transaction usable
until the maturity has been reached.
In this commit, we add the ability to obtain blocking and mutex
profiles. The blocking profile will show which goroutines are
consistently blocked on synchronization primitives like channels, or
I/O. The mutex profile will show which mutexes are very contested.
The blocking profile can be enabled with a new arg: `--blockingprofile`.
The mutex profile can be enabled with a new arg: `--mutexprofile`. These
are both ignored if the profile port isn't set.
Activating these profiles requires the caller to pass in a sampling
rate. For now I've set it just to `1` to test things out. Unfortunately
documentation is rather scarce, so there aren't any good guides re what
these values should be set to. AFAICT, these add more overhead than the
other prowling options, so they shouldn't necessarily be enabled
persistently in production.
In this commit, we update the set of protos to accept the local secret
nonces over RPC. This is actually a 97 byte value, as it includes the
two 32 byte nonces, as well as the 33 byte value of the public key of
the signer.
This is needed in order to be able to open taproot channels over the RPC
interface.
In this commit, we modify the musig2 interfaces to instead use an
explicit value for the local nonces. Before this commit, we used the
functional option, but we want to also support specifying this value
over RPC for the remote signer. The functional option pattern is opaque,
so we can't get the nonce value we need. To get around this, we'll just
make this an explicit pointer, then map this to the functional option at
the very last moment.
In this commit, we increase the max message size for the ws proxy. We
have a similar setting for the normal gRPC server which was tuned to be
able to support decoding `GetNetworkInfo` as the channel graph got
larger. We keep the default buffer size of 64 KB, but allow that to be
expanded to up to 4 MB (current value) to decode larger messages.
One alternative would be to modify the `Split` function to break up
larger lines into smaller ones. We'd need to double check that the
libraries at a higher level of abstraction can handle the chunks. The
scan function would look something like:
```go
splitFunc := func(data []byte, eof bool) (int, []byte, error) {
if len(data) >= chunkSize {
return chunkSize, data[:chunkSize], nil
}
return bufio.ScanLines(data, eof))
}
scanner.Split(splitFunc)
```
In this commit, update the start up logic to gracefully handle a
seemingly rare case. In this case, a peer detects local data loss with a
set of active HTLCs. These HTLCs then eventually expire (they may or may
not actually "exist"), causing a force close decision. Before this PR,
this attempt would fail with a fatal error that can impede start up.
To better handle such a scenario, we'll now catch the error when we fail
to force close due to entering the DLP and instead terminate the state
machine at the broadcast state. When a commitment transaction eventually
confirms, we'll play it as normal.
Fixes https://github.com/lightningnetwork/lnd/issues/7984
In this commit, we introduce the concept of a rogue update. An update is
rogue if we need to ACK it but we have already deleted all the data for
the associated channel due to the channel being closed. In this case, we
now no longer error out and instead keep count of how many rogue updates
a session has backed-up.
This commit adds a new test to the tower client to demonstrate a bug
that can happen if a channel is closed while an update for it has yet to
be acked by the tower server. This will be fixed in an upcomming commit.
The watchtower client test framework currently uses a mock version of
the tower client DB. This can lead to bugs if the mock DB works slightly
differently to the actual bbolt DB. So this commit ensures that we only
use the bbolt db for the tower client tests. We also increment the
`waitTime` used in the tests a bit to account for the slightly longer DB
read and write times. Doing this switch resulted in one bug being
caught: we were not removing sessions from the in-memory set on deletion
of the session and so that is fixed here too.
In this commit, we use exhaustive build tags to ensure that we can
always build the `sqlbase` package, independent of the set build tags.
To do this, we move the type declarations _into_ the parsing functions.
This then allows us to create two versions for each db: with the db, and
without it.
To avoid a module tag round trip to get this working, we use a local
replace for now. Once this is merged in, we can do the tag (along side
rc3), then remove the replace.
hop.Payload and route.Hop are analogs, with onion payloads encoded from
route.Hops and decoded to hop.Payloads. For checking equality of
encoding/decoding, we implement a helper function to convert
hop.Payloads into route.Hops.
The docker image have been updated so we are using another protobuf
version to generate the files. The generate files include the version of
the compiler used to creating them, so we need this commit to pass the
`rpc-check` step in our CI.
Bump all build go versions to v1.21.0
Bump the minimum build package version to v1.19.0
Debian "buster" is not longer supported. Security updates have been
discontinued since June 30th 2022. We will build using the latest
version, "bookworm".
When the numTweaks is zero, we should return a nil instead of
initializing an empty map as we'd get the following error,
```
Diff:
--- Expected
+++ Actual
@@ -11007,4 +11007,3 @@
},
- BreachedHtlcTweaks: (contractcourt.htlcTapTweaks) {
- },
+ BreachedHtlcTweaks: (contractcourt.htlcTapTweaks) <nil>,
```
Fix the following uint test flake,
```
--- FAIL: TestHistoricalConfDetailsTxIndex (0.00s)
--- FAIL: TestHistoricalConfDetailsTxIndex/rpc_polling_enabled (1.16s)
bitcoind_test.go:174: should have found the transaction within the mempool, but did not: TxNotFoundIndex
FAIL
```
This commit fixes the error,
```
go: version constraints conflict:
github.com/dhui/dktest@v0.3.16 requires golang.org/x/time@v0.0.0-20220224211638-0e9765cccd65, but golang.org/x/time@v0.0.0-20210220033141-f8bda1e9f3ba is requested
github.com/golang-migrate/migrate/v4@v4.16.1 requires golang.org/x/time@v0.0.0-20220224211638-0e9765cccd65, but golang.org/x/time@v0.0.0-20210220033141-f8bda1e9f3ba is requested
```
By calling `go get golang.org/x/time@v0.0.0-20220224211638-0e9765cccd65"
and `go mod tidy`.
Fix the following conflict,
```
go: conflicting replacements for github.com/ulikunitz/xz:
github.com/ulikunitz/xz@v0.5.11
github.com/ulikunitz/xz@v0.5.8
```
* lnwallet: fix log output msg
The log message is off by one.
* htlcswitch: fail channel when revoking it fails.
When the revocation of a channel state fails after receiving a new
CommitmentSigned msg we have to fail the channel otherwise we
continue with an unclean state.
* docs: update release-docs
* htlcswitch: tear down connection if revocation processing fails
If we couldn't revoke due to a DB error, then we want to also tear down
the connection, as we don't want the other party to continue to send
updates. That may lead to de-sync'd state an eventual force close.
Otherwise, the database might be able to recover come the next
reconnection attempt.
* kvdb: use sql.LevelSerializable for all backends
In this commit, we modify the default isolation level to be
`sql.LevelSerializable. This is the strictness isolation type for
postgres. For sqlite, there's only ever a single writer, so this doesn't
apply directly.
* kvdb/sqlbase: add randomized exponential backoff for serialization failures
In this commit, we add randomized exponential backoff for serialization
failures. For postgres, we''ll his this any time a transaction set fails
to be linearized. For sqlite, we'll his this if we have many writers
trying to grab the write lock at time same time, manifesting as a
`SQLITE_BUSY` error code.
As is, we'll retry up to 10 times, waiting a minimum of 50 miliseconds
between each attempt, up to 5 seconds without any delay at all. For
sqlite, this is also bounded by the busy timeout set, which applies on
top of this retry logic (block for busy timeout seconds, then apply this
back off logic).
* docs/release-notes: add entry for sqlite/postgres tx retry
---------
Co-authored-by: ziggie <ziggie1984@protonmail.com>
In this commit, we attempt to fix circular waiting scenario introduced
inadvertently when [fixing a race condition
scenario](https://github.com/lightningnetwork/lnd/pull/7856). With that
PR, we added a new channel that would block `Disconnect`, and
`WaitForDisconnect` to ensure that only until the `Start` method has
finished would those calls be allowed to succeed.
The issue is that if the server is trying to disconnect a peer due to a
concurrent connection, but `Start` is blocked on `maybeSendNodeAnn`,
which then wants to grab the main server mutex, then `Start` can never
exit, which causes `startReady` to never be closed, which then causes
the server to be blocked.
This PR attempts to fix the issue by calling `maybeSendNodeAnn` in a
goroutine, so it can grab the server mutex and not block the `Start`
method.
Fixes https://github.com/lightningnetwork/lnd/issues/7924
Fixes https://github.com/lightningnetwork/lnd/issues/7928
Fixes https://github.com/lightningnetwork/lnd/issues/7866
With this commit we unify three existing PSBT channel funding
integration tests into a single one with the goal of testing all three
cases with simple taproot channels as well.
In this commit, we carry out a new notion introduced during a recent
spec meeting to use a feature bit plus 100 before the feature has been
finalized in the spec.
We split into the Final and Staging bits.
We use a helper function to ensure that anytime we're about to make a
normal sighash, we consult the channel type to check if we should use
the default value or sighash all explicitly.
In this commit, we remove the internal call to `InitRemoteMusigNonces`.
We don't need this since when we go to process the remote party's chan
reest message, we'll already call this method. Otherwise, we'll get an
error here since the pending verification nonce has been wiped out after
each call.
Otherwise, in the itests (which are mainly based on implicit
negotiation), we won't try to open taproot chans when we actually need
to.
We may want to revisit this however, since it may lock in parties trying
to use the defaults that aren't currently setting the explicit commit
type during funding.
This is a preparatory change for the upcoming "simple channel close"
feature which'll utilize RBF to allow either side to resign the co-op
close transaction for broadcast at any point.
In this commit, we update the channel state machine to use the new
ScriptDescriptor interface. This fixes some subtle issues with the
existing commits, as for p2wsh we always sign the same witness script,
but for p2tr, the witness script differs depending on which branch is
taken.
With the new abstractions, we can treat p2wsh and p2tr as the same
mostly, right up until we need to obtain a control block or a tap tweak.
All tests have been updated accordingly.
In this commit, we add a new interface, the `ScriptDesciptor` to
abstract over details of a given output script. The purpose of this
interface, and the taproot superset, is to be able to paper over the
differences of a p2wsh vs a p2tr output. With this new interface, we can
treat them as the same, but then use a type assertion to get at any
control block related methods if needed.
In this commit, we update the breach arb to support taproot channels. We
utilize the new taproot briefcase space to store both control blocks,
and also the first+second level scripts for the set of HTLCs.
We pull the information from the sign descriptors and store them in the
resolutions. However, the resolvers created end up duplicating the
resolution data, so we update the sign descs as needed during start up.
In this commit, we add a new taproot specific briefcase to store the
control block and tap tweaks for all taproot outputs. We chose this
route as many of the existing fields are serialized in line, so we
aren't able to serialize this new taproot specific information in the
existing briefcase.
For taproot channels, we need to thread through the control block in the
sign descriptor. We also ensure that the proper sign method is set. We
leverage the new input.Signature generalization be able to support
handling both schnorr and ECDSA signatures for the second level output.
In this commit, we add the tapscript tree to the ScriptInfo struct, as
in many cases the caller needs the tree in order to generate the control
block or obtain the taptweak which is needed to spend revoked outputs.
In this commit, we update all the taproot scripts spends to optionally
make the control block. This is useful in cases where we've already
created the control block, or may not have the items needed to construct
it in the first place.
We also add the control block to the sign descriptor itself.
This ensures that when loading the channel again after a normal chan
reest, we generate the local nonces, which ensures we can then process
nonces the remote party sends us in their chan reest message.
In this commit, we update the chain watcher to be able to generate the
correct pkScript so it can register for confirmation and spend
notifications for taproot channels.
In this commit, we start to set _internally_ a new feature bit in the
channel announcements we generate. As these taproot channels can only be
unadvertised, this will never actually leak to the public network. The
funding manager will then set this field to allow the router to properly
validate these channels.
In this commit, we add support for the new musig2 channel funding flow.
This flow is identical to the existing flow, but not both sides need to
exchange local nonces up front, and then signatures sent are now partial
signatures instead of regular signatures.
The funding manager also gains some new state of the local nonces it
needs to generate in order to send the funding locked message, and also
process the funding locked message from the remote party.
In order to allow the funding manger to generate the nonces that need to
be applied to each channel, then AddNewChannel method has been modified
to accept a set of options that the peer will then use to bind the
nonces to a new channel.
In this commit, we update the logic to handle nonce init in
ProcessChanSyncMsg. Once a channel is already open, this is where we'll
get the new nonce data from the remote party we'll use to gain the nonce
we need to sign for their next state.
Before this commit, we would conditionally generate nonces in
RevokeCurrentCommitment. We move this to generateRevocation as this is
called when doing channel sync, and we want to make sure we send the
correct set of nonces.
In this commit, we update the ChanSyncMsg to populate nonce information.
With this change, we can now hide nonce generation further down in the
pipeline and ensure that all callers will have the expected fields
populated.
In this commit, we fix a bug in the `deriveMusig2Shachain` function
where it didn't actually use the passed in revocation root as part of
the hmac invocation.
We also modify the function to be more generally useable as well, as now
the caller can just pass in the revocation root things should be derived
from.
In this commit, we update the co-op close flow to support the new musig2
keyspend flow. We'll use some new functional options to allow a caller
to pass in an active musig2 session. If this is present, then we'll use
that to complete the musig2 flow by signing with a partial signature,
and then ultimately combining the signatures at the end.
In this commit, we update the genHtlcSigValidationJobs function to be
taproot aware. As we actually need a schnorr signature for the taproot
validation, we need to coerce the entire wire type into a schnorr sig
with the ForceSchnorr() method.
In this commit, we update the genRemoteHtlcSigJobs function to be able
to generate taproot jobs. We also modify the sigpool to now use a
input.Signature everywhere. This'll allow us to pass around both ECDSA
and Schnorr signatures via the same interface.
We use a tapscript sighash in this case, as all the HTLC spends will
actually be script path spends.
In this commit, we update the channel state machine with a new set of
functional options that can be used to create/set the musig session
state. When a channel is made during the funding process, the set of
nonces we want to use is already known, so we allow them to be passed
in. Similarly, once the channel is confirmed, then we'll need to create
another channel instance that this times carries the newly generated
nonces to send along side funding_locked.
We also add some utility methods to permit callers to properly generate
nonces in the various contexts.
In this commit, we add a new NewCommitState struct. This preps us for
the future change wherein a partial signature is also added to the mix.
All related tests and type signatures have also been updated
accordingly.
In this commit, we modify the starting logic to note attempt to add a
tower client for taproot channels. Instead, we'll just log that this
isn't available yet.
In this commit, we build on all the prior commits and integrate the new
taproot channels into the existing internal funding flow. Along the way,
we do some refactoring to unify things like signing and verifying
incoming commitment transaction signatures.
For our local nonce, we use the existing functional option type to
derive the nonce based on the initial shachain pre-image we'll use as
our revocation.
In this commit, we add a new wallet level channel type, along with the
new fields we'll need to accept from both parties within the
contribution messages. In this case, we now have a local nonce, along
with the internal musig session.
By using the multimutex here, we'll no longer rely on a single mutex for
the entire musig session set like we used to. Instead, we can use the
session ID to key into a map of mutexes and use those directly.
In this commit, we add a series of abstractions that'll allow us to
easily do funding and also state updates for the new taproot channels. A
partial session is defined by the knowledge of a verification nonce.
Once the remote party sends a signature, we learn of their signing
nonce, and can then complete a session. By using a JIT nonce approach,
we ensure that the signer can generate their nonces randomly and also
at the very last step to avoid having to maintain state.
For our local nonces, we also have an option to use a counter based
nonce derived from the shachain instead of fully random nonces. This
allows us to not have to store ay additional state. Instead, when we
need to go to broadcast, we can just regenerate the nonce then use that
to broadcast.
In this commit, we update the set of intents and assemblers to recognize
musig2. For this change, we use a new bool, `musig2`, then use that to
determine if we need to use the new taproot funding scripts or not.
In this commit, we extract the musig2 session management into a new
module. This allows us to re-use the session logic elsewhere in unit
tests so we don't need to instantiate the entire wallet.
In this commit, we modify the to_local script to use a script path for
the revocation scenario. With this change, we ensure that the internal
key is always revealed which means the anchor outputs can still always
be swept.
In this commit, we restore usage of the NUMS key for the to remote
output, as this allows a remote party to scan the chain in order to find
their remote output that in emergency recovery scenarios.
Unlike the old HTLC scripts, we now need to handle the various control
block interactions. As is, we opt to simply re-compute the entire tree
when needed, as the tree only has two leaves.
In this commit, we add GenTaprootFundingScript, which'll return the
taproot pkScript and output for a taproot+musig2 channel. This uses
musig2 key aggregation with sorting activated.
The final key produced uses a bip86 tweak, meaning that the output key
provably doesn't commit to any script path. In the future, we may want
to permit this, as then it allows for a greater degree of
programmability of the funding output.
In this commit, we add the new types that'll house musig signatures with
and without their nonces. We send the nonce along with the sig
everywhere but the co-op close flow.
In this commit, we update the Sig type to support ECDSA and schnorr
signatures. We need to do this as the HTLC signatures will become
schnorr sigs for taproot channels. The current spec draft opts to
overload this field since both the sigs are actually 64 bytes in length.
The only consideration with this move is that callers need to "coerce" a
sig to the proper type if they need schnorr signatures.
There is a race condition,
- goroutine_1: performing `stateStep` under `channelReadySent`. Once
`receivedChannelReady` returns true, it will mark the channel state as
`addedToRouterGraph`, which will delete the initial forwarding policy
for private channels.
- goroutine_2: performing `handleChannelReady`, which processes the
remote's channelReady message. It will ask brontide to `AddNewChannel`
in the end.
- goroutine_3: performing `handleNewActiveChannel` in brontide, which
will query the initial forwarding policy and fail to find it because
it's already deleted in goroutine_1.
To fix it, we require `receivedChannelReady` to also check that the
channel's barrier signal in the map `handleChannelReadyBarriers` doesn't
exist, as this signal is removed once `handleChannelReady` finishes
adding the channel in brontide.
This commit is a pure code move. We add a new method
`handleChannelReadyReceived` to handle the channel's state change after
the remote's channel ready message is received.
This commit moves the deletion of the initial forwarding policy to the
end of `stateStep` to make sure the router has persisted it to disk
before the deletion.
Because we now send the correct initial forwarding policy to Brontide,
the correct initial values are inserted into the link and we no longer
need to update the link in a later step.
This will allow us to also set the TimeLockDelta and HTLC settings
when creating the channel. We leave the actual implementation of
the RPC and CLI changes to another PR, this is just to make
things more consistent.
Since the actual wallet backends might be different between the wallet
DB and the actual channel state DB, we pass in the correct struct to the
funding manager.
We remove the publishing of the last published sweep tx during the
startup of the sweeper. This republishing can lead to situations
where funds of the default wallet might be locked for neutrino
backend clients.
Moreover all related tests are removed as well.
This commit changes the name returned from `prepContractResolutions`
from `htlcResolvers` to `resolvers` to avoid confusion as there are
multiple types of resolvers returned.
This commit now sends messages to `chanStream` for both pending and
active channels. If the message is sent to a pending channel, it will be
queued in `chanStream`. Once the channel link becomes active, the early
messages will be processed.
This commit adds a new itest case to check the race condition found in
issue #7401. In order to control the funding manager's state, a new dev
config for the funding manager is introduced to specify a duration we
should hold before processing remote node's channel_ready message.
A new development config, `DevConfig` is introduced in `lncfg` and will
only have effect if built with flag `integration`. This can also be
extended for future integration tests if more dev-only flags are needed.
This commit adds a new interface method, `RemovePendingChannel`, to be
used when the funding flow is failed after calling `AddPendingChannel`
such that the Brontide has the most up-to-date view of the active
channels.
This commit adds a new struct `chanIdentifier` which wraps the pending
channel ID and active channel ID. This struct is then used in
`failFundingFlow` so the channel ID can be access there.
The funding manager has been updated to use `AddPendingChannel`. Note
that we track the pending channel before it's confirmed as the peer may
have a block height in the future(from our view), thus they may start
operating in this channel before we consider it as fully open.
The mocked peers have been updated to implement the new interface method.
This commit adds a new channel `newPendingChannel` and its dedicated
handler `handleNewPendingChannel` to keep track of pending open
channels. This should not affect the original handling of new active
channels, except `addedChannels` is now updated in
`handleNewPendingChannel` such that this new pending channel won't be
reestablished in link.
This commit moves the registration of the State server with a REST proxy
into the `RegisterWithRestProxy` method in order to keep all the REST
registrations in one place.
Adding the ability to stop rebroadcasting a transaction. This is
useful when we want to abandon a channel and grantee that this
transaction will not confirm accidentally.
The new script perfoms the following checks on the sample-lnd.conf file:
1. Checks that all relevant options of lnd are included.
2. Verifies that defaults are labeled if there are also further examples.
3. Checks that all default values of lnd --help are mentioned correctly,
including empty defaults and booleans which are set to false by
default.
If there is only one entry in the sample-lnd.conf, it must reflect the
default value. If additional examples are provided, they should follow
the following format.
; Default:
; option=defaultvalue
; Example
; option=examplevalue
Booleans have been changed to 'false' to align with the default behavior
of Go. Additionally, the description for various parameters has been
adjusted.
Noticed this while updating to latest master in another project:
If there are two notifications for the same output being registered and
the first one does _NOT_ request the block to be included, then the
second one also will not receive the block, even if it explicitly
requests it.
This is caused by the block being removed from the original confirmation
set instead of a copy (as it is done in NotifyHeight and
UpdateConfDetails).
memoryMailBox uses multiple container/list.List objects to track
messages and packets, which use interface{} to accept objects of any
type. go1.18 added generics to the language, which means we could use a
typed list instead, allowing us to stop using forced type assertions
when reading objects from the list.
I'm not aware of any standard library implementation of a typed list yet,
so let's just add a TODO for now.
In this commit, we make sure that all the `wg.Add(1)` calls succeed
before we attempt to wait on the shutdown of all the goroutines. Under
rare scheduling scenarios, if both `Start` and `Disconnect` are called
concurrently, then this internal race error can be hit, causing the
panic to occur.
Fixes https://github.com/lightningnetwork/lnd/issues/7853
This commit provides the scaffolding for using the new sql stores.
The new interfaces, structs and methods are in sync with other projects
like Taproot Assets.
- Transactional Queries: the sqldb package defines the interfaces required
to execute transactional queries to our storage interface.
- Migration Files Embedded: the migration files are embedded into the binary.
- Database Migrations: I kept the use of 'golang-migrate' to ensure our
codebase remains in sync with the other projects, but can be changed.
- Build Flags for Conditional DB Target: flexibility to specify our database
target at compile-time based on the build flags in the same way we do
with our kv stores.
- Update modules: ran `go mod tidy`.
The MapCustomBroadcastError of the rebroadcaster is defined.
This let's us use the Rebroadcaster of the neutrino package (pushtx)
with other backends and guarantee a consistent behaviour. Prior
to this change the rebroadcaster would stop rebroadcasting
transactions because they did not meet the neutrino specific
broadcastErr type.
Add a test where the channel arbitrator starts up correctly
when a prior unilateral close of a channel did not broadcast
for specific reasons.
Also add a test which ensures that when a crib output is
rejected by the bitcoin backend the startup works correctly
for specific errors.
In case the mempool backend signals that our transaction does not
meet fee requirements when publishing it we will continue to
start up now. The transaction will be rebroadcasted in the
background and a specific log message will be printed to let the
user know that he could increase his mempool size to at least
have this transaction in his own mempool.
For our relatively "static" test invoice, add a simple constructor
using the globals defined, rather than having a global invoice which
risks being mutated.
The goroutine is very long and littered with switches on courierType. By
separating the implementations into different methods for each
courierType we eliminate the cruft and improve readability.
It is very difficult for the fuzzer to create a valid checksum for each
serialized invoice, and we were therefore unable to fuzz deeper than
invoice decoding. We can help the fuzzer generate valid serialized
invoices by calculating and appending the checksum ourselves.
We also switch to using mainnet invoices to make it easier to find valid
invoices for seeding the fuzzer. We prepend the required "lnbc" prefix
ourselves to further help the fuzzer generate valid invoices.
The message signer from invoice_test.go is identical to the one created
in the fuzz test. We're already using the private key from
invoice_test.go, so we may as well use the complete message signer for
simplicity.
The fuzz tests call inv.MinFinalCLTVExpiry() and inv.Expiry() supposedly
to ensure the invoice is well-formed. However, those methods can never
panic or return errors and therefore provide no benefit for this
purpose.
In this commit, the bugs demonstrated in prior commits are fixed. In the
case where an session has persisted a CommittedUpdate and the tower is
being removed, the session will now replay that update on to the main
task pipeline so that it can be backed up using a different session.
Add a new DeleteCommittedUpdate method to the wtdb In preparation for an
upcoming commit that will replay committed updates from one session to
another.
This commit demonstrates that if a session has persisted committed
updates and the client is restarted _after_ these committed updates have
been persisted, then removing the tower will fail.
In this commit, we demonstrate the situation where a client has
persisted CommittedUpdates but has not yet recieved Acks for them from
the tower. If this happens and the client attempts to remove the tower,
it will with the "tower has unacked updates" error.
This commit does a few things:
- First, it gives the sessionQueue access to the TowerClient task
pipeline so that it can replay backup tasks onto the pipeline on Stop.
- Given that the above is done, the ForceQuit functionality of the
sessionQueue and TowerClient can be removed.
- The bug demonstrated in a prior commit is now fixed due to the above
changes.
In preparation for an upcoming commit where multiple threads will have
access to the TowerClient sessionQueueSet, we turn it into a thread safe
struct.
This commit demonstrates a bug. It shows that if backup tasks have been
bound to a session with a tower (ie, the tasks are in the session's
pendingQueue) and then the tower is removed and a new one is added, then
the tasks from the pendingQueue are _not_ replayed to the session with
the new tower. Instead, they are silently lost. This will be fixed in an
upcoming commit.
This commit adds a new watchtower client test to demonstrate that a
client is able to successfully switch to a new tower and continue
backing up updates to that new tower.
Test utils currently have two different test invoices - a minimalist
one that is used in a variety of test cases, and a customizable one
that is used specifically for tests concerning invoice expiry. The
latter is renamed and moved closer to the calling code to more clearly
indicate its use.
Details of race:
- A global testInvoice is used in various invoice related tests.
- All tests are running with t.Parallel.
- TestMppPaymentWithOverpayment passes a reference to the testInvoice
to AddInvoice which writes the AddIndex.
- TestMultipleSetHeightExpiry reads the testInvoice to make a copy.
With the latest Golang Docker base image we are using the new gpg
version 2.4 is now being installed in the lnd Docker base image.
Apparently the expected value for the --keyring flag is just a file name
and not an absolute path. The path of the file is indicated either by
the $HOME environment variable or the --homedir flag. It looks like 2.4
now finally stopped supporting an absolute path in the --keyring flag
and we need to update our gpg command to make the script work again.
This should be backward compatible and still work on older versions of
gpg.
Schema for AMP invocies.
AMP invoices can be paid multiple times and each payment to an AMP invoice
is identified by a `set_id`.
The A in AMP stands for `Atomic`. All the htlcs belonging to the same
AMP payment (share the same set_id) will be resolved at the same time
with the same result: settled/canceled.
AMP invoices do not have an "invoice preimage". Instead, each htcl has
its own hash/preimage. When a new htlc is added the hash for that htlc
is attached to it. When all the htlcs of a set_id have been received we
are able to compute the preimage for each one of them.
Set of queries to deal with invoices. A couple of things to take into
account:
- Because the queries are not rewritten at runtime, we cannot have a
generic `INSERT` with different tuples.
- Because the queries are not rewritten at runtime, we cannot build
one query with only the filters that matter for that queries. The
two options are a combinatorial approach (a new query for every
permutation) or a generic query using the pattern
```
SELECT *
FROM table
WHERE (
-- Can be read as:
-- Match the filter 1 value if filter_1 != nil
column_1 >= sqlc.narg('filter_1') OR
sqlc.narg('filter_1') IS NULL
) AND (
column_2 >= sqlc.narg('filter_2') OR
sqlc.narg('filter_2') IS NULL
) ...
```
This is the schema for "ordinal" BOLT11 invoices.
The invoices table aims to keep an entry for each invoice, BOLT11 or not,
that will be supported.
Invoice related HTLCs will be stored in a separete table than forwarded
htlcs.
SQLite does not support `SEQUENCE`. We achieve atomic autoincrementals
using primary keys with autoincrement/serial. An invoice `AddIndex`
translates to `invoices(id)` while `SettleIndex` is `invoice_payments(id)`.
sqlc is a tool that generates fully type-safe idiomatic code from SQL.
The result is Go code can then used execute the queries in the database.
The noraml flow looks like:
- The developer write some sql that will update the schema in the
database: new tables, indices, etc
- The developer updates the set of queries that will use the new schema.
- `sqlc` generates type-safe interfaces to those queries.
- The developer can then write application code that calls the methods
generated by sqlc.
The tool configuration needs to live in the repo's root and its name is
`sqlc.yaml`.
LND will support out of the box sqlite and postgres. The sql code needs to
be (almost) the same for both engines, so we cannot use custom functions
like `ANY` in postgres.
The SQLC config file needs to define what is the target engine, we will
set postgres but the generated code can be executed by sqlite too.
In some specific cases, we will `match and replace` some sql lines to be
sure the table definitions are valid for the targeted engine.
The spec allows the final HTLC value and CLTV expiry to exceed
the value and expiry specified in the payload of the last hop
of the onion packet. We were over-restricting it to require
that it matches exactly.
As new pending channels will be tracked in the following commits, the
`newChannels` is now renamed to `newActiveChannel` to explicitly refer
to active, non-pending channels.
With this commit we avoid calling LookupHost if we already have an IPv4
or IPv6 address, as we can return that directly.
This avoids asking Tor to resolve an IPv6 address, which it cannot do.
This commit fixed a wrong ordering of initializing subsystem loggers. We
need to make sure "third party" loggers are initialized first such that
when we customize the names later in `lnd`'s packages, the overwriting
of the logger names will take effect.
Without this commit, PRs can fail to be merged via the merge queue, as
it'll fail with this error:
```
Run scripts/check-release-notes.sh
PR gh-readonly-queue didn't update release notes
Error: Process completed with exit code 1.
```
In a previous PR we added a Memo field for channels that could be
specified when opening a channel. This was a reference note-to-self
with no bearing on the functioning of the channel. In that PR, the
memo value was returned only through ListChannels. This commit builds
upon that PR by also returning the Memo field for channels returned by
PendingChannels RPC.
This commit removes the `PrivateTowerURIs` member from the `WtClient`
config struct. This field has been deprecated since v0.8.0-beta and
currently, LND would fail to start if the field was specified.
In this commit, a `DefaultWatchtowerCfg` function is added which is used
to construct a default `lncfg.Watchtower` struct. This is then used to
populate the default watchtower config in the main LND config struct.
In this commit, a `DefaultWtClientCfg` function is added which
populates some default values for the `WtClient` struct. This is then
used to populate the wtclient defaults in the main LND config struct.
In theory, it should be only one custom account with a given name. However,
users could have created custom accounts with various key scopes. In that case,
'LookupAccount' has a non deterministic behaviour. To fix that, we browse
through all key scopes (deterministically) and return the first account we found.
In theory, it should be only one custom account with a given name. However,
due to a lack of check, users could have created custom accounts with various
key scopes. In that case, ListAccounts has to list all these accounts.
In this commit we add support for two more custom parameters in the
main entrypoint for the `lnd` Docker image script (`start-lnd.sh`)
The two parameters are:
* RPCHOST to set a custom endpoint for the RPC server
* RPCCRTPATH to set a custom location for the certificate used when
communicating with the RPC server
Verify that the addresses we're decoding when sending coins onchain are
for the correct network. Without this check we'll convert the users
addresses to their equivalent on other networks, which is a gross
violation of the principle of least astonishment.
This change allows users to customize the gRPC keepalive settings. The
server ping values configure when the server sends out a ping by itself
and how quickly the client is expected to respond.
The client ping min wait configures the minimum time a client has to
wait before sending another ping. A connection might be closed by the
server if a client pings more frequently than the allowed min wait time.
We choose lower default values than the gRPC library used before:
- ServerPingTime: 1 minute instead of 2 hours
- ClientPingMinWait: 5 seconds instead of 5 minutes
This should by itself already make long-standing gRPC streams more
stable but also allow clients to set a custom client ping time of 5
seconds or more (a value of slightly more than 5000 milliseconds should
be chosen to avoid the server disconnecting due to slight timing skews).
By defining the full name for the docker image we avoid the issue that
occurs when using `podman` (tested on Fedora 38) where podman will prompt the
user for what docker repository to pull the image from.
We know that onion blobs in lightning are _exactly_ 1366 bytes in
lightning, but they are currently expressed as a byte slice in
channeldb's HTLC struct. Blobs are currently serialized as var bytes,
so we can take advantage of this known length and variable length
to add additional data to the inline serialization of our HTLCs, which
are otherwise not easily extensible (without creating a new bucket).
In this commit, we eliminate some code duplication by removing the old
`HashMutex` struct as it just duplicates all the code with a different
type (uint64 and hash). We then make the main Mutex struct take a type
param, so the key can be parametrized when the struct is instantiated.
Test AddressIterator for the absence of panics, nil addresses, and empty
lists.
This fuzz test finds https://github.com/lightningnetwork/lnd/issues/7552
in seconds. No other panics found after 300+ CPU-hours of fuzzing.
In this commit, we attempt to fix an issue that may lead to force closes due
to small value HTLCs. The sweeper has built in a "negative yield" heuristic
where it won't sweep something that'll result in paying more fees than the
HTLC amount. However for HTLCs, we want to always sweep them, as we don't
cancel back the HTLCs before the outgoing contract is fully resolved.
In the future, we'll start to make more uneconomical decisions about if we
should go to chain at all for small value HTLCs, and also do things like
cancel back early if the HTLC is small and we think we might be contested by
chain fees.
This refactor aims to house the CalculateFeeRate function in a more
"shareable" location. It used to be a non-exported function inside
of rpcserver.go. However, other routines, such as FundPsbt inside
of walletkit_server.go, could also rely on the same fee calculation
functionality. So we move the function to rpc_utils.go and export it,
and we will reuse it in the FundPsbt workflow in a future PR.
This ensures that for transactions where a fee rate is specified
(instead of a confirmation target), lnd doesn't accept transactions
which would be ultimately ignored by the underlying chain's RPC.
In this commit, we add a new LinkFailureDisconnect action that'll be
used if we detect that the remote party hasn't sent a revoke and ack
when it actually should.
Before this commit, we would log our action, tear down the link, but
then not actually force a connection recycle, as we assumed that if the
TCP connection was actually stale, then the read/write timeout would
expire.
In practice this doesn't always seem to be the case, so we make a strong
action here to actually force a disconnection in hopes that either side
will reconnect and keep the good times rollin' 🕺.
In this commit, we add a new LinkFailureAction enum to take over the old
force close bool. Force closing isn't the only thing we might want to do
when we decide to fail the link, so this is a prep refactoring for an
upcoming change.
It is best to have deterministic fuzz targets, so that if a failure
occurs, it can be easily reproduced.
This commit swaps the cryptographically secure RNG for a deterministic
one seeded from fuzzer input.
Move the functionality directly into completeHandshake instead. If a
failure does happen at any point during the handshake, it is beneficial
to know which line it happens on for debugging. The helper function was
hiding this information.
We test both the happy path (valid memo is returned when querying),
as well as the unhappy path (invalid memo rejects the open action).
To accomplish this, we update the OpenChannelParams struct inside
the harness to accept the Memo.
We add a Memo field to the OpenChannel DB struct. We also persist
it using a tlv record. We then pass the Memo value from the
InitFundingReserveMsg when creating a new reservation for the channel.
Finally, we also read Memo field when fetching channel from DB.
This commit simply adds a new memo flag to the openchannel command.
Memo could be any note-to-self kind of useful information to go
along with the channel. The value isn't used yet, will be in the
next commit.
In this commit, we add an Identifier method to the blob.Type struct
which returns a unique identifier for a given blob type. This identifier
is then used for initialising the disk overflow queue of the given
client.
The bug manifests when a nil ChannelType is passed to the funding
manager in InitFundingMsg. A default value for ChannelType is selected
and sent in the OpenChannel message. However, a nil ChannelType is
stored in the reservation context. This causes our ChannelType checks in
handleFundingAccept to be bypassed.
Usually this makes us end up in the "peer unexpectedly sent explicit
ChannelType" case, where we can still recover by reselecting a default
ChannelType and verifying it matches the one the peer sent. But if the
peer sends a nil ChannelType, we miss it.
While fixing the bug, I also tried to simplify the negotiation logic, as
the complexity is likely what hid the bug in the first place.
Now negotiateCommitmentType only returns the ChannelType to be used in
OpenChannel/AcceptChannel and the CommitmentType to pass to the wallet.
It will even return a nil ChannelType when we're supposed to use implicit
negotiation, so we don't need to manually set it to nil for OpenChannel
and AcceptChannel.
This commits uses TestStoreChangePassword to demonstrate that currently
the ChangePassword function only changes the password of the default
root key and not that of other root keys. This will be fixed in an
upcoming commit.
With this commit, GenerateNewRootKey will regenerate the Default root
key and will then also check if any other root keys exist and regenerate
those as well.
This commit adds to the existing TestStoreGenerateNewRootKey to show
that the method only successfully regenerates the root key in the
default root key ID location. This will be fixed in an upcoming commit.
In this commit, a new generic DiskOverflowQueue implementation is added.
This allows a user to specify a maximum number of items that the queue
can hold in-memory. Any new items will then overflow to disk. The
producer and consumer of the queue items will interact with the queue
just like a normal in-memory queue.
This commit adds a test to the wtclient. The test demonstrates that if a
client tries to back up states while it has no active sessions with a
server then those updates are accumlated in memory and lost on restart.
This will be fixed in upcoming commits.
With recent updates to some of our dependencies, the
github.com/dgrijalva/jwt-go package is not referenced in any transitive
dependency anymore which allows us to remove one of the replace
directives which was added as a security precaution before.
The addition of AliasScid in the tlv_stream field of the
funding_locked msg made the requirement for testdata of the
ExtraOpaqueData part for the funding_locked msg type more strict.
Now the input testdata for the funding_locked test is more specific
and includes the new added AliasScid tlv record.
Move merge of our features into the feature manager, rather than
updating our node announcement then retrospectively setting the
feature manger's set to the new vector. This allows us to use the
feature vector as our single source of truth for announcements.
Base 32 encoded bolt 11 invoices only allow 10 bits to express the
length of the feature vector in a tagged field, so there is a much
lower limit on the values invoice custom features can hold.
Other places in the protocol are theoretically limited by the maximum
message size, but since we express a feature bit as u16 we don't need
to be concerned about this.
The decision is made to track maximum per-set in the feature manager,
which is conceptually aware of sets and then validate in lnwire/features
against some arbitrary maximum value provided to the caller to keep
the base features package unaware of sets.
Disallow update of any features that are defined by LND (since
just updating the feature, but not the functionality will result in
strange behavior). All known feature bits should be toggled using
protocol config options.
In preparation for a more complex function signature for set node
announcement, separate get and set so that readonly callers don't need
to handle the extra arguments.
Previously when a block spend is found for the outpoint, our htlc
timeout resolver will do a checkpoint, which implicitly creates a db
record if there isn't one. Now, if the spend is found in mempool,
the resolver will be deleted once the contract is resolved. Later on
when the spend is found in the block again, the resolver will be created
again, but never gets resolved this time.
This modifies the `genMacaroons` logic to indepently check for each of
the three default macaroons (admin, readonly, invoice) and generate
whichever are missing. Previously, this was an all or nothing routine.
In other words, either all three didn't exist on disk and all three are
created, or no macaroons are created. Although that works for the first
run of a new node, it can result in inconsistent states if only one or
two of the macaroons is deleted.
See https://github.com/lightningnetwork/lnd/discussions/7566.
This commit adds a a new `MarshalOutPoint` helper in the `lnrpc` package
that can be used to convert a `wire.Outpoint` to an `lnrpc.Outpoint`.
By using this helper, we are less likely to forget to populate both the
string and byte form of the TXID.
Lock the `backupMu` when accessing `c.chanCommitHeights` in the `New`
function. It is not strictly necessary right now but good to add it so
that there is no accidental oversight if the `perUpdate` method is ever
extracted and reused in future.
Since the retrubution info of a backup task is now only constructed at
the time that the task is being bound to a session, the in-memory queue
only needs to carry the BackupID of the task.
Since the TowerClient now has a callback that it can use to retrieve the
retribution for a certain channel and commit height, let it use this
call back instead of requiring the info to be passed to it through
BackupState.
In this commit, a new BuildBreachRetribution callback is added to the
tower client's Config struct. The main LND server provides the client
with an implementation of the callback.
This commit introduces a new `channelSelector` method and moves all
generic logic from `FetchChannel` to it. This refactor will make it
easier to add new methods that require the same open-channel db
traversal with slightly different channel selection logic.
This commit adds a small optimisation to the FetchChannel method.
Instead of iterating over each channel bucket, an identifiable error is
thrown once the wanted channel is found so that the iteration can stop
early.
In this commit, we an existing gap in our rebroadcast handling logic. As
is, if we're trying to sweep a transaction and a conflicting transaction
is mined (timeout lands on chain, anchor swept), then we'll continue to
try to rebroadcast the tx in the background.
To resolve this, we give the sweeper a new closure function that it can
use to mark conflicted transactions as no longer requiring rebroadcast.
In this commit, we increase the default CTLV value to 80 blocks.
Initially this was set to 144 blocks in the early days, but then was
lowered to 40 blocks as the lnd implementation matured. By setting this
to a higher value, we increase the safety window (MTTR) when it comes to
node downtime, and also add some buffer room around time locks which may
become more stressed in the future assuming the current mempool load
remains persistent.
In this commit, a bug is fixed in the funding manager that could result
in the funding process erroring out if the persisted initial forwarding
policy is not found. This could occur if a node restarts after opening a
channel that is not yet fully confirmed and also upgrades their node
from a pre-0.16 version to 0.16 since the values are only expected to be
persisted after 0.16.
Currently `make lint` creates a new container each time it runs. We can
automatically delete these containers once linting is done by using the
--rm flag.
This commit extends the current htlc timeout resolver to also watch for
preimage spend in mempool for a full node backend.
If mempool enabled, the resolver will watch the spend of the htlc output
in mempool and blocks **concurrently**, as if they are independent.
Ideally, a transaction will first appear in mempool then in a block.
However, there's no guarantee it will appear in **our** mempool since
there's no global mempool, thus we need to watch the spend in two places
in case it doesn't go through our mempool.
The current design favors the spend event found in blocks, that is, when
the tx is confirmed, we'd abort the monitoring and conitnue since the
outpoint cannot be double spent and re-appear in mempool again. This is
not true in the rare case of reorg, and we will handle reorg seperately.
The right way to solve the problem of the link not being up to date with
custom user set forwarding policies once the channel is announced would
be to pass in those custom values when the link is created initially.
This requires a bit more of a refactor and is not addressed in this bug
fix.
This commit enhances the custom fee policy channel open test by adding a
second channel and testing forwarding payments through the channel with
the custom forwarding policies.
This was able to reproduce the bug reported in #5796 which was fixed in
a previous commit of this PR.
With this commit we give the funding manager the ability to inform the
switch about custom channel policies, right after we've announced the
channel to the network.
This change is necessary because before #6753 a channel could only be
opened with the default forwarding policies, so the switch automatically
had the "correct" default values. Since #6753 added the ability to
specify forwarding policies at channel open time, we announced those
policies to the network but never updated the switch to inform it about
the changed policies (previously changing the policies was only possible
through the UpdateChannelPolicy RPC which did call the switch).
In LND v0.15.1, batch_open_channel used p2tr by default
for change output. However, in a later version, a breaking
change has been fixed in FundPSBT to make the change type
configurable (default to p2wkh). As batch_open_channel
uses FundPsbt, we need to put back p2tr as default for
change output
Refactor `PendingSweeps` to use `allWitnessTypes` map, and
return an error from `PendingSweeps` if an unknown witness
type is used, instead of logging a warning.
With this commit we update the docs according to the latest changes that
were necessary to support loop and pool (which requires all 255
internally used accounts to be imported at wallet creation time).
Fixes#7567.
The default allocation of 500 bytes for the script that is
used in NewScriptBuilder is way too much for most of our scripts.
With the new functional option we can tune the allocation to exactly
what we need.
In this commit, we fix a bug that would cause the walletrpc on the
simnet chain to not be able to respond to all RPC calls. The issue is
that for the SimNet backend, within chainparams.go:
6e0a67d05b/chainreg/chainparams.go (L45-L51),
we set a new CoinType for Simnet in order to match the RegTest value.
Later on, when we go to verify the on disk accounts against the
in-memory cointype, we fall through to an error case as the in-memory
cointype is 115, while the on disk cointype is 1.
To fix this, when we know we're doing simnet mode, we'll override the
cointype similar to the way we did/do for LTC.
Fixes https://github.com/lightningnetwork/lnd/issues/6807.
Allows to define a maximum amount to provision a channel
opening with using a new field `FundUpToMaxAmt` on the
`Request` struct. Also adds a new coin select function
`CoinSelectUpToAmount` to select coins up to a maximum
amount respecting a minimum amount.
In this commit, we bump the version of the master branch to v0.16.99.
This reflects the fact that master will be a super set of all the minor
releases until v0.17, and will also include changes towards v0.17.
All the updates type are now catched in a switch statement, we can
delete the rest of the body of this function + add a default path for
not matched flows.
Previous to this commit we were able to call `UpdateInvoice` with an
updated that added and cancelled htlcs at the same time. The function
returned an error if there was overlapping between the two htlc set.
However, that behavior was not used in the LND code itself.
Eventually we want to split this method in multiple ones, among them one
for canceling htlcs and another one for adding them. For that reason,
this behavior is not supported anymore.
This corrects the documentation for the `amt_paid` and `amt_paid_msat`
fields on the `Invoice` message to indicate that the fields will be set
if the state of the invoice is either accepted or settled, not only
settled. This reflects the actual behavior of lnd, as demonstrated in
the below `lncli` output:
```
"amt_paid": "10000",
"amt_paid_sat": "10",
"amt_paid_msat": "10000",
"state": "ACCEPTED",
```
In this commit, we modify the way we compute the starting ideal fee for
the co-op close transaction. Before thsi commit, channel.CalcFee was
used, which'll compute the fee based on the commitment transaction
itself, rathern than the co-op close transaction. As the co-op close
transaction is potentailly bigger (two P2TR outputs) than the commitment
transaction, this can cause us to under estimate the fee, which can
result in the fee rate being too low to propagate.
To remedy this, we now compute a fee estimate from scratch, based on the
delivery fees of the two parties.
We also add a bug fix in the chancloser unit tests that wasn't caught
due to loop variable shadowing.
The wallet import itest has been updated as well, since we'll now pay
600 extra saothis to close the channel, since we're accounting for the
added weight of the P2TR outputs.
Fixes#6953
This commit adds a short guide that explains how a remote signing node
setup should be migrated from lnd v0.14.x-beta to lnd v0.15.x-beta and
adds a note to all 0.15.x release notes.
In order to reduce the number of calls to the db we try to process as
few channels as we can + try to not do extra work for each of them.
- First fetch all the channels. Then, filter all the public ones and
sort the potential candidates by remote balance.
- Filter out each potential candidate as soon as possible.
- Only check the alias if the channel supports scid aliases.
- Because we sort the channels by remote balance, we will hit the
target amount, if possible, as soon as we can.
We do not want to leak information about our remote balances, so we
shuffle the hop hints (the forced ones go always first) so the invoice
receiver does not know which channels have more balance than others.
This fixes a bug where a caller would:
- call NotifyWhenOnline and pass in a channel
- the server's mutex is held and the pubStr is checked in peersByPub
- the peer object is in the peersByPub map
- the peer object has its quit channel closed
- QuitSignal is called in the select statement, and the function
returns
- the caller is still waiting for the channel to be sent on forever
since it was never sent on or added to the peerConnectedListeners
slice.
This patch fixes the above bug by adding the channel to the
peerConnectedListeners slice if the QuitSignal select case is called.
Lists with non-primitive members aren't supported in the query string of
a GET request with the current version of the grpc-gateway library. To
allow route_hints to be set through REST, we also offer a POST endpoint
for that call where the encoding of the request parameter can be
specified as JSON.
This commit adds a default case to the address type switch statement in
the NewAddress rpc server function. This catches any invalid address
types and returns an error.
Bitcoind 23 will use the new `getdeploymentinfo` while versions after 19
(but below 23) will use the `UnifiedSoftForks` field instead of the
`SoftForks UnifiedSoftForks` field.
With this PR the taproot gating logic has been tested on bitcoind
versions: 21, 22, and 23. 21 is when the taproot logic was first added.
This is required by BOLT#07 as otherwise the counter-party will
discard the channel_update as they may not consider the channel
"ready" or reorg-safe. Most other implementations besides eclair
have work-arounds for this, but it is nice to be spec-compliant.
This updates the RPCAcceptor to send the correct commitment type
even if the zero-conf or scid-alias channel types are set. This also
adds two bools to the ChannelAcceptRequest struct that denotes whether
the funder set the zero-conf and scid-alias channel types.
In this commit, we modify the watch tower to use P2TR addrs for just
about anything sweep related.
One eye sore in this diff are the changes to
`backup_task_internal_test.go`. All the values are hard coded, and now
either differ by a value of 48, or needed to be modified to account for
the new assumptions propagated to rewards values and fees.
In this commit, we catch up our logic with the latest version of the
spec that removed support for normal p2kh and p2sh addresses for co-op
closes, in order to make dust calculations more uniform.
In this commit, we add awareness of the option_shutdown_anysegwit that
permits both sides to send newer segwit based addresses. This'll
eventually enable us to send taproot addresses for co-op close.
With this change, transactions created via craftSweepTx will be
standard. Previously, p2wsh/p2pkh scripts passed in via SendCoins would
be weighted as p2wpkh scripts. With a feerate of 1 sat/vbyte,
transactions returned would be non-standard. Luckily, the critical
sweeper subsystem only used p2wpkh scripts so this only affected
callers from the rpcserver.
Also added is an integration test that fails if SendCoins manages
to generate a non-standard transaction. All script types are now
accounted for in getWeightEstimate, which now errors if an unknown
script type is passed in.
In this commit, we parse the new max fee field, and pass it through the
switch, all the way to the peer where it's ultimately passed into the
chan closer state machine.
In this commit, we remove the raw channel state machine pointer from the
chan closer and instead replace that with an interface that captures
*just* the methods we need in order to do the co-op close dance.
This is a preparatory refactoring for some upcoming unit tests.
In this commit, we stop clamping the ideal fee rate to the commitment
fee of the channel. This catches us up to this PR of the spec:
https://github.com/lightning/bolts/pull/847.
We also do away with the old 3x ideal fee "max fee", and replace that
with an explicit max fee. This new max fee will either be the default
multiplier of the ideal fee, or a new user specified max fee value.
In this commit, we add a check during normal node construction to see if
the backend node supports Taproot. If it doesn't, then we want to
shutdown and force the user to take note.
To check if the node supports Taproot, we'll first try the normal
getblockchaininfo call. If this works, cool, then we can rely on the
value. If it doesn't, then we'll fall back to the getdeploymentinfo call
which was added in a recent version of bitcoind [1]. Newer versions of
bitcoind might also have this call, and the getblockchaininfo call, but
not actually populate the softforks field [2]. In this case, we'll fall
back, and we also account for the case when the getblockchaininfo RPC is
removed all together.
[1]: https://github.com/bitcoin/bitcoin/pull/23508
[2]: https://github.com/bitcoin/bitcoin/pull/25114Fixes#6773
The bitcoind .cookie contains an autogenerated user (__cookie__) and
password (random string), which can be used instead of the rpc user name
and password. This commit allows for running against bitcoind without
having to access bitcoin.conf like in the case for pure
user/password/zmq configuration.
This commit changes how we locate the next migration height by including
the scenario where `lnd@v0.15.0` is active. In the new version, we will
see a mixed of new and old logs under the same open channel bucket.
Hence, we need to alter how we locate the next un-migrated height.
This commit enables the db to run optional migrations that are specified
by config flags. To achieve this, an optional meta is introduced to
manage the optional migrations. We distinguish the two types of
migrations here so it's easier to manage them for the concern a future
migration can cause trouble for us to determine the db version if we
don't.
This commit adds the migration that's used to convert the old revocation
logs into the new format. The migration is fault-tolerant, meaning the
process can be interrupted and the migration will pick up what's left
when running again. We also cap how many records to be processed in each
db transaction to prevent OOM.
This commit adds supporting functions that will be used in the unit
test. The testing data are also added as hard-coded. We choose to copy
the most of the testing data from our itest results such that a) they
are "real" data that can be used to calculate scripts and b) we preserve
the result generated by the current code so a future change won't affect
our test.
This commit adds several utility functions to assist the migration. In
particular, an updateLocator is added to gives us the next un-migration
position in our buckets. This locator helps us to continue the job
in case of an interrupted migration. It also serves as an indicator on
whether the migration is finished or not.
This commit adds relevant code from the revocation_log.go and the
package lnwallet. The code is needed to migrate the data, and we choose
to copy the code instead of importing to preserve the version such that
a future change won't affect current migration. An alternative would be
tagging each of the packages imported.
In this commit, we change the flow of the rpc middleware registration
a bit. In order to allow a client to add rpc middleware interceptors in
a deterministic order, we now make the server send a "registration
complete" message to the client after compeleting the registration
process so that the client knows when it can go ahead and register the
next client.
Ensure the wallet has synced the blockchain before attempting to spend
funds.
Prior to this fix, I get the following error:
rpc error: code = Unknown desc = wallet couldn't fund PSBT: error
creating funding TX: insufficient funds available to construct
transaction
This commit removes the old multi shards test for `SendToRoute` as the
method doesn't support sending MPP. The original test passed due to a
flawed mocking method, where the mockers bypassed the public interfaces
to maintain their internal state, which caused a non-exsiting situation
that a temp error wouldn't fail the payment. A new unit test is added to
demonstrate the actual case.
This commit adds a new method `SendToRouteSkipTempErr` that skips
failing the payment unless a terminal error occurred. This is
accomplished by demoting the original `SendToRoute` to a private method
and creating two new methods on top of it to minimize code change.
With this commit we allow a replacement message to be sent by the
middleware for a request type as well as the response type. This allows
an incoming RPC request to be modified before it is forwarded to lnd.
Because of the way the gRPC Receive() method is designed, we need a way
to replace a proto message with the content of another one without
replacing the original instance itself (e.g. overwrite all values in the
existing struct instance).
If we don't flag the /v1/middleware call as request streaming, it can't
be used properly with REST WebSockets because the proxy would close the
connection after the first request message.
With a change in #6379 we made sure that all default scopes are added to
the the wallet. Unfortunately this included the BIP044 key scope that
our wallet doesn't really use. This breaks the remote signing setup
because we don't export the account of the BIP044 scope and therefore
run into an issue on the watch-only side when attempting to create the
wallet.
In this commit, we fix a detected flake: we go to make a payment, and
then want to assert that 2 attempts we required. The existing logic
assumes that the success and the payment will still be on disk. With a
recent PR, we now delete failed payments by default, but after the fact,
in a non-atomic fashion.
We fix this issue simply by having all the nodes keep around failed
payments for the sake of all the old itests that assumed this
information would always be on disk.
Introduced in: https://github.com/lightningnetwork/lnd/pull/6438.
Fixes https://github.com/lightningnetwork/lnd/issues/6711.
In this commit, we let the registered middleware interceptors be stored
in a slice rather than a map so that the order in which the interceptors
are executed is guarenteed to be the same as the order in which they
were registered.
This removes the requirement that the zero-conf channel acceptor
flow use anchors. Also adds a fail-early check for minimum depth
zero in the non zero conf case. It would fail later, but it makes
more sense to fail immediately when receiving AcceptChannel.
This commit modifies the netann subsystem to use the peer's alias
for ChannelUpdates where appropriate (i.e. in case we are sending
the alias to the peer). It also modifies the loadActiveChannels
function in the peer package to handle upgrading a channel when the
scid-alias feature bit is turned on.
AddInvoice,AddHoldInvoice now issue invoices that include our
peer's aliases. Some extra sanity checks are included to ensure we
don't leak our confirmed SCID for a private channel.
feature-bit channels
This allows opening zero-conf chan-type, scid-alias chan-type, and
scid-alias feature-bit channels. scid-alias chan-type channels are
required to be private. Two paths are available for opening a zero-conf
channel:
* explicit chan-type negotiation
* LDK carve-out where chan-types are not used, LND is on the
receiving end, and a ChannelAcceptor is used to enable zero-conf
When a zero-conf channel is negotiated, the funding manager:
* sends a FundingLocked with an alias
* waits for a FundingLocked from the remote peer
* calls addToRouterGraph to persist the channel using our alias in
the graph. The peer's alias is used to send them a ChannelUpdate.
* wait for six confirmations. If public, the alias edge in the
graph is deleted and replaced (not atomically) with the confirmed
edge. Our policy is also read-and-replaced, but the counterparty's
policy won't exist until they send it to us.
When a scid-alias-feature channel is negotiated, the funding manager:
* sends a FundingLocked with an alias:
* calls addToRouterGraph, sends ChannelUpdate with the confirmed SCID
since it exists.
* when six confirmations occurs, the edge is deleted and re-inserted
since the peer may have sent us an alias ChannelUpdate that we are
storing in the graph.
Since it is possible for a user to toggle the scid-alias-feature-bit
to on while channels exist in the funding manager, care has been taken
to ensure that an alias is ALWAYS sent in the funding_locked message
if this happens.
This intent of this change is to prevent privacy leaks when routing
with aliases and also to allow routing when using an alias. The
aliases are our aliases.
Introduces are two maps:
* aliasToReal:
This is an N->1 mapping for a channel. The keys are the set of
aliases and the value is the confirmed, on-chain SCID.
* baseIndex:
This is also an N->1 mapping for a channel. The keys are the set
of aliases and the value is the "base" SCID (whatever is in the
OpenChannel.ShortChannelID field). There is also a base->base
mapping, so not all keys are aliases.
The above maps are populated when a link is added to the switch and
when the channel has confirmed on-chain. The maps are not removed
from if the link is removed, but this is fine since forwarding won't
occur.
* getLinkByMapping
This function is introduced to adhere to the spec requirements that
using the confirmed SCID of a private, scid-alias-feature-bit
channel does not work. Lnd implements a stricter version of the spec
and disallows this behavior if the feature-bit was negotiated, rather
than just the channel type. The old, privacy-leak behavior is
preserved.
The spec also requires that if we must fail back an HTLC, the
ChannelUpdate must use the SCID of whatever was in the onion, to avoid
a privacy leak. This is also done by passing in the relevant SCID to
the mailbox and link. Lnd will also cancel back on the "incoming" side
if the InterceptableSwitch was used or if the link failed to decrypt
the onion. In this case, we are cautious and replace the SCID if an
alias exists.
This allows the router to determine what is and isn't an alias from
lnd's definition of an alias. Any ChannelAnnouncement that has an
alias ShortChannelID field is not verified on-chain. To prevent a
DoS vector from existing, the gossiper ensures that only the local
lnd node can send its ChannelAnnouncements to the router with an
alias ShortChannelID.
An OptionalMsgField has been added that allows outside subsystems
to provide a short channel id we should insert into a ChannelUpdate
that we then sign and send to our peer.
When the gossiper receives a ChannelUpdate, it will query the
alias manager by the passed-in FindBaseByAlias function to determine
if the short channel id in the ChannelUpdate points to a known
channel. If this lookup returns an error, we'll fallback to using
the original id in the ChannelUpdate when querying the router.
The lookup and potential fallback must occur in order to properly
lock the multimutex, query the correct router channels, and rate
limit the correct short channel id. An unfortunate side effect of
receiving ChannelUpdates from our peer that reference on of our
aliases rather than the real SCID is that we must store this policy.
Yet it is not broadcast-able. Care has been taken to ensure the
gossiper does not broadcast *any* ChannelUpdate with an alias SCID.
The cachedNetworkMsg uses the new processedNetworkMsg struct. This
is necessary so that delete-and-reinsert in the funding manager
doesn't process a ChannelUpdate twice and end up in a deadlock since
the err chan is no longer being used.
This introduces the a store for managing all things alias-related.
There are two maps:
* baseToSet:
This stores the "base" short channel id as the key. The value is
the set of all aliases issued for this channel. The "base" SCID is
whatever is stored in the OpenChannel's ShortChannelID member. For
everything other than zero-conf channels, this is the confirmed SCID.
For zero-conf channels, this is the very first alias assigned. This is
used mostly by the Switch to retrieve a set of aliases and determine
if it is safe to forward.
* aliasToBase:
This stores the reverse mapping of baseToSet. Each key is an alias
SCID and the value is the "base" SCID. This is exclusively used by
the gossiper to determine if an alias in a ChannelUpdate our peer
sends actually references a channel we know of.
The functions make use of the above two maps:
* AddLocalAlias:
This persists the {alias, base} pair in the database. The baseToSet
map is populated. The aliasToBase is optionally populated depending on
where this function is called from. Upgrade cases, where the
scid-alias feature bit is toggled and channels already exist, will
not persist to the gossip map. This is mainly to simplify the tangle
of logic that would otherwise occur.
* GetAliases:
This fetches the set of aliases by using the passed-in base SCID. This
is used in the Switch and other places where the alias set is needed.
* FindBaseSCID:
This fetches the base given an alias. This is used in the gossiper to
determine validity of a peer's ChannelUpdate that contains an alias.
* DeleteSixConfs:
This removes the aliasToBase map entry for the given "base". This is
used when the gossiper mappings are no longer needed, i.e. when the
channel has six confirmations and is public.
* PutPeerAlias:
Stores the peer's alias.
* GetPeerAlias:
Fetches the peer's alias.
* RequestAlias:
Generates an alias for us in the range 16000000:0:0 and
16250000:16777215:65535
This extends the Reservation arguments to include whether a pending
channel open has negotiated the zero-conf channel type, the scid-alias
channel type, and/or the scid-alias feature bit. The result of those
negotiates are stored in the OpenChannel's ChanType. The arguments to
NewChannelReservation have also been simplified.
This introduces a BigSize migration that is used to expand the width
of the ChannelStatus and ChannelType fields. Three channel "types"
are added - ZeroConfBit, ScidAliasChanBit, and ScidAliasFeatureBit.
ScidAliasChanBit denotes that the scid-alias channel type was
negotiated for the channel. ScidAliasFeatureBit denotes that the
scid-alias feature bit was negotiated during the *lifetime* of the
channel. Several helper functions on the OpenChannel struct are
exposed to aid callers from different packages.
The RefreshShortChanID has been renamed to Refresh.
A new function BroadcastHeight is used to guard access to the
mutable FundingBroadcastHeight member. This prevents data races.
This allows the zero-conf and scid-alias feature bits to be toggled
using the config. The feature bits are off by default to protect users
from accidentally incurring the risk of a zero-conf channel.
This defines the zero-conf feature bit, the scid-alias feature bit,
the zero-conf channel type, and the scid-alias channel type. It also
defines the dependency "tree" that exists for the feature bits.
The scid-alias feature bit signals that the node requires an alias
short channel id to be sent in funding_locked. The scid-alias channel
type requires that the channel is private, in addition to some other
forwarding-related privacy measures.
With this, extra calls to RemoveLink will wait for the link to
fully stop. This is accomplished by a map that stores a single stop
channel that callers to RemoveLink will listen on. This map is not
consulted when the Switch is shutting down and calls Stop on each
individual link. Though that could be added in the future, it is
not necessary.
Fixes#6626.
If either of the two fields FinalScriptSig or FinalScriptWitness is set
on an input of a PSBT then that results in most of the fields of that
input not to be serialized in the packet anymore, since the input is
considered to be complete.
But because a signer isn't supposed to set any of the Final* fields,
this was wrong from the beginning. Only the finalizer will set those
fields.
Without this, calls to DisconnectPeer would bypass the
peerTerminationWatcher and allow subsequent connect requests to
go through before the peer's links were fully shut down. This could
lead to force closes.
Fixes#6672.
A new metadata tag called build.vcs.modified was added in Go 1.18.x that
indicates whether there were any untracked files present during the
build or not.
Because the `make release` command creates a directory in which the
output packages are created, and because that directory was not added
to git and also not ignored by it, the build.vcs.modified flag was
different to the docker build which resulted in a different digest of
the resulting binary.
In this commit, we increase the legacy fee limit threshold (the amount
below which we'll allow 100% of funds to go to fees for the non-v2 RPC
calls) from 50 sats to 1k sats.
When updating the channel routing policy, we encounter an inaccurate
precision error when calculating the routing fee. The issue stems from
the way the IEEE 754 standard works.
The solution here is to add a uint64 parameter (as mentioned in the
issue) and keep the float64 fee_rate parameter but rounding the product
of the base and fee rate.
This also changes the chain_watcher and breacharbiter handoff. The
new logic ensures that the channel is only marked as pending closed
when the channel arbitrator has persisted the resolutions and commit
set.
This sets the `CustomCaveatCondition` value on rpc middleware requests
if one exists. Previously, this value was always blank even if the
macaroon had a value set for its custom caveat condition.
This adds a `GetCustomCaveatCondition` function that returns the custom
caveat condition for a given macaroon and caveat name. Previously there
was no function for getting the custom caveat condition from a macaroon,
only for setting one.
In this commit, we modify the implementation of ForEachChannel to
utilize the new kvdb method ForAll. This greatly reduces the number of
round-trips to the database needed to iterate over all channels
in the graph.
This commit adds the `force` flag to the `XImportMissionControl` RPC
which allows skipping rules around the pair import except for what is
mandatory to make values meaningful. This can be useful for when clients
would like to forcibly override MC state in order to manipulate routing
results.
For older nodes, this bucket was never created, so we'll get an error if
we try and query it. In this commit, we catch this error like we do when
a given channel doesn't have the information (but the bucket actually
exists).
Fixes#6155
This was not properly enforced and would be a spec violation on the
peer's end. Also re-use a pong buffer to save on heap allocations if
there are a lot of peers. The pong buffer is only read from, so this
is concurrent safe.
In this commit, we fix an inadvertent memory leak by ensuring we always
use `defer` to clean up the allocated objects/memory we use to be
notified of new blocks to update what we send within the set of ping
headers.
A further optimization here would be using a single global block epoch
housed within the server, that all peer `pingHandler` goroutines use
directly.
Fixes#6143.
To enable converting an existing wallet with private key material into a
watch-only wallet on first startup with remote signing enabled, we add a
new flag. Since the conversion is a destructive process, this shouldn't
happen automatically just because remote signing is enabled.
This fixeslightninglabs/loop#437 by adding all accounts that are used
in liquidity products such as Loop or Pool. Since both of these products
use key families below 255, we can get by with that number.
The alternative to creating way too many accounts (which increases the
default wallet size by ~250kB) would be to hard code the exact accounts
used by Loop (99) and Pool (210). But that sounds like a bad idea given
that there could always be more accounts being added to those (or other)
products. By making sure the first 255 accounts exist, we have a lot
more flexibility in those products for choosing key families.
With the remote signing instance now not needing to know anything about
addresses or current derivation indices, we don't need to forward any
such calls to that instance and can simplify the RPCKeyRing
considerably.
We use SignOutputRaw which expects a witness script being set, even for
P2WKH. There is a special case in SignOutputRaw for the case where the
script is a p2wkh script, then the input script is reconstructed
correctly for the sighash.
This commit adds a new component, harness miner, to the itest. This
newly added component is responsible for checking the mempool and blocks
for the itest.
This commit adds a new struct RPCClients to better handle rpc clients.
A private field, rpc, is added to HarnessNode to prevent direct access
to its clients. Inside RPCClients, all clients are exported in case a
test case need access to a specific client.
This commit removes the context as a param needed when calling methods
of HarnessNode. This change moves the context management inside
HarnessNode, aside from saving us a few lines, it makes the context
creation/timeout less error-prone.
This commit adds a running context to HarnessNode which replaces all the
background context used and also serves as a way to signal quit when the
test is shutting down.
This commit adds a method to resend premature when new blocks arrive.
Previously when a message has specified a block+delta in the future, we
would ignore the message and never process it again, causing an open
channel never being broadcast under fast blocks generation. This commit
fixes it by saving the future messages and resending them once the
required block height is reached.
This commit moves syncing blocks into a dedicated goroutine to avoid the
race condition where several go channels are ready and the block height
update is pushed after a network message is processed.
This commit fixes the issue where duplicate peers are used both in
making persistent connections and bootstrap connections. When we init
bootstrapping, we need to ignore peers that have connections already
made so far plus peers which we are attempting to make connections with,
hence the persistent peers.
In this commit, we switch to always sending a channel type when we're in
explicit mode. This is compatible with prior versions of lnd as they
won't send a channel type, and we'll just arrive at the same type via
the existing implicit funding.
Fixes https://github.com/lightningnetwork/lnd/issues/6067
The minimum relay fee is always ensured to be above our fee floor except
in the very first min relay fee query to bitcoind. This commit ensures
that the fee floor is respected in this first query.
This adds an integration test that makes sure channel can be funded from
empty wallet using PSBT if the funding transaction contains an output
belonging to the wallet, satisfying the reserve.
This change avoids enforcing new reserved value when PSBT funding is not
finished yet as new inputs and outputs may still be added that could
change the outcome of the check.
This originally failed in the scenario when funding a channel from
external wallet *and depositing to on-chain wallet* was done
simultaneously in a single transaction. If such transaction confirms
then reserved UTXO is guaranteed to be available but the check didn't
take it into account.
The enforcement still occurs in the final step of PSBT funding flow, so
it is safe. It also occurs in case of non-PSBT funding.
Since bbolt returns references to internally stored data when storing
locally it's best to copy the byte slices returned or alternatively
convert them to string (which also makes a copy) to avoid crashes casued
by memory corruption.
when payment_hash or final_cltv_delta and payment_request was set, the error message showed that the parameters shouldn't be set with dest instead of payment_request
[skip ci]
Instead of hard coding a commit to use for a binary tool that we use
during the build process, we now only use "go install" to install the
binaries and the golang builtin versioning system to pin the exact
version/commit we want to use in go.mod.
Because the PGP keys are no longer downloaded from Keybase but are used
from the repo directly, we also have to copy them to the Docker image so
we can run the script without needing to supply them through a volume.
With this commit we add a new restriction that checks that the
username in the signature file (manifest-<username>-<version>.sig) actually
does have a signing key and that the signature was created with that key.
The signature is only counted towards the minimum of 5 signatures if
that check is successful.
The changes in this commit were inspired by @kixunil in #5048.
Instead of importing the keys from Keybase, we add all the signing keys
to the repository. In addition to having the key file present each key's
ID must also be added to the verification script. This acts as a double
check that the correct key is added and makes a key change more
explicit.
This removes a vulnerability brought up by @Kixunil where both the lnd
and lncli binaries are executed to obtain their version before they have
been verified against the release. A malicious binary could have already
compromised the user's system before any of the checks had been
performed.
This commit it split out from the doc changes so that it can easily be
cherry-picked to master/0.13.0.
This commit makes SendHTLC (we are the source) evaluate the dust
threshold of the outgoing channel against the default threshold of
500K satoshis. If the threshold is exceeded by adding this HTLC, we
fail backwards. It also makes handlePacketForward (we are forwarding)
evaluate the dust threshold of the incoming channel and the outgoing
channel and fails backwards if either channel's dust sum exceeds the
default threshold.
This commit extends the Mailbox interface with the SetDustClosure,
SetFeeRate, and DustPackets methods. This enables the mailbox to
report the dust exposure to the Switch when the Switch decides whether
to forward a dust packet. The dust is counted from the time an Add is
introduced via AddPacket until it is removed via AckPacket. This can
lead to some packets being counted twice before they are signed for,
but this is a trade-off between accuracy and simplicity.
It over-estimates the local or remote commitment's dust sum by
counting all updates in both updateLogs that are dust using the
trimmed-to-dust mechanism if applicable. The over-estimation is done
because ensuring an accurate counting is a trade-off between code
simplicity and accuracy.
This is necessary and is implied by BOLT#02. Both ChannelReserve
parameters should be above both DustLimit parameters. Otherwise,
it is possible for one side to have nothing at stake.
This commit updates call-sites to use the proper dust limits for
various script types. This also updates the default dust limit used
in the funding flow to be 354 satoshis instead of 573 satoshis.
In this commit, we fix a bug that would cause attempts to re-use an AMP
invoice to fail. Without this commit, we would only attempt to parse the
payment addr if no invoice was specified, so a user manually specifying the
pubkey of the detonation. The fix is straight forward: always parse the
`pay_addr` field as the user may be attempting to re-use an AMP invoice w/o
open coding each of the sections.
This removes a vulnerability brought up by @Kixunil where both the lnd
and lncli binaries are executed to obtain their version before they have
been verified against the release. A malicious binary could have already
compromised the user's system before any of the checks had been
performed.
This commit it split out from the doc changes so that it can easily be
cherry-picked to master/0.13.0.
This produces a race condition when reading AssumeChannelValid from a
different goroutine. Instead we isolate the test cases and initial
AssumeChannelValid properly.
This commit reduces the number of concurrent validation operations the
router will perform when fully validating the channel graph. Reports
from several users indicate that GetInfo would hang for several minutes,
which is believed to be caused by attempting to validate massive amounts
of channels in parallel. This commit returns the limit back to its
original state before adding the batched gossip improvements.
We keep the 1000 concurrent validation request limit for
AssumeChannelValid, since we don't fetch blocks in that case. This
allows us to still keep the performance benefits on mobile/low-resource
devices.
This reverts commit 25b90832e3d74e62176a9c71b0b724d6fb49a39d, as
well as reverting some changes in other commits in the PR that weren't
fully attomic with the primary commit.
The shasum command isn't available in Alpine linux while the sha256sum
command isn't available on MacOS. We add a simple switch that tries to
detect which one is available.
To fix an issue where the golang version would be picked up from the
host system if the docker-release command was used, we switch over to
using make inside of the container as well instead of feeding the
parameters into the release script manually.
We only pass in the flags that we might actually want to overwrite.
In this commit, we thread through the necessary state to allow users to
set a max shard amount. If this value is set, then this'll effectively
serve as a ceiling for all our split attempts. If we need to split,
we'll first try to use `paymentAmt/2`, if that's bigger than
`MaxShardAmt, then we'll use the latter instead.
Ideally in the future we have a dynamic way to automatically set both
the `MaxShardAmt` as well as `MaxParts` for users. Until then exposing
these two new fields will allow us to experiment with setting them
automatically using the RPC interface, and also give users a bit more
control over how we attempt to route payments, akin to coin control for
on-chain payments.
Fixes#4730
In this commit, we raise the default value for the `MaxParts` field from
1 to 16. This change was motivated by the fact that many users either
forget, or don't even know this field is there in the first place. A
value of 16 was chosen rather arbitraliy (other than power of 2). In
the future, we should tune this value based on the expected number of
payment attempts for a given payment amount.
Due to a misunderstanding of how the gpg command line options work, we
didn't actually create detached signatures because the --clear-sign
flag would overwrite that. We update our verification script to now only
download the detached signatures and verify them against the main
manifest file.
We also update the signing instructions.
Instead of only allowing the installed versions of lnd and lncli to be
verified, we now also support specifying explicit paths to binaries that
we want to verify.
In this commit, we fix a bug that would cause resolution of items in
/etc/hosts (or the like) to fail as Tor wouldn't recognize them as
proper host names. With this commit, we'll catch this error and fall
back to the system's resolver.
With this commit, if --tor.active is specified, then IPv6 addresses
will no longer go through the connmgr.TorLookupIP function from btcd.
This function does not have proper IPv6 support and would fail with
the error "tor general error". Instead, use the system resolver.
The recently added gossip throttling was shown to be too aggressive,
especially with our auto channel enable/disable signaling. We switch to
a token bucket based system instead as it's based on time, rather than a
block height which isn't constantly updated at a given rate.
Since the batch interval can potentially be long, adding local updates
to the graph could be slow. This would slow down operations like adding
our own channel update and announcements during the funding process, and
updating edge policies for local channels.
Now we instead check whether the update is remote or not, and only for
remote updates use the SchedulerOption to lazily add them to the graph.
Currently when numgraphsyncpeers=0, lnd will still attempt to perform
an initial historical sync. We change this behavior here to forgoe
historical sync entirely when numgraphsyncpeers is zero, since the
routing table isn't being updated anyway while the node is active.
This permits a no-graph lnd mode where no syncing occurs at all.
This PR updates the hold invoice itest to create a private
channel, and sets the private option on the invoices created
to add coverage for the addition of hop hints.
We do this instead of using the source of the AnnounceSignatures
message, as we filter out the source when broadcasting any
announcements, leading to the remote node not receiving our channel
update. Note that this is done more for the sake of correctness and to
address a flake within the integration tests, as channel updates are
sent directly and reliably to channel counterparts.
To fix an issue where the vendor.tar.gz in a release build had a
different hash if the mobile RPC stubs were in the mobile/ folder, we
clean those out first.
The culprit was the `google.golang.org/grpc/test/bufconn` package which
is currently only used in the mobile RPC stubs and nowhere else.
Therefore the vendor/module.txt was different when vendoring with the
generated mobile RPC stubs being around.
This allows users to see progress whenever the docker image is
[re]built, and (esp on non-linux hosts) track the size of the build
context being uploaded. Currently no output is displayed, so it's hard
to attribute the source of latency, e.g. network latency, building
layers, a large work directory, etc.
AFAICT it's not possible to flip back from bein synced_to_chain, so we
remove the underlying call that could reflect this. The method is moved
into the test file since it's still used to test correctness of other
portions of the flow.
Rather than performing this call in the SyncManager, we give each
gossipSyncer the ability to mark the first sync completed. This permits
pinned syncers to contribute towards the rpc-level synced_to_graph
value, allowing the value to be true after the first pinned syncer or
regular syncer complets. Unlinke regular syncers, pinned syncers can
proceed in parallel possibly decreasing the waiting time if consumers
rely on this field before proceeding to load their application.
A pinned syncer is an ActiveSyncer that is configured to always remain
active for the lifetime of the connection. Pinned syncers do not count
towards the total NumActiveSyncer count, which are rotated periodically.
This features allows nodes to more tightly synchronize their routing
tables by ensuring they are always receiving gossip from distinguished
subset of peers.
Because we now build a docker image for the RPC compilation, we can save
some execution minutes if we run the mobile RPC and code compilation check in the
same step of the CI workflow.
Now that we have a base docker image that has all our RPC compilation
dependencies installed, we can also run the mobile RPC compilation
there. This removes the need to install falafel and goimports on the
local machine.
Because we compile the REST code from the rest-annotations.yaml and no
longer import the annotations in the proto files, we don't need to
specify the custom import path anymore.
This commit aims to make it easier for developers to compile our
protobuf definitions. They now only need to have docker installed
instead of a whole set of binaries and libraries all pinned to very
specific versions.
When verifying the release signatures, we don't want to fail if a
signer's signature is not available in the gpg key ring. Instead we just
don't want to count the signature for now and still succeed if there's
at least one other good sig with a known key.
To avoid leaking any sensitive information like Docker Hub credentials
because of compromised actions repositories, we use our own, vendored
actions for all steps that potentially touch sensitive information.
This commit moves the db calls for retrieving add and settle backlogs
outide of the main event loop. All other db operations are performed
outside of the event loop and synchronized via the invoice registry's
mutex, which also synchronizes the order in which events submitted to be
processed.
This resolves various concurrency issues where notifications can be
missed of inconsistent reads against the databse. This is especially
important in this case because we are actually making two separate
database calls.
These race conditions originate from the mock database storing and
returning pointers, rather than returning a copy.
Observed on Travis:
WARNING: DATA RACE
Read at 0x00c0003222b8 by goroutine 149:
github.com/lightningnetwork/lnd/watchtower/wtclient.(*sessionQueue).drainBackups()
/home/runner/work/lnd/lnd/watchtower/wtclient/session_queue.go:288 +0xed
github.com/lightningnetwork/lnd/watchtower/wtclient.(*sessionQueue).sessionManager()
/home/runner/work/lnd/lnd/watchtower/wtclient/session_queue.go:281 +0x450
Previous write at 0x00c0003222b8 by goroutine 93:
github.com/lightningnetwork/lnd/watchtower/wtclient.getClientSessions()
/home/runner/work/lnd/lnd/watchtower/wtclient/client.go:365 +0x24f
github.com/lightningnetwork/lnd/watchtower/wtclient.(*TowerClient).handleNewTower()
/home/runner/work/lnd/lnd/watchtower/wtclient/client.go:1063 +0x23e
github.com/lightningnetwork/lnd/watchtower/wtclient.(*TowerClient).backupDispatcher()
/home/runner/work/lnd/lnd/watchtower/wtclient/client.go:784 +0x10b9
The logger string used to identify the wtclient and wtclientrpc loggers
was the same, leading to being unable to modify the log level of the
wtclient logger as it would be overwritten with the wtclientrpc's one.
To simplify things, we decide to use the existing RPC logger for
wtclientrpc.
The new table format for the pay command started to use the
`Millisecond()` method on `time.Duration`. However, this method was only
added in Go 1.13, so this breaks the build for Go 1.12. We replace this
by manual division. `time.Duration` "natively" is in nanoseconds, so we
covert to milli seconds by dividing my `time.Millisecond`, which is
1,000,000.
In this commit, we eliminate an extraneous copy in the `QueryPayments`
method. Before this commit, we would copy each payment from the initial
FetchPayments call into a new slice. However, pointers to payments are
return from `FetchPayments`, so we can just maintain that same reference
rather than copying again when we want to limit our response.
If the server hasn't fully started yet, it's possible that the channel
event store hasn't either, so it won't be able to consume any requests
until then. To prevent blocking, we'll just omit the uptime related
fields for now.
In this commit, we update to a new version of `x/crypto` that drops
broken ARM assembly that can cause a segfault in systems like raspis.
The broken assembly was removed in this commit to the runtime:
8b774103d3.
Fixes#4052.
After renewing the certificate, the new certificate wasn't actually
loaded and used, causing the old one to be used until lnd was restarted.
This fixes that by reloading it after it has been written.
This commit makes lnd recreate its TLS certificate if the config's
tlsextradomains or tlsextraips changed. This is useful, since earlier
user would have to manually delete the files to trigger lnd to recreate
them.
To ensure users don't accidentally have their TLS certificate recreated,
we gate it behind a flag --tlsautorefresh that defaults to false.
This commit creates a new utility method IsOutdated that can be used to
check whether a TLS certificate mathces the extra IPs and domains given
in the lnd config.
Since our HTLC must also be added to the remote commitment, we do the
balance caluclation also from the remote chain perspective and report
our minimum balance from the two commit views as our available balance.
When we send non-dust HTLCs as the non-initiator, the remote party will
have to pay the extra commitment fee. To account for this we figure out
if they can afford paying this fee, if not we report that we only have
balance available for dust HTLCs, since these HTLCs won't increase the
commitment fee.
Since we want to handle the edge case where paying the HTLC fee would
take the initiator below the reserve, we move the subtraction of the
reserve into availableBalance where this calculation will be performed.
This commit adds an extra validation step when adding HTLCs. Previously
we would only validate the remote commitment resulting from adding an
HTLC, which in most cases is enough. However, there are situations where
the dustlimits are different, which could lead to the resulting remote
commitment from adding the HTLC being valid, but not the local
commitment.
Now we also validate the local commitment. A test to trigger the case is
added.
add
To ba able to validate the commitment sanity both for remote and local
commitments, and at the same time predict both our and their add, we let
validateCommitmentSanity take an extra payment descriptor to make this
possible.
This commit fixes the TestMaxAcceptedHTLCs, TestMaxPendingAmount,
TestMinHTLC, & TestChanReserve unit tests to pass with the new
ReceiveHTLC logic. Instead of asserting specific failures upon
receiving a new commitment signature, the various assertions were
moved to assert on the error returned from ReceiveHTLC.
This commit checks the commitment sanity when receiving an HTLC so
that if a commitment transaction will overflow from an ADD, it is
caught earlier rather than in ReceiveNewCommitment.
The unit test TestNewBreachRetributionSkipsDustHtlcs triggered a state
transition from Bob, even though it was Alice that had added the HTLCs.
This is wrong since it will lead to Bob still owing Alice a commitment,
which is not accounted for in the unit tests.
We add a sanity check that the add heights has been set for all entries
found in the logs, and return an error otherwise. This won't happen
during normal operation, but it does reveal the mistake in the unit
test, which is fixed by making Alice trigger the transition.
In addition we resolve a long standing TODO by removing a (purposeful)
panic in the channel state machine. Old version of lnd had a bug that
could lead to the parent entries being lost during channel restore. A
panic was added to get to the bottom of if.
This is now fixed, so new nodes shouldn't encounter it. However, to be
on the safe side, instead of panicking we return an error back to
gracefully exit the channel state machine.
This commit enables the user to specify he is not interested in
automatically close channels with pending payments that their
corresponding htlcs have timed-out.
By requiring a configurable grace period uptime of our node
before closing such channels, we give a chance to the other node to
properly cancel the htlc and avoid unnecessary on-chain transaction.
In mobile it is very important for the user experience as otherwise
channels will be force closed more frequently.
This commit changes the shouldGoOnChain signature to get the htlc
as parameter. I will allow the function to take decisions based on
whether the htlc is Incoming or Outgoing.
This change instructs the REST proxy server to overwrite its default
JSON marshaler settings. That allows us to set EmitDefaults to true
which will result in all JSON fields returned in REST responses to
be fully populated, even if their values are falsey.
This commit intends to fix slow first startup time when there are many
invoices that need to be canceled. The slowdown is caused by a combination
of adding invoices to the expiry watcher one-by-one and slow
cancellation. Due to slow cancellation and the unbuffered channel which
we use to pass invoices to the expiry watcher blocks the registry.
With this fix we'll instead batch add invoices to the expiry watcher and
thereby won't block the registry startup.
If the provided ChainHash in a QueryChannelRange message does not match
that of our current chain, then we should send a blank response, rather
than reply with channels for the wrong chain.
Updates were always restored with the same log index. This could cause a
crash when the logs were compacted and possibly other problems
elsewhere.
Extended unit test to cover the crash scenario.
This commit updates the channel state machine to
persistently store remote updates that we have received a
signature for, but that we haven't yet included in a commit
signature of our own.
Previously those updates were only stored in memory and
dropped across restarts. This lead to the production of
an invalid signature and channel force closure. The remote
party expects us to include those updates.
This test asserts that remote updates that are locked-in on the local
commitment, but haven't been signed for on the remote commitment, are
properly restored after a restart.
In this commit, we update to the latest `btcwallet` version that
includes a fix for how we perform rescans. Before this commit, the
wallet would load ALL the created keys into the wallet to perform a
rescan. This is unnecessary, as many of the keys we create are actually
used in contracts, so the wallet can't spend them directly anyway.
For neutrino nodes, this would've caused them to attempt o match more
items in the filter than necessary, possibly resulting in an increased
number of false positive block fetches.
In this commit, we start to clamp the max HTLC forwarding policy to the
current register max HTLC payment size. By doing this, we ensure that
any links that have a advertised max HTLC transit size above the max
payment size will reject any incoming or outgoing attempts for such
large payments.
In this commitment, we make the `SendToRoute` RPC call consistent with
all the other payment RPCs which will properly adhere to the current max
payment sat limit. This is a prep commit for the future wumbo soft cap
that will eventually land in lnd.
The btcwallet update includes some edge-case wallet bug fixes and an
optimization on ZMQ connections for bitcoind backends.
The btcd update allows for compatibility with bitcoind v0.19.0 backends.
2019-11-14 16:31:56 -08:00
1056 changed files with 164550 additions and 51925 deletions
@ -14,6 +14,7 @@ Steps for reviewers to follow to test the change.
- [ ] The change obeys the [Code Documentation and Commenting](https://github.com/lightningnetwork/lnd/blob/master/docs/code_contribution_guidelines.md#CodeDocumentation) guidelines, and lines wrap at 80.
- [ ] Commits follow the [Ideal Git Commit Structure](https://github.com/lightningnetwork/lnd/blob/master/docs/code_contribution_guidelines.md#IdealGitCommitStructure).
- [ ] Any new logging statements use an appropriate subsystem and logging level.
- [ ] [There is a change description in the release notes](https://github.com/lightningnetwork/lnd/tree/master/docs/release-notes), or `[skip ci]` in the commit message for small changes.
- [ ] Any new lncli commands have appropriate tags in the comments for the rpc in the proto file.
- [ ] [There is a change description in the release notes](https://github.com/lightningnetwork/lnd/tree/master/docs/release-notes), or `[skip ci]` in the commit message for small changes.
📝 Please see our [Contribution Guidelines](https://github.com/lightningnetwork/lnd/blob/master/docs/code_contribution_guidelines.md) for further guidance.
📝 Please see our [Contribution Guidelines](https://github.com/lightningnetwork/lnd/blob/master/docs/code_contribution_guidelines.md) for further guidance.
From this new version onwards, in addition time-stamping the _git tag_ with [OpenTimeStamps](https://opentimestamps.org/), we'll also now timestamp the manifest file along with its signature. Two new files are now included along with the rest of our release artifacts:` manifest-roasbeef-${{ env.RELEASE_VERSION }}.txt.asc.ots`.
From this new version onwards, in addition time-stamping the _git tag_ with [OpenTimestamps](https://opentimestamps.org/), we'll also now timestamp the manifest file along with its signature. Two new files are now included along with the rest of our release artifacts:` manifest-roasbeef-${{ env.RELEASE_VERSION }}.txt.asc.ots`.
Assuming you have the opentimestamps client installed locally, the timestamps can be verified with the following commands:
Alternatively, [the open timestamps website](https://opentimestamps.org/) can be used to verify timestamps if one doesn't have a `bitcoind` instance accessible locally.
Alternatively, [the OpenTimestamps website](https://opentimestamps.org/) can be used to verify timestamps if one doesn't have a `bitcoind` instance accessible locally.
These timestamps should give users confidence in the integrity of this release even after the key that signed the release expires.
## Verifying the Release Binaries
Our release binaries are fully reproducible. Third parties are able to verify that the release binaries were produced properly without having to trust the release manager(s). See our [reproducible builds guide](https://github.com/lightningnetwork/lnd/tree/master/build/release) for how this can be achieved.
Our release binaries are fully reproducible. Third parties are able to verify that the release binaries were produced properly without having to trust the release manager(s). See our [reproducible builds guide](https://github.com/lightningnetwork/lnd/blob/master/docs/release.md) for how this can be achieved.
The release binaries are compiled with `go${{ env.GO_VERSION }}`, which is required by verifiers to arrive at the same ones.
They include the following build tags:`autopilotrpc`, `signrpc`, `walletrpc`, `chainrpc`, `invoicesrpc`, `neutrinorpc`, `routerrpc`, `watchtowerrpc`, `monitoring`, `peersrpc`, `kvdb_postrgres`, `kvdb_etcd` and `kvdb_sqlite`. Note that these are already included in the release script, so they do not need to be provided.
#? sample-conf-check: Make sure default values in the sample-lnd.conf file are set correctly
sample-conf-check:
@$(call print, "Making sure every flag has an example in the sample-lnd.conf file")
for flag in $$(GO_FLAGS_COMPLETION=1 go run -tags="$(RELEASE_TAGS)"$(PKG)/cmd/lnd -- | grep -v help| cut -c3-);doif ! grep -q $$flag sample-lnd.conf;thenecho"Command line flag --$$flag not added to sample-lnd.conf";exit 1;fi;done
@$(call print, "Checking that default values in the sample-lnd.conf file are set correctly")
- [Other versions are tagged](https://github.com/btcpayserver/lnd/tags), but obsoleted and not supported.
- All LND versions prior to 0.15.4 contain a consensus bug that prevents them from properly parsing transactions with more than 500,000 witness items per input (https://github.com/btcsuite/btcd/issues/1906)
- All LND versions prior to 0.15.2 contain a bug that prevents them from properly parsing Taproot transactions with script size over 11000 bytes (https://github.com/lightningnetwork/lnd/issues/7002)
- LND version 0.14.0-beta shipped with check that made it incompatable with c-lightning and eclair (https://github.com/lightningnetwork/lnd/issues/5890)
- All LND versions prior to 0.13.3 contain specification-level vulnerability (https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-October/003257.html)
- All LND versions prior to 0.7 contain critical vulnerability (https://lists.linuxfoundation.org/pipermail/lightning-dev/2019-September/002174.html)
## Lightning Network Specification Compliance
`lnd`_fully_ conforms to the [Lightning Network specification
Basis of Lightning Technology. The specifications are currently being drafted
by several groups of implementers based around the world including the
developers of `lnd`. The set of specification documents as well as our
implementation of the specification are still a work-in-progress. With that
said, the current status of `lnd`'s BOLT compliance is:
Each version is marked with appropriate `basedon-vX.X.X-beta` tags. We are using `basedon` prefix in order not to conflict with LND tags from source repository.
- [X] BOLT 1: Base Protocol
- [X] BOLT 2: Peer Protocol for Channel Management
- [X] BOLT 3: Bitcoin Transaction and Script Formats
- [X] BOLT 4: Onion Routing Protocol
- [X] BOLT 5: Recommendations for On-chain Transaction Handling
- [X] BOLT 7: P2P Node and Channel Discovery
- [X] BOLT 8: Encrypted and Authenticated Transport
- [X] BOLT 9: Assigned Feature Flags
- [X] BOLT 10: DNS Bootstrap and Assisted Node Location
- [X] BOLT 11: Invoice Protocol for Lightning Payments
## Updating LND version in BTCPay Server
## Developer Resources
1. **Update https://github.com/btcpayserver/lnd**
The daemon has been designed to be as developer friendly as possible in order
to facilitate application development on top of `lnd`. Two primary RPC
interfaces are exported: an HTTP REST API, and a [gRPC](https://grpc.io/)
service. The exported API's are not yet stable, so be warned: they may change
drastically in the near future.
a) Go to https://github.com/lightningnetwork/lnd/releases and find the commit on which we should add our resources.
b) Checkout a new branch for that commit, usually in the format of `lnd/v0.18.3-beta`.
c) Cherry-pick the `Adding BtcPayServer related files and resources` commit. [Example commit](https://github.com/btcpayserver/lnd/commit/ae4bb33c6a3db8b7cc01d18fdf46e600ead9bed4).
d) Tag it with the `basedon-v*` prefix name and push it. For v0.18.1, the tag name was `basedon-v0.18.3-beta`.
i. Before you push the tag to CircleCI to build and publish image to Docker Hub, you can test if building the image works locally.
ii. You can do this for linuxamd64 for example by using command `docker build --pull -t local-lnd:test_version -f linuxamd64.Dockerfile .`
e) The build process will start (it [matches on tag format](.circleci/config.yml#L11)). Here is [an example CircleCI build](https://app.circleci.com/pipelines/github/btcpayserver/lnd/202/workflows/b90b5888-c0b8-4207-860e-a63ce21077af).
f) The resulting image will be published to Docker Hub. Example [Docker Hub image for v0.18.3](https://hub.docker.com/layers/btcpayserver/lnd/v0.18.3-beta/images/sha256-513ddd55a5af44a14e27110ee14cb28f1c7a69205bcaa2fba4e66275c1f725e5?context=repo).
An automatically generated set of documentation for the RPC APIs can be found
at [api.lightning.community](https://api.lightning.community). A set of developer
resources including guides, articles, example applications and community resources can be found at:
- Versioning of base Docker images used for building Go binaries. You may need to bump that base image - [example commit](https://github.com/btcpayserver/lnd/commit/c841954c515a9d067c24987291316b093b91c2f2).
- [Updating Loop](https://github.com/lightninglabs/loop) as part of the package, which needs to happen occasionally. [Example bump of Loop version](https://github.com/btcpayserver/lnd/commit/b3aecc7ac58280ef662e39ba99461573a30fe79a
Finally, we also have an active
[Slack](https://lightning.engineering/slack.html) where protocol developers, application developers, testers and users gather to
discuss various aspects of `lnd` and also Lightning in general.
In order to build from source, please see [the installation
instructions](docs/INSTALL.md).
Now we need to update the dependency in our Lightning library project. This library has tests, so we will know if something is broken.
## Docker
To run lnd from Docker, please see the main [Docker instructions](docs/DOCKER.md)
a) Modify the `docker-compose.yml` file to reference the new LND version. [Example](https://github.com/btcpayserver/BTCPayServer.Lightning/pull/162/commits/413784ef9b2a8e7aa0496eb91f792ff0086c0ef7).
b) Checkout a new branch for that commit, usually in the format of `feat/lnd-0.18.1`.
c) Title the commit `Bumping LND to 0.18.1-beta`.
d) Open a pull request and reference Docker Hub and Tag. [Example PR](https://github.com/btcpayserver/BTCPayServer.Lightning/pull/162).
This will give access to LND to the whole dev team and allow for further testing on their dev machines if everything works as expected.
When operating a mainnet `lnd` node, please refer to our [operational safety
guidelines](docs/safety.md). It is important to note that `lnd` is still
**beta** software and that ignoring these operational guidelines can lead to
loss of funds.
a) Modify 4 `docker-compose.yml` files in `BTCPayServer.Tests`. [Example pull request to emulate](https://github.com/btcpayserver/btcpayserver/pull/6795).
b) When you open the PR, include the version and link to the BTCPayServer.Lightning PR.
a) Now that everything is prepared, open a PR in the btcpayserver-docker repository to allow these changes to propagate to everyone. [Example pull request](https://github.com/btcpayserver/btcpayserver-docker/pull/911).
b) Open the PR in DRAFT mode and tag @NicolasDorier and @Pavlenex as reviewers. They typically handle releases, and once they test that the LND version update works on their server, they can ACK the update and merge it as part of the release process.
## Further reading
* [Step-by-step send payment guide with docker](https://github.com/lightningnetwork/lnd/tree/master/docker)
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
