Compare commits

...

56 Commits

Author SHA1 Message Date
Nicolas Dorier
5147b5f261
bump deps 2026-06-10 10:12:06 +09:00
Nicolas Dorier
6ad4941712
Merge pull request #544 from s373nZ/docker-curl
Add `curl` to Docker image to support health checks
2026-06-02 09:07:36 +09:00
se7enz
98d89f924a
docker: Add curl to Docker image to support health checks 2026-05-26 09:52:03 +02:00
Nicolas Dorier
36e1596b5c
Faster tests 2026-04-24 14:14:09 +09:00
Nicolas Dorier
83e0ab4f68
Relax xpub validation 2026-04-24 12:25:52 +09:00
Nicolas Dorier
c24b47bb03
Deprecate import into RPC 2026-04-24 09:05:55 +09:00
Nicolas Dorier
b9f324b2eb
bump testframework 2026-04-23 15:33:14 +09:00
Nicolas Dorier
e0af3bf493
Fix broken links 2026-04-23 15:23:49 +09:00
Nicolas Dorier
06349f9818
Trying to speedup tests 2026-04-23 14:51:28 +09:00
Nicolas Dorier
c46949ac60
Merge pull request #542 from dgarage/chunk
Add chunk fee and weight to transaction metadata
2026-04-22 18:14:34 +09:00
Nicolas Dorier
5e570a84fb
Add chunk fee and weight to transaction metadata 2026-04-22 18:03:29 +09:00
Nicolas Dorier
060d19f6a8
bump 2026-04-22 11:13:19 +09:00
Nicolas Dorier
b83178be8d
Fix tests 2026-04-22 10:51:31 +09:00
Nicolas Dorier
f72fc4d321
Remove some error during tests 2026-04-22 09:59:01 +09:00
Nicolas Dorier
9242d645b6
Trying to speed up tests 2026-04-22 09:25:43 +09:00
Nicolas Dorier
05df9b5037
Decrease spams in test logs 2026-04-22 09:06:09 +09:00
Nicolas Dorier
456717935c
Fix tests 2026-04-22 09:03:32 +09:00
Nicolas Dorier
98df6b7fdd
Improve error messages for RPC wallet errors 2026-04-22 08:54:33 +09:00
Nicolas Dorier
449568be9a
Bump test to 31.0 2026-04-22 08:47:51 +09:00
Nicolas Dorier
abb5fd3a6c
bump bitcoin core in tests 2026-04-21 21:21:35 +09:00
Nicolas Dorier
deb868b2c0
Add PushNuget.sh 2026-04-21 10:38:46 +09:00
Nicolas Dorier
f0000ceab6
bump libs 2026-04-21 10:30:00 +09:00
Nicolas Dorier
d9cba8b0b5
bump nbt 2026-04-11 09:17:57 +09:00
Nicolas Dorier
becdc3f47d
Fix DOGE getting stuck 2026-04-10 22:33:18 +09:00
Nicolas Dorier
aa462e7af1
bump 2026-04-10 11:28:07 +09:00
Nicolas Dorier
34d6b0dcf3
Potentially fix dogecoin bug, improve logs when node connection fail with exception 2026-04-10 11:27:47 +09:00
Nicolas Dorier
87037e17e1
Improve logs at startup 2026-04-10 10:44:46 +09:00
Nicolas Dorier
c806e9022e
Do not use static logger in tests 2026-04-10 09:44:04 +09:00
Nicolas Dorier
39b1861d7a
Fix test 2026-04-10 09:27:13 +09:00
Nicolas Dorier
a379506b46
Port code to dotnet10.0 style 2026-04-10 09:18:44 +09:00
Nicolas Dorier
68513b4668
Remove message broker leftover 2026-04-10 09:09:44 +09:00
Nicolas Dorier
802945291a
Remove message broker dependencies 2026-04-10 08:55:28 +09:00
Nicolas Dorier
3756b54138
Fix compatibility with core 31.0 2026-04-10 08:17:05 +09:00
Nicolas Dorier
9eeaf71195
bump 2026-04-03 11:05:43 +09:00
Nicolas Dorier
0e4222a1f2
Merge pull request #539 from Unknown-Kush/master
Add Pepecoin
2026-04-03 11:04:13 +09:00
Unknown-Kush
34831b2d85 Add Pepecoin support 2026-04-02 09:51:55 -04:00
Unknown-Kush
aba8cdc8c6 bump NBitcoin.Altcoins 2026-04-02 09:24:52 -04:00
Nicolas Dorier
ba76d3b059
bump dockerfile 2026-04-01 10:07:47 +09:00
Nicolas Dorier
26b3dddc28
bump nbx 2026-03-11 09:54:42 +09:00
Nicolas Dorier
1a38ffa497
Bump libraries 2026-03-11 09:54:42 +09:00
Nicolas Dorier
2fb63ae18e
Merge pull request #537 from NicolasDorier/retry-flaky-rpc
Retry flaky idempotent RPC requests
2026-03-11 09:44:29 +09:00
Nicolas Dorier
206fd0017f
Fix .net10 mention 2026-03-11 09:42:01 +09:00
Nicolas Dorier
5d78c7034c
Retry flaky idempotent RPC requests 2026-03-11 09:32:27 +09:00
Nicolas Dorier
f226238779
Add publish-docker.sh 2026-03-11 09:15:22 +09:00
Nicolas Dorier
05d8ed054e bump 2025-12-26 11:05:07 +09:00
Nicolas Dorier
4ebd7c4c2f
Update to .NET10.0 and bump libs (#534) 2025-12-20 23:29:30 +09:00
Nicolas Dorier
8670e58923 bump postgres 2025-12-18 16:51:01 +09:00
nicolas.dorier
5ea76f64bb
bump NBitcoin.Testframework 2025-09-09 22:06:10 +09:00
nicolas.dorier
be086282ae
Allow feature tag to version on docker 2025-09-06 11:29:45 +09:00
nicolas.dorier
58bde3c916
bump asp.net 2025-09-06 11:25:36 +09:00
nicolas.dorier
8af448f724
bump 2025-08-27 19:41:21 +09:00
Nicolas Dorier
273333b040
Fix documentation AI slop, allow more format for broadcasting (#529) 2025-08-27 11:40:53 +01:00
Nicolas Dorier
d03465c240
Fix incorrect doc for GetUTXOs (#530) 2025-08-27 11:40:33 +01:00
nicolas.dorier
58e9bba5a2
Fix: Unable to delete a ADDRESS: item in a group 2025-08-27 19:40:09 +09:00
nicolas.dorier
c58c60c9fa
bump 2025-08-25 10:09:38 +09:00
nicolas.dorier
939a575c51
Fix: Periodic tasks would sometimes stop firing 2025-08-25 10:09:37 +09:00
53 changed files with 635 additions and 1473 deletions

View File

@ -36,5 +36,8 @@ workflows:
filters:
branches:
ignore: /.*/
# only act on version tags v1.0.0.88 or v1.0.2-1
# OR feature tags like abc
# OR features on specific versions like v1.0.0.88-abc-1
tags:
only: /v[0-9]+(\.[0-9]+)*/
only: /(v[1-9]+(\.[0-9]+)*(-[a-z0-9-]+)?)|(v[a-z0-9-]+)/

View File

@ -1,4 +1,4 @@
FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:8.0.404-bookworm-slim AS builder
FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:10.0.301-noble AS builder
WORKDIR /source
COPY NBXplorer/NBXplorer.csproj NBXplorer/NBXplorer.csproj
COPY NBXplorer.Client/NBXplorer.Client.csproj NBXplorer.Client/NBXplorer.Client.csproj
@ -8,12 +8,14 @@ COPY . .
RUN cd NBXplorer && \
dotnet publish --output /app/ --configuration Release
FROM mcr.microsoft.com/dotnet/aspnet:8.0.11-bookworm-slim
FROM mcr.microsoft.com/dotnet/aspnet:10.0.9-noble
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends curl && rm -rf /var/lib/apt/lists/*
RUN mkdir /datadir
ENV NBXPLORER_DATADIR=/datadir
VOLUME /datadir
COPY --from=builder "/app" .
ENTRYPOINT ["dotnet", "NBXplorer.dll"]
ENTRYPOINT ["dotnet", "NBXplorer.dll"]

View File

@ -1,4 +1,5 @@
using Newtonsoft.Json;
using System;
using Newtonsoft.Json;
using System.Collections.Generic;
namespace NBXplorer.Models
@ -14,6 +15,7 @@ namespace NBXplorer.Models
[JsonConverter(typeof(NBXplorer.JsonConverters.ScriptPubKeyTypeConverter))]
public NBitcoin.ScriptPubKeyType? ScriptPubKeyType { get; set; }
public string Passphrase { get; set; }
[Obsolete("We will remove this feature in a future release.")]
public bool ImportKeysToRPC { get; set; }
public bool SavePrivateKeys { get; set; }
public Dictionary<string, string> AdditionalOptions { get; set; }

View File

@ -127,8 +127,6 @@ namespace NBXplorer.Models
public static bool TryParse(ReadOnlySpan<char> trackedSource, out GroupTrackedSource walletTrackedSource)
{
if (trackedSource == null)
throw new ArgumentNullException(nameof(trackedSource));
walletTrackedSource = null;
if (!trackedSource.StartsWith("GROUP:".AsSpan(), StringComparison.Ordinal))
return false;
@ -177,8 +175,6 @@ namespace NBXplorer.Models
public static bool TryParse(ReadOnlySpan<char> strSpan, out TrackedSource addressTrackedSource, Network network)
{
if (strSpan == null)
throw new ArgumentNullException(nameof(strSpan));
if (network == null)
throw new ArgumentNullException(nameof(network));
addressTrackedSource = null;
@ -216,8 +212,6 @@ namespace NBXplorer.Models
public static bool TryParse(ReadOnlySpan<char> strSpan, out DerivationSchemeTrackedSource derivationSchemeTrackedSource, NBXplorerNetwork network)
{
if (strSpan == null)
throw new ArgumentNullException(nameof(strSpan));
if (network == null)
throw new ArgumentNullException(nameof(network));
derivationSchemeTrackedSource = null;

View File

@ -10,6 +10,18 @@ namespace NBXplorer.Models
{
public class TransactionMetadata
{
public class ChunkMetadata
{
[JsonProperty("fees", DefaultValueHandling = DefaultValueHandling.Ignore)]
[JsonConverter(typeof(NBXplorer.JsonConverters.MoneyJsonConverter))]
public Money Fees { get; set; }
[JsonProperty("weight", DefaultValueHandling = DefaultValueHandling.Ignore)]
public int Weight { get; set; }
[JsonProperty("feeRate", DefaultValueHandling = DefaultValueHandling.Ignore)]
[JsonConverter(typeof(NBitcoin.JsonConverters.FeeRateJsonConverter))]
public FeeRate FeeRate { get; set; }
}
[JsonProperty("vsize", DefaultValueHandling = DefaultValueHandling.Ignore)]
public int? VirtualSize { get; set; }
[JsonProperty("fees", DefaultValueHandling = DefaultValueHandling.Ignore)]
@ -18,10 +30,10 @@ namespace NBXplorer.Models
[JsonProperty("feeRate", DefaultValueHandling = DefaultValueHandling.Ignore)]
[JsonConverter(typeof(NBitcoin.JsonConverters.FeeRateJsonConverter))]
public FeeRate FeeRate { get; set; }
public ChunkMetadata Chunk { get; set; }
public static TransactionMetadata Parse(string json) => JsonConvert.DeserializeObject<TransactionMetadata>(json);
public string ToString(bool indented) => JsonConvert.SerializeObject(this, indented ? Formatting.Indented : Formatting.None);
public override string ToString() => ToString(true);
[JsonExtensionData]
public IDictionary<string, JToken> AdditionalData { get; set; } = new Dictionary<string, JToken>();
}

View File

@ -1,16 +1,16 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net8.0;netstandard2.1</TargetFrameworks>
<TargetFrameworks>net10.0;netstandard2.1</TargetFrameworks>
<Company>Digital Garage</Company>
<Version>5.0.5</Version>
<Version>5.0.6</Version>
<Copyright>Copyright © Digital Garage 2017</Copyright>
<Description>Client API for the minimalist HD Wallet Tracker NBXplorer</Description>
<PackageIcon>Bitcoin.png</PackageIcon>
<PackageTags>bitcoin</PackageTags>
<PackageProjectUrl>https://github.com/dgarage/NBXplorer/</PackageProjectUrl>
<PackageProjectUrl>https://github.com/btcpayserver/NBXplorer/</PackageProjectUrl>
<PackageLicenseExpression>MIT</PackageLicenseExpression>
<RepositoryUrl>https://github.com/dgarage/NBXplorer</RepositoryUrl>
<RepositoryUrl>https://github.com/btcpayserver/NBXplorer</RepositoryUrl>
<RepositoryType>git</RepositoryType>
<PackageReadmeFile>README.md</PackageReadmeFile>
<LangVersion>12</LangVersion>
@ -27,11 +27,17 @@
<NoWarn>$(NoWarn);1591;1573;1572;1584;1570;3021</NoWarn>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="NBitcoin" Version="9.0.0" />
<PackageReference Include="NBitcoin.Altcoins" Version="5.0.0" />
<PackageReference Include="NBitcoin" Version="10.0.6" />
<PackageReference Include="NBitcoin.Altcoins" Version="6.0.3" />
</ItemGroup>
<ItemGroup>
<InternalsVisibleTo Include="NBXplorer.Tests" />
</ItemGroup>
<ItemGroup>
<None Include="..\README.md" Pack="true" PackagePath="\" />
<None Include="Bitcoin.png" Pack="true" PackagePath="\" />
</ItemGroup>
<ItemGroup Condition="'$(TargetFramework)' == 'net10.0'">
<PackageReference Include="Microsoft.AspNetCore.Mvc.NewtonsoftJson" Version="10.0.9" />
</ItemGroup>
</Project>

View File

@ -0,0 +1,24 @@
using NBitcoin;
using System;
namespace NBXplorer
{
public partial class NBXplorerNetworkProvider
{
private void InitPepecoin(ChainName networkType)
{
Add(new NBXplorerNetwork(NBitcoin.Altcoins.Pepecoin.Instance, networkType)
{
MinRPCVersion = 10000,
ChainLoadingTimeout = TimeSpan.FromHours(1),
ChainCacheLoadingTimeout = TimeSpan.FromMinutes(2),
CoinType = NetworkType == ChainName.Mainnet ? new KeyPath("3434'") : new KeyPath("1'")
});
}
public NBXplorerNetwork GetPEPE()
{
return GetFromCryptoCode(NBitcoin.Altcoins.Pepecoin.Instance.CryptoCode);
}
}
}

View File

@ -13,6 +13,7 @@ namespace NBXplorer
InitBitcore(networkType);
InitLitecoin(networkType);
InitDogecoin(networkType);
InitPepecoin(networkType);
InitBCash(networkType);
InitGroestlcoin(networkType);
InitBGold(networkType);

9
NBXplorer.Client/PushNuget.sh Executable file
View File

@ -0,0 +1,9 @@
#!/bin/bash
set -euo pipefail
rm -rf "bin/Release/"
dotnet pack --configuration Release --include-symbols -p:SymbolPackageFormat=snupkg
package=$(find ./bin/Release -name "*.nupkg" -type f | head -n 1)
dotnet nuget push "${package[0]}" --source "https://api.nuget.org/v3/index.json" --api-key "$NUGET_API_KEY"
ver=$(basename "${package[0]}" | sed -E 's/NBXplorer\.Client\.([0-9]+(\.[0-9]+){1,3}).*/\1/')
git tag -a "Client/v$ver" -m "Client/$ver"
git push origin "Client/v$ver"

View File

@ -1,84 +0,0 @@
using Microsoft.AspNetCore.Hosting;
using System.Linq;
using System;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Builder;
using System.Threading.Tasks;
using System.Threading;
using Microsoft.AspNetCore.Hosting.Server.Features;
using System.Net;
using System.Net.Sockets;
namespace NBXplorer.Tests
{
public class CustomServer : IDisposable
{
public static int FreeTcpPort()
{
TcpListener l = new TcpListener(IPAddress.Loopback, 0);
l.Start();
int port = ((IPEndPoint)l.LocalEndpoint).Port;
l.Stop();
return port;
}
TaskCompletionSource<bool> _Evt = null;
IWebHost _Host = null;
CancellationTokenSource _Closed = new CancellationTokenSource();
public CustomServer()
{
var port = FreeTcpPort();
_Host = new WebHostBuilder()
.Configure(app =>
{
app.Run(req =>
{
while(_Act == null)
{
Thread.Sleep(10);
_Closed.Token.ThrowIfCancellationRequested();
}
_Act(req);
_Act = null;
_Evt.TrySetResult(true);
req.Response.StatusCode = 200;
return Task.CompletedTask;
});
})
.UseKestrel()
.UseUrls("http://127.0.0.1:" + port)
.Build();
_Host.Start();
}
public Uri GetUri()
{
return new Uri(_Host.ServerFeatures.Get<IServerAddressesFeature>().Addresses.First());
}
Action<HttpContext> _Act;
public void ProcessNextRequest(Action<HttpContext> act)
{
var source = new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
CancellationTokenSource cancellation = new CancellationTokenSource(20000);
cancellation.Token.Register(() => source.TrySetCanceled());
source = new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
_Evt = source;
_Act = act;
try
{
_Evt.Task.GetAwaiter().GetResult();
}
catch(TaskCanceledException)
{
throw new Xunit.Sdk.XunitException("Callback to the webserver was expected, check if the callback url is accessible from internet");
}
}
public void Dispose()
{
_Closed.Cancel();
_Host.Dispose();
}
}
}

View File

@ -1,4 +1,4 @@
FROM mcr.microsoft.com/dotnet/sdk:8.0.404-bookworm-slim AS builder
FROM mcr.microsoft.com/dotnet/sdk:10.0.301-noble AS builder
WORKDIR /source
COPY . .
RUN cd NBXplorer.Tests && dotnet build

View File

@ -77,16 +77,4 @@ namespace NBXplorer.Tests
catch { }
}
}
public class Logs
{
public static ILog Tester
{
get; set;
}
public static XUnitLoggerProvider LogProvider
{
get;
set;
}
}
}

View File

@ -8,18 +8,13 @@ using Xunit.Abstractions;
namespace NBXplorer.Tests
{
public class MaintenanceUtilities
public class MaintenanceUtilities(ITestOutputHelper helper) : UnitTestBase(helper)
{
public MaintenanceUtilities(ITestOutputHelper helper)
{
Logs.Tester = new XUnitLog(helper) { Name = "Tests" };
Logs.LogProvider = new XUnitLoggerProvider(helper);
}
[Fact]
[Trait("Maintenance", "Maintenance")]
public async Task GenerateFullSchema()
{
using var t = ServerTester.Create();
using var t = CreateTester();
var script = await GenerateDbScript(t);
File.WriteAllText(GetFullSchemaFile(), script);
}

View File

@ -1,6 +1,6 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework Condition="'$(TargetFrameworkOverride)' == ''">net8.0</TargetFramework>
<TargetFramework Condition="'$(TargetFrameworkOverride)' == ''">net10.0</TargetFramework>
<LangVersion>12</LangVersion>
<TargetFramework Condition="'$(TargetFrameworkOverride)' != ''">$(TargetFrameworkOverride)</TargetFramework>
</PropertyGroup>
@ -11,10 +11,11 @@
<EmbeddedResource Include="Scripts\generate-whale.sql" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="NBitcoin.TestFramework" Version="4.0.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.12.0" />
<PackageReference Include="xunit" Version="2.9.2" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.8.0">
<PackageReference Include="Microsoft.AspNetCore.Mvc.NewtonsoftJson" Version="10.0.9" />
<PackageReference Include="NBitcoin.TestFramework" Version="5.0.2" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.6.0" />
<PackageReference Include="xunit" Version="2.9.3" />
<PackageReference Include="xunit.runner.visualstudio" Version="3.1.5">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
</PackageReference>

View File

@ -29,6 +29,12 @@ namespace NBXplorer.Tests
//Network = NBitcoin.Altcoins.Dogecoin.Instance.Regtest;
//RPCStringAmount = false;
//Tests of PEPE are broken because it outpoint locking seems to work differently
//CryptoCode = "PEPE";
//nodeDownloadData = NodeDownloadData.Pepecoin.v1_1_0;
//Network = NBitcoin.Altcoins.Pepecoin.Instance.Regtest;
//RPCStringAmount = false;
//CryptoCode = "DASH";
//nodeDownloadData = NodeDownloadData.Dash.v0_12_2;
//Network = NBitcoin.Altcoins.Dash.Instance.Regtest;

View File

@ -1,7 +1,7 @@
using System.Linq;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Hosting;
using NBXplorer.Configuration;
using Microsoft.AspNetCore.Hosting;
using NBitcoin;
using NBitcoin.Tests;
using System;
@ -16,9 +16,11 @@ using NBitcoin.RPC;
using System.Net;
using NBXplorer.DerivationStrategy;
using System.Net.Http;
using System.Net.Sockets;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.DependencyInjection;
using NBitcoin.WalletPolicies;
using Newtonsoft.Json.Linq;
using NBitcoin.Scripting;
namespace NBXplorer.Tests
{
@ -26,14 +28,14 @@ namespace NBXplorer.Tests
{
private readonly string _Directory;
public static ServerTester Create([CallerMemberNameAttribute] string caller = null)
public static ServerTester Create(TesterLogs logs, [CallerMemberNameAttribute] string caller = null)
{
return new ServerTester(caller, true);
return new ServerTester(logs, caller, true);
}
public static ServerTester CreateNoAutoStart([CallerMemberNameAttribute] string caller = null)
public static ServerTester CreateNoAutoStart(TesterLogs logs, [CallerMemberNameAttribute] string caller = null)
{
return new ServerTester(caller, false);
return new ServerTester(logs, caller, false);
}
public void Dispose()
@ -57,11 +59,13 @@ namespace NBXplorer.Tests
get; set;
}
public TesterLogs Logs { get; }
public string Caller { get; }
public ServerTester(string directory, bool autoStart = true)
public ServerTester(TesterLogs logs, string directory, bool autoStart = true)
{
_Name = directory;
SetEnvironment();
Logs = logs;
Caller = directory;
var rootTestData = "TestData";
directory = Path.Combine(rootTestData, directory);
@ -76,7 +80,7 @@ namespace NBXplorer.Tests
{
get;
set;
} = NBitcoin.Tests.RPCWalletType.Legacy;
}
public void Start()
{
@ -109,16 +113,24 @@ namespace NBXplorer.Tests
throw;
}
}
static int FreeTcpPort()
{
TcpListener l = new TcpListener(IPAddress.Loopback, 0);
l.Start();
int port = ((IPEndPoint)l.LocalEndpoint).Port;
l.Stop();
return port;
}
public int TrimEvents { get; set; } = -1;
public bool UseRabbitMQ { get; set; } = false;
public List<(string key, string value)> AdditionalConfiguration { get; set; } = new List<(string key, string value)>();
public List<string> AdditionalFlags = new List<string>();
internal string PostgresConnectionString;
private void StartNBXplorer()
{
var additionalFlags = new List<string>();
var port = CustomServer.FreeTcpPort();
var port = FreeTcpPort();
List<(string key, string value)> keyValues = new List<(string key, string value)>();
keyValues.Add(("conf", Path.Combine(datadir, "settings.config")));
PostgresConnectionString ??= GetTestPostgres(null, _Name);
@ -132,32 +144,17 @@ namespace NBXplorer.Tests
keyValues.Add(("verbose", "1"));
keyValues.Add(($"{CryptoCode.ToLowerInvariant()}rpcauth", Explorer.GetRPCAuth()));
keyValues.Add(($"{CryptoCode.ToLowerInvariant()}rpcurl", Explorer.CreateRPCClient().Address.AbsoluteUri));
keyValues.Add(($"{CryptoCode.ToLowerInvariant()}rpcdefaultwallet", "default"));
keyValues.Add(("exposerpc", "1"));
keyValues.Add(("rpcnotest", "1"));
keyValues.Add(("trimevents", TrimEvents.ToString()));
keyValues.Add(("mingapsize", "3"));
keyValues.Add(("maxgapsize", "8"));
keyValues.Add(($"{CryptoCode.ToLowerInvariant()}nodeendpoint", $"{Explorer.Endpoint.Address}:{Explorer.Endpoint.Port}"));
keyValues.Add(("asbcnstr", AzureServiceBusTestConfig.ConnectionString));
keyValues.Add(("asbblockq", AzureServiceBusTestConfig.NewBlockQueue));
keyValues.Add(("asbtranq", AzureServiceBusTestConfig.NewTransactionQueue));
keyValues.Add(("asbblockt", AzureServiceBusTestConfig.NewBlockTopic));
keyValues.Add(("asbtrant", AzureServiceBusTestConfig.NewTransactionTopic));
if (UseRabbitMQ)
{
keyValues.Add(("rmqhost", RabbitMqTestConfig.RabbitMqHostName));
keyValues.Add(("rmqvirtual", RabbitMqTestConfig.RabbitMqVirtualHost));
keyValues.Add(("rmquser", RabbitMqTestConfig.RabbitMqUsername));
keyValues.Add(("rmqpass", RabbitMqTestConfig.RabbitMqPassword));
keyValues.Add(("rmqtranex", RabbitMqTestConfig.RabbitMqTransactionExchange));
keyValues.Add(("rmqblockex", RabbitMqTestConfig.RabbitMqBlockExchange));
}
var args = keyValues.SelectMany(kv => new[] { $"--{kv.key}", kv.value })
.Concat(AdditionalFlags)
.Concat(additionalFlags).ToArray();
Host = new WebHostBuilder()
.UseConfiguration(new DefaultConfiguration().CreateConfiguration(args))
.UseKestrel()
Host = Microsoft.Extensions.Hosting.Host.CreateDefaultBuilder()
.ConfigureLogging(l =>
{
l.SetMinimumLevel(LogLevel.Information)
@ -165,9 +162,16 @@ namespace NBXplorer.Tests
.AddFilter("Microsoft", LogLevel.Error)
.AddFilter("Hangfire", LogLevel.Error)
.AddFilter("NBXplorer.Authentication.BasicAuthenticationHandler", LogLevel.Critical)
.ClearProviders()
.AddProvider(Logs.LogProvider);
})
.UseStartup<Startup>()
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder
.UseKestrel()
.UseConfiguration(new DefaultConfiguration().CreateConfiguration(args))
.UseStartup<Startup>();
})
.Build();
NBXplorer.Logging.Logs.Configure(Host.Services.GetRequiredService<ILoggerFactory>());
NBXplorerNetwork = ((NBXplorerNetworkProvider)Host.Services.GetService(typeof(NBXplorerNetworkProvider))).GetFromCryptoCode(CryptoCode);
@ -208,7 +212,8 @@ namespace NBXplorer.Tests
string datadir;
public void ResetExplorer(bool deleteAll = true)
{
Host.Dispose();
_ = Host.StopAsync();
Host.WaitForShutdown();
if (deleteAll)
{
PostgresConnectionString = null;
@ -231,7 +236,7 @@ namespace NBXplorer.Tests
{
get
{
var address = Host.ServerFeatures.Get<IServerAddressesFeature>().Addresses.FirstOrDefault();
var address = Host.GetServerFeatures<IServerAddressesFeature>().Addresses.First();
return new Uri(address);
}
}
@ -262,7 +267,7 @@ namespace NBXplorer.Tests
}
public IWebHost Host
public IHost Host
{
get; set;
}
@ -374,9 +379,11 @@ namespace NBXplorer.Tests
var k = PrivateKeyOf(key, path);
try
{
#pragma warning disable CS0618 // Type or member is obsolete
await RPC.ImportPrivKeyAsync(k).ConfigureAwait(false);
#pragma warning restore CS0618 // Type or member is obsolete
}
catch (RPCException ex) when (ex.RPCCode == RPCErrorCode.RPC_WALLET_ERROR)
catch (RPCException ex) when (ex.RPCCode is RPCErrorCode.RPC_WALLET_ERROR or RPCErrorCode.RPC_METHOD_NOT_FOUND)
{
string[] desc;
if (this.RPC.Capabilities.SupportSegwit)
@ -395,8 +402,8 @@ namespace NBXplorer.Tests
new JArray(
new JObject()
{
["desc"] = OutputDescriptor.AddChecksum(d),
["timestamp"] = this.RPC.Network.Consensus.CoinbaseMaturity
["desc"] = Miniscript.AddChecksum(d),
["timestamp"] = "now"
})
}
}).ConfigureAwait(false);

View File

@ -1,74 +0,0 @@
namespace NBXplorer.Tests
{
public static class RabbitMqTestConfig
{
//Put your rabbit mq settings here
public static string RabbitMqHostName => "localhost";
public static string RabbitMqVirtualHost => "/";
public static string RabbitMqUsername => "guest";
public static string RabbitMqPassword => "guest";
public static string RabbitMqBlockExchange => "NewBlock";
public static string RabbitMqTransactionExchange => "NewTransaction";
}
public static class AzureServiceBusTestConfig
{
public static string ConnectionString
{
get
{
//Put your service bus connection string here - requires READ / WRITE permissions
return "";
}
}
public static string NewBlockQueue
{
get
{
return "newblock";
}
}
public static string NewBlockTopic
{
get
{
return "newbitcoinblock";
}
}
public static string NewBlockSubscription
{
get
{
return "NewBlock";
}
}
public static string NewTransactionQueue
{
get
{
return "newtransaction";
}
}
public static string NewTransactionTopic
{
get
{
return "newbitcointransaction";
}
}
public static string NewTransactionSubscription
{
get
{
return "NewTransaction";
}
}
}
}

View File

@ -0,0 +1,9 @@
using Xunit.Abstractions;
namespace NBXplorer.Tests;
public class TesterLogs(ITestOutputHelper helper)
{
public XUnitLog Tester { get; } = new XUnitLog(helper) { Name = "Tests" };
public XUnitLoggerProvider LogProvider { get; } = new(helper);
}

View File

@ -14,7 +14,7 @@ namespace NBXplorer.Tests
[Fact]
public async Task CanCRUDGroups()
{
using var tester = ServerTester.Create();
using var tester = CreateTester();
var g1 = await tester.Client.CreateGroupAsync();
void AssertG1Empty()
{
@ -90,7 +90,7 @@ namespace NBXplorer.Tests
[Fact]
public async Task CanAliceAndBobShareWallet()
{
using var tester = ServerTester.Create();
using var tester = CreateTester();
var bobW = tester.Client.GenerateWallet(new GenerateWalletRequest() { ScriptPubKeyType = ScriptPubKeyType.Segwit });
var aliceW = tester.Client.GenerateWallet(new GenerateWalletRequest() { ScriptPubKeyType = ScriptPubKeyType.Segwit });

View File

@ -1,24 +1,17 @@
using Microsoft.Azure.ServiceBus;
using Microsoft.Azure.ServiceBus.Core;
using NBitcoin;
using NBitcoin.RPC;
using NBXplorer.DerivationStrategy;
using NBXplorer.Models;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using RabbitMQ.Client;
using System.Net.Http;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using NBitcoin.Altcoins.Elements;
using Xunit;
using Xunit.Abstractions;
using System.Net.Http;
using System.IO;
using Dapper;
using NBXplorer.Configuration;
using NBXplorer.Backend;
@ -28,19 +21,12 @@ using System.Globalization;
using System.Net;
using NBXplorer.HostedServices;
using static NBXplorer.Backend.DbConnectionHelper;
using NBitcoin.Altcoins;
using NBitcoin.WalletPolicies;
using System.Diagnostics.Metrics;
namespace NBXplorer.Tests
{
public partial class UnitTest1
public partial class UnitTest1(ITestOutputHelper helper) : UnitTestBase(helper)
{
public UnitTest1(ITestOutputHelper helper)
{
Logs.Tester = new XUnitLog(helper) { Name = "Tests" };
Logs.LogProvider = new XUnitLoggerProvider(helper);
}
NBXplorerNetworkProvider _Provider = new NBXplorerNetworkProvider(ChainName.Regtest);
private NBXplorerNetwork GetNetwork(INetworkSet network)
{
@ -281,7 +267,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanEasilySpendUTXOs()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var userExtKey = new ExtKey();
var userDerivationScheme = tester.Client.Network.DerivationStrategyFactory.CreateDirectDerivationStrategy(userExtKey.Neuter(), new DerivationStrategyOptions()
@ -349,7 +335,7 @@ namespace NBXplorer.Tests
[InlineData("tr(@0/**)", 0, 1)]
public async Task CanCreatePSBTInMiniscript(string template, int depositIndex, int changeIndex)
{
using var tester = ServerTester.Create();
using var tester = CreateTester();
var root = new ExtKey();
var path = new KeyPath("86'/1'/0'");
@ -420,7 +406,7 @@ namespace NBXplorer.Tests
public async Task CanCreatePSBT(PSBTVersion v, bool useMiniscript)
{
var version = v == PSBTVersion.PSBTv0 ? 0 : 2;
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
// We need to check if we can get utxo information of segwit utxos
var segwit = await tester.RPC.GetNewAddressAsync(new GetNewAddressRequest()
@ -496,9 +482,8 @@ namespace NBXplorer.Tests
}
}
private static void CanCreatePSBTCore(ServerTester tester, int psbtVersion, ScriptPubKeyType type, bool useMiniscript = false)
private void CanCreatePSBTCore(ServerTester tester, int psbtVersion, ScriptPubKeyType type, bool useMiniscript = false)
{
var userExtKey = new ExtKey();
var userExtKey2 = new ExtKey();
@ -1224,7 +1209,7 @@ namespace NBXplorer.Tests
[InlineData(false)]
public async Task CanDoubleSpend(bool onConfirmedUTXO)
{
using var tester = ServerTester.Create();
using var tester = CreateTester();
var bobW = await tester.Client.GenerateWalletAsync(new GenerateWalletRequest() { ScriptPubKeyType = ScriptPubKeyType.Segwit });
var bob = bobW.DerivationScheme;
var bobAddr = await tester.Client.GetUnusedAsync(bob, DerivationFeature.Deposit, 0);
@ -1291,7 +1276,7 @@ namespace NBXplorer.Tests
// Mine a block without B.
// Double-Spend A with B'
// Check that B' is replacing B in events
using var tester = ServerTester.Create();
using var tester = CreateTester();
var bobW = tester.Client.GenerateWallet();
var bob = bobW.DerivationScheme;
var bobAddr = tester.Client.GetUnused(bob, DerivationFeature.Deposit, 0);
@ -1365,7 +1350,7 @@ namespace NBXplorer.Tests
// Else, B' just bump the fees.
// We should make sure that B' is still saved in the database, and B properly marked as replaced.
// If cancelB is true, then B' output shouldn't be related to Bob.
using var tester = ServerTester.Create();
using var tester = CreateTester();
var bobW = await tester.Client.GenerateWalletAsync();
var bob = bobW.DerivationScheme;
@ -1435,7 +1420,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task ShowRBFedTransaction()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var bob = tester.CreateDerivationStrategy();
var bobSource = new DerivationSchemeTrackedSource(bob);
@ -1568,7 +1553,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanGetUnusedAddresses()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var bob = tester.CreateDerivationStrategy();
var utxo = tester.Client.GetUTXOs(bob); //Track things do not wait
@ -1617,364 +1602,6 @@ namespace NBXplorer.Tests
CancellationToken Cancel => new CancellationTokenSource(5000).Token;
[Fact]
[Trait("Azure", "Azure")]
public async Task CanSendAzureServiceBusNewBlockEventMessage()
{
Assert.False(string.IsNullOrWhiteSpace(AzureServiceBusTestConfig.ConnectionString), "Please Set Azure Service Bus Connection string in TestConfig.cs AzureServiceBusTestConfig Class. ");
Assert.False(string.IsNullOrWhiteSpace(AzureServiceBusTestConfig.NewBlockQueue), "Please Set Azure Service Bus NewBlockQueue name in TestConfig.cs AzureServiceBusTestConfig Class. ");
Assert.False(string.IsNullOrWhiteSpace(AzureServiceBusTestConfig.NewBlockTopic), "Please Set Azure Service Bus NewBlockTopic name in TestConfig.cs AzureServiceBusTestConfig Class. ");
Assert.False(string.IsNullOrWhiteSpace(AzureServiceBusTestConfig.NewBlockSubscription), "Please Set Azure Service Bus NewBlock Subscription name in TestConfig.cs AzureServiceBusTestConfig Class. ");
using (var tester = ServerTester.Create())
{
tester.Client.WaitServerStarted();
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.CreateDerivationStrategy(key.Neuter(), true);
tester.Client.Track(pubkey);
IQueueClient blockClient = new QueueClient(AzureServiceBusTestConfig.ConnectionString, AzureServiceBusTestConfig.NewBlockQueue);
ISubscriptionClient subscriptionClient = new SubscriptionClient(AzureServiceBusTestConfig.ConnectionString, AzureServiceBusTestConfig.NewBlockTopic, AzureServiceBusTestConfig.NewBlockSubscription);
//Configure Service Bus Subscription callback
//We may have existing messages from other tests - push all message to a LIFO stack
var busMessages = new ConcurrentStack<Microsoft.Azure.ServiceBus.Message>();
var messageHandlerOptions = new MessageHandlerOptions((e) =>
{
throw e.Exception;
})
{
// Maximum number of Concurrent calls to the callback `ProcessMessagesAsync`, set to 1 for simplicity.
// Set it according to how many messages the application wants to process in parallel.
MaxConcurrentCalls = 1,
// Indicates whether MessagePump should automatically complete the messages after returning from User Callback.
// False below indicates the Complete will be handled by the User Callback as in `ProcessMessagesAsync` below.
AutoComplete = false
};
//Service Bus Topic Message Handler
subscriptionClient.RegisterMessageHandler(async (m, t) =>
{
busMessages.Push(m);
await subscriptionClient.CompleteAsync(m.SystemProperties.LockToken);
}, messageHandlerOptions);
//Test Service Bus Queue
//Retry 10 times
var retryPolicy = new RetryExponential(new TimeSpan(0, 0, 0, 0, 500), new TimeSpan(0, 0, 1), 10);
var messageReceiver = new MessageReceiver(AzureServiceBusTestConfig.ConnectionString, AzureServiceBusTestConfig.NewBlockQueue, ReceiveMode.ReceiveAndDelete, retryPolicy);
Microsoft.Azure.ServiceBus.Message msg = null;
//Clear any existing messages from queue
while (await messageReceiver.PeekAsync() != null)
{
// Batch the receive operation
var brokeredMessages = await messageReceiver.ReceiveAsync(300);
}
await messageReceiver.CloseAsync(); //Close queue , otherwise receiver will consume our test message
messageReceiver = new MessageReceiver(AzureServiceBusTestConfig.ConnectionString, AzureServiceBusTestConfig.NewBlockQueue, ReceiveMode.ReceiveAndDelete, retryPolicy);
//Create a new Block - AzureServiceBus broker will receive a message from EventAggregator and publish to queue
var expectedBlockId = tester.Explorer.CreateRPCClient().Generate(1)[0];
msg = await messageReceiver.ReceiveAsync();
JsonSerializerSettings settings = new JsonSerializerSettings();
new Serializer(tester.Client.Network).ConfigureSerializer(settings);
Assert.True(msg != null, $"No message received on Azure Service Bus Block Queue : {AzureServiceBusTestConfig.NewBlockQueue} after 10 read attempts.");
Assert.Equal(msg.ContentType, typeof(NewBlockEvent).ToString());
var blockEventQ = JsonConvert.DeserializeObject<NewBlockEvent>(Encoding.UTF8.GetString(msg.Body), settings);
Assert.IsType<Models.NewBlockEvent>(blockEventQ);
Assert.Equal(expectedBlockId.ToString().ToUpperInvariant(), msg.MessageId.ToUpperInvariant());
Assert.Equal(expectedBlockId, blockEventQ.Hash);
Assert.NotEqual(0, blockEventQ.Height);
await Task.Delay(1000);
Assert.True(busMessages.Count > 0, $"No message received on Azure Service Bus Block Topic : {AzureServiceBusTestConfig.NewBlockTopic}.");
Microsoft.Azure.ServiceBus.Message busMsg = null;
busMessages.TryPop(out busMsg);
var blockEventS = JsonConvert.DeserializeObject<Models.NewBlockEvent>(Encoding.UTF8.GetString(busMsg.Body), settings);
Assert.IsType<Models.NewBlockEvent>(blockEventS);
Assert.Equal(expectedBlockId.ToString().ToUpperInvariant(), busMsg.MessageId.ToUpperInvariant());
Assert.Equal(expectedBlockId, blockEventS.Hash);
Assert.NotEqual(0, blockEventS.Height);
}
}
[Fact]
[Trait("Azure", "Azure")]
public async Task CanSendAzureServiceBusNewTransactionEventMessage()
{
Assert.False(string.IsNullOrWhiteSpace(AzureServiceBusTestConfig.ConnectionString), "Please Set Azure Service Bus Connection string in TestConfig.cs AzureServiceBusTestConfig Class.");
Assert.False(string.IsNullOrWhiteSpace(AzureServiceBusTestConfig.NewTransactionQueue), "Please Set Azure Service Bus NewTransactionQueue name in TestConfig.cs AzureServiceBusTestConfig Class.");
Assert.False(string.IsNullOrWhiteSpace(AzureServiceBusTestConfig.NewTransactionSubscription), "Please Set Azure Service Bus NewTransactionSubscription name in TestConfig.cs AzureServiceBusTestConfig Class.");
using (var tester = ServerTester.Create())
{
tester.Client.WaitServerStarted();
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.CreateDerivationStrategy(key.Neuter(), true);
tester.Client.Track(pubkey);
IQueueClient tranClient = new QueueClient(AzureServiceBusTestConfig.ConnectionString, AzureServiceBusTestConfig.NewTransactionQueue);
ISubscriptionClient subscriptionClient = new SubscriptionClient(AzureServiceBusTestConfig.ConnectionString, AzureServiceBusTestConfig.NewTransactionTopic, AzureServiceBusTestConfig.NewTransactionSubscription);
//Configure Service Bus Subscription callback
//We may have existing messages from other tests - push all message to a LIFO stack
var busMessages = new ConcurrentStack<Microsoft.Azure.ServiceBus.Message>();
var messageHandlerOptions = new MessageHandlerOptions((e) =>
{
throw e.Exception;
})
{
// Maximum number of Concurrent calls to the callback `ProcessMessagesAsync`, set to 1 for simplicity.
// Set it according to how many messages the application wants to process in parallel.
MaxConcurrentCalls = 1,
// Indicates whether MessagePump should automatically complete the messages after returning from User Callback.
// False below indicates the Complete will be handled by the User Callback as in `ProcessMessagesAsync` below.
AutoComplete = false
};
//Service Bus Topic Message Handler
subscriptionClient.RegisterMessageHandler(async (m, t) =>
{
busMessages.Push(m);
await subscriptionClient.CompleteAsync(m.SystemProperties.LockToken);
}, messageHandlerOptions);
//Test Service Bus Queue
//Retry 10 times
var retryPolicy = new RetryExponential(new TimeSpan(0, 0, 0, 0, 500), new TimeSpan(0, 0, 1), 10);
//Setup Message Receiver and clear queue
var messageReceiver = new MessageReceiver(AzureServiceBusTestConfig.ConnectionString, AzureServiceBusTestConfig.NewTransactionQueue, ReceiveMode.ReceiveAndDelete, retryPolicy);
while (await messageReceiver.PeekAsync() != null)
{
// Batch the receive operation
var brokeredMessages = await messageReceiver.ReceiveAsync(300);
}
await messageReceiver.CloseAsync();
//New message receiver to listen to our test event
messageReceiver = new MessageReceiver(AzureServiceBusTestConfig.ConnectionString, AzureServiceBusTestConfig.NewTransactionQueue, ReceiveMode.ReceiveAndDelete, retryPolicy);
//Create a new UTXO for our tracked key
tester.SendToAddress(tester.AddressOf(pubkey, "0/1"), Money.Coins(1.0m));
//Check Queue Message
Microsoft.Azure.ServiceBus.Message msg = null;
msg = await messageReceiver.ReceiveAsync();
Assert.True(msg != null, $"No message received on Azure Service Bus Transaction Queue : {AzureServiceBusTestConfig.NewTransactionQueue} after 10 read attempts.");
var isCrptoCodeExist = msg.UserProperties.TryGetValue("CryptoCode", out object cryptoCode);
Assert.True(isCrptoCodeExist, "No crypto code information in user properties.");
Assert.Equal(tester.Client.Network.CryptoCode, (string)cryptoCode);
//Configure JSON custom serialization
NBXplorerNetwork networkForDeserializion = new NBXplorerNetworkProvider(ChainName.Regtest).GetFromCryptoCode((string)cryptoCode);
JsonSerializerSettings settings = new JsonSerializerSettings();
new Serializer(networkForDeserializion).ConfigureSerializer(settings);
var txEventQ = JsonConvert.DeserializeObject<NewTransactionEvent>(Encoding.UTF8.GetString(msg.Body), settings);
Assert.Equal(txEventQ.DerivationStrategy, pubkey);
await Task.Delay(1000);
Assert.True(busMessages.Count > 0, $"No message received on Azure Service Bus Transaction Topic : {AzureServiceBusTestConfig.NewTransactionTopic}.");
//Check Service Bus Topic Payload
Microsoft.Azure.ServiceBus.Message busMsg = null;
busMessages.TryPop(out busMsg);
var blockEventS = JsonConvert.DeserializeObject<NewTransactionEvent>(Encoding.UTF8.GetString(busMsg.Body), settings);
Assert.IsType<NewTransactionEvent>(blockEventS);
Assert.Equal(blockEventS.DerivationStrategy, pubkey);
}
}
[Fact]
[Trait("Broker", "RabbitMq")]
public async Task CanSendRabbitMqNewTransactionEventMessage()
{
using (var tester = ServerTester.CreateNoAutoStart())
{
tester.UseRabbitMQ = true;
tester.Start();
tester.Client.WaitServerStarted();
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.CreateDerivationStrategy(key.Neuter(), true);
tester.Client.Track(pubkey);
// RabbitMq connection
var factory = new ConnectionFactory()
{
HostName = RabbitMqTestConfig.RabbitMqHostName,
VirtualHost = RabbitMqTestConfig.RabbitMqVirtualHost,
UserName = RabbitMqTestConfig.RabbitMqUsername,
Password = RabbitMqTestConfig.RabbitMqPassword
};
IConnection connection = factory.CreateConnection();
var channel = connection.CreateModel();
channel.ExchangeDeclare(RabbitMqTestConfig.RabbitMqTransactionExchange, ExchangeType.Topic);
// Setup a queue for all transactions
var allTransactionsQueue = "allTransactions";
var allTransactionsRoutingKey = $"transactions.#";
channel.QueueDeclare(allTransactionsQueue, true, false, false);
channel.QueueBind(allTransactionsQueue, RabbitMqTestConfig.RabbitMqTransactionExchange, allTransactionsRoutingKey);
while (channel.BasicGet(allTransactionsQueue, true) != null) { } // Empty the queue
// Setup a queue for all [CryptoCode] transactions
var allBtcTransactionsQueue = "allBtcTransactions";
var allBtcTransactionsRoutingKey = $"transactions.{tester.Client.Network.CryptoCode}.#";
channel.QueueDeclare(allBtcTransactionsQueue, true, false, false);
channel.QueueBind(allBtcTransactionsQueue, RabbitMqTestConfig.RabbitMqTransactionExchange, allBtcTransactionsRoutingKey);
while (channel.BasicGet(allBtcTransactionsQueue, true) != null) { }
// Setup a queue for all unconfirmed transactions
var allUnConfirmedTransactionsQueue = "allUnConfirmedTransactions";
var allUnConfirmedTransactionsRoutingKey = $"transactions.*.unconfirmed";
channel.QueueDeclare(allUnConfirmedTransactionsQueue, true, false, false);
channel.QueueBind(allUnConfirmedTransactionsQueue, RabbitMqTestConfig.RabbitMqTransactionExchange, allUnConfirmedTransactionsRoutingKey);
while (channel.BasicGet(allUnConfirmedTransactionsQueue, true) != null) { }
//Create a new UTXO for our tracked key
tester.SendToAddress(tester.AddressOf(pubkey, "0/1"), Money.Coins(1.0m));
await Task.Delay(5000);
BasicGetResult result = channel.BasicGet(allTransactionsQueue, true);
Assert.True(result != null, $"No message received from RabbitMq Queue : {allTransactionsQueue}.");
result = channel.BasicGet(allBtcTransactionsQueue, true);
Assert.True(result != null, $"No message received from RabbitMq Queue : {allBtcTransactionsQueue}.");
result = channel.BasicGet(allUnConfirmedTransactionsQueue, true);
Assert.True(result != null, $"No message received from RabbitMq Queue : {allUnConfirmedTransactionsQueue}.");
var isCrptoCodeExist = result.BasicProperties.Headers.TryGetValue("CryptoCode", out object cryptoCodeValue);
Assert.True(isCrptoCodeExist, "No crypto code information in user properties.");
var cryptoCode = Encoding.UTF8.GetString((cryptoCodeValue as byte[]));
Assert.Equal(tester.Client.Network.CryptoCode, cryptoCode);
var contentType = result.BasicProperties.ContentType;
Assert.Equal(typeof(NewTransactionEvent).ToString(), contentType);
var message = Encoding.UTF8.GetString(result.Body);
//Configure JSON custom serialization
NBXplorerNetwork networkForDeserializion = new NBXplorerNetworkProvider(ChainName.Regtest).GetFromCryptoCode((string)cryptoCode);
JsonSerializerSettings settings = new JsonSerializerSettings();
new Serializer(networkForDeserializion).ConfigureSerializer(settings);
var txEventQ = JsonConvert.DeserializeObject<NewTransactionEvent>(message, settings);
Assert.Equal(txEventQ.DerivationStrategy, pubkey);
// Setup a queue for all confirmed transactions
var allConfirmedTransactionsQueue = "allConfirmedTransactions";
var allConfirmedTransactionsRoutingKey = $"transactions.*.confirmed";
channel.QueueDeclare(allConfirmedTransactionsQueue, true, false, false);
channel.QueueBind(allConfirmedTransactionsQueue, RabbitMqTestConfig.RabbitMqTransactionExchange, allConfirmedTransactionsRoutingKey);
while (channel.BasicGet(allConfirmedTransactionsQueue, true) != null) { }
tester.RPC.EnsureGenerate(1);
await Task.Delay(5000);
result = channel.BasicGet(allConfirmedTransactionsQueue, true);
Assert.True(result != null, $"No message received from RabbitMq Queue : {allConfirmedTransactionsQueue}.");
}
}
[Fact]
[Trait("Broker", "RabbitMq")]
public async Task CanSendRabbitMqNewBlockEventMessage()
{
using (var tester = ServerTester.CreateNoAutoStart())
{
tester.UseRabbitMQ = true;
tester.Start();
tester.Client.WaitServerStarted();
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.CreateDerivationStrategy(key.Neuter(), true);
tester.Client.Track(pubkey);
// RabbitMq connection
var factory = new ConnectionFactory()
{
HostName = RabbitMqTestConfig.RabbitMqHostName,
VirtualHost = RabbitMqTestConfig.RabbitMqVirtualHost,
UserName = RabbitMqTestConfig.RabbitMqUsername,
Password = RabbitMqTestConfig.RabbitMqPassword
};
IConnection connection = factory.CreateConnection();
var channel = connection.CreateModel();
channel.ExchangeDeclare(RabbitMqTestConfig.RabbitMqBlockExchange, ExchangeType.Topic);
// Setup a queue for all blocks
var allBlocksQueue = "allBlocks";
var allBlocksRoutingKey = $"blocks.#";
channel.QueueDeclare(allBlocksQueue, true, false, false);
channel.QueueBind(allBlocksQueue, RabbitMqTestConfig.RabbitMqBlockExchange, allBlocksRoutingKey);
while (channel.BasicGet(allBlocksQueue, true) != null) { } // Empty the queue
// Setup a queue for all [CryptoCode] blocks
var allBtcBlocksQueue = "allBtcblocks";
var allBtcBlocksRoutingKey = $"blocks.{tester.Client.Network.CryptoCode}";
channel.QueueDeclare(allBtcBlocksQueue, true, false, false);
channel.QueueBind(allBtcBlocksQueue, RabbitMqTestConfig.RabbitMqBlockExchange, allBtcBlocksRoutingKey);
while (channel.BasicGet(allBtcBlocksQueue, true) != null) { }
var expectedBlockId = tester.Explorer.CreateRPCClient().Generate(1)[0];
await Task.Delay(5000);
BasicGetResult result = channel.BasicGet(allBlocksQueue, true);
Assert.True(result != null, $"No message received from RabbitMq Queue : {allBlocksQueue}.");
result = channel.BasicGet(allBtcBlocksQueue, true);
Assert.True(result != null, $"No message received from RabbitMq Queue : {allBtcBlocksQueue}.");
var isCrptoCodeExist = result.BasicProperties.Headers.TryGetValue("CryptoCode", out object cryptoCodeValue);
Assert.True(isCrptoCodeExist, "No crypto code information in user properties.");
var cryptoCode = Encoding.UTF8.GetString((cryptoCodeValue as byte[]));
Assert.Equal(tester.Client.Network.CryptoCode, cryptoCode);
var contentType = result.BasicProperties.ContentType;
Assert.Equal(typeof(NewBlockEvent).ToString(), contentType);
var message = Encoding.UTF8.GetString(result.Body);
//Configure JSON custom serialization
NBXplorerNetwork networkForDeserializion = new NBXplorerNetworkProvider(ChainName.Regtest).GetFromCryptoCode((string)cryptoCode);
JsonSerializerSettings settings = new JsonSerializerSettings();
new Serializer(networkForDeserializion).ConfigureSerializer(settings);
var blockEventQ = JsonConvert.DeserializeObject<NewBlockEvent>(message, settings);
Assert.IsType<Models.NewBlockEvent>(blockEventQ);
Assert.Equal(expectedBlockId.ToString().ToUpperInvariant(), result.BasicProperties.MessageId.ToUpperInvariant());
Assert.Equal(expectedBlockId, blockEventQ.Hash);
Assert.NotEqual(0, blockEventQ.Height);
}
}
class TestMetadata
{
public string Message { get; set; }
@ -1982,7 +1609,7 @@ namespace NBXplorer.Tests
[Fact]
public void CanTrimEvents()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
tester.Client.WaitServerStarted();
var ids = tester.Explorer.Generate(100);
@ -2007,7 +1634,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanGetAndSetMetadata()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
tester.Client.WaitServerStarted();
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
@ -2040,7 +1667,7 @@ namespace NBXplorer.Tests
// In this test we have fundingTxId with 2 output and spending1
// We make sure that only once the 2 outputs of fundingTxId have been consumed
// fundingTxId get pruned
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
tester.Client.WaitServerStarted();
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
@ -2149,7 +1776,7 @@ namespace NBXplorer.Tests
// In this test we have fundingTxId with 2 output and spending1
// We make sure that if only 1 outputs of fundingTxId have been consumed
// spending1 does not get pruned, even if its output got consumed
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
tester.Client.WaitServerStarted();
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
@ -2218,7 +1845,7 @@ namespace NBXplorer.Tests
[InlineData(true)]
public async Task CanUseWebSockets(bool legacyAPI)
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
tester.Client.WaitServerStarted();
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
@ -2267,7 +1894,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanUseLongPollingNotifications()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
tester.Client.WaitServerStarted();
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
@ -2310,7 +1937,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanUseWebSockets2()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
tester.Client.WaitServerStarted();
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
@ -2425,7 +2052,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task DoNotLoseTimestampForLongConfirmations()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var bob = new BitcoinExtKey(new ExtKey(), tester.Network);
var bobPubKey = tester.CreateDerivationStrategy(bob.Neuter());
@ -2450,7 +2077,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanTrack4()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var bob = new BitcoinExtKey(new ExtKey(), tester.Network);
var alice = new BitcoinExtKey(new ExtKey(), tester.Network);
@ -2511,7 +2138,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanTrack3()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.CreateDerivationStrategy(key.Neuter());
@ -2573,7 +2200,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanTrackSeveralTransactions()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.CreateDerivationStrategy(key.Neuter());
@ -2597,6 +2224,22 @@ namespace NBXplorer.Tests
await tester.Client.GetUnusedAsync(pubkey, DerivationFeature.Deposit, reserve: true);
}
uint256 lastTx = null;
Logs.Tester.LogInformation($"Importing 20 descriptions...");
await tester.RPC.SendCommandAsync(new RPCRequest()
{
Method = "importdescriptors",
ThrowIfRPCError = true,
Params = new JArray[]{new JArray(Enumerable.Range(0, 20)
.Select(i => tester.PrivateKeyOf(key, $"0/{i + 1}"))
.Select(k =>
new JObject()
{
["desc"] = Miniscript.AddChecksum($"wpkh({k})"),
["timestamp"] = "now"
}))}
}).ConfigureAwait(false);
for (i = 0; i < 20; i++)
{
LockTestCoins(tester.RPC, addresses);
@ -2604,7 +2247,6 @@ namespace NBXplorer.Tests
coins = coins - Money.Coins(0.001m);
var path = $"0/{i + 1}";
var destination = tester.AddressOf(key, path);
await tester.ImportPrivKeyAsync(key, path);
var txId = await tester.SendToAddressAsync(destination, coins);
Logs.Tester.LogInformation($"Sent to {path} in {txId}");
addresses.Add(destination.ScriptPubKey);
@ -2626,7 +2268,7 @@ namespace NBXplorer.Tests
[InlineData(true)]
public async Task CanUseWebSocketsOnAddress(bool legacyAPI)
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
tester.Client.WaitServerStarted();
var key = new Key();
@ -2678,7 +2320,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanUseWebSocketsOnAddress2()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
tester.Client.WaitServerStarted();
var key = new Key();
@ -2713,7 +2355,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanTrackAddress()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var extkey = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.NBXplorerNetwork.DerivationStrategyFactory.Parse($"{extkey.Neuter()}-[legacy]");
@ -2804,7 +2446,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanTrack2()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.CreateDerivationStrategy(key.Neuter());
@ -2846,7 +2488,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanReserveAddress()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
//WaitServerStarted not needed, just a sanity check
var bob = tester.CreateDerivationStrategy();
@ -2995,7 +2637,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanGetStatus()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
tester.Client.WaitServerStarted(Timeout);
var status = await tester.Client.GetStatusAsync();
@ -3022,7 +2664,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanGetTransactionsOfDerivation()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.CreateDerivationStrategy(key.Neuter());
@ -3082,13 +2724,21 @@ namespace NBXplorer.Tests
Assert.NotNull(metadata.Fees);
Assert.NotNull(metadata.FeeRate);
Assert.NotNull(metadata.VirtualSize);
Assert.NotNull(metadata.Chunk);
Assert.NotNull(metadata.Chunk.Fees);
Assert.NotEqual(0, metadata.Chunk.Fees.Satoshi);
Assert.NotEqual(0, metadata.Chunk.Weight);
Assert.NotNull(metadata.Chunk.FeeRate);
// Those are equal because the chunk is composed of a single transaction
Assert.Equal(metadata.Chunk.FeeRate, metadata.FeeRate);
Assert.Equal((metadata.Chunk.Weight + 3) / 4, metadata.VirtualSize);
}
}
[FactWithTimeout]
public async Task CanTrack5()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.CreateDerivationStrategy(key.Neuter());
@ -3148,7 +2798,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanRescan()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
tester.Client.WaitServerStarted(Timeout);
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
@ -3204,7 +2854,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanTrackManyAddressesAtOnce()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.CreateDerivationStrategy(key.Neuter());
@ -3232,7 +2882,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanTrack()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.CreateDerivationStrategy(key.Neuter());
@ -3410,7 +3060,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanCacheTransactions()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.CreateDerivationStrategy(key.Neuter());
@ -3430,7 +3080,7 @@ namespace NBXplorer.Tests
[Fact(Timeout = 60 * 1000)]
public async Task CanUseLongPollingOnEvents()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
//WaitServerStarted not needed, just a sanity check
tester.Client.WaitServerStarted(Timeout);
@ -3612,7 +3262,7 @@ namespace NBXplorer.Tests
}
/// <summary>
/// To understand this test, read https://github.com/dgarage/NBXplorer/blob/master/docs/Design.md
/// To understand this test, read https://github.com/btcpayserver/NBXplorer/blob/master/docs/Design.md
/// This create a specific graph of transaction and make sure that it computes the UTXO set as expected.
/// </summary>
[Fact]
@ -3768,7 +3418,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanBroadcast()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
tester.Client.WaitServerStarted();
var tx = tester.Network.Consensus.ConsensusFactory.CreateTransaction();
@ -3778,17 +3428,52 @@ namespace NBXplorer.Tests
var result = await tester.Client.BroadcastAsync(signed);
Assert.True(result.Success);
signed.Inputs[0].PrevOut.N = 999;
result = await tester.Client.BroadcastAsync(signed);
Assert.False(result.Success);
var ex = await Assert.ThrowsAsync<NBXplorerException>(() => tester.Client.GetFeeRateAsync(5));
Assert.Equal("fee-estimation-unavailable", ex.Error.Code);
for (int i = 0; i < 6; i++)
{
result = await Broadcast(i, tester, signed);
Assert.False(result.Success);
var ex = await Assert.ThrowsAsync<NBXplorerException>(() => tester.Client.GetFeeRateAsync(5));
Assert.Equal("fee-estimation-unavailable", ex.Error.Code);
}
}
}
private static async Task<BroadcastResult> Broadcast(int format, ServerTester tester, Transaction signed)
{
BroadcastResult result;
if (format == 0)
result = await tester.Client.BroadcastAsync(signed);
else
{
var val = format switch
{
1 => "{\"hex\":\"TX\"}",
2 => "{\"psbt\":\"PSBT\"}",
3 => "{\"hex\":\"PSBT\"}",
4 => "\"PSBT\"",
5 => "\"TX\"",
_ => throw new Exception()
};
var noSig = signed.Clone();
noSig.RemoveSignatures();
var psbt = PSBT.FromTransaction(noSig, Network.RegTest);
for (int j = 0; j < psbt.Inputs.Count; j++)
{
psbt.Inputs[j].FinalScriptSig = signed.Inputs[j].ScriptSig;
psbt.Inputs[j].FinalScriptWitness = signed.Inputs[j].WitScript;
}
val = val.Replace("TX", signed.ToHex());
val = val.Replace("PSBT", psbt.ToBase64());
result = await tester.Client.SendAsync<BroadcastResult>(HttpMethod.Post, JToken.Parse(val), $"v1/cryptos/BTC/transactions", default);
}
return result;
}
[FactWithTimeout]
public async Task CanGetKeyInformations()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.CreateDerivationStrategy(key.Neuter());
@ -3892,7 +3577,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanRescanFullyIndexedTransaction()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.CreateDerivationStrategy(key.Neuter());
@ -3928,7 +3613,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanScanUTXOSet()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
var key = new BitcoinExtKey(new ExtKey(), tester.Network);
var pubkey = tester.CreateDerivationStrategy(key.Neuter());
@ -4163,7 +3848,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task ElementsTests()
{
using (var tester = ServerTester.CreateNoAutoStart())
using (var tester = CreateTesterNoAutoStart())
{
if (tester.Network.NetworkSet != NBitcoin.Altcoins.Liquid.Instance)
{
@ -4298,7 +3983,7 @@ namespace NBXplorer.Tests
[Fact]
public async Task CanGenerateWithRPCTracking()
{
using (var tester = ServerTester.CreateNoAutoStart())
using (var tester = CreateTesterNoAutoStart())
{
tester.RPCWalletType = RPCWalletType.Descriptors;
tester.Start();
@ -4348,10 +4033,9 @@ namespace NBXplorer.Tests
[TheoryWithTimeout]
[InlineData(RPCWalletType.Descriptors)]
[InlineData(RPCWalletType.Legacy)]
public async Task CanGenerateWallet(RPCWalletType walletType)
{
using (var tester = ServerTester.CreateNoAutoStart())
using (var tester = CreateTesterNoAutoStart())
{
tester.CreateWallet = true;
tester.RPCWalletType = walletType;
@ -4503,7 +4187,7 @@ namespace NBXplorer.Tests
[FactWithTimeout]
public async Task CanUseRPCProxy()
{
using (var tester = ServerTester.Create())
using (var tester = CreateTester())
{
Assert.NotNull(await tester.Client.RPCClient.GetBlockchainInfoAsync());
@ -4541,7 +4225,7 @@ namespace NBXplorer.Tests
[Fact]
public async Task DoNotHangDuringReorg()
{
using var tester = ServerTester.Create();
using var tester = CreateTester();
var wallet = await tester.Client.GenerateWalletAsync(new GenerateWalletRequest());
var addr = await tester.Client.GetUnusedAsync(wallet.DerivationScheme, DerivationFeature.Deposit);
var txId = tester.SendToAddress(addr.Address, Money.Coins(1.0m));
@ -4571,7 +4255,7 @@ namespace NBXplorer.Tests
[Fact]
public async Task IsTrackedTests()
{
using var tester = ServerTester.Create();
using var tester = CreateTester();
var xpub = new DerivationSchemeTrackedSource(new DirectDerivationStrategy(
new BitcoinExtPubKey(new Mnemonic(Wordlist.English).DeriveExtKey().Neuter(), tester.Network), true));
Assert.False(await tester.Client.IsTrackedAsync(xpub, Cancel));
@ -4592,7 +4276,7 @@ namespace NBXplorer.Tests
[Fact]
public async Task CanImportUTXOs()
{
using var tester = ServerTester.Create();
using var tester = CreateTester();
var wallet1 = await tester.Client.CreateGroupAsync();
var wallet1TS = new GroupTrackedSource(wallet1.GroupId);

View File

@ -0,0 +1,12 @@
using System.Runtime.CompilerServices;
using Xunit.Abstractions;
namespace NBXplorer.Tests;
public class UnitTestBase(ITestOutputHelper helper)
{
public TesterLogs Logs { get; set; } = new TesterLogs(helper);
public ServerTester CreateTester([CallerMemberName] string caller = null) => ServerTester.Create(Logs, caller);
public ServerTester CreateTesterNoAutoStart([CallerMemberName] string caller = null) => ServerTester.CreateNoAutoStart(Logs, caller);
}

View File

@ -17,9 +17,9 @@ services:
- postgres
- pgadmin
postgres:
image: postgres:13
image: postgres:18.1
container_name: nbxplorertests_postgres_1
command: [ "-c", "random_page_cost=1.0", "-c", "shared_preload_libraries=pg_stat_statements" ]
command: [ "-c", "random_page_cost=1.0", "-c", "shared_preload_libraries=pg_stat_statements", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
environment:
POSTGRES_HOST_AUTH_METHOD: trust
ports:

View File

@ -1,4 +1,4 @@
#!/bin/sh
set -e
dotnet test --filter "Azure!=Azure&Broker!=RabbitMq&Benchmark!=Benchmark&Maintenance!=Maintenance" --no-build -v n --logger "console;verbosity=normal" < /dev/null
dotnet test --filter "Benchmark!=Benchmark&Maintenance!=Maintenance" --no-build -v n --logger "console;verbosity=normal" < /dev/null

View File

@ -33,7 +33,6 @@ namespace NBXplorer.Backend
action?.Invoke(connStrBuilder);
var builder = new NpgsqlDataSourceBuilder(connStrBuilder.ConnectionString);
DbConnectionHelper.Register(builder);
builder.Build();
return builder;
}

View File

@ -291,7 +291,7 @@ namespace NBXplorer.Backend
}
var node = await Node.ConnectAsync(network.NBitcoinNetwork, ChainConfiguration.NodeEndpoint, nodeParams);
Logger.LogInformation($"TCP Connection succeed, handshaking...");
node.VersionHandshake(handshakeTimeout.Token);
await node.VersionHandshakeAsync(handshakeTimeout.Token);
Logger.LogInformation($"Handshaked");
await node.SendMessageAsync(new SendHeadersPayload());
@ -547,7 +547,14 @@ namespace NBXplorer.Backend
private void Node_Disconnected(Node node)
{
Logger.LogInformation($"Node disconnected ({node.DisconnectReason.Reason})");
if (node.DisconnectReason.Exception != null)
{
Logger.LogError(node.DisconnectReason.Exception, $"Node disconnected with exception");
}
else
{
Logger.LogInformation($"Node disconnected ({node.DisconnectReason.Reason})");
}
_Connection?.Dispose();
node.MessageReceived -= Node_MessageReceived;
node.Disconnected -= Node_Disconnected;

View File

@ -14,11 +14,12 @@ using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using NBitcoin.Altcoins.Elements;
using NBXplorer.Client;
using NBitcoin.Scripting;
using System.Text.RegularExpressions;
using Npgsql;
using static NBXplorer.Backend.DbConnectionHelper;
using NBitcoin.DataEncoders;
using NBitcoin.WalletPolicies;
using Derivation = NBXplorer.DerivationStrategy.Derivation;
namespace NBXplorer.Backend
@ -267,7 +268,7 @@ namespace NBXplorer.Backend
}
descriptor = ReplaceBase58(descriptor, $"$0/{keyTemplate}");
// descriptor: tr([abcdefaa/49'/0'/0']xpriv/0/*)
await rpc.ImportDescriptors(OutputDescriptor.AddChecksum(descriptor), fromIndex, fromIndex + toGenerate - 1, default);
await rpc.ImportDescriptors(Miniscript.AddChecksum(descriptor), fromIndex, fromIndex + toGenerate - 1, default);
}
}
}

View File

@ -43,19 +43,6 @@ namespace NBXplorer.Configuration
app.Option($"--{crypto}exposerpc", $"Expose the node RPCs through the REST API (default: false)", CommandOptionType.SingleValue);
}
app.Option("--asbcnstr", "[For Azure Service Bus] Azure Service Bus Connection string. New Block and New Transaction messages will be pushed to queues when this values is set", CommandOptionType.SingleValue);
app.Option("--asbblockq", "[For Azure Service Bus] Name of Queue to push new block message to. Leave blank to turn off", CommandOptionType.SingleValue);
app.Option("--asbtranq", "[For Azure Service Bus] Name of Queue to push new transaction message to. Leave blank to turn off", CommandOptionType.SingleValue);
app.Option("--asbblockt", "[For Azure Service Bus] Name of Topic to push new block message to. Leave blank to turn off", CommandOptionType.SingleValue);
app.Option("--asbtrant", "[For Azure Service Bus] Name of Topic to push new transaction message to. Leave blank to turn off", CommandOptionType.SingleValue);
app.Option("--rmqhost", "[For RabbitMq] RabbitMq host name. Leave blank to turn off", CommandOptionType.SingleValue);
app.Option("--rmquser", "[For RabbitMq] RabbitMq username. Leave blank to turn off", CommandOptionType.SingleValue);
app.Option("--rmqpass", "[For RabbitMq] RabbitMq password. Leave blank to turn off", CommandOptionType.SingleValue);
app.Option("--rmqvirtual", "[For RabbitMq] RabbitMq virtual host.", CommandOptionType.SingleValue);
app.Option("--rmqtranex", "[For RabbitMq] Name of exchange to push transaction messages.", CommandOptionType.SingleValue);
app.Option("--rmqblockex", "[For RabbitMq] Name of exchange to push block messages.", CommandOptionType.SingleValue);
app.Option("--customkeypathtemplate", $"Define an additional derivation path tracked by NBXplorer (Format: m/1/392/*/29, default: empty)", CommandOptionType.SingleValue);
app.Option("--maxgapsize", $"The maximum gap address count on which the explorer will track derivation schemes (default: 30)", CommandOptionType.SingleValue);
app.Option("--mingapsize", $"The minimum gap address count on which the explorer will track derivation schemes (default: 20)", CommandOptionType.SingleValue);
@ -179,15 +166,6 @@ namespace NBXplorer.Configuration
builder.AppendLine("#port=" + settings.DefaultPort);
builder.AppendLine("#bind=127.0.0.1");
builder.AppendLine($"#{networkType.ToString().ToLowerInvariant()}=1");
builder.AppendLine();
builder.AppendLine();
builder.AppendLine("####Azure Service Bus####");
builder.AppendLine("## Azure Service Bus configuration - set connection string to use Service Bus. Set Queue and / or Topic names to publish message to queues / topics");
builder.AppendLine("#asbcnstr=Endpoint=sb://<yourdomain>.servicebus.windows.net/;SharedAccessKeyName=<your key name here>;SharedAccessKey=<your key here>");
builder.AppendLine("#asbblockq=<new block queue name>");
builder.AppendLine("#asbtranq=<new transaction queue name>");
builder.AppendLine("#asbblockt=<new block topic name>");
builder.AppendLine("#asbtrant=<new transaction topic name>");
return builder.ToString();
}

View File

@ -187,19 +187,6 @@ namespace NBXplorer.Configuration
CustomKeyPathTemplate = v;
}
AzureServiceBusConnectionString = config.GetOrDefault<string>("asbcnstr", "");
AzureServiceBusBlockQueue = config.GetOrDefault<string>("asbblockq", "");
AzureServiceBusTransactionQueue = config.GetOrDefault<string>("asbtranq", "");
AzureServiceBusBlockTopic = config.GetOrDefault<string>("asbblockt", "");
AzureServiceBusTransactionTopic = config.GetOrDefault<string>("asbtrant", "");
RabbitMqHostName = config.GetOrDefault<string>("rmqhost", "");
RabbitMqVirtualHost = config.GetOrDefault<string>("rmqvirtual", "");
RabbitMqUsername = config.GetOrDefault<string>("rmquser", "");
RabbitMqPassword = config.GetOrDefault<string>("rmqpass", "");
RabbitMqTransactionExchange = config.GetOrDefault<string>("rmqtranex", "");
RabbitMqBlockExchange = config.GetOrDefault<string>("rmqblockex", "");
var obsolete = string.Join(", ",
new[] { "dbtrie", "automigrate", "nomigrateevts", "nomigraterawtxs", "cachechain", "deleteaftermigration", "dbcache" }
.Where(o => !string.IsNullOrEmpty(config[o])));
@ -207,7 +194,7 @@ namespace NBXplorer.Configuration
if (obsolete != string.Empty)
{
if (Directory.Exists(Path.Combine(DataDir, "db")))
throw new ConfigException($"Options '{obsolete}' are not supported anymore, if you need to migrate an old instance to the new postgres backend, please use NBXplorer v2.5.2 and follow https://github.com/dgarage/NBXplorer/blob/master/docs/Postgres-Migration.md.");
throw new ConfigException($"Options '{obsolete}' are not supported anymore, if you need to migrate an old instance to the new postgres backend, please use NBXplorer v2.5.2 and follow https://github.com/btcpayserver/NBXplorer/blob/master/docs/Postgres-Migration.md.");
else
Logs.Explorer.LogWarning($"Options '{obsolete}' is obsolete and ignored...");
}
@ -239,41 +226,6 @@ namespace NBXplorer.Configuration
set;
}
public int TrimEvents { get; set; }
public string AzureServiceBusConnectionString
{
get;
set;
}
public string AzureServiceBusBlockQueue
{
get;
set;
}
public string AzureServiceBusBlockTopic
{
get;
set;
}
public string AzureServiceBusTransactionQueue
{
get;
set;
}
public string AzureServiceBusTransactionTopic
{
get;
set;
}
public string RabbitMqHostName { get; set; }
public string RabbitMqVirtualHost { get; set; }
public string RabbitMqUsername { get; set; }
public string RabbitMqPassword { get; set; }
public string RabbitMqTransactionExchange { get; set; }
public string RabbitMqBlockExchange { get; set; }
public KeyPathTemplate CustomKeyPathTemplate { get; set; }
public EndPoint SocksEndpoint { get; set; }

View File

@ -9,10 +9,10 @@ using System.Globalization;
using System;
using System.Threading.Tasks;
using NBitcoin.RPC;
using NBitcoin.Scripting;
using System.Linq;
using NBXplorer.Logging;
using Microsoft.Extensions.Logging;
using NBitcoin.WalletPolicies;
namespace NBXplorer.Controllers
{
@ -351,7 +351,7 @@ namespace NBXplorer.Controllers
ScriptPubKeyType.TaprootBIP86 => $"tr({imported})",
_ => throw new NotSupportedException($"Bug of NBXplorer (ERR 3082), please notify the developers ({scriptPubKeyType})")
};
return OutputDescriptor.AddChecksum(descriptor);
return Miniscript.AddChecksum(descriptor);
}
}
}

View File

@ -161,7 +161,9 @@ namespace NBXplorer.Controllers
if (c?.TrackedSource is null)
return null;
var net = c.CryptoCode is null ? null : NetworkProvider.GetFromCryptoCode(c.CryptoCode);
if (c.TrackedSource.StartsWith("ADDRESS:") || c.TrackedSource.StartsWith("DERIVATIONSCHEME:") && c.CryptoCode is null)
if (c.CryptoCode is not null && net is null)
throw new NBXplorerException(new NBXplorerError(400, "invalid-group-child", "Invalid cryptoCode"));
if ((c.TrackedSource.StartsWith("ADDRESS:") || c.TrackedSource.StartsWith("DERIVATIONSCHEME:")) && net is null)
throw new NBXplorerException(new NBXplorerError(400, "invalid-group-child", "ADDRESS: and DERIVATIONSCHEME: tracked sources must also include a cryptoCode parameter"));
if (!TrackedSource.TryParse(c.TrackedSource, out var ts, net))
throw new NBXplorerException(new NBXplorerError(400, "invalid-group-child", "Invalid tracked source format"));

View File

@ -20,6 +20,7 @@ using NBXplorer.Configuration;
using System.Net.WebSockets;
using Newtonsoft.Json;
using System.Reflection;
using NBitcoin.DataEncoders;
using NBXplorer.Analytics;
using NBXplorer.Backend;
using static NBXplorer.Backend.DbConnectionHelper;
@ -288,9 +289,7 @@ namespace NBXplorer.Controllers
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(10.0));
blockchainInfo = await rpc2.GetBlockchainInfoAsyncEx(cts.Token);
}
catch (HttpRequestException ex) when (ex.InnerException is IOException) { } // Sometimes "The response ended prematurely."
catch (IOException) { } // Sometimes "The response ended prematurely."
catch (OperationCanceledException) // Timeout, can happen if core is really busy
catch
{
}
}
@ -876,11 +875,7 @@ namespace NBXplorer.Controllers
bool testMempoolAccept = false)
{
var network = trackedSourceContext.Network;
var tx = network.NBitcoinNetwork.Consensus.ConsensusFactory.CreateTransaction();
var buffer = new MemoryStream();
await Request.Body.CopyToAsync(buffer);
buffer.Position = 0;
tx.FromBytes(buffer.ToArrayEfficient());
var tx = await ParseTx(network);
if (testMempoolAccept && !trackedSourceContext.RpcClient.Capabilities.SupportTestMempoolAccept)
throw new NBXplorerException(new NBXplorerError(400, "not-supported", "This feature is not supported for this crypto currency"));
@ -946,6 +941,83 @@ namespace NBXplorer.Controllers
}
}
private async Task<Transaction> ParseTx(NBXplorerNetwork network)
{
Transaction ParsePSBT(string psbtStr)
{
PSBT psbt = null;
try
{
psbt = PSBT.Parse(psbtStr, network.NBitcoinNetwork);
}
catch (FormatException)
{
throw;
}
catch (Exception ex)
{
throw new FormatException(ex.Message, ex);
}
try
{
return psbt.ExtractTransaction();
}
catch (Exception ex)
{
throw new FormatException("Unable to finalize the PSBT: " + ex.Message, ex);
}
}
Transaction ParseTxOrPSBT(string txOrPSBT)
{
try
{
return Transaction.Parse(txOrPSBT, network.NBitcoinNetwork);
}
catch
{
return ParsePSBT(txOrPSBT);
}
}
var buffer = new MemoryStream();
await Request.Body.CopyToAsync(buffer);
buffer.Position = 0;
var body = buffer.ToArrayEfficient();
JToken tok = null;
try
{
tok = JToken.Parse(Encoding.UTF8.GetString(body));
}
catch
{
}
if (tok is JObject json)
{
if ((json["hex"] as JValue)?.Value is string hex)
return ParseTxOrPSBT(hex);
if ((json["psbt"] as JValue)?.Value is string psbtStr)
return ParsePSBT(psbtStr);
}
if (tok is JValue { Value: string hex2 })
return ParseTxOrPSBT(hex2);
try
{
var tx = network.NBitcoinNetwork.Consensus.ConsensusFactory.CreateTransaction();
tx.FromBytes(body);
return tx;
}
catch
{
throw new FormatException("Invalid transaction format");
}
}
private RPCErrorCode? GetRPCCodeFromReason(string rejectReason)
{
return rejectReason switch

View File

@ -1,4 +1,4 @@
-- For documentation see https://github.com/dgarage/NBXplorer/tree/master/docs/Postgres-Schema.md
-- For documentation see https://github.com/btcpayserver/NBXplorer/tree/master/docs/Postgres-Schema.md
-- This file contains additional comments about column meaning.
-- The main tables are blks, blks_txs, txs, ins, outs, ins_outs, descriptors, desriptors_scripts, scripts, wallets, wallets_descriptors, wallets_scripts.

View File

@ -19,14 +19,10 @@ using Microsoft.Extensions.Hosting;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Authentication;
using NBXplorer.Authentication;
using NBXplorer.MessageBrokers;
using NBXplorer.HostedServices;
using NBXplorer.Controllers;
using NBXplorer.Backend;
using NBitcoin.Altcoins.Elements;
using Npgsql;
using NBitcoin.Altcoins;
using System.Threading;
using Newtonsoft.Json.Linq;
using static NBXplorer.Backend.Repository;
@ -217,7 +213,6 @@ namespace NBXplorer
services.AddHostedService<RPCReadyFileHostedService>();
services.AddSingleton<IHostedService, ScanUTXOSetService>();
services.TryAddSingleton<ScanUTXOSetServiceAccessor>();
services.AddSingleton<IHostedService, BrokerHostedService>();
services.AddSingleton<Analytics.FingerprintHostedService>();
services.AddSingleton<IHostedService, Analytics.FingerprintHostedService>(o => o.GetRequiredService<Analytics.FingerprintHostedService>());

View File

@ -22,6 +22,7 @@ namespace NBXplorer.HostedServices
Channel<ScheduledTask> jobs = Channel.CreateBounded<ScheduledTask>(100);
CancellationTokenSource cts;
public Task StartAsync(CancellationToken cancellationToken)
{
cts = new CancellationTokenSource();
@ -31,33 +32,29 @@ namespace NBXplorer.HostedServices
loop = Task.WhenAll(Enumerable.Range(0, 3).Select(_ => Loop(cts.Token)).ToArray());
return Task.CompletedTask;
}
Task loop;
private async Task Loop(CancellationToken token)
{
try
{
await foreach (var job in jobs.Reader.ReadAllAsync(token))
{
if (job.NextScheduled <= DateTimeOffset.UtcNow)
var t = (IPeriodicTask)ServiceProvider.GetService(job.PeriodicTaskType);
try
{
var t = (IPeriodicTask)ServiceProvider.GetService(job.PeriodicTaskType);
try
{
await t.Do(token);
}
catch when (token.IsCancellationRequested)
{
throw;
}
catch (Exception ex)
{
Logger.LogError(ex, $"Unhandled error in job {job.PeriodicTaskType.Name}");
}
finally
{
job.NextScheduled = DateTimeOffset.UtcNow + job.Every;
}
await t.Do(token);
}
catch when (token.IsCancellationRequested)
{
throw;
}
catch (Exception ex)
{
Logger.LogError(ex, $"Unhandled error in job {job.PeriodicTaskType.Name}");
}
_ = Wait(job, token);
}
}
@ -68,16 +65,17 @@ namespace NBXplorer.HostedServices
private async Task Wait(ScheduledTask job, CancellationToken token)
{
var timeToWait = job.NextScheduled - DateTimeOffset.UtcNow;
try
{
await Task.Delay(timeToWait, token);
await Task.Delay(job.Every, token);
while (await jobs.Writer.WaitToWriteAsync(token))
{
if (jobs.Writer.TryWrite(job))
break;
}
}
catch { }
while (await jobs.Writer.WaitToWriteAsync())
catch when (token.IsCancellationRequested)
{
if (jobs.Writer.TryWrite(job))
break;
}
}
@ -89,4 +87,4 @@ namespace NBXplorer.HostedServices
await loop;
}
}
}
}

View File

@ -11,6 +11,5 @@ namespace NBXplorer.HostedServices
}
public Type PeriodicTaskType { get; set; }
public TimeSpan Every { get; set; } = TimeSpan.FromMinutes(5.0);
public DateTimeOffset NextScheduled { get; set; } = DateTimeOffset.UtcNow;
}
}

View File

@ -1,76 +0,0 @@
using Microsoft.Azure.ServiceBus;
using Microsoft.Azure.ServiceBus.Core;
using NBXplorer.Models;
using System;
using System.Text;
using System.Threading.Tasks;
using System.Security.Cryptography;
namespace NBXplorer.MessageBrokers
{
public class AzureBroker : IBrokerClient
{
const int MaxMessageIdLength = 128;
public AzureBroker(ISenderClient client, NBXplorerNetworkProvider networks)
{
Client = client;
Networks = networks;
}
public ISenderClient Client
{
get;
}
public NBXplorerNetworkProvider Networks { get; }
static Encoding UTF8 = new UTF8Encoding(false);
public async Task Send(NewTransactionEvent transactionEvent)
{
string jsonMsg = transactionEvent.ToJson(Networks.GetFromCryptoCode(transactionEvent.CryptoCode).JsonSerializerSettings);
var bytes = UTF8.GetBytes(jsonMsg);
var message = new Message(bytes);
string msgIdHash = HashMessageId($"{transactionEvent.TrackedSource}-{transactionEvent.TransactionData.Transaction.GetHash()}-{(transactionEvent.TransactionData.BlockId?.ToString() ?? string.Empty)}");
ValidateMessageId(msgIdHash);
message.MessageId = msgIdHash;
message.ContentType = transactionEvent.GetType().ToString();
message.UserProperties.Add("CryptoCode", transactionEvent.CryptoCode);
await Client.SendAsync(message);
}
public async Task Send(NewBlockEvent blockEvent)
{
string jsonMsg = blockEvent.ToJson(Networks.GetFromCryptoCode(blockEvent.CryptoCode).JsonSerializerSettings);
var bytes = UTF8.GetBytes(jsonMsg);
var message = new Message(bytes);
message.MessageId = blockEvent.Hash.ToString();
message.ContentType = blockEvent.GetType().ToString();
message.UserProperties.Add("CryptoCode", blockEvent.CryptoCode);
await Client.SendAsync(message);
}
public async Task Close()
{
if(!Client.IsClosedOrClosing)
await Client.CloseAsync();
}
private string HashMessageId(string messageId)
{
HashAlgorithm algorithm = SHA256.Create();
return Encoding.UTF8.GetString( algorithm.ComputeHash(Encoding.UTF8.GetBytes(messageId)));
}
private void ValidateMessageId(string messageId)
{
if (string.IsNullOrEmpty(messageId) )
{
throw new ArgumentException("MessageIdIsNullOrEmpty");
}
else if (messageId.Length > MaxMessageIdLength)
{
throw new ArgumentException($"MessageIdIsOverMaxLength ({MaxMessageIdLength}) : {messageId} ");
}
}
}
}

View File

@ -1,140 +0,0 @@
using Microsoft.Azure.ServiceBus;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Options;
using NBXplorer.Configuration;
using RabbitMQ.Client;
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
namespace NBXplorer.MessageBrokers
{
public class BrokerHostedService : IHostedService
{
EventAggregator _EventAggregator;
bool _Disposed = false;
CompositeDisposable _subscriptions = new CompositeDisposable();
IBrokerClient _senderBlock = null;
IBrokerClient _senderTransactions = null;
ExplorerConfiguration _config;
public BrokerHostedService(EventAggregator eventAggregator, IOptions<ExplorerConfiguration> config, NBXplorerNetworkProvider networks)
{
_EventAggregator = eventAggregator;
Networks = networks;
_config = config.Value;
}
public Task StartAsync(CancellationToken cancellationToken)
{
if (_Disposed)
throw new ObjectDisposedException(nameof(BrokerHostedService));
_senderBlock = CreateClientBlock();
_senderTransactions = CreateClientTransaction();
_subscriptions.Add(_EventAggregator.Subscribe<Models.NewBlockEvent>(async o =>
{
await _senderBlock.Send(o);
}));
_subscriptions.Add(_EventAggregator.Subscribe<Models.NewTransactionEvent>(async o =>
{
await _senderTransactions.Send(o);
}));
return Task.CompletedTask;
}
IBrokerClient CreateClientTransaction()
{
var brokers = new List<IBrokerClient>();
if (!string.IsNullOrEmpty(_config.AzureServiceBusConnectionString))
{
if (!string.IsNullOrWhiteSpace(_config.AzureServiceBusTransactionQueue))
brokers.Add(CreateAzureQueue(_config.AzureServiceBusConnectionString, _config.AzureServiceBusTransactionQueue));
if (!string.IsNullOrWhiteSpace(_config.AzureServiceBusTransactionTopic))
brokers.Add(CreateAzureTopic(_config.AzureServiceBusConnectionString, _config.AzureServiceBusTransactionTopic));
}
if(!string.IsNullOrEmpty(_config.RabbitMqHostName) &&
!string.IsNullOrEmpty(_config.RabbitMqUsername) &&
!string.IsNullOrEmpty(_config.RabbitMqPassword))
{
if(!string.IsNullOrEmpty(_config.RabbitMqTransactionExchange))
{
brokers.Add(CreateRabbitMqExchange(
hostName: _config.RabbitMqHostName,
virtualHost: _config.RabbitMqVirtualHost,
username: _config.RabbitMqUsername,
password: _config.RabbitMqPassword,
newTransactionExchange: _config.RabbitMqTransactionExchange,
newBlockExchange: string.Empty));
}
}
return new CompositeBroker(brokers);
}
IBrokerClient CreateClientBlock()
{
var brokers = new List<IBrokerClient>();
if (!string.IsNullOrEmpty(_config.AzureServiceBusConnectionString))
{
if (!string.IsNullOrWhiteSpace(_config.AzureServiceBusBlockQueue))
brokers.Add(CreateAzureQueue(_config.AzureServiceBusConnectionString, _config.AzureServiceBusBlockQueue));
if (!string.IsNullOrWhiteSpace(_config.AzureServiceBusBlockTopic))
brokers.Add(CreateAzureTopic(_config.AzureServiceBusConnectionString, _config.AzureServiceBusBlockTopic));
}
if(!string.IsNullOrEmpty(_config.RabbitMqHostName) &&
!string.IsNullOrEmpty(_config.RabbitMqUsername) &&
!string.IsNullOrEmpty(_config.RabbitMqPassword))
{
if(!string.IsNullOrEmpty(_config.RabbitMqBlockExchange))
{
brokers.Add(CreateRabbitMqExchange(
hostName: _config.RabbitMqHostName,
virtualHost: _config.RabbitMqVirtualHost,
username: _config.RabbitMqUsername,
password: _config.RabbitMqPassword,
newTransactionExchange: string.Empty,
newBlockExchange: _config.RabbitMqBlockExchange));
}
}
return new CompositeBroker(brokers);
}
private IBrokerClient CreateAzureQueue(string connnectionString, string queueName)
{
return new AzureBroker(new QueueClient(connnectionString, queueName), Networks);
}
private IBrokerClient CreateAzureTopic(string connectionString, string topicName)
{
return new AzureBroker(new TopicClient(connectionString, topicName), Networks);
}
private IBrokerClient CreateRabbitMqExchange(
string hostName, string virtualHost,
string username, string password,
string newTransactionExchange, string newBlockExchange)
{
return new RabbitMqBroker(
Networks,
new ConnectionFactory() {
HostName = hostName, VirtualHost = virtualHost,
UserName = username, Password = password },
newTransactionExchange, newBlockExchange );
}
public async Task StopAsync(CancellationToken cancellationToken)
{
if (_senderBlock is null)
return;
_Disposed = true;
_subscriptions.Dispose();
await Task.WhenAll(_senderBlock.Close(), _senderTransactions.Close());
}
public NBXplorerNetworkProvider Networks { get; }
}
}

View File

@ -1,41 +0,0 @@
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using NBXplorer.Models;
namespace NBXplorer.MessageBrokers
{
public class CompositeBroker : IBrokerClient
{
public CompositeBroker(IEnumerable<IBrokerClient> clients)
{
Clients = clients.ToArray();
}
public IBrokerClient[] Clients
{
get;
}
public Task Close()
{
if(Clients.Length == 0)
return Task.CompletedTask;
return Task.WhenAll(Clients.Select(c => c.Close()));
}
public Task Send(NewTransactionEvent transactionEvent)
{
if(Clients.Length == 0)
return Task.CompletedTask;
return Task.WhenAll(Clients.Select(c => c.Send(transactionEvent)));
}
public Task Send(NewBlockEvent blockEvent)
{
if(Clients.Length == 0)
return Task.CompletedTask;
return Task.WhenAll(Clients.Select(c => c.Send(blockEvent)));
}
}
}

View File

@ -1,11 +0,0 @@
using System.Threading.Tasks;
namespace NBXplorer.MessageBrokers
{
public interface IBrokerClient
{
Task Send(Models.NewTransactionEvent transactionEvent);
Task Send(Models.NewBlockEvent blockEvent);
Task Close();
}
}

View File

@ -1,126 +0,0 @@
using System;
using System.Collections.Generic;
using System.Security.Cryptography;
using System.Text;
using System.Threading.Tasks;
using NBXplorer.Models;
using RabbitMQ.Client;
namespace NBXplorer.MessageBrokers
{
internal class RabbitMqBroker : IBrokerClient
{
private readonly NBXplorerNetworkProvider Networks;
private readonly ConnectionFactory ConnectionFactory;
private readonly string NewTransactionExchange;
private readonly string NewBlockExchange;
private IConnection Connection;
private IModel Channel;
public RabbitMqBroker(
NBXplorerNetworkProvider networks, ConnectionFactory connectionFactory,
string newTransactionExchange, string newBlockExchange)
{
Networks = networks;
ConnectionFactory = connectionFactory;
NewTransactionExchange = newTransactionExchange;
NewBlockExchange = newBlockExchange;
}
private void CheckAndOpenConnection()
{
if(Channel == null)
{
Connection = ConnectionFactory.CreateConnection();
Channel = Connection.CreateModel();
if(!string.IsNullOrEmpty(NewTransactionExchange))
Channel.ExchangeDeclare(NewTransactionExchange, ExchangeType.Topic);
if(!string.IsNullOrEmpty(NewBlockExchange))
Channel.ExchangeDeclare(NewBlockExchange, ExchangeType.Topic);
}
}
Task IBrokerClient.Close()
{
if(Connection != null && Connection.IsOpen)
Connection.Close();
if(Channel != null && Channel.IsOpen)
Channel.Close();
return Task.CompletedTask;
}
Task IBrokerClient.Send(NewTransactionEvent transactionEvent)
{
CheckAndOpenConnection();
string jsonMsg = transactionEvent.ToJson(Networks.GetFromCryptoCode(transactionEvent.CryptoCode).JsonSerializerSettings);
var body = Encoding.UTF8.GetBytes(jsonMsg);
var conf = (transactionEvent.BlockId == null ? "unconfirmed" : "confirmed");
var routingKey = $"transactions.{transactionEvent.CryptoCode}.{conf}";
string msgIdHash = HashMessageId($"{transactionEvent.TrackedSource}-{transactionEvent.TransactionData.Transaction.GetHash()}-{(transactionEvent.TransactionData.BlockId?.ToString() ?? string.Empty)}");
ValidateMessageId(msgIdHash);
IBasicProperties props = Channel.CreateBasicProperties();
props.MessageId = msgIdHash;
props.ContentType = typeof(NewTransactionEvent).ToString();
props.Headers = new Dictionary<string, object>();
props.Headers.Add("CryptoCode", transactionEvent.CryptoCode);
Channel.BasicPublish(
exchange: NewTransactionExchange,
routingKey: routingKey,
basicProperties: props,
body: body);
return Task.CompletedTask;
}
Task IBrokerClient.Send(NewBlockEvent blockEvent)
{
CheckAndOpenConnection();
string jsonMsg = blockEvent.ToJson(Networks.GetFromCryptoCode(blockEvent.CryptoCode).JsonSerializerSettings);
var body = Encoding.UTF8.GetBytes(jsonMsg);
var routingKey = $"blocks.{blockEvent.CryptoCode}";
IBasicProperties props = Channel.CreateBasicProperties();
props.MessageId = blockEvent.Hash.ToString();
props.ContentType = typeof(NewBlockEvent).ToString();
props.Headers = new Dictionary<string, object>();
props.Headers.Add("CryptoCode", blockEvent.CryptoCode);
Channel.BasicPublish(
exchange: NewBlockExchange,
routingKey: routingKey,
basicProperties: props,
body: body);
return Task.CompletedTask;
}
const int MaxMessageIdLength = 128;
private string HashMessageId(string messageId)
{
HashAlgorithm algorithm = SHA256.Create();
return Encoding.UTF8.GetString( algorithm.ComputeHash(Encoding.UTF8.GetBytes(messageId)));
}
private void ValidateMessageId(string messageId)
{
if (string.IsNullOrEmpty(messageId) )
{
throw new ArgumentException("MessageIdIsNullOrEmpty");
}
else if (messageId.Length > MaxMessageIdLength)
{
throw new ArgumentException($"MessageIdIsOverMaxLength ({MaxMessageIdLength}) : {messageId} ");
}
}
}
}

View File

@ -1,9 +1,9 @@
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework Condition="'$(TargetFrameworkOverride)' == ''">net8.0</TargetFramework>
<TargetFramework Condition="'$(TargetFrameworkOverride)' == ''">net10.0</TargetFramework>
<TargetFramework Condition="'$(TargetFrameworkOverride)' != ''">$(TargetFrameworkOverride)</TargetFramework>
<Version>2.5.28</Version>
<Version>2.6.8</Version>
<DocumentationFile>bin\$(Configuration)\$(TargetFramework)\NBXplorer.xml</DocumentationFile>
<NoWarn>1701;1702;1705;1591;CS1591</NoWarn>
<LangVersion>12</LangVersion>
@ -35,17 +35,18 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="Dapper" Version="2.1.35" />
<PackageReference Include="Microsoft.Azure.ServiceBus" Version="4.2.1" />
<PackageReference Include="Npgsql" Version="8.0.6" />
<PackageReference Include="RabbitMQ.Client" Version="5.1.2" />
<PackageReference Include="Dapper" Version="2.1.79" />
<PackageReference Include="Npgsql" Version="10.0.3" />
<PackageReference Include="NicolasDorier.CommandLine" Version="2.0.0" />
<PackageReference Include="NicolasDorier.CommandLine.Configuration" Version="2.0.0" />
<PackageReference Include="NicolasDorier.StandardConfiguration" Version="2.0.0" />
<PackageReference Include="Microsoft.AspNetCore.Mvc.NewtonsoftJson" Version="8.0.11"></PackageReference>
<PackageReference Include="NicolasDorier.StandardConfiguration" Version="2.0.1" />
<PackageReference Include="Microsoft.AspNetCore.Mvc.NewtonsoftJson" Version="10.0.9" />
<ProjectReference Include="..\NBXplorer.Client\NBXplorer.Client.csproj" />
</ItemGroup>
<ItemGroup>
<InternalsVisibleTo Include="NBXplorer.Tests" />
</ItemGroup>
<ItemGroup>
<Resource Include="DBScripts\001.Migrations.sql" />
</ItemGroup>
@ -63,4 +64,4 @@
</EmbeddedResource>
</ItemGroup>
<ProjectExtensions><VisualStudio><UserProperties wwwroot_4api_1json__JsonSchema="https://raw.githubusercontent.com/OAI/OpenAPI-Specification/refs/tags/3.1.0/schemas/v3.0/schema.json" /></VisualStudio></ProjectExtensions>
</Project>
</Project>

View File

@ -8,20 +8,25 @@ using Microsoft.Extensions.Configuration;
using CommandLine;
using System.Runtime.CompilerServices;
using System.Reflection;
using Microsoft.AspNetCore.Hosting.Server.Features;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using NBitcoin;
[assembly: InternalsVisibleTo("NBXplorer.Tests")]
namespace NBXplorer
{
public class Program
{
public static void Main(string[] args)
public static async Task Main(string[] args)
{
ExtPubKey.SkipInvalidMasterExtPubKeyCheck = true;
var version = typeof(Program).Assembly.GetCustomAttribute<AssemblyInformationalVersionAttribute>();
var processor = new ConsoleLoggerProcessor();
Logs.Configure(new FuncLoggerFactory(i => new CustomerConsoleLogger(i, (a, b) => true, null, processor)));
if (version is { InformationalVersion: { } v })
Logs.Configuration.LogInformation($"NBXplorer version {v.Split('+')[0]}");
IWebHost host = null;
IHost host = null;
try
{
var conf = new DefaultConfiguration() { Logger = Logs.Configuration }.CreateConfiguration(args);
@ -32,13 +37,9 @@ namespace NBXplorer
// However, a bug in .NET Core fixed in 2.1 will prevent the app from stopping if an exception is thrown by the host
// at startup. We need to remove this line later
new ExplorerConfiguration().LoadArgs(conf);
ConfigurationBuilder builder = new ConfigurationBuilder();
host = new WebHostBuilder()
host = Host.CreateDefaultBuilder()
.UseContentRoot(Directory.GetCurrentDirectory())
.UseKestrel()
.UseIISIntegration()
.UseConfiguration(conf)
.ConfigureLogging(l =>
{
l.AddFilter("Microsoft", LogLevel.Error);
@ -48,11 +49,25 @@ namespace NBXplorer
{
l.SetMinimumLevel(LogLevel.Debug);
}
l.ClearProviders();
l.AddProvider(new CustomConsoleLogProvider(processor));
})
.UseStartup<Startup>()
.ConfigureWebHostDefaults(webBuilder => {
webBuilder
.UseKestrel()
.UseConfiguration(conf)
.UseStartup<Startup>();
})
.Build();
host.Run();
await host.StartAsync();
var logger = host.Services.GetRequiredService<ILoggerFactory>().CreateLogger("Configuration");
var urls = host.GetServerFeatures<IServerAddressesFeature>().Addresses;
foreach (var url in urls)
{
// Some tools such as dotnet watch parse this exact log to open the browser
logger.LogInformation("Now listening on: " + url);
}
await host.WaitForShutdownAsync();
}
catch (ConfigException ex)
{

View File

@ -7,7 +7,7 @@
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development",
"NBXPLORER_POSTGRES": "User ID=postgres;Application Name=test;Include Error Detail=true;Host=localhost;Port=39383",
"NBXPLORER_BTCSTARTHEIGHT": "860200",
"NBXPLORER_BTCSTARTHEIGHT": "911478",
"NBXPLORER_BTCRESCAN": "1",
"NBXPLORER_BTCRPCCOOKIEFILE": "D:\\Bitcoin\\.cookie"
},

View File

@ -14,6 +14,7 @@ using System.Net.Http;
using System.Net;
using System.Net.Http.Headers;
using System.Collections.Generic;
using System.IO;
using System.Runtime.CompilerServices;
using NBitcoin.Crypto;
using NBXplorer.Models;
@ -100,11 +101,11 @@ namespace NBXplorer
goto retry;
}
}
public static async Task<bool?> SupportTxIndex(this RPCClient rpc)
public static async Task<bool?> SupportTxIndex(this RPCClient rpc, CancellationToken cancellationToken = default)
{
try
{
var result = await rpc.SendCommandAsync(new RPCRequest("getindexinfo", new[] { "txindex" }) { ThrowIfRPCError = false });
var result = await WithRetry(() => rpc.SendCommandAsync(new RPCRequest("getindexinfo", new[] { "txindex" }) { ThrowIfRPCError = false }, cancellationToken), cancellationToken);
if (result.Error != null)
return null;
return result.Result["txindex"] is not null;
@ -116,7 +117,7 @@ namespace NBXplorer
}
public static async Task<bool> WarmupBlockchain(this RPCClient rpc, ILogger logger)
{
if (await rpc.GetBlockCountAsync() < rpc.Network.Consensus.CoinbaseMaturity)
if (await WithRetry(() => rpc.GetBlockCountAsync()) < rpc.Network.Consensus.CoinbaseMaturity)
{
logger.LogInformation($"Less than {rpc.Network.Consensus.CoinbaseMaturity} blocks, mining some block for regtest (you can disable with NBXPLORER_NOWARMUP=1)");
await rpc.EnsureGenerateAsync(rpc.Network.Consensus.CoinbaseMaturity + 1);
@ -124,16 +125,16 @@ namespace NBXplorer
}
else
{
var hash = await rpc.GetBestBlockHashAsync();
var hash = await WithRetry(() => rpc.GetBestBlockHashAsync());
BlockHeader header = null;
try
{
header = await rpc.GetBlockHeaderAsync(hash);
header = await WithRetry(() => rpc.GetBlockHeaderAsync(hash));
}
catch (RPCException ex) when (ex.RPCCode == RPCErrorCode.RPC_METHOD_NOT_FOUND)
{
header = (await rpc.GetBlockAsync(hash)).Header;
header = (await WithRetry(() => rpc.GetBlockAsync(hash))).Header;
}
if ((DateTimeOffset.UtcNow - header.BlockTime) > TimeSpan.FromSeconds(24 * 60 * 60))
{
@ -151,8 +152,15 @@ namespace NBXplorer
=> new()
{
Fees = entry.BaseFee,
Chunk = entry.ChunkFees is null || entry.ChunkWeight is 0 ? null :
new TransactionMetadata.ChunkMetadata()
{
Fees = entry.ChunkFees,
Weight = entry.ChunkWeight,
FeeRate = GetFeeRate(entry.ChunkFees, ((entry.ChunkWeight + 3) / 4)),
},
VirtualSize = entry.VirtualSizeBytes,
FeeRate = GetFeeRate(entry.BaseFee, entry.VirtualSizeBytes)
FeeRate = GetFeeRate(entry.BaseFee, entry.VirtualSizeBytes),
};
// This method fetch some information from getmempoolentry which may be useful for analysis, it's not critical to have it, so we don't want to fail the whole thing if it fails.
@ -169,7 +177,7 @@ namespace NBXplorer
return metadatas;
try
{
await batch.SendBatchAsync(cancellationToken);
await WithRetry(() => batch.SendBatchAsync(cancellationToken), cancellationToken);
foreach (var t in tasks)
{
try
@ -193,8 +201,10 @@ namespace NBXplorer
{
if (peer is null)
return false;
#pragma warning disable CS0618 // Type or member is obsolete
if (peer.IsWhiteListed)
return true;
#pragma warning restore CS0618 // Type or member is obsolete
if (peer.Permissions.Contains("noban", StringComparer.OrdinalIgnoreCase))
return true;
return false;
@ -218,10 +228,29 @@ namespace NBXplorer
return blockchainInfo.Headers - blockchainInfo.Blocks > 6;
}
public static async Task<GetBlockchainInfoResponse> GetBlockchainInfoAsyncEx(this RPCClient client, CancellationToken cancellationToken = default)
public static Task<GetBlockchainInfoResponse> GetBlockchainInfoAsyncEx(this RPCClient client, CancellationToken cancellationToken = default)
=> WithRetry(async () =>
{
var result = await client.SendCommandAsync("getblockchaininfo", cancellationToken).ConfigureAwait(false);
var result = await client.SendCommandAsync("getblockchaininfo", cancellationToken)
.ConfigureAwait(false);
return JsonConvert.DeserializeObject<GetBlockchainInfoResponse>(result.ResultString);
}, cancellationToken);
static Task WithRetry(Func<Task> func, CancellationToken cancellationToken = default)
=> WithRetry<string>(async () => { await func(); return null; }, cancellationToken);
static async Task<T> WithRetry<T>(Func<Task<T>> func, CancellationToken cancellationToken = default)
{
try
{
return await func();
}
catch (HttpRequestException ex) when (ex.InnerException is IOException) { } // Sometimes "The response ended prematurely."
catch (IOException) { } // Sometimes "The response ended prematurely."
catch (OperationCanceledException) // Timeout can happen if Bitcoin core is really busy
{
}
cancellationToken.ThrowIfCancellationRequested();
return await func();
}
public static async Task EnsureWalletCreated(this RPCClient client, ILogger logger)
@ -229,6 +258,7 @@ namespace NBXplorer
var network = client.Network.NetworkSet;
var walletName = client.CredentialString.WalletName ?? "";
bool created = false;
retry:
try
{
await client.CreateWalletAsync(walletName, new CreateWalletOptions()
@ -244,19 +274,28 @@ namespace NBXplorer
// Not supported
return;
}
catch (RPCException ex) when (ex.RPCCode == RPCErrorCode.RPC_WALLET_ERROR || ex.RPCCode == RPCErrorCode.RPC_WALLET_ALREADY_EXISTS)
catch (RPCException ex) when (ex.RPCCode == RPCErrorCode.RPC_INVALID_PARAMETER && walletName == "")
{
logger.LogInformation($"{network.CryptoCode}: RPC wallet features are disabled because neither `{network.CryptoCode}.rpc.defaultwallet` nor `NBXPLORER_{network.CryptoCode.ToUpperInvariant()}_RPC_DEFAULTWALLET` is configured.");
return;
}
catch (RPCException ex) when (ex.RPCCode == RPCErrorCode.RPC_WALLET_ERROR ||
ex.RPCCode == RPCErrorCode.RPC_WALLET_ALREADY_EXISTS)
{
// Already exists, let's load it
}
catch (HttpRequestException ex) when (ex.StatusCode is HttpStatusCode.Unauthorized || ex.StatusCode is HttpStatusCode.Forbidden)
catch (HttpRequestException ex) when (ex.StatusCode is HttpStatusCode.Unauthorized ||
ex.StatusCode is HttpStatusCode.Forbidden)
{
// Not allowed, which is fine
logger.LogInformation($"{network.CryptoCode}: RPC wallet features are disabled due to the node's policy");
return;
}
catch (Exception ex)
{
logger.LogWarning(ex, $"{network.CryptoCode}: Failed to create a RPC wallet with unknown error, skipping...");
logger.LogWarning(ex,
$"{network.CryptoCode}: Failed to create a RPC wallet with unknown error, skipping...");
}
try
{
await client.LoadWalletAsync(walletName, true);
@ -276,16 +315,17 @@ namespace NBXplorer
}
}
public static async Task<GetNetworkInfoResponse> GetNetworkInfoAsync(this RPCClient client)
{
var result = await client.SendCommandAsync("getnetworkinfo").ConfigureAwait(false);
return JsonConvert.DeserializeObject<GetNetworkInfoResponse>(result.ResultString);
}
public static Task<GetNetworkInfoResponse> GetNetworkInfoAsync(this RPCClient client, CancellationToken cancellationToken = default)
=> WithRetry(async () =>
{
var result = await client.SendCommandAsync("getnetworkinfo", cancellationToken).ConfigureAwait(false);
return JsonConvert.DeserializeObject<GetNetworkInfoResponse>(result.ResultString);
}, cancellationToken);
public static async Task<PSBT> UTXOUpdatePSBT(this RPCClient rpcClient, PSBT psbt)
public static async Task<PSBT> UTXOUpdatePSBT(this RPCClient rpcClient, PSBT psbt, CancellationToken cancellationToken = default)
{
if (psbt == null) throw new ArgumentNullException(nameof(psbt));
var response = await rpcClient.SendCommandAsync("utxoupdatepsbt", new object[] { psbt.ToBase64() });
var response = await WithRetry(() => rpcClient.SendCommandAsync("utxoupdatepsbt", new object[] { psbt.ToBase64() }, cancellationToken), cancellationToken);
response.ThrowIfError();
if (response.Error == null && response.Result is JValue rpcResult && rpcResult.Value is string psbtStr)
{
@ -298,32 +338,32 @@ namespace NBXplorer
{
var batch = rpc.PrepareBatch();
var hashes = blockHeights.Select(h => batch.GetBlockHashAsync(h, cancellationToken)).ToArray();
await batch.SendBatchAsync(cancellationToken);
await WithRetry(() => batch.SendBatchAsync(cancellationToken), cancellationToken);
batch = rpc.PrepareBatch();
var headers = hashes.Select(async h => await batch.GetBlockHeaderAsyncEx(await h, cancellationToken)).ToArray();
await batch.SendBatchAsync(cancellationToken);
await WithRetry(() => batch.SendBatchAsync(cancellationToken), cancellationToken);
return new BlockHeaders(headers.Select(h => h.GetAwaiter().GetResult()).Where(h => h is not null).ToList());
}
public static async Task<BlockHeaders> GetBlockHeadersAsync(this RPCClient rpc, IList<uint256> hashes, CancellationToken cancellationToken)
{
var batch = rpc.PrepareBatch();
await batch.SendBatchAsync(cancellationToken);
await WithRetry(() => batch.SendBatchAsync(cancellationToken), cancellationToken);
batch = rpc.PrepareBatch();
var headers = hashes.Select(async h => await batch.GetBlockHeaderAsyncEx(h, cancellationToken)).ToArray();
await batch.SendBatchAsync(cancellationToken);
await WithRetry(()=> batch.SendBatchAsync(cancellationToken), cancellationToken);
return new BlockHeaders(headers.Select(h => h.GetAwaiter().GetResult()).Where(h => h is not null).ToList());
}
public static async Task<RPCBlockHeader> GetBlockHeaderAsyncEx(this RPCClient rpc, uint256 blk, CancellationToken cancellationToken)
{
var header = await rpc.SendCommandAsync(new NBitcoin.RPC.RPCRequest("getblockheader", new[] { blk.ToString() })
var header = await WithRetry(() => rpc.SendCommandAsync(new NBitcoin.RPC.RPCRequest("getblockheader", new[] { blk.ToString() })
{
ThrowIfRPCError = false
}, cancellationToken);
}, cancellationToken), cancellationToken);
if (header.Result is null || header.Error is not null)
return null;
var response = header.Result;
@ -343,7 +383,7 @@ namespace NBXplorer
public static async Task<SavedTransaction> TryGetRawTransaction(this RPCClient client, uint256 txId, CancellationToken cancellationToken)
{
var request = new RPCRequest(RPCOperations.getrawtransaction, new object[] { txId, true }) { ThrowIfRPCError = false };
var response = await client.SendCommandAsync(request);
var response = await WithRetry(() => client.SendCommandAsync(request, cancellationToken), cancellationToken);
if (response.Error == null && response.Result is JToken rpcResult && rpcResult["hex"] != null)
{
uint256 blockHash = null;
@ -376,7 +416,7 @@ namespace NBXplorer
return null;
}
public async static Task<Dictionary<uint256, Transaction>> GetTransactionFromBlocks(this RPCClient rpc, HashSet<(uint256 BlockId, uint256 TransactionId)> txBlockIds, CancellationToken cancellationToken = default)
public static async Task<Dictionary<uint256, Transaction>> GetTransactionFromBlocks(this RPCClient rpc, HashSet<(uint256 BlockId, uint256 TransactionId)> txBlockIds, CancellationToken cancellationToken = default)
{
async Task<Dictionary<uint256, Transaction>> GetTransactionFromStoredBlocks(HashSet<(uint256 BlockId, uint256 TransactionId)> txBlockIds)
{
@ -385,7 +425,7 @@ namespace NBXplorer
return result;
var batch = rpc.PrepareBatch();
var fetching = txBlockIds.Select(b => (b.TransactionId, Fetching: batch.GetRawTransactionAsync(b.TransactionId, b.BlockId, false, cancellationToken))).ToArray();
await batch.SendBatchAsync(cancellationToken);
await WithRetry(()=> batch.SendBatchAsync(cancellationToken), cancellationToken);
foreach (var f in fetching)
{
Transaction tx = null;
@ -406,7 +446,7 @@ namespace NBXplorer
var downloaded = new HashSet<uint256>();
if (blocks.Count == 0)
return downloaded;
var peers = (await rpc.GetPeersInfoAsync(cancellationToken))
var peers = (await WithRetry(() => rpc.GetPeersInfoAsync(cancellationToken), cancellationToken))
.Where(p => p.ServicesNames?.Contains("NETWORK") is true)
.ToArray();
NBitcoin.Utils.Shuffle(peers);
@ -503,13 +543,13 @@ namespace NBXplorer
}
throw new NotSupportedException($"Bug of NBXplorer (ERR 3083), please notify the developers");
}
public static async Task<Dictionary<uint256, Transaction>> GetRawTransactions(this RPCClient rpc, HashSet<uint256> txIds)
public static async Task<Dictionary<uint256, Transaction>> GetRawTransactions(this RPCClient rpc, HashSet<uint256> txIds, CancellationToken cancellationToken = default)
{
if (txIds.Count == 0)
return new();
var batch = rpc.PrepareBatch();
var txs = txIds.Select(t => (Id: t, Tx: batch.GetRawTransactionAsync(t, false))).ToArray();
await batch.SendBatchAsync();
var txs = txIds.Select(t => (Id: t, Tx: batch.GetRawTransactionAsync(t, false, cancellationToken))).ToArray();
await WithRetry(() => batch.SendBatchAsync(cancellationToken), cancellationToken);
var res = new Dictionary<uint256, Transaction>();
foreach (var txAsync in txs)
{
@ -519,11 +559,11 @@ namespace NBXplorer
}
return res;
}
public static async Task<Dictionary<OutPoint, GetTxOutResponse>> GetTxOuts(this RPCClient rpc, IList<OutPoint> outpoints)
public static async Task<Dictionary<OutPoint, GetTxOutResponse>> GetTxOuts(this RPCClient rpc, IList<OutPoint> outpoints, CancellationToken cancellationToken = default)
{
var batch = rpc.PrepareBatch();
var txOuts = outpoints.Select(o => batch.GetTxOutAsync(o.Hash, (int)o.N, true)).ToArray();
await batch.SendBatchAsync();
var txOuts = outpoints.Select(o => batch.GetTxOutAsync(o.Hash, (int)o.N, true, cancellationToken)).ToArray();
await WithRetry(() => batch.SendBatchAsync(cancellationToken), cancellationToken);
var result = new Dictionary<OutPoint, GetTxOutResponse>();
int i = 0;
foreach (var txOut in txOuts)

View File

@ -12,9 +12,7 @@ using NBitcoin.RPC;
using NBXplorer.DerivationStrategy;
using NBXplorer.Models;
using NBXplorer.Logging;
using NBitcoin.Scripting;
using NBXplorer.Backend;
using NBitcoin.Altcoins;
using static NBXplorer.Backend.DbConnectionHelper;
using NBitcoin.Altcoins.Elements;
@ -61,7 +59,7 @@ namespace NBXplorer
class ScannedItems
{
public Dictionary<Script, KeyPathInformation> KeyPathInformations = new Dictionary<Script, KeyPathInformation>();
public List<OutputDescriptor> Descriptors = new List<OutputDescriptor>();
public List<string> Descriptors = new List<string>();
}
public ScanUTXOSetService(ScanUTXOSetServiceAccessor accessor,
@ -317,7 +315,7 @@ namespace NBXplorer
.GenerateBlindingKey(derivationStrategy.DerivationStrategy, keyPath, derivation.ScriptPubKey, network.NBitcoinNetwork).PubKey;
info.Address = new BitcoinBlindedAddress(blindingPubKey, info.Address);
}
items.Descriptors.Add(OutputDescriptor.NewRaw(info.ScriptPubKey, network.NBitcoinNetwork));
items.Descriptors.Add($"raw({info.ScriptPubKey.ToHex()})");
items.KeyPathInformations.TryAdd(info.ScriptPubKey, info);
return info;
}).All(_ => true);
@ -329,7 +327,7 @@ namespace NBXplorer
public Task StopAsync(CancellationToken cancellationToken)
{
_Cts.Cancel();
_Channel.Writer.Complete();
_Channel.Writer.TryComplete();
return _Task;
}

View File

@ -3,11 +3,20 @@ using NBXplorer.Models;
using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.AspNetCore.Hosting.Server;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
namespace NBXplorer
{
public static class Utils
{
public static T GetServerFeatures<T>(this IHost host) where T : class
{
var server = host.Services.GetRequiredService<IServer>();
var features = server.Features;
return features.Get<T>();
}
public static ICollection<AnnotatedTransaction> TopologicalSort(this ICollection<AnnotatedTransaction> transactions)
{
var confirmed = new MultiValueDictionary<long, AnnotatedTransaction>();

View File

@ -2,7 +2,7 @@
"openapi": "3.0.0",
"info": {
"title": "NBXplorer API",
"description": "NBXplorer is a multi-cryptocurrency lightweight block explorer that does not index the whole blockchain. \nInstead, it listens to transactions and blocks from a trusted full node and indexes only addresses and \ntransactions that belong to tracked DerivationSchemes.\n\nBy default, NBXplorer is are using [Basic Authentication with a cookie file](https://github.com/dgarage/NBXplorer/blob/master/docs/API.md#authentication) for authentication.",
"description": "NBXplorer is a multi-cryptocurrency lightweight block explorer that does not index the whole blockchain. \nInstead, it listens to transactions and blocks from a trusted full node and indexes only addresses and \ntransactions that belong to tracked DerivationSchemes.\n\nBy default, NBXplorer is are using [Basic Authentication with a cookie file](https://github.com/btcpayserver/NBXplorer/blob/master/docs/API.md#authentication) for authentication.",
"version": "1.0.0"
},
"servers": [
@ -14,11 +14,11 @@
"tags": [
{
"name": "Groups",
"description": "A group is a tracked source which serves as a logical method for grouping several tracked sources into a single entity. You can add or remove tracked sources to and from a group.\n\nFor more details, check out the documentation on [GitHub](https://github.com/dgarage/NBXplorer/blob/master/docs/API.md#groups)."
"description": "A group is a tracked source which serves as a logical method for grouping several tracked sources into a single entity. You can add or remove tracked sources to and from a group.\n\nFor more details, check out the documentation on [GitHub](https://github.com/btcpayserver/NBXplorer/blob/master/docs/API.md#groups)."
},
{
"name": "Derivations",
"description": "A derivation scheme, (also called derivationStrategy) is a flexible way to define how to generate deterministic addresses for a wallet. NBXplorer will track any addresses on the `0/x`, `1/x` and `x` path.\n\nFor more details, check out the documentation on [GitHub](https://github.com/dgarage/NBXplorer/blob/master/docs/API.md#derivation-scheme)."
"description": "A derivation scheme, (also called derivationStrategy) is a flexible way to define how to generate deterministic addresses for a wallet. NBXplorer will track any addresses on the `0/x`, `1/x` and `x` path.\n\nFor more details, check out the documentation on [GitHub](https://github.com/btcpayserver/NBXplorer/blob/master/docs/API.md#derivation-scheme)."
}
],
"components": {
@ -47,7 +47,7 @@
"type": "string"
},
"example": "2-of-xpub1-xpub2",
"description": "The derivation scheme ([See documentation](https://github.com/dgarage/NBXplorer/blob/master/docs/API.md#derivation-scheme))"
"description": "The derivation scheme ([See documentation](https://github.com/btcpayserver/NBXplorer/blob/master/docs/API.md#derivation-scheme))"
},
"GroupId": {
"name": "groupId",
@ -107,6 +107,31 @@
"description": "The estimated fee rate in satoshis per vByte. Only available if the transaction has been indexed when it was in the mempool.",
"example": 20.0,
"nullable": true
},
"chunk": {
"type": "object",
"description": "Fee metadata for the transaction chunk, as reported by Bitcoin Core mempool entry data.",
"nullable": true,
"properties": {
"fees": {
"type": "integer",
"description": "Total fee of this chunk in satoshi.",
"example": 2820,
"nullable": true
},
"weight": {
"type": "integer",
"description": "Total chunk weight in weight units.",
"example": 564,
"nullable": true
},
"feeRate": {
"type": "number",
"description": "Chunk fee rate in satoshis per vByte.",
"example": 20.0,
"nullable": true
}
}
}
}
},
@ -678,11 +703,6 @@
"type": "string",
"description": "Optional. The [BIP39](https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki) passphrase to use with the mnemonic. This is also known as the 'wallet password'. Default is an empty string."
},
"importKeysToRPC": {
"type": "boolean",
"default": false,
"description": "Optional. If **true**, every time a new unused address is generated, the corresponding private key will be imported into the underlying node via RPC's `importprivkey`. Useful if you need to manage your wallet via the node's command-line interface or RPC calls. Default is **false**."
},
"savePrivateKeys": {
"type": "boolean",
"description": "Optional. If **true**, the private keys (mnemonic seed and derived keys) will be saved in NBXplorer's metadata under `Mnemonic`, `MasterHDKey`, and `AccountHDKey`. Be cautious when enabling this option, as storing private keys increases security risks."
@ -958,17 +978,49 @@
"UTXOChanges": {
"type": "object",
"properties": {
"spentUTXOs": {
"trackedSource": {
"type": "string",
"description": "The tracked source identifier, which can be a group or a derivation scheme.",
"example": "DERIVATIONSCHEME:tpubD6NzVbkrYhZ4W..."
},
"currentHeight": {
"type": "integer",
"description": "The current height of the blockchain.",
"example": 104
},
"unconfirmed": {
"$ref": "#/components/schemas/UTXOChange"
},
"confirmed": {
"$ref": "#/components/schemas/UTXOChange"
},
"spentUnconfirmed": {
"type": "array",
"description": "List of still unconfirmed outpoints that were spent. For a transaction chain `A->B->C` where `B` spends `A` and `C` spends `B`, outpoints `A` and `B` will appear here since they remain unconfirmed until the entire chain confirms.",
"items": {
"type": "string"
},
"example": [ "b1ceb33c783788d7c9d4c1417ffe3490eb0c83abd9267d470eaa06a1d3be0475-1", "8224f00a3c12f19b89155b6e4a5ed2320f6407d37d1a6ca7da8def0953fecc3e-2" ]
}
}
},
"UTXOChange": {
"type": "object",
"description": "The list of unconfirmed UTXOs and of spent confirmed UTXOs",
"properties": {
"utxOs": {
"type": "array",
"items": {
"$ref": "#/components/schemas/UTXO"
}
},
"receivedUTXOs": {
"spentOutpoints": {
"type": "array",
"description": "List of confirmed outpoints that were spent. (Only available for the unconfirmed UTXOChange)",
"items": {
"$ref": "#/components/schemas/UTXO"
}
"type": "string"
},
"example": [ "f3e34c3ae29c79ddf807de0ab3b40da7862adb1b175b7421d89ec78446a52b13-1", "fd97fb1de63fcdb9fa1614a74775e84818b2dbd79fe36aef9e0c18f9fad03742-2" ]
}
}
},
@ -1755,16 +1807,9 @@
"properties": {
"hex": {
"type": "string",
"description": "The raw transaction in hexadecimal format."
},
"psbt": {
"type": "string",
"description": "The PSBT in Base64 encoding."
"description": "The raw transaction in hexadecimal format, or finalized PSBT in Base64 or hex.",
"example": "0200000001..."
}
},
"example": {
"hex": "0200000001abcd1234...",
"psbt": "cHNidP8BAHECAAAA..."
}
}
}
@ -1778,16 +1823,26 @@
"schema": {
"type": "object",
"properties": {
"transactionId": {
"success": {
"type": "boolean",
"description": "Whether the transaction was successfully broadcast.",
"example": false
},
"rpcCode": {
"type": "integer",
"description": "The RPC code returned by the node.",
"example": -25
},
"rpcCodeMessage": {
"type": "string",
"description": "The ID of the transaction."
"description": "The error code message returned during transaction submission.",
"example": "General error during transaction submission"
},
"rpcMessage": {
"type": "string",
"description": "The detailed RPC message returned by the node.",
"example": "bad-txns-inputs-missingorspent"
}
},
"required": [
"transactionId"
],
"example": {
"transactionId": "abcd1234..."
}
}
}
@ -3677,77 +3732,6 @@
}
}
},
"/v1/cryptos/{cryptoCode}/broadcast": {
"post": {
"summary": "Broadcast a transaction",
"operationId": "Broadcast",
"description": "Broadcasts a raw transaction to the network for the specified cryptocurrency.",
"tags": [
"Transactions"
],
"parameters": [
{
"$ref": "#/components/parameters/CryptoCode"
}
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"hex": {
"type": "string",
"description": "The raw transaction in hexadecimal format."
}
},
"required": [
"hex"
]
},
"example": {
"hex": "0200000001abcd1234..."
}
}
}
},
"responses": {
"200": {
"description": "Transaction broadcast successfully.",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"transactionId": {
"type": "string",
"description": "The ID of the broadcasted transaction."
}
},
"required": [
"transactionId"
],
"example": {
"transactionId": "abcd1234..."
}
}
}
}
},
"400": {
"description": "Bad request. Invalid transaction data.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ErrorResponse"
}
}
}
}
}
}
},
"/v1/cryptos/{cryptoCode}/derivations/{derivationScheme}/transactions/{txId}": {
"get": {
"summary": "Get a specific transaction",
@ -3800,7 +3784,7 @@
"/cryptos/{cryptoCode}/node/rpc": {
"post": {
"summary": "Proxy RPC call to the underlying node",
"description": "Allows you to make RPC calls to the underlying cryptocurrency node (e.g., Bitcoin Core) via NBXplorer. This endpoint acts as a proxy, forwarding your RPC requests to the node and returning the node's responses.\n\nBy default, only a handful of RPC methods are authorized. You can use [configuration](https://github.com/dgarage/NBXplorer/blob/master/docs/API.md#configuration) `--btcexposerpc` or `NBXPLORER_BTCEXPOSERPC=1` to allow all methods.\n\nGo to [The JSON-RPC specification page](https://www.jsonrpc.org/specification) and the [Bitcoin Core RPC page](https://developer.bitcoin.org/reference/rpc) for more information.",
"description": "Allows you to make RPC calls to the underlying cryptocurrency node (e.g., Bitcoin Core) via NBXplorer. This endpoint acts as a proxy, forwarding your RPC requests to the node and returning the node's responses.\n\nBy default, only a handful of RPC methods are authorized. You can use [configuration](https://github.com/btcpayserver/NBXplorer/blob/master/docs/API.md#configuration) `--btcexposerpc` or `NBXPLORER_BTCEXPOSERPC=1` to allow all methods.\n\nGo to [The JSON-RPC specification page](https://www.jsonrpc.org/specification) and the [Bitcoin Core RPC page](https://developer.bitcoin.org/reference/rpc) for more information.",
"tags": [
"Blockchain"
],

112
README.md
View File

@ -2,7 +2,7 @@
[![NuGet](https://img.shields.io/nuget/v/NBxplorer.Client.svg)](https://www.nuget.org/packages/NBxplorer.Client)
[![Docker Automated buil](https://img.shields.io/docker/automated/jrottenberg/ffmpeg.svg)](https://hub.docker.com/r/nicolasdorier/nbxplorer/)
[![CircleCI](https://circleci.com/gh/dgarage/NBXplorer.svg?style=svg)](https://circleci.com/gh/dgarage/NBXplorer)
[![CircleCI](https://circleci.com/gh/btcpayserver/NBXplorer.svg?style=svg)](https://circleci.com/gh/btcpayserver/NBXplorer)
A minimalist UTXO tracker for HD wallets.
The goal is to provide a flexible, .NET-based UTXO tracker for HD wallets.
@ -14,24 +14,24 @@ This explorer is not intended to be exposed to the internet; it should be used a
## Typical usage
You start by [Creating a wallet (hot wallet)](https://dgarage.github.io/NBXplorer/#tag/Derivations/operation/GenerateWallet), or [Tracking a derivation scheme (cold wallet)](https://dgarage.github.io/NBXplorer/#tag/Derivations/operation/Track).
You start by [Creating a wallet (hot wallet)](https://btcpayserver.github.io/NBXplorer/#tag/Derivations/operation/GenerateWallet), or [Tracking a derivation scheme (cold wallet)](https://btcpayserver.github.io/NBXplorer/#tag/Derivations/operation/Track).
Second, [Get the next unused address](https://dgarage.github.io/NBXplorer/#tag/Derivations/operation/GetUnused) to get paid.
Second, [Get the next unused address](https://btcpayserver.github.io/NBXplorer/#tag/Derivations/operation/GetUnused) to get paid.
Listen to events through [Polling](https://dgarage.github.io/NBXplorer/#tag/Events/operation/GetLatest), [Long Polling](https://dgarage.github.io/NBXplorer/#tag/Events/operation/EventStream) or [Web Sockets](https://dgarage.github.io/NBXplorer/#tag/Events/operation/WebSocket).
Listen to events through [Polling](https://btcpayserver.github.io/NBXplorer/#tag/Events/operation/GetLatest), [Long Polling](https://btcpayserver.github.io/NBXplorer/#tag/Events/operation/EventStream) or [Web Sockets](https://btcpayserver.github.io/NBXplorer/#tag/Events/operation/WebSocket).
You can then [List transactions](https://dgarage.github.io/NBXplorer/#tag/Derivations/operation/ListTransactionDerivationScheme), [List UTXOs](https://dgarage.github.io/NBXplorer/#tag/Derivations/operation/ListUTXOsDerivationScheme), or [Create a PSBT](https://dgarage.github.io/NBXplorer/#tag/Derivations/operation/CreatePSBT) for your app to sign.
You can then [List transactions](https://btcpayserver.github.io/NBXplorer/#tag/Derivations/operation/ListTransactionDerivationScheme), [List UTXOs](https://btcpayserver.github.io/NBXplorer/#tag/Derivations/operation/ListUTXOsDerivationScheme), or [Create a PSBT](https://btcpayserver.github.io/NBXplorer/#tag/Derivations/operation/CreatePSBT) for your app to sign.
When the transaction is signed, [Broadcast it](https://dgarage.github.io/NBXplorer/#tag/Transactions/operation/Broadcast).
When the transaction is signed, [Broadcast it](https://btcpayserver.github.io/NBXplorer/#tag/Transactions/operation/Broadcast).
You can also track multiple derivation schemes or individual addresses by [Creating a group](https://dgarage.github.io/NBXplorer/#tag/Groups/operation/Create).
You can also track multiple derivation schemes or individual addresses by [Creating a group](https://btcpayserver.github.io/NBXplorer/#tag/Groups/operation/Create).
## General features
* Miniscript support via [Wallet Policies (BIP0388)](https://github.com/bitcoin/bips/blob/master/bip-0388.mediawiki).
* Can pass arguments via environment variable, command line or configuration file
* Automatically reconnect to your node if the connection goes temporarily down
* An easy to use [REST API](https://dgarage.github.io/NBXplorer/) ([Overview](./docs/API.md).)
* An easy to use [REST API](https://btcpayserver.github.io/NBXplorer/) ([Overview](./docs/API.md).)
* Persistence via [Postgres](./docs/docs/Postgres-Schema.md)
* Connect via RPC to broadcast transaction instead of using the P2P protocol like this example
* Connect via RPC to your trusted node to get the proper fee rate.
@ -60,17 +60,18 @@ It currently supports the following altcoins:
* Monacoin
* MonetaryUnit
* Monoeci
* Pepecoin
* Polis
* Qtum
* Terracoin
* Ufo
* Viacoin
Read our [API Specification](https://dgarage.github.io/NBXplorer/).
Read our [API Specification](https://btcpayserver.github.io/NBXplorer/).
## Prerequisite
* Install [.NET Core SDK v8.0 or above](https://www.microsoft.com/net/download)
* Install [.NET SDK v10.0 or above](https://www.microsoft.com/net/download)
* Bitcoin Core instance synched and running (at least 24.0).
* PostgresSQL v13+
@ -118,7 +119,7 @@ Example, if you have ltc node and btc node on regtest (default configuration), a
## How to use the API?
Check [the API documentation](https://dgarage.github.io/NBXplorer/), you can then use any client library:
Check [the API documentation](https://btcpayserver.github.io/NBXplorer/), you can then use any client library:
* [NBXplorer.NodeJS](https://github.com/junderw/NBXplorer.NodeJS) for NodeJS clients.
* [NBXplorer.Client](https://www.nuget.org/packages/NBxplorer.Client) for .NET clients.
@ -215,7 +216,7 @@ If you need to see old payments, you need to configure `--[crypto]startheight` t
[Postman](https://www.getpostman.com) is a useful tool for testing and experimenting with REST API's.
You can test the [NBXplorer API](https://dgarage.github.io/NBXplorer/) quickly and easily using Postman.
You can test the [NBXplorer API](https://btcpayserver.github.io/NBXplorer/) quickly and easily using Postman.
If you use cookie authentication (enabled by default) in your locally run NBXplorer, you need to set that up in Postman:
@ -242,94 +243,13 @@ You are now ready to test the API - it is easiest to start with something simple
this should return a JSON payload e.g.
```json
{
"feeRate": 9,
"blockCount": 3
}
## Message Brokers
### Azure Service Bus
Support has been added for Azure Service Bus as a message broker. Currently 2 Queues and 2 Topics are supported
### Queues
* New Block
* New Transaction
### Topics
* New Block
* New Transaction
Filters should be applied on the client, if required.
To activate Azure Service Bus Mesages you should add an Azure Service Bus Connection string to your config file or on the command line.
* To use queues you should specify the queue names you wish to use
* To use topics you should specify the topic names you wish to use
You can use both queues and topics at the same time.
#### Config Settings
If you use the Configuration file to setup your NBXplorer options:
```ini
asbcnstr=Your Azure Service Bus Connection string
asbblockq=Name of queue to send New Block message to
asbtranq=Name of queue to send New Transaction message to
asbblockt=Name of topic to send New Block message to
asbtrant=Name of queue to send New Transaction message to
```
### RabbitMq
Support has been added for RabbitMq as a message broker. Currently 2 exchanges supported;
* New Block
* New Transaction
Filters can be applied on the client by defining routing keys;
For transactions;
* `transactions.#` to get all transactions.
* `transactions.[BTC].#` to get all [Bitcoin] transactions.
* `transactions.[BTC].confirmed` to get only confirmed [Bitcoin] transactions.
* `transactions.[BTC].unconfirmed` to get only unconfirmed [Bitcoin] transactions.
* `transactions.*.confirmed` to get all confirmed transactions.
* `transactions.*.unconfirmed` to get all unconfirmed transactions.
For blocks;
* `blocks.#` to get all blocks.
* `blocks.[BTC]` to get all [Bitcoin] blocks.
To activate RabbitMq mesages you should add following settings to your config file or on the command line.
* rmqhost, rmquser, rmqpass
#### Config Settings
If you use the Configuration file to setup your NBXplorer options:
```ini
rmqhost= RabbitMq host name
rmqvirtual= RabbitMq virtual host
rmquser= RabbitMq username
rmqpass= RabbitMq password
rmqtranex= Name of exchange to send transaction messages
rmqblockex= Name of exchange to send block messages
```
Payloads are JSON and map to `NewBlockEvent`, `NewTransactionEvent` in the `NBXplorer.Models` namespace. There is no support in NBXplorer client for message borkers at the current time. You will need to use the `Serializer` in `NBXplorer.Client` to de-serialize the objects or then implement your own JSON de-serializers for the custom types used in the payload.
For configuring serializers you can get crypto code info from `BasicProperties.Headers[CryptoCode]` of RabbitMq messages or `UserProperties[CryptoCode]` of Azure Service Bus messages.
Examples can be found in unit tests.
#### Troubleshooting
If you receive a 401 Unauthorized then your cookie data is not working. Check you are using the current cookie by opening the cookie file again - also check the date/time of the cookie file to ensure it is the latest cookie (generated when you launched NBXplorer).
@ -343,7 +263,7 @@ If you receive a 404 or timeout then Postman cannot see the endpoint
A better documentation is on the way, for now the only documentation is the client API in C# on [nuget](https://www.nuget.org/packages/NBxplorer.Client).
The `ExplorerClient` classes allows you to query unused addresses, and the UTXO of an HD PubKey.
You can take a look at [the tests](https://github.com/dgarage/NBXplorer/blob/master/NBXplorer.Tests/UnitTest1.cs) to see how it works.
You can take a look at [the tests](https://github.com/btcpayserver/NBXplorer/blob/master/NBXplorer.Tests/UnitTest1.cs) to see how it works.
There is a simple use case documented on [Blockchain Programming in C#](https://programmingblockchain.gitbooks.io/programmingblockchain/content/wallet/web-api.html).
@ -376,7 +296,7 @@ Then run the tests.
4. `Electrum protocol` is cumbersome for HD wallets.
5. `Bitcoin Core RPC` is inflexible and difficult to use. It also scales poorly when a wallet has too many addresses or UTXOs.
6. `Bitcoin Core RPC` supports multiple wallets but isn't designed to handle thousands of them. Having too many wallets will not scale.
7. While NBXplorer exposes an [API](https://dgarage.github.io/NBXplorer/), it also allows you to query the data using the most expressive and flexible language designed for this purpose: [SQL](./docs/docs/Postgres-Schema.md).
7. While NBXplorer exposes an [API](https://btcpayserver.github.io/NBXplorer/), it also allows you to query the data using the most expressive and flexible language designed for this purpose: [SQL](./docs/docs/Postgres-Schema.md).
8. Alternative SaaS infrastructure providers depend on third parties, forcing you to compromise your privacy by sharing financial information while relinquishing control over API changes and service level agreements (SLAs).
## Licence

View File

@ -27,18 +27,9 @@ services:
- "39388:39388"
volumes:
- "bitcoin_datadir:/data"
rabbitmq:
image: rabbitmq:3-management
hostname: rabbitmq
ports:
- 4369:4369
- 5671:5671
- 5672:5672
- 15672:15672
postgres:
image: postgres:13
image: postgres:18.1
container_name: nbxplorertests_postgres_1
command: [ "-c", "random_page_cost=1.0", "-c", "shared_preload_libraries=pg_stat_statements" ]
environment:

View File

@ -14,12 +14,6 @@ services:
NBXPLORER_CHAINS: "btc,lbtc"
NBXPLORER_BTCRPCURL: http://bitcoind:43782/
NBXPLORER_BTCNODEENDPOINT: bitcoind:39388
NBXPLORER_RMQHOST: rabbitmq
NBXPLORER_RMQVIRTUAL: /
NBXPLORER_RMQUSER: guest
NBXPLORER_RMQPASS: guest
NBXPLORER_RMQTRANEX: NewTransaction
NBXPLORER_RMQBLOCKEX: NewBlock
NBXPLORER_LBTCRPCURL: http://elementsd-liquid:43783/
NBXPLORER_LBTCNODEENDPOINT: elementsd-liquid:39389
volumes:
@ -28,7 +22,6 @@ services:
- "elementsd_liquid_datadir:/root/.elements"
links:
- bitcoind
- rabbitmq
bitcoind:
restart: always
@ -50,15 +43,6 @@ services:
- "39388:39388"
volumes:
- "bitcoin_datadir:/data"
rabbitmq:
image: rabbitmq:3-management
hostname: rabbitmq
ports:
- 4369:4369
- 5671:5671
- 5672:5672
- 15672:15672
elementsd-liquid:
restart: always

View File

@ -4,7 +4,7 @@ NBXplorer is a multi crypto currency lightweight block explorer.
NBXplorer does not index the whole blockchain, rather, it listens transactions and blocks from a trusted full node and index only addresses and transactions which belongs to a `DerivationScheme` that you decide to track.
This document describes the concepts, while the [API endpoints are documented here](https://dgarage.github.io/NBXplorer/).
This document describes the concepts, while the [API endpoints are documented here](https://btcpayserver.github.io/NBXplorer/).
## Table of content
@ -58,7 +58,7 @@ And there are three different types: `Derivation Schemes`, `Groups` and `Standal
A derivation scheme, also called `derivationStrategy` internally, is a flexible way to define how to generate deterministic addresses for a wallet.
A derivation scheme tracked source's format is `DERIVATIONSCHEME:derivationScheme` (eg. `DERIVATIONSCHEME:xpub1`).
You can create one by calling [Tracking derivation scheme](https://dgarage.github.io/NBXplorer/#tag/Derivations/operation/Track).
You can create one by calling [Tracking derivation scheme](https://btcpayserver.github.io/NBXplorer/#tag/Derivations/operation/Track).
There are two types of derivation schemes:
* [Standard](#standard-derivation-scheme), for simple and standard use cases (single sig, or multi sig)
@ -130,7 +130,7 @@ Where:
wsh(and_v(or_c(pk([973a74ba/48'/1'/0']tprv8gZh1wDxtsw28saCHAXAKGRStbAZpWAzuT13AVv5erS3CxHMNLmmJUFfpUKtvfBzQx5qajkzcHkghNp8kuPknDhMTwxPdzPA29NZQcr17ZT/**),or_c(pk([39bad04c/48'/1'/1']tprv8fkJnxvpWbmwWd8Ep6YdXoYrhp1WNDFLvVJKsioFwatwA2kyZtfpFhSqi84QGb83VXqPHWBiH4xV8rcrHz1WeWtk1gWS1MwJsE3dJWfp1Fj/**),v:older(1000))),pk([f19e9416/48'/1'/2']tprv8fuFjBofiYNzDW3GAyPeoMqDf2QdrQZxozJjAT74CxaeYkQv7cKZvrPLSTuB2Z6qX8nxfBbTQaGCzDpPzaH1jyV9RiRj8xVFmo34hFzsKb8/**)))
```
When passing this derivation scheme to the API via a URL path (like when calling [Tracking derivation scheme](https://dgarage.github.io/NBXplorer/#tag/Derivations/operation/Track)), dont forget to escape it properly.
When passing this derivation scheme to the API via a URL path (like when calling [Tracking derivation scheme](https://btcpayserver.github.io/NBXplorer/#tag/Derivations/operation/Track)), dont forget to escape it properly.
For example, if you need to track this policy, replace:
@ -146,7 +146,7 @@ For example, if you need to track this policy, replace:
| `:` | `%3A` |
| `*` | `%2A` |
The call to [Tracking derivation scheme](https://dgarage.github.io/NBXplorer/#tag/Derivations/operation/Track) would then be:
The call to [Tracking derivation scheme](https://btcpayserver.github.io/NBXplorer/#tag/Derivations/operation/Track) would then be:
```
HTTP POST /v1/cryptos/BTC/derivations/wsh%28and_v%28or_c%28pk%28%5B973a74ba%2F48%27%2F1%27%2F0%27%5Dtprv8gZh1wDxtsw28saCHAXAKGRStbAZpWAzuT13AVv5erS3CxHMNLmmJUFfpUKtvfBzQx5qajkzcHkghNp8kuPknDhMTwxPdzPA29NZQcr17ZT%2F%2A%2A%29%2Cor_c%28pk%28%5B39bad04c%2F48%27%2F1%27%2F1%27%5Dtprv8fkJnxvpWbmwWd8Ep6YdXoYrhp1WNDFLvVJKsioFwatwA2kyZtfpFhSqi84QGb83VXqPHWBiH4xV8rcrHz1WeWtk1gWS1MwJsE3dJWfp1Fj%2F%2A%2A%29%2Cv%3Aolder%281000%29%29%29%2Cpk%28%5Bf19e9416%2F48%27%2F1%27%2F2%27%5Dtprv8fuFjBofiYNzDW3GAyPeoMqDf2QdrQZxozJjAT74CxaeYkQv7cKZvrPLSTuB2Z6qX8nxfBbTQaGCzDpPzaH1jyV9RiRj8xVFmo34hFzsKb8%2F%2A%2A%29%29%29%23adgpkx0s
```
@ -160,11 +160,11 @@ Additionally, specific addresses can be tracked through the group.
Every address attached by a child tracked source will be added to the group, including all related UTXOs and transactions.
A group can have any number of children, and a group can also be a child of another group.
Please note that all the children are returned by [Get a group](https://dgarage.github.io/NBXplorer/#tag/Groups/operation/Get). As such, it is advised not to add too many children to avoid slowing down this call.
Please note that all the children are returned by [Get a group](https://btcpayserver.github.io/NBXplorer/#tag/Groups/operation/Get). As such, it is advised not to add too many children to avoid slowing down this call.
A group tracked source's format is `GROUP:groupid`.
You can create a new group by calling [Create a group](https://dgarage.github.io/NBXplorer/#tag/Groups/operation/Create).
You can create a new group by calling [Create a group](https://btcpayserver.github.io/NBXplorer/#tag/Groups/operation/Create).
### <a name="addresses"></a>Standalone addresses
@ -172,7 +172,7 @@ This refers to a tracked source that monitors a single address. It functions sim
The address tracked source's format is `ADDRESS:bc1...`.
You can create one by calling [Tracking an address](https://dgarage.github.io/NBXplorer/#tag/Legacy/operation/TrackSingleAddress).
You can create one by calling [Tracking an address](https://btcpayserver.github.io/NBXplorer/#tag/Legacy/operation/TrackSingleAddress).
## Authentication

View File

@ -1,6 +1,6 @@
{
"sdk": {
"version": "8.0.0",
"version": "10.0.0",
"rollForward": "latestMinor",
"allowPrerelease": true
}

14
publish-docker.sh Executable file
View File

@ -0,0 +1,14 @@
#!/usr/bin/env bash
set -euo pipefail
suffix="${1-}"
if [[ -n "$suffix" ]]; then
suffix="-$suffix"
fi
ver="$(sed -n 's/.*<Version>\([^<]*\)<.*/\1/p' NBXplorer/NBXplorer.csproj | head -n 1)"
git tag -a "v${ver}${suffix}" -m "${ver}${suffix}"
git checkout master
git push origin "v${ver}${suffix}" --force