DashRadar development thread

UdjinM6

Official Dash Dev
Dash Core Team
Moderator
May 20, 2014
3,638
3,538
1,183
np :) you got hit with an improvised community driven stress test which brought down some nodes in our insight cluster for some time too (not to mention poloniex and shapeshift :D)
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
The issue was that determining the transaction type (privatesend, create denomination, mixing, ..) was too slow. I optimized the code and it seems to be working fine now. Just processed a 1.91 MB block without an issue. The block explorer should catch up withing an hour or two.
 
  • Like
Reactions: UdjinM6

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Update
  • Block explorer
    • InstantSend icon is now displayed for InstantSend transactions in the block explorer
    • Added a summary of the last 24 hours to the front page of the block explorer
      • Number of transactions
      • Number of InstantSend transactions
      • Number of PrivateSend transactions
      • Hashrate
    • Changed the recent blocks table to also display date instead of only time, because viewing old blocks is possible.
    • Added PrivateSend analysis page. Link available from the transaction page of a PrivateSend transaction.
      • Example 1, Example 2
      • Currently I'm running the analysis on my own computer. When I'm running it, the analysis should be available a few minutes after the PrivateSend transaction is confirmed, but I won't be able to run it 24/7 currently.
      • Todo: Add explanation, add basic transaction info, create analysis where PrivateSend transactions are tracked back to guesstimated wallets instead of create denominations transactions (group together create denominations transactions belonging in the same wallet)
  • Backend
    • Updated to Dash Core v0.12.3
    • Updated to the latest insight-api. Also wrote a tutorial on how to install it.
    • InstantSend locks are stored in the database
    • Had to do some performance optimizations thanks to the recent stress test by the dash community
      • Now able to handle ~2mb blocks with ease
      • Initial sync time should also be greatly reduced
Source code of the PrivateSend analysis program: https://github.com/Antti-Kaikkonen/PrivateSendAnalysis.

As recommended to me by Dash Watch, I also added an analytics software matomo to the site. It's open source and self hosted (unlike something like google analytics), so no data is provided to third parties. I may, however, share some statistics here and with the Dash Watch team. So far only 60 unique visitors since I installed it in July 11th.

There is currently an issue with the search functionality of the block explorer because of this bug.
 
  • Like
Reactions: UdjinM6

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Search functionality is now again working properly in the block explorer. Also updated the privatesend analysis page with loading indicators, basic transaction details and visual improvements.

Currently working on changing the privatesend analysis by combining create denominations transactions originating from the same wallet. Soon the privatesend analysis page will show guesstimated wallets (based on the multi-input heuristic) instead of create denominations transactions.

I also want to inform everyone that I will keep working on the ongoing proposal in August so that I can finish the remaining proposal objectives.
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Update:
Changed the PrivateSend analysis page to use guesstimated wallets as mixing sources. New analysis is available in https://dashradar.com/explorer/privatesend/[txid]. Old analysis is still available in https://dashradar.com/explorer/privatesend-old/[txid]. The new analysis typically has less mixing sources and higher percentages because it combines create denominations transactions belonging in the same wallet. To get a list of example analysis pages you can run this query. It fetches privatesend transactions between blocks 850000 and 855000 because the old analysis isn't available for recent transactions.

I want to emphasize that it's not possible for someone to determine the number of rounds used by just analyzing the blockchain. Different inputs can even use a different number if you interrupt the mixing process. So the results are more like leads rather than something conclusive.
 
Last edited:

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
I found a bug which caused mixing transactions to affect the guesstimated wallets feature although mixing transactions should have been ignored.

I'm going to remove all of the guesstimated wallets and then recompute them. This will cause a few hours of downtime. Edit: Done

I will also need to recompute the PrivateSend analysis results which were using the guesstimated wallets feature.
 
Last edited:

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
I found another bug where a trasaction included in an orphaned block re-appearing in the mempool caused the transaction to be duplicated. This has so far happened to only 1 transaction but it already scewed the average transaction fee chart for August 8th. Currently reprocessing the data and I will edit this post when fixed. Edit: done
 
Last edited:

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
  • Like
Reactions: UdjinM6

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Here is the number of unique visitors for July, August and September
dashradar unique visutors.png

I'm using an open-source self hosted analytics software called Matomo, so that your information is not leaked to Google or other third parties.
 
  • Like
Reactions: solarguy

jeffh

Member
May 8, 2017
108
45
78
@colofi this was some testing done by @DeltaEngine for a stress test a few months ago and then more recently in preparation for a network stress-test (theoretical max tx/s to beat any production blockchain record) scheduled to happen around the 12.4 launch.
 
  • Like
Reactions: colofi

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Nodes page added to dashradar: https://dashradar.com/nodes

There is still some things that could be added/improved:
  • Make the map size dynamic depending on screen resolution
  • Add an option to filter active/inactive nodes in the "All nodes" tab
  • (Individual node page by clicking the ip address)
  • (Add historical charts for country/version/organization distribution)
  • (Add link to collateral txid/address for masternodes)
  • (Show voting data for masternodes)
 
  • Like
Reactions: InhumanPerfection

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Changed the default url (https://dashradar.com) from the graph visualization to a simple landing page containing links to explorer, graph, charts and nodes pages. In the future I might add more content to the landing page to make it more like a dashboard.

Also upgraded from Angular 6 to Angular 7.
 

Gaby_64

New Member
Sep 21, 2016
16
3
3
27
how long does it take for a new node to appear in the nodes list?

I have been running a non masternode node for 2 days now.
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
how long does it take for a new node to appear in the nodes list?

I have been running a non masternode node for 2 days now.
It only lists nodes that accept new connections.

If you are running the node at home then it might be behind a NAT router. In that case you should forward port 9999 from the router to the LAN IP address of the computer running the node.

It's also possible that there could be something wrong with the configuration of the node (dash.conf).

You can check if your node is accepting connections with the node checker found in http://nodes.blakeanderson.ca/networks/dash.
 

qwizzie

Well-known Member
Aug 6, 2014
1,609
764
183
Any chance you can include charts for mining hashrate & mining difficulty & mining pools spread?
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
I recently updated the website to use angular universal server side rendering. This means that the first page the user is loading is rendered to HTML on the server and after that it will continue to function as a single page application. It can make the initial page load slightly faster but the main benefit is that it makes the website crawlable by search engines.

Also moved the navigation from the top to the side. This solution should work better for mobile users and allows fitting more navigation elements in the future.

Any chance you can include charts for mining hashrate & mining difficulty & mining pools spread?
I think that is a good idea but I'm not sure when I will be able to do that.
 

qwizzie

Well-known Member
Aug 6, 2014
1,609
764
183
I recently updated the website to use angular universal server side rendering. This means that the first page the user is loading is rendered to HTML on the server and after that it will continue to function as a single page application. It can make the initial page load slightly faster but the main benefit is that it makes the website crawlable by search engines.

Also moved the navigation from the top to the side. This solution should work better for mobile users and allows fitting more navigation elements in the future.


I think that is a good idea but I'm not sure when I will be able to do that.
No pressure, no hurry.

Good to hear about your website improvements.
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
@qwizzie It's currently possible to get generate those charts using the data explorer but it can be quite slow for long time periods. Adding them to the charts page would require storing a running total of the difficulty so that average difficulty of a day can be computed very quickly. Hashrate by pool would require even more changes.

Here are some examples using the data explorer:
You can modify the queries to get more data but if you make the period too long then the query will not finish because the is a time limit (1 minute if I remember correctly).
 

qwizzie

Well-known Member
Aug 6, 2014
1,609
764
183
@qwizzie It's currently possible to get generate those charts using the data explorer but it can be quite slow for long time periods. Adding them to the charts page would require storing a running total of the difficulty so that average difficulty of a day can be computed very quickly. Hashrate by pool would require even more changes.

Here are some examples using the data explorer:
You can modify the queries to get more data but if you make the period too long then the query will not finish because the is a time limit (1 minute if I remember correctly).
i like the line chart of the daily hashrate last 3 months, it would fit well with the other line charts i think.
And maybe think of ways implementing mining difficulty and pools for a later date ? Or skip if its getting to be too much of a hassle
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
I have had some issues with the Neo4j database that this project is using to store the blockchain:
  • Poor write performance
  • Issues with the cost planner sometimes generating highly inefficient query execution plans
  • Community edition lacking some useful features
  • Other issues
I'm probably not going to build any new features on top of Neo4j and I might start replacing some or all of the current features using other databases.

I'm currently building an improved version of the address clustering a.k.a guesstimated wallets feature. In the new version users will be able to browse an ordered list of the wallet's transactions. Wallet balance after each transaction will also be available as well as a historical balance chart.
 

qwizzie

Well-known Member
Aug 6, 2014
1,609
764
183
Quick question about two charts :

Transactions per day
Mixing transactions per day

Is the "mixing transactions per day" data included or excluded from the "transactions per day" chart ?
I ask this because i wonder if a lot of mixing could inflate the transactions per day chart.
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Quick question about two charts :

Transactions per day
Mixing transactions per day

Is the "mixing transactions per day" data included or excluded from the "transactions per day" chart ?
I ask this because i wonder if a lot of mixing could inflate the transactions per day chart.
Transactions per day includes all transactions including mixing transactions, privatesend transactions and create denominations transactions.
 
  • Like
Reactions: vazaki

tiger-HZX

New Member
Apr 28, 2019
9
0
1
20
Oh sorry about that. I just moved to a diffent city so I couln't work for 2 days.. I will fix asap.

Edit: It got stuck at block 905186 which has 4785 transactions. I probably need to change the code to save in a single database transaction one transaction at a time instead of one block at a time. Meanwhile I'm trying to give the database more memory to work with to get it rolling again.
I found that in block 206980,there are two mixing trx, since their trx fee is 0, but dashradar recognize them as create denomination trx, Is it a bug? If I am wrong, is there something about mixing trx I don't know?
 

thephez

Member
Dash Core Team
Jan 23, 2016
113
49
78
I found that in block 206980,there are two mixing trx, since their trx fee is 0, but dashradar recognize them as create denomination trx, Is it a bug? If I am wrong, is there something about mixing trx I don't know?
In 2015 when that tx happened it was possible to send any tx with a fee of 0. Also, the txs I looked at (mentioned here) cannot be mixing txs. One of them does not have an equal number of inputs/outputs. The other cannot be for multiple reasons including - 1. too many inputs/outputs, and 2. Not all inputs/outputs have the same value (unless the PS rules were different back then (@UdjinM6 ?).
 

tiger-HZX

New Member
Apr 28, 2019
9
0
1
20
In 2015 when that tx happened it was possible to send any tx with a fee of 0. Also, the txs I looked at (mentioned here) cannot be mixing txs. One of them does not have an equal number of inputs/outputs. The other cannot be for multiple reasons including - 1. too many inputs/outputs, and 2. Not all inputs/outputs have the same value (unless the PS rules were different back then (@UdjinM6 ?).
Thanks for your reply, but there are still are something I did not understand. why a PrivateSend tx can not tcontains too many inputs/outputs, if there is a rule, what is the limit of inputs, I remember there are at most 200 inputs/outputs in a tx, but I didn't hear about the inputs/outputs limits of PS tx.

In the follow links(please replace 。 with .) , we can see someone said that inputs/outputs in dash's PrivateSend tx can be different denominations once,
https://forum。bitrad。io/index。php?thread/953-what-is-darksend-and-how-to-use-it/ , maybe it is wrong, maybe it is right.

There is another msg, the 5 PS denominations was introduced in protocal version 72013, it means there may be another rules about PrivateSend before 72013, I searchd a lot and did not find any information about it.

I am still confused, if you know, please tell me a detail about it. @thephez @UdjinM6
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
I found that in block 206980,there are two mixing trx, since their trx fee is 0, but dashradar recognize them as create denomination trx, Is it a bug? If I am wrong, is there something about mixing trx I don't know?
Dashradar does not currently recognize old mixing transactions where multiple different denominations in the same mixing transaction was allowed.