DashRadar development thread

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Just looking at the inputs I would guess that the second transaction is more likely to be fraud because the inputs contain only a few different denominations (same denominations that getfreedash pays I assume). The first transaction seems to have more different denominations so it might be an exchange? Just guessing..

I created csv file that can be imported into gephi. The csv files have one line per address. The first column is the address and the rest of the columns are txids that spend from the address.
Here is the csv files:
gephi.csv
gephi_filtered.csv (doesn't contain addresses that have zero outgoing transactions)

When importing the files to gephi, choose the following options:
Separator: Comma
Import as: Adjacency list
Graph Type: Directed

So basically those files should allow visualizing all the "green concentrations". I'm still new to using gephi. I got some promising results using the "Force Atas2" layout.
 
  • Like
Reactions: demo

demo

Well-known Member
Apr 23, 2016
3,114
263
153
Dash Address
XnpT2YQaYpyh7F9twM6EtDMn1TCDCEEgNX
It is very imporant to export the data to html/web, for anyone in the dash community (who is unable to use gephi) to be able to check the integrity of the data, and thus spot and report potential fraud in the getfreedash dividend.

A way to export the gephi files to html/web is this.
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
It is very imporant to export the data to html/web, for anyone in the dash community (who is unable to use gephi) to be able to check the integrity of the data, and thus spot and report potential fraud in the getfreedash dividend.

A way to export the gephi files to html/web is this.
I guess I'm too lazy to do that but here is an image of the transactions that spend outputs from 100 or more of the payout addresses:screenshot_213953.png
 
Last edited:
  • Like
Reactions: demo

demo

Well-known Member
Apr 23, 2016
3,114
263
153
Dash Address
XnpT2YQaYpyh7F9twM6EtDMn1TCDCEEgNX
I guess I'm too lazy to do that but here is an image of the transactions that spend outputs from 100 or more of the payout addresses:
If you calculate the potential fraud of the dividend, your lazyness may cost a lot of money to the Dash community.
If the MNOs want the dividend to flourish, they should also allocate an amount of dash in order for someone to investigate potential frauds in the dividend and in order to built appropriate applications which facilitate fraud detection.
Otherwise the dividend is doomed, and if the dividend is doomed then the dash cryptocurrency is also doomed. Because the only crypto that will survive in the future, will be the one which will provide dividend to all of its members that can proove their individuality somehow.
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
If you calculate the potential fraud of the dividend, your lazyness may cost a lot of money to the Dash community.
If the MNOs want the dividend to flourish, they should also allocate an amount of dash in order for someone to investigate potential frauds in the dividend and in order to built appropriate applications which facilitate fraud detection.
Otherwise the dividend is doomed, and if the dividend is doomed then the dash cryptocurrency is also doomed. Because the only crypto that will survive in the future, will be the one which will provide dividend to all of its members that can proove their individuality somehow.
And what prevents you from doing it?
 

demo

Well-known Member
Apr 23, 2016
3,114
263
153
Dash Address
XnpT2YQaYpyh7F9twM6EtDMn1TCDCEEgNX
And what prevents you from doing it?
I am not as good as you are, in that field.
For example. although I managed to install gephi, I cannot produce the graph from the data, like you did.
Why should I start doing something, when there is someone else who can do it better than me, and faster than me?
I am mediocre in most of the fields, and I know it. Whenever I try to do something, I always discover a better than me.
Instead of trying to re-invent the wheel, I am trying to encourage and help the one who is better than me to built more wheels.
 
Last edited:

demo

Well-known Member
Apr 23, 2016
3,114
263
153
Dash Address
XnpT2YQaYpyh7F9twM6EtDMn1TCDCEEgNX
And what prevents you from doing it?
I try hard, but I still cannot display the data the way you do, in gephi.
Could you please explain me why my graph appears like this?
gephi.jpg
 

demo

Well-known Member
Apr 23, 2016
3,114
263
153
Dash Address
XnpT2YQaYpyh7F9twM6EtDMn1TCDCEEgNX
Have you tried choosing and running "ForceAtas2" in the "Layout" part of "Overview" tab.
I did it and it is much better.
But how can I paint the incoming red and the outgoing green like you do in your dashradar site?

Can we create the superb colored image of dashradar, by using gephi ???







I am lost....o_O:confused:
 
Last edited:

demo

Well-known Member
Apr 23, 2016
3,114
263
153
Dash Address
XnpT2YQaYpyh7F9twM6EtDMn1TCDCEEgNX

This tools seems great in visualizing getfreedash of @Dashmaximalist, but someone should teach us how to use it, or point to the correct youtube video that will help us accomplish our task, without the need of learning all the gephi enviroment!
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
I did it and it is much better.
But how can I paint the incoming red and the outgoing green like you do in your dashradar site?

Can we create the superb colored image of dashradar, by using gephi ???
Looks like colors might be possible using the gdf import format. It should also be possible to add timestamps to input and create an animated timeline. But I have to do other things now..
 
  • Like
Reactions: demo

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Update
  • Blocks are automatically reloaded in recent blocks page (block explorer)
  • Transactions are automatically reloaded in address page (block explorer)
  • Added loading indicator to charts page (charts)
  • Added two new connection types to make writing privatesend related queries easier (data explorer)
    • FIRST_ROUND connection from a "PrivateSend transaction" to a "mixing transaction" (PrivateSend input)
    • MIXING_SOURCE connection from a "mixing transaction" to a "create denominations transaction"
    • This example query shows how to get the number of "create denominations transactions" reachable from each input of a given PrivateSend transaction. (2-8 rounds assumed)
      • Note the huge difference in the anonyminity sets (6 vs 821)! The 6 is caused by certain mixing transactions only having create denominations transactions as an input. As a result there is no path to a 3rd round for that specific PrivateSend input. More information can found in this thread.
      • Edit: fixed error in the query
 
Last edited:
  • Like
Reactions: demo

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
After seeing this Reddit post, I decided to add two new charts to the charts page:
  • Unspent mixed Dash by denomination
  • Mixed unspent transaction output count by denomination
I think I should organize the charts into groups somehow as there is already quite many of them...
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Update
Next I'm planning to add guesstimated wallet pages in the block explorer and change the PrivateSend analysis tool (demo) to use guesstimated wallets as the mixing sources.

If you are a masternode and haven't voted on my proposal yet, then please take a look and vote.

Edit: looks like there is something wrong with the address cluster clustering feature because the example queries no longer work. Investigating.fixed
 
Last edited:
  • Like
Reactions: UdjinM6 and thephez
Apr 9, 2018
44
20
48
Hello @Antti Kaikkonen
Your graph tool is just wonderful! Want to thank you for this.

I have one improvement suggestion
I'm constantly hit these limitations:
"Transactions has X inputs and Y output. Max limit is 200 inputs and outputs combined."
"This address has X transactions. Max limit is 200"
Can these limits be moved to settings so I can lift them when necessary?
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Hello @Antti Kaikkonen
Your graph tool is just wonderful! Want to thank you for this.

I have one improvement suggestion
I'm constantly hit these limitations:
"Transactions has X inputs and Y output. Max limit is 200 inputs and outputs combined."
"This address has X transactions. Max limit is 200"
Can these limits be moved to settings so I can lift them when necessary?
Hi and thanks for the feedback!

I like your idea of making the limit configurable in the settings. Increasing the limit will probably cause massive lag issues as well as a visual mess. But I guess it should be a choice if the user wants to. Maybe I will make the limit configurable and change the message to a "do you want to proceed anyway" question.

In the future I'm planning to move away from svg rendering which should drastically improve the rendering performance. 3D visualization is also planned which should help with more than 200 connections taking so much space. I might also add address clustering support to the graph visualization which should also help with these issues.
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Update:
  • Made the limit of max 200 connections/node configurable in the graph visualization as requested by @InhumanPerfection.
  • Added wallet pages to the block explorer.
    • Addresses are computed using only the multi-input heuristic. I might add other heuristics in the future but doing that increases the risk of generating false positives.
    • Currently just a list of addresses but current balance, balance chart etc is planned for the future.
    • Accessible from the address pages. For example https://dashradar.com/explorer/address/XdsaHNxXYBR4De7Ui59K8PYiqJdrfwUdUM seems to have 28 other associated addresses (which seem to also be masternodes).
  • Upgraded to Angular 6 from Angular 5
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Update:

dashradar address balances.png
 
  • Like
Reactions: InhumanPerfection
Apr 9, 2018
44
20
48
Suggestion for block explorer:
In transaction view: under addresses add "Previous output" and "Redeemed in" (like on chainz.cryptoid.info)
dr-explorer.png
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Suggestion for block explorer:
In transaction view: under addresses add "Previous output" and "Redeemed in" (like on chainz.cryptoid.info)
View attachment 7429
Previous output and redeemed in is already included in the links. "<" on the left side is a link to the previous output and ">" on the right side is a link to the redeeming input. The block numbers are not shown but can be accessed through the previous output and redeeming input links.
 
Apr 9, 2018
44
20
48
Previous output and redeemed in is already included in the links. "<" on the left side is a link to the previous output and ">" on the right side is a link to the redeeming input.
I know. This suggestion is for visualization improvement. Look at this pic. I can immediately visually identify clusters without going through the links.
dr-explorer2.png
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
I know. This suggestion is for visualization improvement. Look at this pic. I can immediately visually identify clusters without going through the links.
View attachment 7430
Ok. I don't think it's necessarily a good idea to display the txids by default. Maybe it could be added as an option or maybe there could be other ways to achieve the same thing. For example it should be possible to make it so that when mouse is hovered over one of the arrows then all of the arrows pointing to the same txid are highlighted.
 
Apr 9, 2018
44
20
48
I'm not sure how to make it to look good... I'm just saying that I'm finding this feature on cryptoid as very useful. Maybe small identicons from txid can be used...
 

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Update
Next I'm planning to add some median charts to the charts page.
 
Last edited:
Apr 9, 2018
44
20
48
And... another suggestion :rolleyes:
In the graph:
1) when mouse over the transaction - on the top left along with txid show the number of inputs, outputs and valueOut.
2) when mouse over the address - on the top left along with address show the number of txApperances, balance, and (if any) unconfirmedTxApperances and unconfirmedBalance
 
  • Like
Reactions: Antti Kaikkonen

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Update

  • Refactored how the blockchain is imported to the Neo4j graph database
    • Mempool (a.k.a unconfirmed transactions) is imported (example query: get mempool transactions)
    • Delay of importing a new block reduced from ~1 minute to less than a second
    • Orphaned blocks are saved as :OrphanedBlock
    • Previously each block was imported in a single very complex query. Now each transaction, txin and txout is a separate query. This slowed down the initial import but made the code much easier to maintain.
    • Required database indexes are now added automatically
  • Rewrote https://github.com/Antti-Kaikkonen/Neo4jReadOnly using express.js
    • Less code
    • Better caching support (HTPP Not Modified). Should result in a better user experience especially on mobile connections
  • Rewrote https://github.com/Antti-Kaikkonen/chartjs-cypher using express.js
    • Less code
    • Better caching support (HTPP Not Modified). Should result in a better user experience especially on mobile connections
  • Charts page: Image width and height is set before the image is loaded to make loading less cluttered.
These changes do not include many visible changes but should make future development easier and enables using the neo4j graph database for things that require real-time data.
 
  • Like
Reactions: UdjinM6

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Update
  • Block explorer
    • Front page changes
      • Added unconfirmed transactions to front page
      • Removed block hash from recent blocks and added block time
    • Address balance chart changes
      • Should load faster but can still take a while if there are 10k+ transactions
      • Better downsampling by using a chartjs plugin. (only if more than 2000 data points)
      • Hidden if there is only 1 data point.
    • Charts page bugfix: date was off by 1 day in some charts.
    • Many bugfixes and performance optimizations
  • Backend (database)
    • Fixed many bugs with how blockchain reorganizations are handled. Hopefully it's working now..
 
  • Like
Reactions: UdjinM6

Antti Kaikkonen

Active Member
Jun 20, 2017
257
167
103
dashradar.com
Dash Address
XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
Blockchain explorer https://dashradar.com/explorer is not moving anymore... :rolleyes:
Oh sorry about that. I just moved to a diffent city so I couln't work for 2 days.. I will fix asap.

Edit: It got stuck at block 905186 which has 4785 transactions. I probably need to change the code to save in a single database transaction one transaction at a time instead of one block at a time. Meanwhile I'm trying to give the database more memory to work with to get it rolling again.
 
Last edited:
  • Like
Reactions: UdjinM6