Welcome to the Dash Forum!

Please sign up to discuss the most innovative cryptocurrency!

DashRadar development thread

Discussion in 'Projects' started by Antti Kaikkonen, Jun 20, 2017.

  1. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Just looking at the inputs I would guess that the second transaction is more likely to be fraud because the inputs contain only a few different denominations (same denominations that getfreedash pays I assume). The first transaction seems to have more different denominations so it might be an exchange? Just guessing..

    I created csv file that can be imported into gephi. The csv files have one line per address. The first column is the address and the rest of the columns are txids that spend from the address.
    Here is the csv files:
    gephi.csv
    gephi_filtered.csv (doesn't contain addresses that have zero outgoing transactions)

    When importing the files to gephi, choose the following options:
    Separator: Comma
    Import as: Adjacency list
    Graph Type: Directed

    So basically those files should allow visualizing all the "green concentrations". I'm still new to using gephi. I got some promising results using the "Force Atas2" layout.
     
    • Like Like x 1
  2. demo

    demo Well-known Member

    Joined:
    Apr 23, 2016
    Messages:
    3,116
    Likes Received:
    262
    Trophy Points:
    153
    Dash Address:
    XnpT2YQaYpyh7F9twM6EtDMn1TCDCEEgNX
    It is very imporant to export the data to html/web, for anyone in the dash community (who is unable to use gephi) to be able to check the integrity of the data, and thus spot and report potential fraud in the getfreedash dividend.

    A way to export the gephi files to html/web is this.
     
  3. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    I guess I'm too lazy to do that but here is an image of the transactions that spend outputs from 100 or more of the payout addresses: screenshot_213953.png
     
    #64 Antti Kaikkonen, Mar 18, 2018
    Last edited: Mar 18, 2018
    • Like Like x 1
  4. demo

    demo Well-known Member

    Joined:
    Apr 23, 2016
    Messages:
    3,116
    Likes Received:
    262
    Trophy Points:
    153
    Dash Address:
    XnpT2YQaYpyh7F9twM6EtDMn1TCDCEEgNX
    If you calculate the potential fraud of the dividend, your lazyness may cost a lot of money to the Dash community.
    If the MNOs want the dividend to flourish, they should also allocate an amount of dash in order for someone to investigate potential frauds in the dividend and in order to built appropriate applications which facilitate fraud detection.
    Otherwise the dividend is doomed, and if the dividend is doomed then the dash cryptocurrency is also doomed. Because the only crypto that will survive in the future, will be the one which will provide dividend to all of its members that can proove their individuality somehow.
     
  5. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    And what prevents you from doing it?
     
  6. demo

    demo Well-known Member

    Joined:
    Apr 23, 2016
    Messages:
    3,116
    Likes Received:
    262
    Trophy Points:
    153
    Dash Address:
    XnpT2YQaYpyh7F9twM6EtDMn1TCDCEEgNX
    I am not as good as you are, in that field.
    For example. although I managed to install gephi, I cannot produce the graph from the data, like you did.
    Why should I start doing something, when there is someone else who can do it better than me, and faster than me?
    I am mediocre in most of the fields, and I know it. Whenever I try to do something, I always discover a better than me.
    Instead of trying to re-invent the wheel, I am trying to encourage and help the one who is better than me to built more wheels.
     
    #67 demo, Mar 18, 2018
    Last edited: Mar 18, 2018
  7. demo

    demo Well-known Member

    Joined:
    Apr 23, 2016
    Messages:
    3,116
    Likes Received:
    262
    Trophy Points:
    153
    Dash Address:
    XnpT2YQaYpyh7F9twM6EtDMn1TCDCEEgNX
    I try hard, but I still cannot display the data the way you do, in gephi.
    Could you please explain me why my graph appears like this?
    gephi.jpg
     
  8. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Have you tried choosing and running "ForceAtas2" in the "Layout" part of "Overview" tab.
     
    • Like Like x 1
  9. demo

    demo Well-known Member

    Joined:
    Apr 23, 2016
    Messages:
    3,116
    Likes Received:
    262
    Trophy Points:
    153
    Dash Address:
    XnpT2YQaYpyh7F9twM6EtDMn1TCDCEEgNX
    I did it and it is much better.
    But how can I paint the incoming red and the outgoing green like you do in your dashradar site?

    Can we create the superb colored image of dashradar, by using gephi ???

    [​IMG]








    I am lost....o_O:confused:
     
    #70 demo, Mar 21, 2018
    Last edited: Mar 21, 2018
  10. demo

    demo Well-known Member

    Joined:
    Apr 23, 2016
    Messages:
    3,116
    Likes Received:
    262
    Trophy Points:
    153
    Dash Address:
    XnpT2YQaYpyh7F9twM6EtDMn1TCDCEEgNX


    This tools seems great in visualizing getfreedash of @Dashmaximalist, but someone should teach us how to use it, or point to the correct youtube video that will help us accomplish our task, without the need of learning all the gephi enviroment!
     
  11. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Looks like colors might be possible using the gdf import format. It should also be possible to add timestamps to input and create an animated timeline. But I have to do other things now..
     
    • Like Like x 1
  12. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Update
    • Blocks are automatically reloaded in recent blocks page (block explorer)
    • Transactions are automatically reloaded in address page (block explorer)
    • Added loading indicator to charts page (charts)
    • Added two new connection types to make writing privatesend related queries easier (data explorer)
      • FIRST_ROUND connection from a "PrivateSend transaction" to a "mixing transaction" (PrivateSend input)
      • MIXING_SOURCE connection from a "mixing transaction" to a "create denominations transaction"
      • This example query shows how to get the number of "create denominations transactions" reachable from each input of a given PrivateSend transaction. (2-8 rounds assumed)
        • Note the huge difference in the anonyminity sets (6 vs 821)! The 6 is caused by certain mixing transactions only having create denominations transactions as an input. As a result there is no path to a 3rd round for that specific PrivateSend input. More information can found in this thread.
        • Edit: fixed error in the query
     
    #73 Antti Kaikkonen, Apr 4, 2018
    Last edited: Apr 4, 2018
    • Like Like x 1
  13. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    After seeing this Reddit post, I decided to add two new charts to the charts page:
    • Unspent mixed Dash by denomination
    • Mixed unspent transaction output count by denomination
    I think I should organize the charts into groups somehow as there is already quite many of them...
     
    • Like Like x 3
    • Informative Informative x 1
  14. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Update
    Next I'm planning to add guesstimated wallet pages in the block explorer and change the PrivateSend analysis tool (demo) to use guesstimated wallets as the mixing sources.

    If you are a masternode and haven't voted on my proposal yet, then please take a look and vote.

    Edit: looks like there is something wrong with the address cluster clustering feature because the example queries no longer work. Investigating.fixed
     
    #75 Antti Kaikkonen, May 14, 2018
    Last edited: May 15, 2018
    • Like Like x 2
  15. InhumanPerfection

    Joined:
    Apr 9, 2018
    Messages:
    43
    Likes Received:
    19
    Trophy Points:
    48
    Hello @Antti Kaikkonen
    Your graph tool is just wonderful! Want to thank you for this.

    I have one improvement suggestion
    I'm constantly hit these limitations:
    "Transactions has X inputs and Y output. Max limit is 200 inputs and outputs combined."
    "This address has X transactions. Max limit is 200"
    Can these limits be moved to settings so I can lift them when necessary?
     
    • Agree Agree x 1
  16. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Hi and thanks for the feedback!

    I like your idea of making the limit configurable in the settings. Increasing the limit will probably cause massive lag issues as well as a visual mess. But I guess it should be a choice if the user wants to. Maybe I will make the limit configurable and change the message to a "do you want to proceed anyway" question.

    In the future I'm planning to move away from svg rendering which should drastically improve the rendering performance. 3D visualization is also planned which should help with more than 200 connections taking so much space. I might also add address clustering support to the graph visualization which should also help with these issues.
     
    • Like Like x 2
  17. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Update:
    • Made the limit of max 200 connections/node configurable in the graph visualization as requested by @InhumanPerfection.
    • Added wallet pages to the block explorer.
      • Addresses are computed using only the multi-input heuristic. I might add other heuristics in the future but doing that increases the risk of generating false positives.
      • Currently just a list of addresses but current balance, balance chart etc is planned for the future.
      • Accessible from the address pages. For example https://dashradar.com/explorer/address/XdsaHNxXYBR4De7Ui59K8PYiqJdrfwUdUM seems to have 28 other associated addresses (which seem to also be masternodes).
    • Upgraded to Angular 6 from Angular 5
     
    • Like Like x 3
    • Friendly Friendly x 1
  18. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Update:

    dashradar address balances.png
     
    • Like Like x 1
  19. InhumanPerfection

    Joined:
    Apr 9, 2018
    Messages:
    43
    Likes Received:
    19
    Trophy Points:
    48
    Suggestion for block explorer:
    In transaction view: under addresses add "Previous output" and "Redeemed in" (like on chainz.cryptoid.info)
    dr-explorer.png
     
  20. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Previous output and redeemed in is already included in the links. "<" on the left side is a link to the previous output and ">" on the right side is a link to the redeeming input. The block numbers are not shown but can be accessed through the previous output and redeeming input links.
     
  21. InhumanPerfection

    Joined:
    Apr 9, 2018
    Messages:
    43
    Likes Received:
    19
    Trophy Points:
    48
    I know. This suggestion is for visualization improvement. Look at this pic. I can immediately visually identify clusters without going through the links.
    dr-explorer2.png
     
  22. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Ok. I don't think it's necessarily a good idea to display the txids by default. Maybe it could be added as an option or maybe there could be other ways to achieve the same thing. For example it should be possible to make it so that when mouse is hovered over one of the arrows then all of the arrows pointing to the same txid are highlighted.
     
  23. InhumanPerfection

    Joined:
    Apr 9, 2018
    Messages:
    43
    Likes Received:
    19
    Trophy Points:
    48
    I'm not sure how to make it to look good... I'm just saying that I'm finding this feature on cryptoid as very useful. Maybe small identicons from txid can be used...
     
  24. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Update
    Next I'm planning to add some median charts to the charts page.
     
    #85 Antti Kaikkonen, May 23, 2018
    Last edited: May 23, 2018
  25. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Update:

    Revamped the charts page with static preview images as suggested by @UdjinM6 (on page 1). The old charts page is still available here: https://dashradar.com/charts/old

    Please let me know if you like the new version better or worse and let me know if you would like any specific charts added.
     
    #86 Antti Kaikkonen, May 26, 2018
    Last edited: May 26, 2018
  26. InhumanPerfection

    Joined:
    Apr 9, 2018
    Messages:
    43
    Likes Received:
    19
    Trophy Points:
    48
    And... another suggestion :rolleyes:
    In the graph:
    1) when mouse over the transaction - on the top left along with txid show the number of inputs, outputs and valueOut.
    2) when mouse over the address - on the top left along with address show the number of txApperances, balance, and (if any) unconfirmedTxApperances and unconfirmedBalance
     
    • Like Like x 1
  27. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Update

    • Refactored how the blockchain is imported to the Neo4j graph database
      • Mempool (a.k.a unconfirmed transactions) is imported (example query: get mempool transactions)
      • Delay of importing a new block reduced from ~1 minute to less than a second
      • Orphaned blocks are saved as :OrphanedBlock
      • Previously each block was imported in a single very complex query. Now each transaction, txin and txout is a separate query. This slowed down the initial import but made the code much easier to maintain.
      • Required database indexes are now added automatically
    • Rewrote https://github.com/Antti-Kaikkonen/Neo4jReadOnly using express.js
      • Less code
      • Better caching support (HTPP Not Modified). Should result in a better user experience especially on mobile connections
    • Rewrote https://github.com/Antti-Kaikkonen/chartjs-cypher using express.js
      • Less code
      • Better caching support (HTPP Not Modified). Should result in a better user experience especially on mobile connections
    • Charts page: Image width and height is set before the image is loaded to make loading less cluttered.
    These changes do not include many visible changes but should make future development easier and enables using the neo4j graph database for things that require real-time data.
     
    • Like Like x 1
    • Informative Informative x 1
  28. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    250
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Update
    • Block explorer
      • Front page changes
        • Added unconfirmed transactions to front page
        • Removed block hash from recent blocks and added block time
      • Address balance chart changes
        • Should load faster but can still take a while if there are 10k+ transactions
        • Better downsampling by using a chartjs plugin. (only if more than 2000 data points)
        • Hidden if there is only 1 data point.
      • Charts page bugfix: date was off by 1 day in some charts.
      • Many bugfixes and performance optimizations
    • Backend (database)
      • Fixed many bugs with how blockchain reorganizations are handled. Hopefully it's working now..
     
    • Like Like x 1
    • Informative Informative x 1
  29. UdjinM6

    UdjinM6 Official Dash Dev
    Dash Core Team Moderator

    Joined:
    May 20, 2014
    Messages:
    3,631
    Likes Received:
    3,532
    Trophy Points:
    1,183

Share This Page