Welcome to the Dash Forum!

Please sign up to discuss the most innovative cryptocurrency!

DashRadar development thread

Discussion in 'Projects' started by Antti Kaikkonen, Jun 20, 2017.

  1. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Thanks. Let me know if you need help creating a specific chart. And yes I'm planning to create a page with the most interesting charts. When the feature is more ready I'll probably create a landing page with links to 1) The interactive graph explorer 2) The cypher module 3) Regular table based block explorer(TODO). The cypher module would have pre-made charts but there would a link to the editor for each of them.
     
    • Like Like x 2
  2. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    #32 Antti Kaikkonen, Nov 22, 2017
    Last edited: Nov 22, 2017
    • Like Like x 2
  3. UdjinM6

    UdjinM6 Official Dash Dev
    Dash Core Team Moderator

    Joined:
    May 20, 2014
    Messages:
    3,631
    Likes Received:
    3,532
    Trophy Points:
    1,183
    Yay! I can replace 5 bookmarks with a single one now! :)
    Thanks!

    PS. not to mention that this give a much better overview of the mixing process ofc ;)
     
  4. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Update
    • Added a collection of daily charts
      • Start date and end date can be changed
      • 7 day and 30 day simple moving averages (SMA) can be used
    • Added navbar for navigating between charts and the transaction graph visualizer
    • Added toggle fullscreen button to transaction visualizer and and individual charts
    • Neo4j database changes
      • Added "Day" nodes that are linked to last "BlockChainTotals" node of each day.
        • Makes querying daily information much faster. Also makes many queries shorter.
        • Example in the default query
      • Changed "total_generated_sat" of BlockChainTotals to "total_block_rewards_sat" because it also includes fees.
      • Added "total_fees_sat" to BlockChainTotals
     
    • Like Like x 1
  5. UdjinM6

    UdjinM6 Official Dash Dev
    Dash Core Team Moderator

    Joined:
    May 20, 2014
    Messages:
    3,631
    Likes Received:
    3,532
    Trophy Points:
    1,183
    Awesome! :)

    Few suggestions:
    - "fees per kb (Dash)" is 0.000001234323432 or say 7.9234234345345e-7 (just some random examples but you get the idea) and it's not human friendly imo :) IMO it would be much better to have "fees per byte (duff)" like sat/byte in Bitcoin (should be integers, because you can't divide duffs or satoshis into smaller units);
    - also "for fees per kb" - it's not clear is it avg or median, would be nice to have both if possible;
    - would be nice to see avg/median for number of ins/outs for all/ps/non-ps txes - the last graph shows avg for all txes, so it's mostly a ps/non-ps ratio which is interesting too but says nothing about actual trends for various types of txes.

    Thanks :)

    EDIT: also gray background and gray font is smth to tweak imo, it's hard to read the text...
     
    #35 UdjinM6, Dec 15, 2017
    Last edited: Dec 15, 2017
  6. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Thanks for suggestions. I will definitely change fee chart to show in bytes, maybe also duffs.

    Everything is currently using mean but I will think about adding medians. I might need to precompute daily medians to the neo4j database as median can't be quickly computed from the current format.

    I just updated with a version that should fix fullscreen issues with other browsers than firefox.
     
    • Like Like x 1
  7. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Just changed "Fees per kB (Dash)" to "Average fee per byte (Duffs)". It still has decimals because it's an average.
     
    • Like Like x 1
  8. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    • Like Like x 1
  9. fresh calendar

    fresh calendar New Member

    Joined:
    Dec 18, 2017
    Messages:
    4
    Likes Received:
    0
    Trophy Points:
    1
    Looks pretty good.
     
  10. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
  11. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    The Neo4j database is now automatically updated every 5 minutes so the charts are now near real time.

    It doesn't currently handle blockchain reorganizations correctly which might cause some inaccuracies in the data. This should happen quite infrequently but I will try to fix the issue.
     
  12. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Update
    • Block explorer module (beta)
      • https://dashradar.com/explorer
      • TODO: Automatically load new blocks in recent blocks page
      • TODO: Automatically load new transactions in address page
      • TODO: Highlight current address in transaction list of address page
        • Thinking it could be similar to blocktrail but if you think that some other format would be better then please let know.
        TODO: Add ability to search txid, block hash, block height, address
      • TODO: Add historical balance chart to address page
    • Charts page visual improvements
      • Changed background color, added margin etc
    • Graph page right click bug fix
      • Right click menu position was messed up after I had added navigation bar
    • Page title is dynamically updated according to page being viewed
    All feedback and suggestions welcome.
     
    #42 Antti Kaikkonen, Jan 10, 2018
    Last edited: Feb 7, 2018
    • Like Like x 1
  13. UdjinM6

    UdjinM6 Official Dash Dev
    Dash Core Team Moderator

    Joined:
    May 20, 2014
    Messages:
    3,631
    Likes Received:
    3,532
    Trophy Points:
    1,183
    I like how it shows tx styles e.g.
    - PS tx https://dashradar.com/explorer/tx/74b66a0dac36c085e298f5187d790a54683214821abed4e2a7025c8903578299
    - mixing txes even have different colors depending on denom!
    -- 0.01 https://dashradar.com/explorer/tx/7c0a0c65bfa06735fdf003c63f39c2124b1c132ce9e1380452511d375a3130df
    -- 0.1 https://dashradar.com/explorer/tx/91bc76beefaa882c33f7bde0833449a80e7e4897540fdefd05b8b0511abfa78f

    Awesome!

    Fee is a bit crazy though - "0.0007400000000075124" o_O We have only 8 decimal places and _technically_ fee is ("out" - "in"), so it should never have more than 8 decimal places too.

    Also it's not clear how to get older blocks. I can only see 6 latest blocks on the index page, no pagination or search.

    But anyway, good job! :)
     
  14. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Thanks for the feedback!
    The transaction images are the same that are used in the transaction graph visualizer. Transaction fee issue will be easy to fix. I think I will add both pagination to recent blocks and a search bar.
     
    • Like Like x 1
  15. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Update
    • Added addresses to the neo4j database. Now it's possible to create for example address based balance charts. Here is an example of a random address I picked from a block explorer: example.
      • This kind of chart will be added to address pages in the explorer (edit: done)
      • Size of the database grew from 7.6GB to 39GB.
    • The issue with displaying wrong fee in the explorer has been fixed
    • Added pagination to recent blocks page
      • TODO: Add page number to url
     
    #45 Antti Kaikkonen, Jan 18, 2018
    Last edited: Feb 7, 2018
    • Like Like x 1
  16. Dash-Boarder

    Dash-Boarder New Member

    Joined:
    Dec 27, 2017
    Messages:
    19
    Likes Received:
    1
    Trophy Points:
    3
    Very nicely done! I am working on a Treasury proposal to do something for exchange data and sentiment data. http://dash-board.info More of a global dashboard of what is going on in the DASH world - one stop location for all big data and more. No plans to integrate transaction related data that you are modeling.
     
  17. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Update (several updates but I haven't posted them all here)
    • Address page (block explorer)
      • Current address is highlighted, removed link from current address
      • Address balance chart has loading indicator
      • Dramatically increased loading time of address balance chart (should load reasonably fast even for addresses with ~30k transactions, like XvhExSNNr97U1ZenWFjJmgD8wc7v88ZUF7)
      • Fixed address balance chart for addresses with only incoming transactions
    • Block explorer has search field in the navigation bar
      • Search for address, txid, block hash, height
    • Added page numbers to url where pagination is used
    Next I will probably change links to insight.dash.org in the transaction graph visualizer to dashradar.com/explorer. Maybe it could just open a window inside the same page.

    I think I will also add dash amounts next to the arrows in the graph visualization. Maybe even address/txid next to the nodes although it might look messy. There would be option to hide these in the settings.

    I also want to add balance and change to transaction list of the address page of the block explorer. Each transaction would show balance after the transaction and balance change compared to the previous balance. Positive change should be in green and negative change should be in red with a - sign.
     
    • Like Like x 1
  18. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Update
    • Added values next to arrows in graph visualization
      • Can be hidden in settings
    • Moved links from insight.dash.org to dashradar,com/explorer in the graph visualization
    • Added downsampling to address balance chart if there is more than 10000 data points to render. (reduces lagg)
     
    • Like Like x 2
  19. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Update
    • Transaction graph visualization performance improvements
      • Displaying dash values next to arrows is now disabled by default and can be enabled in settings
      • Text elements related to dash values are not included in the svg when the feature is disabled in settings. (previously those were just hidden)
      • Addresses and transactions use svg image element instead of a rectangle filled with image pattern
        • This made a huge difference in performance on my android tablet with chrome browser
        • Selected addresses and transactions now have their own image while previously it was easy to just draw the perimeter of the rectangle.
      • Used images are now much smaller and more optimized.
    • Fixed a bug that caused dash values to appear for new transactions even when displaying values was disabled in settings
    The transaction graph visualizer is still quite laggy on my android tablet computer but at least it doesn't as easily crash the whole browser anymore or the whole tablet like it did to my friend. I have learned that Scalable Vector Graphics (SVG) is not well suited for rendering dynamic things with lots of elements. To fix the performance issues I should completely rewrite the rendering to use html canvas element. I'm planning to do it in the future probably using three.js library which should also allow me to make a 3D version.
     
  20. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    I have also been thinking about adding a visualization like BitConeView to DashRadar. Unfortunately the paper is not free unless you are for example in an university that has paid for it. http://www.bitconeview.info also doesn't seem to work but there is a slideshare that might give you an idea about it. I have also been thinking about implementing a tree visualization like this. Also if you have other visualization ideas then please let me know.
     
    • Like Like x 1
  21. thephez

    thephez Member
    Dash Core Team

    Joined:
    Jan 23, 2016
    Messages:
    109
    Likes Received:
    46
    Trophy Points:
    78
    I have used the blockchain.info tree visualization a number of times and found it to be helpful for certain things. A couple other ideas:
    • Could you add InstantSend Transactions to the Transactions chart (or as a separate chart like the PrivateSend transactions one)? I think it would be great to have some stats on our distinguishing features.
    • The block rewards per day chart scale is skewed by superblocks (and also can be confusing if you don't know about superblocks :)). Don't know if there is a way you could show them distinctly in a way that would be more clear? For instance, with superblocks charted as a bar instead of a line on their own axis or something.
    Just a few thoughts. Great site!
     
    • Like Like x 1
  22. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    Thanks for the suggestions. I haven't included any InstantSend charts so far because the information isn't stored in the blockchain and I have on purpose avoided having to maintain a separate database that can't be reconstructed from the blockchain. It's possible that I will add it in the future along with other information that isn't stored in the blockchain like masternode statistics, orphaned blocks, average time to get first confirmation etc..

    It's definitely possible to make separate bar chart for superblock rewards and other one for regular block rewards (excluding superblocks). One option could be to add some kind "exclude superblocks" button. I will think about it.
     
    • Like Like x 1
  23. jeffh

    jeffh Member

    Joined:
    May 8, 2017
    Messages:
    108
    Likes Received:
    45
    Trophy Points:
    78
    I would think that to exclude superblock rewards, it would be as simple as excluding outliers (aka rewards above a certain threshold that could never otherwise exist).
     
  24. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    DashRadar source code is now available via GitHub.

    Running instructions will be provided later in the readme files or wiki.
     
    • Like Like x 1
  25. demo

    demo Well-known Member

    Joined:
    Apr 23, 2016
    Messages:
    3,114
    Likes Received:
    262
    Trophy Points:
    153
    Dash Address:
    XnpT2YQaYpyh7F9twM6EtDMn1TCDCEEgNX
    There is a list of wallets that received dividend from getfreedash project of @Dashmaximalist.
    Here is the list: https://demodun.github.io/dividendwatch/p25.html

    Is it possible to apply to them your graph ? We want to see whether they spend the dividend they received or not, and where. The star-like presentation of your graph can help a lot in order to spot the points where the (initially dispersed) dividend concentrates. So we may spot potential fraud that way.

    We would like something like this

     
    #55 demo, Mar 18, 2018
    Last edited: Mar 18, 2018
  26. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    That is a lot of addresses and it will cause too much lag to import them all at once.

    I created this json containing a couple hundred of the first addresses. You can import it by clicking the green + icon on the right and choosing "Import from JSON".

    After the import is finished simply expand the addresses. This can be done with right click or ctrl+left click shortcut. After expanding all the addresses it can be observed that for example quite often many addresses are used as input in the same transaction. This most likely means that the addresses are controlled by the same person.


    Something that could really help with this kind of analysis is address clustering. Similar to the "guesstimated wallet" feature on chainz.cryptoid.info. I'm planning to implement it in the future but it might take a while.

    Edit: One thing that might be usefull is selecting all addresses after the import. This way if you expand more addresses you will be able to distinguish them from the imported addresses. You can select addresses by clicking the Dash icon on the right.
     
    #56 Antti Kaikkonen, Mar 18, 2018
    Last edited: Mar 18, 2018
    • Like Like x 1
  27. demo

    demo Well-known Member

    Joined:
    Apr 23, 2016
    Messages:
    3,114
    Likes Received:
    262
    Trophy Points:
    153
    Dash Address:
    XnpT2YQaYpyh7F9twM6EtDMn1TCDCEEgNX
    I did what you said, and there are two kind of concentrations.
    The Red ones, which seem to be the legitimate payments batch jobs of @Dashmaximalist.
    And the Green ones, which seem to be either a potential fraud or a merchant.

    radar.jpg


    @Dashmaximalist should have a look at it ASAP!!!
     
    #57 demo, Mar 18, 2018
    Last edited: Mar 18, 2018
  28. Antti Kaikkonen

    Antti Kaikkonen Active Member

    Joined:
    Jun 20, 2017
    Messages:
    256
    Likes Received:
    164
    Trophy Points:
    103
    Dash Address:
    XnZdwT1w2kGeH6RujwoyJ7BBNrukdyTBRB
    It looks like you didn't fully expand all the addresses. Actually, I just realized that there is an easier way to expand all the addresses. After importing simply click the Dash icon on the right, select all addresses and click "EXPAND SELECTED".

    Final result should look like this:
    asd1.png


    I agree with your conclusions except that I would replace merchant with an exchange. Because who would withdraw directly to a merchant address? Also could be someone collecting dashes for all their family members etc and then sharing the fiat money with them..
     
    #58 Antti Kaikkonen, Mar 18, 2018
    Last edited: Mar 18, 2018
    • Agree Agree x 1
  29. demo

    demo Well-known Member

    Joined:
    Apr 23, 2016
    Messages:
    3,114
    Likes Received:
    262
    Trophy Points:
    153
    Dash Address:
    XnpT2YQaYpyh7F9twM6EtDMn1TCDCEEgNX
    You are right, they can be either merchants, or family members , or exchanges, with the last being the most probable.

    The conclusion is that your chart is a superb tool for analyzing the dividend (especially if you manage to expand it in order to support a higher number of wallets), and @Dashmaximalist should definitely use it.
     

Share This Page