Welcome to the Dash Forum!

Please sign up to discuss the most innovative cryptocurrency!

Interactive blockchain visualization tool development thread

Discussion in 'Projects' started by Antti Kaikkonen, Jun 20, 2017.

  1. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    dashradar example.png
    I'm planning to post updates here. All feedback and suggestions appreciated.

    https://dashradar.com

    You can right click addresses and transactions to expand or delete them. Click blue info icon for more information.


    TODO
    • New block explorer module
      • Most recent blocks

      • Block page

      • Address page

      • Transaction page
    • Chars page changes
      • Pop out individual charts
        • To cypher editor

        • To page with just the chart

        • To image for embedding in forums etc
      • New chart: Median fee per byte (Duffs)

      • New chart: Average transaction size by transaction type (Regular, Privatesend, Mixing 10.0, Mixing 1.0, Mixing 0.1, Mixing 0.01, Create denominations)

      • New chart: Median transaction size by transaction type (Regular, Privatesend, Mixing 10.0, Mixing 1.0, Mixing 0.1, Mixing 0.01, Create denominations)
    • Transaction graph visualizer changes
      • Implement undo/redo functionality

      • Add “Create collateral inputs” transaction type

      • Add “Privatesend collateral payment” transaction type

      • Eventually fetch data from the neo4j graph database for increased speed.
    • Neo4j Database changes
      • Automatically update database with latest blockchain data happens every ~5min
        • Make it update in real time including unconfirmed transactions
      • Add addresses to database
      • Add “Create collateral inputs” transaction type
      • Add “Privatesend collateral payment” transaction type
      • Precompute daily medians
        • Transaction fee per byte (Duffs)

        • Transaction size for each transaction type (Regular, Privatesend, Mixing 10.0, Mixing 1.0, Mixing 0.1, Mixing 0.01, Create denominations)
    • Transaction graph visualizer also in 3D
      • Should be better for visualizing huge graphs, especially nodes with lots of connections

      • WASD navigation
     
    #1 Antti Kaikkonen, Jun 20, 2017
    Last edited: Feb 7, 2018
    • Like Like x 7
  2. tungfa

    tungfa Administrator
    Dash Core Team Foundation Member Masternode Owner/Operator Moderator

    Joined:
    Apr 9, 2014
    Messages:
    8,233
    Likes Received:
    6,451
    Trophy Points:
    1,283
    whow
    keep mixing i would say :rolleyes:
    (Edited link in OP)
     
  3. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    Updated!
    • Live mode. Click the red "Start live" button and new transactions should appear soon.
    • Added icons for
      • PrivateSend create denominations transaction
      • PrivateSend transaction
     
    • Like Like x 2
  4. UdjinM6

    UdjinM6 Official Dash Dev
    Dash Core Team Moderator

    Joined:
    May 20, 2014
    Messages:
    3,586
    Likes Received:
    3,509
    Trophy Points:
    1,183
    Live mode is super cool, nice! :)
     
    • Like Like x 2
  5. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    Small update:
    • You can now use https://dashradar.com (encrypted connection). TODO: redirect http to https
    • Changed layout of icons
    • Added favicon (just a dash icon for now)
     
    #5 Antti Kaikkonen, Jul 14, 2017
    Last edited: Jul 15, 2017
    • Like Like x 3
  6. stan.distortion

    stan.distortion Active Member

    Joined:
    Oct 30, 2014
    Messages:
    812
    Likes Received:
    490
    Trophy Points:
    133
  7. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    Just removed caching from addresses because it was causing certain connections to be missing in the live mode. I think live mode looks better now.
     
  8. stan.distortion

    stan.distortion Active Member

    Joined:
    Oct 30, 2014
    Messages:
    812
    Likes Received:
    490
    Trophy Points:
    133
    You've got both TX and IX? Played around with one of these before, didn't get IX working but always wanted to see what PrivateSend looked like on it :)
     
  9. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    I tried to add InstantSend support but it isn't working in the live mode yet.

    I think I know why though. The instantsend lock messages are usually broadcasted after the transaction so I would need to listen to the lock messages. I will let you know when I get it fixed. Also I can't fix InstantSend for old transactions because the information isn't stored in the blockchain I think.
     
    • Like Like x 1
  10. stan.distortion

    stan.distortion Active Member

    Joined:
    Oct 30, 2014
    Messages:
    812
    Likes Received:
    490
    Trophy Points:
    133
    Sorry, I wasn't clear there, I meant both inputs and outputs, I'd tried it with the insight websocket API and it only broadcast outputs so everything was disjointed, PrivateSend transactions where obvious but they weren't connected. I see now you've got both and there's a nice nest of tangled up furballs forming as PrivateSend does its thing :) Really nice seeing that, it doesn't leave a whole lot of room for doubt that PrivateSend transactions are untraceable.

    Good luck showing InstantSend, looking forward to watching this develop :)
     
  11. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    Yeah at some point i deployed a broken version with that problem and it seems you happened use it at the same time :)

    Edit: Current problem is that it sometimes displays a transaction with wrong icon (dash logo). I don't know why but I'll try to find out..
     
  12. stan.distortion

    stan.distortion Active Member

    Joined:
    Oct 30, 2014
    Messages:
    812
    Likes Received:
    490
    Trophy Points:
    133
    Yeah, I'd seen that here too, no idea what might be causing it though :/ Another I get is with poor network connectivity, for some reason the missed transactions when network traffic is stalled seem to come together as a single transaction when it's re-established. I was getting the same before, iirc it was from insight but I didn't get as far as seeing whether they could be detected and queried and I doubt it's much of a problem for most.
     
  13. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    Update:
    • Bugfixes (switched from webgl graphics to svg graphics because webgl implementation of VivaGraph library was buggy for images)
    • Added icon for coinbase transaction
    Please let me know if you find any bugs.

    Edit: Another update: You can now change settings by clicking the settings icon in top right corner.
     
    #13 Antti Kaikkonen, Jul 17, 2017
    Last edited: Jul 17, 2017
    • Like Like x 1
  14. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    I made a tool for analyzing/visualizing private send transactions. It currently finds the all the shortest paths to create denominations transactions behind a set number of mixing transactions. It's not ready for public release but you can preview the results by importing the JSON data in one of the links below.

    Exactly 3 rounds
    Exactly 4 rounds (slow to import, might lagg)
    Both links are for the privatesend transaction 59d51690d4b56ddbf1e393fa8d3a49bcfc3247f270f36be3b6ee411802666cba

    Other updates:
    • Fixed exporting to json
    • PrivateSend Create Denomination transactions that also created PrivateSend collateral inputs were not recognized as Create Denomination transactions.
    Edit: improved stability and speed of importing JSON data.
     
    #14 Antti Kaikkonen, Jul 26, 2017
    Last edited: Jul 26, 2017
    • Like Like x 1
  15. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    Updated
    • Mixing transactions are now different color for different denominations.
    • Added list views for currently displayed transactions and addresses
      • Addresses and transactions can be selected and selected nodes ares highlighted in the graph.
        • TODO: Add ability to select nodes in the graph by right clicking. Maybe add rectangular selection tool.
      • Addresses and transactions can be sorted by their properties by clicking the column header.
        • TODO: Add more properties
      • Can perform group actions on selected addresses/transactions (delete all, expand all, etc)
     
    • Like Like x 1
  16. Chuck Williams

    Chuck Williams Official Dash Dev
    Core Developer

    Joined:
    Feb 10, 2017
    Messages:
    61
    Likes Received:
    64
    Trophy Points:
    58
    I love this thing! Well done, sir or madam.
     
  17. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    Thanks. Just updated it with edge arrows to make it more intuitive.
     
    • Friendly Friendly x 1
  18. alexblack

    alexblack New Member

    Joined:
    Jul 25, 2017
    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    1
  19. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    Sorry but I don't understand what you are referring to? What you mean by "it"?
     
  20. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    Status update:
    I'm currently developing a backend project to replace or at least complement the insight-api-dash that I'm currently using. I'm currently importing the dash blockchain to a neo4j graph database. The reason I'm doing this is that it will hopefully allow me to develop some high performance analysis tools. I'm particularly interested in tracking PrivateSend transactions to determine how much anonymity is there depending on the number of rounds, number of inputs, different denominations etc.
     
    • Winner Winner x 3
    • Like Like x 2
  21. solarguy

    solarguy Active Member

    Joined:
    Mar 15, 2017
    Messages:
    569
    Likes Received:
    239
    Trophy Points:
    113
    Is there a donate button???

    I would also love the option of sound effects at each transaction with a choice of perhaps 4-6 sounds:

    wind chimes
    simple bell
    old fashioned cash register
    bubbles popping


    I have noticed that if I have Dashradar on a tab in Chrome, it doesn't run if it's not the active tab. Is that a Chrome thing or a DashRadar thing? No big deal either way.

    Cool tool.

    Carry on.
     
  22. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    Haven't had much time to develop this but here is something that you can play with if you want: http://dashradar.com:7474/browser/. With that you can query the dash blockchain using the neo4j cypher query language.

    Here is some example queries that you can try executing:

    Count the number of transactions between blocks 600000 and 700000:
    Code:
    MATCH (tx:Transaction)-[:INCLUDED_IN]->(b:Block) where (b.height > 600000 AND b.height < 700000) RETURN count(tx);

    Count the number of unspent outputs:
    Code:
    MATCH (n:TransactionOutput) where NOT (n)-[:SPENT_IN]->(:TransactionInput) return count(n);

    Return the distribution of how many previous mixing transactions mixing transactions have for a given denomination. Only consider transactions that happened after block 600000. Denominations(pstype=denom): (7=0.01), (6=0.1), (5=1.0), (4=10.0) .
    Code:
    MATCH (tx:Transaction {pstype:7})-[conn:PREVIOUS_ROUND]->(:Transaction)-[:INCLUDED_IN]->(b:Block) where b.height>600000 with tx, count(conn) as connection_count with connection_count, count(tx) as tx_count return connection_count, tx_count ORDER BY connection_count;

    To get an idea of the database schema you can click the icon on the top left. As you can see the database does not currently contain information about addresses. Just blocks and the transaction graph.

    The database is currently not automatically updated. To find the last block height in the database you can run
    Code:
    MATCH (b:Block) RETURN b.height ORDER BY b.height DESC LIMIT 1;
     
    #22 Antti Kaikkonen, Sep 30, 2017
    Last edited: Sep 30, 2017
    • Like Like x 2
  23. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    Update
    • Added sound effects to live mode as suggested by @solarguy
      • Sound can be disabled in settings
    • Added "max transactions in live mode" to settings to prevent lag when running live mode for a long time
      • Oldest transactions are removed when this limit is reached
    • Added more stuff to information page/tab
      • Used technologies
      • Link to this thread
      • Contact email
      • Donation address
     
    • Like Like x 2
    • Winner Winner x 1
  24. solarguy

    solarguy Active Member

    Joined:
    Mar 15, 2017
    Messages:
    569
    Likes Received:
    239
    Trophy Points:
    113
    Excellent...
    And I noticed a donate button....incoming....

    finest regards,

    carry on to victory!
     
    • Friendly Friendly x 1
  25. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    Update

    Check out https://dashradar.com/cypher. It's related to my previous post where I was talking about neo4j and cypher queries.

    In that page you can write (or use someone else's) cypher queries and instantly display the results in a line, bar or pie chart. You can share your query with others by copying the link next to the execute button. The example on the page shows how to get average privatesend transaction amount in dash for each day.

    Currently first column of the results is used for x axis in line chart and labels in bar and pie charts. Second column is used for y axis / label values. If the first column name is 'date' then the column values are assumed to be unix timestamps and are converted to dates in the charts.

    It's still a work in progress. In the future I'm planning to make the individual results (table, line chart, bar chart, ...) shareable with a similar link. I might also make sharing images to charts possible with a similar link (better for embedding in pages, forum posts, etc). Also better support for multiple data series isplanned.

    I moved the neo4j browser from http://dashradar.com:7474/browser/ to https://1.dashradar.com:7473/browser/. You can view that page to view the database chema. The database is read-only, it's not automatically updated and there is a 60 second execution limit for the queries to avoid someone accidentally jamming the server.
     
    #25 Antti Kaikkonen, Oct 23, 2017
    Last edited: Oct 23, 2017
    • Like Like x 3
  26. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    #26 Antti Kaikkonen, Oct 23, 2017
    Last edited: Oct 24, 2017
    • Like Like x 1
  27. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    I've noticed that querying certain stuff from the database is quite slow and the queries can timeout (over 60sec). As a result I've decided to add hourly data points that represent the total value of things since the first block.
    It would contain for example the following things:
    • timestamp
    • total number of blocks
    • total number of transactions
    • total number of inputs
    • total number of outputs
    • total output volume (in duffs)
    • total fees volume (in duffs)
    • total block size (in bytes)
    • total transaction size (in bytes)
    In order to calculate to calculate a value (e.g. total fees) in a time period (e.g. day) you could just find the starting hour and ending hour and then subtract the value of starting hour from the value of the ending hour. This should hopefully allow certain things to be queried over 100x faster.

    No ETA on this update. It's just a plan.
     
    • Like Like x 1
  28. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    Update
    • Added cumulative data to the neo4j database
    • Added nodes with 'BlockChainTotals' label. For each of these nodes there exists a block with the same height and time properties
      • BlockChainTotals
        • height
        • time
        • tx_count
        • input_count
        • output_count
        • total_block_size
        • total_transaction_size
        • total_generated_sat
        • total_output_volume_sat
        • privatesend_tx_count
        • privatesend_mixing_10_0_count
        • privatesend_mixing_1_0_count
        • privatesend_mixing_0_1_count
        • privatesend_mixing_0_01_count
    • All properties except height and time are cumulated values since the genesis block.
    • Now it is also possible to generate image link from the charts
    Below are some examples how to use BlockChainTotals to generate graphs.

    Historical unspent output count
    [​IMG]

    Transactions per day
    [​IMG]
    Transactions per day (2017)
    [​IMG]


    Coins generated per day
    [​IMG]

    There is probably still many bugs. It currently only works well for time series data where first column in result must be called 'date' and represent epoch timestamp. Second column column should be a number. Third column (optional) is a series label that can be used for displaying multi series charts. I will add example with 3 columns later.
     
    #28 Antti Kaikkonen, Nov 19, 2017
    Last edited: Nov 21, 2017
    • Like Like x 1
  29. Antti Kaikkonen

    Joined:
    Jun 20, 2017
    Messages:
    72
    Likes Received:
    47
    Trophy Points:
    58
    For some reason the embedded images don't seem to load. Here is the same images as links:
    Historical unspent output count
    Transactions per day
    Coins generated per day

    Edit: It seems that the embedded images are not loading because it takes a little bit too long to generate them and the image proxy on this forum doesn't like that. When I limited transactions per day to 2017 it works.
     
    #29 Antti Kaikkonen, Nov 19, 2017
    Last edited: Nov 19, 2017
  30. UdjinM6

    UdjinM6 Official Dash Dev
    Dash Core Team Moderator

    Joined:
    May 20, 2014
    Messages:
    3,586
    Likes Received:
    3,509
    Trophy Points:
    1,183
    Nice! I created few graphs based on your examples (mostly a shameless copy-paste with few tweaks :oops:) and it's very interesting to see some trends.
    On a side note, I think most common/popular requests deserve a dedicated page/website like https://blockchain.info/charts with static previews :rolleyes:
     

Share This Page