codablock
Active member
Hello,
I was already thinking about working for Dash since some time, and now got some time between 2 projects (I'm a freelancer) that I could use for this. In the end, I'd like to deliver something valuable and later create a proposal to fund this work.
I've talked with two core devs on Slack about where I could support and one suggestion was to look into performance improvements of dashd and backporting perf related changes from bitcoin 0.13+.
I started to read through the performance related changes in bitcoin and one thing that I immediately assumed to be worth of backporting is the "assumed valid blocks" feature from the 0.14 release, with an optional addition (which I describe later). Other changes, especially network code related, might be hard to backport as I read that Dash had some major refactoring in the network code.
Is anyone already working on this or a comparable task? Or are there maybe any documents describing the state of backporting tasks?
Regarding "assumed valid blocks": The basic idea is to specify a block hash that is known/trusted to be valid and part of the longest chain. This knowledge lifts the node from full block/transaction validation when doing the initial block download for each block that was mined before the specified block, decreasing load on the CPU and thus reducing IBD time. Bitcoin has also added a default block hash which was confirmed by multiple developers to be valid and part of the longest chain.
As of https://bitcoincore.org/en/2017/03/08/release-0.14.0/#ibd, this change combined with another improvement (Memory-sharing between mempool and UTXO DB cache) reduced the IBD time to around 1/6 of the time needed in previous versions. I'm not sure how much impact the 2 individual improvements have on it's own, but I may put some research into the second one later if needed.
To change the default hash, users may specify the hash on startup, but the problem now is "how to figure out which is a valid block?". Users can check other nodes that they are in possession of (and thus fully trust) or use 3rd party services like block explorers to find a valid block. Everyone not willing to take this hurdle will probably just stick with the default hash, making them dependent on frequent releases to have recent hashes and thus accept slow syncing for each block that comes after the default hash.
And that's where I think the MN network could come into play. Instead of manually specifying the hash (and providing a default), clients/nodes could ask a random MN for a valid block that is part of the longest chain. The MN would check the latest block and go back in time by a few blocks (to make sure the block is really hardened and no reorg/orphanage may happen) and return the hash. After this hash is received by the node/client, it would ask multiple other random MNs if this block is REALLY a good one. Only if all MNs give a positive response, the client/node would start to use the block as an "assumed valid block".
If I'm not missing anything, this should be as safe as what the bitcoin core developers did to manually choose the default hash, but being completely automatic and thus resulting in always recent "assumed valid blocks" used by clients/nodes.
I'm not sure about the upgrade process needed for such a change, as I didn't dig through versioning and upgrade processes in Bitcoin/Dash until now. I'd assume that some kind of protocol upgrade would be required for the MNs before clients could start asking for valid blocks, as all MNs must be able to verify the chosen block when they (the MNs) are chosen randomly. Client side upgrade should be backwards compatible however. I'm happy for suggestions and pointers here
Disclaimer: I'm in the middle of digging through code and documentation as I'm quite new to blockchain/bitcoin/Dash. I assume that I already got a good understanding of how the overall tech works, but of course there may still be details still missing in my head. So please, tell me if I'm wrong with anything that I may have misunderstood or missed
I was already thinking about working for Dash since some time, and now got some time between 2 projects (I'm a freelancer) that I could use for this. In the end, I'd like to deliver something valuable and later create a proposal to fund this work.
I've talked with two core devs on Slack about where I could support and one suggestion was to look into performance improvements of dashd and backporting perf related changes from bitcoin 0.13+.
I started to read through the performance related changes in bitcoin and one thing that I immediately assumed to be worth of backporting is the "assumed valid blocks" feature from the 0.14 release, with an optional addition (which I describe later). Other changes, especially network code related, might be hard to backport as I read that Dash had some major refactoring in the network code.
Is anyone already working on this or a comparable task? Or are there maybe any documents describing the state of backporting tasks?
Regarding "assumed valid blocks": The basic idea is to specify a block hash that is known/trusted to be valid and part of the longest chain. This knowledge lifts the node from full block/transaction validation when doing the initial block download for each block that was mined before the specified block, decreasing load on the CPU and thus reducing IBD time. Bitcoin has also added a default block hash which was confirmed by multiple developers to be valid and part of the longest chain.
As of https://bitcoincore.org/en/2017/03/08/release-0.14.0/#ibd, this change combined with another improvement (Memory-sharing between mempool and UTXO DB cache) reduced the IBD time to around 1/6 of the time needed in previous versions. I'm not sure how much impact the 2 individual improvements have on it's own, but I may put some research into the second one later if needed.
To change the default hash, users may specify the hash on startup, but the problem now is "how to figure out which is a valid block?". Users can check other nodes that they are in possession of (and thus fully trust) or use 3rd party services like block explorers to find a valid block. Everyone not willing to take this hurdle will probably just stick with the default hash, making them dependent on frequent releases to have recent hashes and thus accept slow syncing for each block that comes after the default hash.
And that's where I think the MN network could come into play. Instead of manually specifying the hash (and providing a default), clients/nodes could ask a random MN for a valid block that is part of the longest chain. The MN would check the latest block and go back in time by a few blocks (to make sure the block is really hardened and no reorg/orphanage may happen) and return the hash. After this hash is received by the node/client, it would ask multiple other random MNs if this block is REALLY a good one. Only if all MNs give a positive response, the client/node would start to use the block as an "assumed valid block".
If I'm not missing anything, this should be as safe as what the bitcoin core developers did to manually choose the default hash, but being completely automatic and thus resulting in always recent "assumed valid blocks" used by clients/nodes.
I'm not sure about the upgrade process needed for such a change, as I didn't dig through versioning and upgrade processes in Bitcoin/Dash until now. I'd assume that some kind of protocol upgrade would be required for the MNs before clients could start asking for valid blocks, as all MNs must be able to verify the chosen block when they (the MNs) are chosen randomly. Client side upgrade should be backwards compatible however. I'm happy for suggestions and pointers here
Disclaimer: I'm in the middle of digging through code and documentation as I'm quite new to blockchain/bitcoin/Dash. I assume that I already got a good understanding of how the overall tech works, but of course there may still be details still missing in my head. So please, tell me if I'm wrong with anything that I may have misunderstood or missed
