Overview of Mainnet Stress Test and Dash Core v0.14.0.3 Release

Status
Not open for further replies.

tungfa

Administrator
Dash Core Team
Moderator
Foundation Member
Masternode Owner/Operator
Apr 9, 2014
8,898
6,725
1,283
By @Pasta
https://blog.dash.org/overview-of-mainnet-stress-test-and-dash-core-v0-14-0-3-release-db6b978da42c

Last week there was an abnormally high network load on the Dash network consisting of around a million 1 input 1 output transactions with a fee just higher than 1 duff per byte. After contacting all likely community members who could have executed this and finding that none of them were involved, it appears that the artificial load on the Dash network was either a stress test by someone outside of the Dash developer community or was an attempted attack.

Over the course of 2 days, there were two large spikes in transactions:

  • Wed, 2019–08–07 0000–0830 (8.5 hrs) — ~650k txs
  • Wed/Thu, 2019–08–07/08 2300–0230 (3.5 hrs) — ~300k txs
The Dash mempool spiked to 46 MB around 11:30AM on the 7th of August before being cleared in about 3 hours. Before that, at around 3AM on the 7th of August, the mempool spiked to around 17 MB and was sustained for around 4 hours. After analyzing the Dash blockchain as well as MN logs and their PoSe score, there have been multiple interesting discoveries.

1. Some pools are capped at 1MB blocks
This is likely due to custom implementations. Mining pools will change this naturally once usage approaches 1MB in an effort to capture more fees and be more profitable.

2. Some pools seems not to be emptying the mempool and are producing near empty blocks
We are in active communication with these pools in order to figure out what we can do such that the pools include transactions while not including what appears to be spam (1 input, 1 output, very low fee).

3. Some Masternodes crashed due to not having enough storage space
This was a result of having to store the IS Lock for every transaction. This resulted in nodes running on minimal storage (around 15–20 GB) not having enough space to store all of the IS Locks. Previously, locks were removed after around 7 days. After 0.14.0.3 ("); background-size: 1px 1px; background-position: 0px calc(1em + 1px);">PR#3048), IS Locks will be removed as soon as they are confirmed via a ChainLock. This should reduce the size of the `llmq` directory significantly over time.

4. Some Masternodes banned other Masternodes under high load when on the brink of a new quorum becoming active
In order to fix this, we need to detect this situation and then re-verify failed sigs with the previous active set. This is being implemented by "); background-size: 1px 1px; background-position: 0px calc(1em + 1px);">PR#3052 and will also be released in 0.14.0.3. This issue resulted in some blocks not receiving a ChainLock and some transactions not receiving IS Locks.

5. Some user transactions did not get included in a block rapidly since they used the default fee
The fee selection algorithm and UI in the Dash Core wallet is being updated in 0.14.1 thanks to the backporting of Bitcoin 0.15. The fee selection algorithms in the mobile wallets are being reviewed as well.

We are exploring ways that we can try to minimize the load that spam and an attacker can have on the network. There are many potential ideas, however, each of these ideas have their benefits and problems and we are still investigating what we can do in order to ensure that users can send funds at a sub cent fee while also preventing spam and attacks on the Dash network.

In response to the discoveries outlined above, 0.14.0.3 is being released. The release includes various bug fixes and improvements. The upgrade is strongly recommended for all Masternodes and is also recommended for all users, exchanges, partners and full node operators.

Notable Changes in Dash Core v0.14.0.3
  • Database space usage improvements
  • As described above in point 3
  • DKG and LLMQ signing failures fixed
  • As described above in point 4
  • Signed binaries for Windows
  • MacOS: disable AppNap during sync and mixing
  • New RPC command: quorum memberof <proTxHash>
  • More information about number of InstantSend locks
Please see the full "); background-size: 1px 1px; background-position: 0px calc(1em + 1px);">changelog for more details.
 

t0dd

Active Member
Mar 21, 2016
150
132
103
keybase.io
Dash Address
XyxQq4qgp9B53QWQgSqSxJb4xddhzk5Zhh
RPMs built for Fedora 29 and 30 and running on my MNs... So... Available for update if you have been using these in the past...
https://github.com/taw00/dashcore-rpm

UPDATE: I appreciate the thumbsup and such from a few folks. But I posted an incorrect link that lead to a entire different project!!!! You all fail! ;) Ha! Anyway... the link has been corrected.
 
Last edited:

splawik21

Grizzled Member
Dash Core Team
Moderator
Foundation Member
Dash Support Group
Apr 8, 2014
1,923
1,280
1,283
Great to have these kind of stress tests so we can improve the Dash code.
Top work Dash devs!
Updated all before this post appeared
 

camosoul

Grizzled Member
Sep 19, 2014
2,263
1,130
1,183
By @Pasta...it appears that the artificial load on the Dash network was either a stress test by someone outside of the Dash developer community or was an attempted attack.
Did you ever figure it out?
By @PastaWe are exploring ways that we can try to minimize the load that spam and an attacker can have on the network. There are many potential ideas, however, each of these ideas have their benefits and problems and we are still investigating what we can do in order to ensure that users can send funds at a sub cent fee while also preventing spam and attacks on the Dash network.
Can you name some actual examples of these "many potential ideas" you have?

It pains me to say it, but BTC has terrible fees for a reason... Fees were the only mechanism preventing spam, and I pointed out ages ago that a replacement spam prevention mechanism would be needed back when you cut the fees... Nobody even thought about it until now? And, how do you plan to pay the bills? With the low volume of TXes, paying DCG and keeping the MNs incentivized seems a problem made orders of magnitude worse, not better... In your race to be maximum overprogressive, you seem to have lost your grip on reality... Is it even possible to carry TX volume sufficient to pay the bills? That'd be 100MB blocks, full to the brim. MNs aren't paid enough to handle that. Running an MN is already not worth it from an ROI perspective. 3 years from now? I don't think DASH is going to exist 3 years from now... Or, even 1...
 
Status
Not open for further replies.