Ongoing DDoS attack on masternode network

Figlmüller

Member
Sep 2, 2014
85
45
58
Vienna, Austria
i tried this command, lets see whats gonna happen thx for reply.

update; after i tried this command my masternode status at dashninja turned to unknown masternode. i think ip tables good choise :)
Dashninja is also affected by the attack and may not display your MN status at all. Currently, all their nodes are "not responding".
 

tungfa

Administrator
Dash Core Team
Moderator
Foundation Member
Masternode Owner/Operator
Apr 9, 2014
8,895
6,723
1,283
Dashninja is also affected by the attack and may not display your MN status at all. Currently, all their nodes are "not responding".
Wallet sais
 

Sapereaude

Well-known Member
Foundation Member
Apr 30, 2014
191
235
203
For those less technologically savvey there are step by step instructions below for Ubuntu 16.04. If you followed TAO's set up guide this will work for you.

**************
Enter root and enter the following commands

******* First Remove ufw
sudo ufw disable
sudo apt-get -y remove ufw
sudo apt-get -y purge ufw

****** Now install persistant ip tables and say yes when the purple screen appears
apt-get install -y iptables-persistent
invoke-rc.d netfilter-persistent save
service netfilter-persistent stop
service netfilter-persistent start

***** Now remove the old iptables file and paste in the new rules
rm /etc/iptables/rules.v4
joe /etc/iptables/rules.v4

************** Now paste in these rules and save, Note- change port 22 if you moved ssh to another port.
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:LOGNDROP - [0:0]
:OUTPUT ACCEPT [0:0]
#
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m conntrack --ctstate INVALID -j DROP
-A INPUT -p udp -j REJECT --reject-with icmp-port-unreachable
-A INPUT -p tcp -j REJECT --reject-with tcp-reset
-A INPUT -j REJECT --reject-with icmp-proto-unreachable
-A INPUT -p tcp -m tcp --dport 9999 -j ACCEPT
# some tcp ddos
-A INPUT -i eth0 -p tcp -f -m tcp -j DROP
-A INPUT -i eth0 -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,SYN,RST,ACK -j DROP
-A INPUT -i eth0 -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j DROP
-A INPUT -i eth0 -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP
-A INPUT -i eth0 -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,SYN,RST,PSH,ACK,URG -j DROP
-A INPUT -i eth0 -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j DROP
-A INPUT -i eth0 -p tcp -m tcp --dport 0 -j DROP
#
-A INPUT -i eth0 -p tcp -m tcp --dport 9998 -j REJECT --reject-with tcp-reset
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 --tcp-flags FIN,SYN,RST,ACK SYN -m connlimit --connlimit-above 8 --connlimit-mask 24 --connlimit-saddr -j DROP
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 --tcp-flags FIN,SYN,RST,ACK SYN -m connlimit --connlimit-above 2 --connlimit-mask 32 --connlimit-saddr -j DROP
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 -m conntrack --ctstate NEW -m recent --set --name DEFAULT --mask 255.255.255.255 --rsource
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 -m conntrack --ctstate NEW -m recent --update --seconds 30 --hitcount 3 --name DEFAULT --mask 255.255.255.255 --rsource -j DROP
#
-A INPUT -i eth0 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW -m recent --set --name DEFAULT --mask 255.255.255.255 --rsource
-A INPUT -i eth0 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW -m recent --update --seconds 30 --hitcount 3 --name DEFAULT --mask 255.255.255.255 --rsource -j DROP
#
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 -j ACCEPT
#
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
#
-A INPUT -j LOGNDROP
-A LOGNDROP -p tcp -m limit --limit 32/min -j LOG --log-prefix "Denied TCP: " --log-level 7
-A LOGNDROP -p udp -m limit --limit 32/min -j LOG --log-prefix "Denied UDP: " --log-level 7
-A LOGNDROP -p icmp -m limit --limit 32/min -j LOG --log-prefix "Denied ICMP: " --log-level 7
-A LOGNDROP -j DROP
COMMIT
#Remember to leave an extra space at the bottom

******* and save, then to check it is working
iptables -L

Viola :D

Edit- Credit to chaeplin for the far superior rules
 
Last edited:

nightowl

Member
Dec 30, 2015
67
105
73
HI Guys

Attached a very nice firewall script that will help you out. It already blocks a few bad behaving IP addresses (I got those bad behaving ip's from Dash's debug log). It limits connections to your SSH and Dash ports (allows only 2 per second). It looks out for bad SYN packets (only some of them, not all).

Copy the file to your server, save in root directly "dashfirewall.sh" (remove the txt at the end, I was forced to add the .txt at the end because the forum won't allow me to upload files withtout an extension)

The execute

chmod a+x dashfirewall.sh

Then run it with

./dashfirewall.sh

If you're on Ubuntu/Debain, you can also add add "/root/dashfirewall.sh" to /etc/rc.local before "exit 0" to ensure the firewall runs when your server starts.

Hope this helps!
 

Attachments

chaeplin

Active Member
Core Developer
Mar 29, 2014
749
356
133
here new rule
- limit concurrent connection 2 per ip, 8 per c class
- limit 3 syn per ip with in 30 sec
- limit some tcp ddos
- limit ssh conn

Code:
# /etc/default/iptables
## Firewall configuration written by system-config-firewall
## Manual customization of this file is not recommended.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
#
#-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
# allow established
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
# allow some ip always
#-A INPUT -m state --state NEW -m tcp -p tcp -s white_ip_or_my_ip -j ACCEPT
# some tcp ddos
-A INPUT -i eth0 -p tcp -f -m tcp -j DROP
-A INPUT -i eth0 -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,SYN,RST,ACK -j DROP
-A INPUT -i eth0 -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j DROP
-A INPUT -i eth0 -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP
-A INPUT -i eth0 -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,SYN,RST,PSH,ACK,URG -j DROP
-A INPUT -i eth0 -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j DROP
-A INPUT -i eth0 -p tcp -m tcp --dport 0 -j DROP
# deny connection to rpc port
-A INPUT -i eth0 -p tcp -m tcp --dport 9998 -j DROP
# drop udp to p2p 9999
-A INPUT -i eth0 -p udp -m udp --dport 9999 -j DROP
# limit concurrent connection 2 per ip, 8 per c class
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 --tcp-flags FIN,SYN,RST,ACK SYN -m connlimit --connlimit-above 8 --connlimit-mask 24 --connlimit-saddr -j DROP
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 --tcp-flags FIN,SYN,RST,ACK SYN -m connlimit --connlimit-above 2 --connlimit-mask 32 --connlimit-saddr -j DROP
# limit syn to 3 / 30 sec / p2p 9999
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 -m conntrack --ctstate NEW -m recent --set --name DEFAULT --mask 255.255.255.255 --rsource
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 -m conntrack --ctstate NEW -m recent --update --seconds 30 --hitcount 3 --name DEFAULT --mask 255.255.255.255 --rsource -j DROP
# limit syn to 3 / 30 sec / ssh 22
-A INPUT -i eth0 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW -m recent --set --name DEFAULT --mask 255.255.255.255 --rsource
-A INPUT -i eth0 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW -m recent --update --seconds 30 --hitcount 3 --name DEFAULT --mask 255.255.255.255 --rsource -j DROP
# allow
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 -j ACCEPT
#
-A INPUT -i eth0 -p tcp -j ACCEPT
# output allow
-A OUTPUT -o eth0 -j ACCEPT
# deny forward
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
#
COMMIT
#

if you have --mask error, you are using old kernel.
change
Code:
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 -m conntrack --ctstate NEW -m recent --update --seconds 30 --hitcount 3 --name DEFAULT --mask 255.255.255.255 --rsource -j DROP
#
-A INPUT -i eth0 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW -m recent --update --seconds 30 --hitcount 3 --name DEFAULT --mask 255.255.255.255 --rsource -j DROP
to
Code:
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 -m conntrack --ctstate NEW -m recent --update --seconds 30 --hitcount 3 --name DEFAULT --rsource -j DROP
#
-A INPUT -i eth0 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW -m recent --update --seconds 30 --hitcount 3 --name DEFAULT --rsource -j DROP
 
Last edited:

nightowl

Member
Dec 30, 2015
67
105
73
@Sapereaude yea I saw that. You actually posted as I was writing the reply.

For those of you who are not technically inclined, take note that my firewall script also filters out a few ip addresses that takes part in the DDOS, which should help keep your masternode ports open for real Dash connections. The attacker will probably change IP's during the course of the day. I will update a list of blocked IP's later tonight and post an updated version.
 

Dash4Ever

Active Member
Sep 24, 2015
105
105
93
Sweden
Dash Address
XybaxnhtFBih2g4M2F71rWKBt5USzo8R
Was affected by the attack, woke up this morning with a email from my VPS provider that my VPS was temporary suspended.
So bought another one, so far so good!

Specs:

2048MB Memory
70GB SSD Space
4 Core Processor
Unmetered Network
3 IPv4 Addresses
3 IPv6 Addresses *

Pretty fair price 13.99 USD paid with dash debit ofc :)
 

Dash4Ever

Active Member
Sep 24, 2015
105
105
93
Sweden
Dash Address
XybaxnhtFBih2g4M2F71rWKBt5USzo8R
I'm a little curious, did they handpicked some IP addresses to attack? or all 4176 ip?
Some serious amount of computer power there.

In any case this demonstrates the strength of the Dash network :) they failade hard. ;)
 
  • Like
Reactions: tungfa and Walter

David

Well-known Member
Dash Support Group
Jun 21, 2014
618
628
163
Does Vultr's DDOS protection ($10 extra per month) help with these kind of DDOS attacks?
 

AndyDark

Well-known Member
Sep 10, 2014
353
705
163
I'm a little curious, did they handpicked some IP addresses to attack? or all 4176 ip?
Some serious amount of computer power there.

In any case this demonstrates the strength of the Dash network :) they failade hard. ;)
The attack was against all MNs as far as we know, from 2,000 individual IPs mostly in Asia, i.e. a botnet
 
  • Like
Reactions: RichardAO

RichardAO

New Member
Jan 17, 2017
34
8
8
NYS
Dash Address
XmTSEYFTG5dF7N68mEZwtpVUQPAPQ
Would it be helpful to report the IP hosts regarding their customers being used as ddos bots?
 

edificio

New Member
Jan 30, 2017
28
8
3
What exactly happens when a server is DDoS'ed? Does it just crash and restart? If that's the case, if i have my VPS set up with auto restart of dashd upon crash with dashcentral will the problem automatically solve itself if i get DDoS'ed?
 

crowning

Well-known Member
May 29, 2014
1,415
1,997
183
Alpha Centauri Bc
What exactly happens when a server is DDoS'ed? Does it just crash and restart? If that's the case, if i have my VPS set up with auto restart of dashd upon crash with dashcentral will the problem automatically solve itself if i get DDoS'ed?
It won't be able to answer any request from other peers/masternodes, so after a while it'll disappear form the masternode list due inactivity.

A possible crash is just the extra bonus you might get.
 

fuzzyduck

Active Member
Feb 19, 2015
134
113
93
be careful when copy pasting iptables scripts. You might end up locking yourself out if you got ssh on another port like i do. So look through the script and look for port 22 and change it to your own.
 

qwizzie

Well-known Member
Aug 6, 2014
1,576
736
183
It won't be able to answer any request from other peers/masternodes, so after a while it'll disappear form the masternode list due inactivity.

A possible crash is just the extra bonus you might get.
this DDoS attack seem to focus on just port 9999, maybe if we all choose different ports instead of the default port this type of attack could be mitigated in the future ?
 

edificio

New Member
Jan 30, 2017
28
8
3
It won't be able to answer any request from other peers/masternodes, so after a while it'll disappear form the masternode list due inactivity.

A possible crash is just the extra bonus you might get.
Ah i see. Is there a way to make it reboot automatically if all system resources are suddenly being used up?

Is it likely that the attacker would immediately start DDoS'ing the server again after that reboot? Of course ufw or iptables is the better option - But it would be nice to know the server would just reboot and start dashd again, should an attack slip through
 

lynx

Active Member
Dec 11, 2015
364
250
133
this DDoS attack seem to focus on just port 9999, maybe if we all choose different ports instead of the default port this type of attack could be mitigated in the future ?
This wouldn't help in the slightest, since every node (and hence everyone) would still have a list of all masternodes and ports.
 

ichigo13

Member
Masternode Owner/Operator
Jul 6, 2014
42
30
58
For those less technologically savvey there are step by step instructions below for Ubuntu 16.04. If you followed TAO's set up guide this will work for you.

**************
Enter root and enter the following commands

******* First Remove ufw
sudo ufw disable
sudo apt-get -y remove ufw
sudo apt-get -y purge ufw

****** Now install persistant ip tables and say yes when the purple screen appears
apt-get install -y iptables-persistent
invoke-rc.d netfilter-persistent save
service netfilter-persistent stop
service netfilter-persistent start

***** Now remove the old iptables file and paste in the new rules
rm /etc/iptables/rules.v4
joe /etc/iptables/rules.v4

************** Now paste in these rules and save
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:LOGNDROP - [0:0]
:OUTPUT ACCEPT [0:0]
#
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m conntrack --ctstate INVALID -j DROP
-A INPUT -p udp -j REJECT --reject-with icmp-port-unreachable
-A INPUT -p tcp -j REJECT --reject-with tcp-reset
-A INPUT -j REJECT --reject-with icmp-proto-unreachable
# some tcp ddos
-A INPUT -i eth0 -p tcp -f -m tcp -j DROP
-A INPUT -i eth0 -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,SYN,RST,ACK -j DROP
-A INPUT -i eth0 -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j DROP
-A INPUT -i eth0 -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP
-A INPUT -i eth0 -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,SYN,RST,PSH,ACK,URG -j DROP
-A INPUT -i eth0 -p tcp -m tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j DROP
-A INPUT -i eth0 -p tcp -m tcp --dport 0 -j DROP
#
-A INPUT -i eth0 -p tcp -m tcp --dport 9998 -j REJECT --reject-with tcp-reset
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 --tcp-flags FIN,SYN,RST,ACK SYN -m connlimit --connlimit-above 8 --connlimit-mask 24 --connlimit-saddr -j DROP
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 --tcp-flags FIN,SYN,RST,ACK SYN -m connlimit --connlimit-above 2 --connlimit-mask 32 --connlimit-saddr -j DROP
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 -m conntrack --ctstate NEW -m recent --set --name DEFAULT --mask 255.255.255.255 --rsource
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 -m conntrack --ctstate NEW -m recent --update --seconds 30 --hitcount 3 --name DEFAULT --mask 255.255.255.255 --rsource -j DROP
#
-A INPUT -i eth0 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW -m recent --set --name DEFAULT --mask 255.255.255.255 --rsource
-A INPUT -i eth0 -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW -m recent --update --seconds 30 --hitcount 3 --name DEFAULT --mask 255.255.255.255 --rsource -j DROP
#
-A INPUT -i eth0 -p tcp -m tcp --dport 9999 -j ACCEPT
#
-A INPUT -i eth0 -p tcp -j ACCEPT
-A OUTPUT -o eth0 -j ACCEPT
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
#
-A INPUT -j LOGNDROP
-A LOGNDROP -p tcp -m limit --limit 32/min -j LOG --log-prefix "Denied TCP: " --log-level 7
-A LOGNDROP -p udp -m limit --limit 32/min -j LOG --log-prefix "Denied UDP: " --log-level 7
-A LOGNDROP -p icmp -m limit --limit 32/min -j LOG --log-prefix "Denied ICMP: " --log-level 7
-A LOGNDROP -j DROP
COMMIT
#Remember to leave an extra space at the bottom

******* and save, then to check it is working
iptables -L

Viola :D

Edit- Credit to chaeplin for the far superior rules
What should I see when I execute the iptables command below? I saved the rules you posted above.
iptables -L
 

Figlmüller

Member
Sep 2, 2014
85
45
58
Vienna, Austria
Ah i see. Is there a way to make it reboot automatically if all system resources are suddenly being used up?

Is it likely that the attacker would immediately start DDoS'ing the server again after that reboot? Of course ufw or iptables is the better option - But it would be nice to know the server would just reboot and start dashd again, should an attack slip through
No, rebooting will make it even worse because you will lose all existing, valid masternode connections. When rebooting, you will get, as before, immediately spammed with invalid connection attempts or garbage data up to the connection limit of the dashd, thus preventing connections from legitimate nodes.
The attack will usually not stop if the server turns out to be unreachable.

So limit the rate of connection attempts using a firewall and also limit the amount of new connections per IP and subnet as mentioned in the thread. One may also think about proper kernel configuration, such as usage of syn cookies, enabling of some checks, etc. This will squash a lot of invalid TCP connections, allowing the masternode to "breathe". A way larger attack (this current one only caused peak traffic at around 0.15 Mb/s at our nodes) will still either fill up your connection queue in the daemon or exhaust the server resources, even if packets get dropped at the local firewall. Thus, a DDoS mitigation service, as offered by many server providers, may help by blocking attacks before those packets reach your host (in addition to security measures on your server).
 
  • Like
Reactions: Cofresí

Figlmüller

Member
Sep 2, 2014
85
45
58
Vienna, Austria
...
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:LOGNDROP - [0:0]
:OUTPUT ACCEPT [0:0]
...
The default behaviour for the INPUT chain is ACCEPT, which, at the end is redundantly repeated by the rule -A INPUT -i eth0 -p tcp -j ACCEPT

Basic invalid TCP packets are blocked, but you can still reach any other socket listening to the interface eth0 on any port. You can also flood any port other than 9999, which may cause a ton of half open connections on the host.

Also, I think this part, causing a jump to LOGNDROP will never be reached:
-A INPUT -j LOGNDROP
Why? Because A INPUT -i eth0 -p tcp -j ACCEPT at the end accepts any TCP packet on eth0 not matched by any rules above, and thus leaving the chain (if matched) and proceeding to the other tables (nat, etc.).

I would like to suggest, that you test your configuration before you simply trust a copy&paste solution.