HOWTO: Stop NTP amplification attacks from reaching your nodes!

Discussion in 'Tutorials and Guides' started by Francisco, Feb 18, 2014.

  1. Francisco

    Francisco Company Lube Verified Provider

    2,476
    1,770
    May 15, 2013
    Hello everyone,

    **If you share this information on your blog, forum, etc, be kind and please link back to this topic!**

    Normally I'm not one to share stallion code, but after a discussion with a couple staffers we came to the conclusion that the following work must be made public for the 'greater good' and all that righteous crap.

    What this does

    The following blocks NTP monlist packets at the node (or router level if you're using a linux based setup), before they ever get to your customers. This means that it provides preemptive filtering, instead of after-the-fact-oh-god-my-bandwidth-bills. Stopping NTP amplification floods before the user gets them was the only way for us morally address users from being used in NTP floods be it now or later on.

    What this doesn't do

    This does not patch the users configuration files by any means. This is entirely node side done with iptables. You should still make it an effort to inform your customers about the dangers of using a bad version of NTP.

    Lets get started!

    First, you must add the following entry to your /etc/sysctl.conf. This makes it so all packets sent over a bridge (for XEN & KVM based VM's) are also filtered.
     


    net.bridge.bridge-nf-call-iptables = 1

    Once this is done, apply the changes
     


    sysctl -p

    The following rule is what does all the magic. You'll want to put this in /etc/rc.local above the exit 0 so it gets applied on reboot. You should also look at using iptables-save as well.
     


    iptables -I FORWARD -p udp --dport 123 -m u32 --u32 "0x1C=0x1700032a && 0x20=0x00000000" -m comment --comment "NTP amplification packets" -j DROP

    You can change the chain from FORWARD to INPUT in the off chance that you want to use this inside a VPS or something like that. It'd be smarter to simply ACL monlist or upgrade your version, but to each their own.

    You should feel no performance impact from this rule being in place. Your node will still be smacked with the packets, but nothing will be sent out.

    For obvious reasons, I won't be talking to Phill about including this in his node side SolusVM code, but if someone wishes to point him this way, they have my permission to include this.

    For the greater good,

    Francisco
    Your friendly neighborhood hairyman
     
     
    EDIT: Added the below quoted response upon request of OP. -MD
     

     
    Last edited by a moderator: Feb 19, 2014
  2. bzImage

    bzImage New Member

    40
    39
    May 19, 2013
    scale.jpg
     
    HalfEatenPie likes this.
  3. Kakashi

    Kakashi New Member Verified Provider

    133
    37
    Sep 30, 2013
    Very nice, appreciate the help.
     
  4. splitice

    splitice Just a little bit crazy... Verified Provider

    550
    252
    Jun 16, 2013
    Good work Fran / Delta,

    When doing this for the sake of your customers flexibility (and sanity when debugging) either ensure your conntrack size is suitably large or disable connection tracking for these packets when enabling iptables for a KVM emulated network interface. Most people assume that a KVM virtual nic is the same as a bare metal one and not limited in terms of connections.

    Disabling conntrack can be done with a rule in the PREROUTING chain -j NOTRACK or by not having the module loaded / compiled in.
     
    Last edited by a moderator: Feb 18, 2014
  5. Mun

    Mun Never Forget

    892
    237
    May 9, 2013
    I think I wrote up a bash script that should automate the process. Here is the source!

    http://cdn.content-network.net/Mun/apps/frantp/0.1/source.txt

    You should be able to run it via:


    wget http://www.cdn.content-network.net/Mun/apps/frantp/0.1/script.sh -O - | bash

    or for https://

    wget https://www.cdn.content-network.net/Mun/apps/frantp/0.1/script.sh -O - | bash



    I am also looking for suggestions on how to improve the code.

    Please note, that unless you run this as a rc.local task the IPtables will be flushed on reboot! I suggest you save the file to a local directory, chmod +x it, and then make a rc.local task.

    Please note I have tested it on my dedicated server, but not on a VPS node, can someone please confirm it works?

    Mun
     
    Last edited by a moderator: Feb 18, 2014
    drmike and Francisco like this.
  6. HostUS-Alexander

    HostUS-Alexander New Member Verified Provider

    255
    48
    May 15, 2013
    <3
     
  7. kaniini

    kaniini Beware the bunny-rabbit! Verified Provider

    497
    236
    Jun 18, 2013
    If you want the inverse of this (to filter the crap out that you're receiving), simply block UDP packets to port 123 which are 468 octets long. You can do this with a route-map on most routers or a policer.
     
    drmike likes this.
  8. Mun

    Mun Never Forget

    892
    237
    May 9, 2013
    Should I add this to the script, or it that bad? If so, then what would the iptables command look like?

    Mun
     
  9. mcmyhost

    mcmyhost New Member

    49
    11
    Nov 22, 2013
    What about IPv6?
     
  10. Mun

    Mun Never Forget

    892
    237
    May 9, 2013
    Snipped just so now one does anything stupid.
     
    Last edited by a moderator: Feb 18, 2014
  11. Francisco

    Francisco Company Lube Verified Provider

    2,476
    1,770
    May 15, 2013
    This is most likely incorrect.

    I'll poke around with tcpdump later and see what it actually is.

    Francisco
     
  12. Francisco

    Francisco Company Lube Verified Provider

    2,476
    1,770
    May 15, 2013
    For sure!

    With this iptables rule, though, it allows users that are renting all their gear and don't have switch ACL access to defend from it as well.

    Francisco
     
  13. Francisco

    Francisco Company Lube Verified Provider

    2,476
    1,770
    May 15, 2013
    We've had no performance issues to date with it. We actually had a few users get banned by autonull because people were sending so many packets at them to reflect.

    Francisco
     
  14. Mun

    Mun Never Forget

    892
    237
    May 9, 2013
    When I was reading this, I was thinking about it in a whole different way.... oppps.
     
  15. FLDataTeK

    FLDataTeK New Member Verified Provider

    48
    23
    Jan 24, 2014
    Thank you for figuring out how to block it at the node level I am sure it will help everyone rest better at night knowing that their nodes won't be assisting in the DDOS...
     
  16. eva2000

    eva2000 Active Member

    325
    90
    May 22, 2013
  17. Francisco

    Francisco Company Lube Verified Provider

    2,476
    1,770
    May 15, 2013
    Same concept just I hope they aren't doing literal string matches :)

    Francisco
     
  18. Mun

    Mun Never Forget

    892
    237
    May 9, 2013
    @Francisco, do you see anything I should add to my script?
     
  19. Francisco

    Francisco Company Lube Verified Provider

    2,476
    1,770
    May 15, 2013
    Looks good to me.

    Francisco
     
  20. Mun

    Mun Never Forget

    892
    237
    May 9, 2013
    KK, think I will make another one without the bridge adapter for people whom wont be using it on a VPS setup.