amuck-landowner

BGP IPs being hijacked...

Jack

Active Member
Hey; 

 

Just wondering if this has happened to any other providers on here as it was a first for me when I read the reason for the outage;.

 

https://my.securedragon.net/announcements.php?id=313

 

 

 

Today we received some network disruption in our Tampa data center due to our IPs being hijacked by a data center in Latin America. While we do not know if this was a malicious attempt to impact our network or an honest mistake (fat fingered an IP?), it was still disruptive none-the-less. Luckily our data center reacted quickly to resolve this issue and the downtime was limited to about 30 minutes according to our external monitoring. In reality, some of our upstream providers should have been routing normally so the outage was not a complete outage (check drgn.biz and you'll see no downtime reported but our other monitoring service shows downtime for certain nodes in Tampa).

 

 

The "fix".

 

Our data center had to announce our IPs as a /24 to overwrite their announcement.

 

So how is this possible?

(An excellent question from a longtime client of ours that prompted me to do more research about it.)

 

It's possible because BGP requires no authentication or confirmation of IP ownership. It's up to the upstream providers (Level3, Cogent, HE, nLayer, Verizon, Comcast, etc...) to verify IP ownership but some of them accept any routes provided to them without even manual intervention (Cogent lets all data centers who pay for a commit to announce any IP via a web interface, Level3 requires a phone call or an e-mail/ticket along with a Letter of Authority from the IP owners).

 

Unfortunately there is no real method to prevent this which is why BGP monitoring services are pretty popular although they just send an e-mail AFTER the damage is done and the IPs are announced. Hopefully with the smaller announcements it will prevent future hijacks.

 

-The Secure Dragon Staff
 

mikho

Not to be taken seriously, ever!
Never heard of a hijack like this before. Not surprised on the setup, meaning it's not the only place that allows for updates without requesting confirmation.


One should learn atleast one new thing each day. This was mine for today.
 

HalfEatenPie

The Irrational One
Retired Staff
Oh yeah if I remember EDIS has a huge IP block to them that they just finally got back and majority of it was blacklisted.  
 

KuJoe

Well-Known Member
Verified Provider
Doing a search for "BCP hijack" brings a lot of interesting RFOs of large attacks. I will go into more detail of these attacks and how they work when I get to work tonight.
 

KuJoe

Well-Known Member
Verified Provider
I meant "BGP hijack". I wish I wasn't forced to use the mobile version on my phone. :(
 

Nick_A

Provider of the year (2014)
I'm pretty sure we determined someone in Michigan briefly hijacked some of our IPs way back.
 

MCH-Phil

New Member
Verified Provider
I've seen this happen and have discussed some of the dangers on other sites etc.  It can be kind of scary when ya really think about it.  BGP hijacking has made the news a few times also.
 

KuJoe

Well-Known Member
Verified Provider
@Francisco, Doesn't work on my phone in any of my browsers for some reason. Not sure why.

Now for some of the info I've found out about BGP hijacking and some info based on my limited networking knowledge (feel free to correct me if I state something incorrectly because this information is based on about 30 minutes of research without my glasses on while this hijacking happened and a lot of the high level documents were way over my head).

To start off with, as I said in the announcement reposted above, there is little protection against a BGP hijacking. The only real protection is to announce each subnet as a /24 from your data center since the most specific route wins (and a /24 is the smallest route you can announce).

One thing we noticed during this hijack is that our NodePing monitoring and custom external monitoring DID NOT report and packet loss or downtime and our StatusCake monitoring only showed a few nodes offline for a few minutes in Tampa (not even the full duration of the hijacking). To my understanding, this is because the hijacker (malicious or accident, doesn't matter) was only announcing our IPs to his upstream providers where as some of our own upstream providers already knew about our IPs in Tampa. For example, my dad (partner) lives in the Tampa area and during the hijacking was able to get to our network and everything worked normally for him because his home ISP peers with our data center and we filled out paperwork for them to accept our routes so it did not need to go out of their network to get to our IPs.

Some of the sites I read had people asking "Why not only announce /24s?", this is fine for a small company like us with only 12 /24s in different data centers, but other larger companies with hundreds of /24s can be problematic for routing tables.

I highly recommend BGPMon to anybody who has their own IP space, it's a useful tool although it's only helpful for letting you know after your IPs have been hijacked though.

Ideally I would like somebody with more knowledge on the subject to chime in.
 

Jack

Active Member
we noticed during this hijack is that our NodePing monitoring and custom external monitoring DID NOT report and packet loss or downtime


Well mine did, that's why I opened a ticket, my VM had 91.8% packetloss from my terminal(home) then dropped out completely to live to exceeded with an ENET IP.
 
Last edited by a moderator:

mikho

Not to be taken seriously, ever!

turfhosting

New Member
I think this will become more and more common and eventually your going to be defending your IP's with your life! Hah just kidding, but that is crazy. I don't even understand how its possible to "hijack" a IP...
 
Last edited by a moderator:

MannDude

Just a dude
vpsBoard Founder
Moderator
I think this will become more and more common and eventually your going to be defending your IP's with your life! Hah just kidding, but that is crazy. I don't even understand how its possible to "hijack" a IP...
I just got a new chrome S&W .45 last week. Come and get my IPs, you damned dirty bastards! (I have no IPs)
 

fusa

New Member
Verified Provider
The real problem is when they "proxy" your traffic. Then the bastards could see all the traffic.


There are some ranges announced that are used for spamming and hacking, pre-ARIN/RIPE blocks


As a provider you could only monitor this and block those routes
 
Top
amuck-landowner