Some fun info about Colocrossing for today. The reason Colocrossing can't offer IPv6 and has delayed setting up BGP sessions (See Crissic's LA departure thread) is because they don't actually have a large core router. Instead, they use switches similar to the photo below as their "edge network". These routers can bring up a BGP session but they can't hold "an internet worth of routes". These switches top out around ~10,000 total routes. So how does Colocrossing get around this? They use multipath between all of their peers. Multipath does round-robin blasting packets down each peer in equal weight. Ever wonder why you have really ugly traceroutes like this? 3 host.colocrossing.com (18.104.22.168) 0.652 ms host.colocrossing.com (22.214.171.124) 0.708 ms host.colocrossing.com (126.96.36.199) 0.826 ms 4 188.8.131.52 (184.108.40.206) 0.288 ms 0.287 ms buf-b1-link.telia.net (220.127.116.11) 0.326 ms 5 nyk-bb1-link.telia.net (18.104.22.168) 9.606 ms 9.604 ms 9.545 ms 6 tinet.yyz02.atlas.cogentco.com (22.214.171.124) 12.793 ms 12.846 ms nyk-b3-link.telia.net (126.96.36.199) 9.808 ms 7 tmobile-ic-302276-war-b1.c.telia.net (188.8.131.52) 27.436 ms xe-4-3-0.atl11.ip4.tinet.net (184.108.40.206) 49.765 ms xe-8-0-0.atl11.ip4.tinet.net (220.127.116.11) 49.862 ms 8 eth2-1.r1.ash1.us.atrato.net (18.104.22.168) 21.874 ms total-server-solutions-gw.ip4.tinet.net (22.214.171.124) 49.968 ms eth2-1.r1.ash1.us.atrato.net (126.96.36.199) 23.592 ms 9 eth3-1.r1.atl1.us.atrato.net (188.8.131.52) 46.245 ms total-server-solutions-gw.ip4.tinet.net (184.108.40.206) 61.983 ms Notice how you have Cogent & Telia on the same hop, as well as Atrato once you start getting to [email protected]? That's because each packet of the traceroute is going through a different peer in round robin. If they were running full routing tables they would have a single path through a single provider to get to a destination (since that ISP/path would have been picked to be the best, assuming they aren't doing cost based balancing). It's pretty sad and this setup actually causes random speed issues for some users, especially Comcast customers. You'll have good speed one one transfer then garbage the next. I'm not sure where they "invested" $1,000,000 into their network but it's obviously a lie.