# Low Latency Networking Across USA



## imperio (Jun 30, 2013)

I have a project which requires low latency access to cable/dsl providers in USA.I need recommendations for virtualized(LEB and non-LEB) and dedicated server providers which are hosted on datacenters which have low latency upstreams(directly peers preferred) to most used cable/dsl providers like comcast,time warner,cox,charter,cablevision,at&t,verizon,centurylink,frontier etc.


----------



## serverian (Jun 30, 2013)

Peer1


----------



## TheLinuxBug (Jun 30, 2013)

For East Coast find someone who is in Equinix or Latisys in Ashburn, VA as they directly peer Comcast.  One such provider, though I have no real experience with them is: Bitcable who say directly in their Advertisement they peer with Comcast.

Cheers!


----------



## SeriesN (Jun 30, 2013)

Correct me if I am wrong but I beleive Atlantic metro directly peers with CableVision (I need to look it up though and can't bother from my cell phone).


----------



## imperio (Jun 30, 2013)

Thanks for the suggestions.

Let me clarify more.What i expect to see is direct peerings(no tier1 or tier2 provider between) like below example on comcast looking glass.

 

 

Traceroute from Comcast to Highwinds

 

    route-server.newyork.ny.ibone>traceroute 209.197.26.100

    

    Type escape sequence to abort.

    Tracing the route to oct020-ipmi.at1.hwcdn.net (209.197.26.100)

    

      1 te-1-4-0-6-102-cr01.newyork.ny.ibone.comcast.net (66.208.229.6) 4 msec 4 msec 4 msec

      2 be-10-pe02.111eighthave.ny.ibone.comcast.net (68.86.82.66) 0 msec 0 msec 4 msec

      3 173.167.58.46 0 msec 0 msec 0 msec

      4 e8-1.r1.dc.hwng.net (209.197.0.41) [AS 12989] 8 msec 4 msec 20 msec

      5 3-1.r1.at.hwng.net (69.16.191.126) [AS 12989] 20 msec 20 msec 48 msec

      6 oct020-ipmi.at1.hwcdn.net (209.197.26.100) [AS 12989] 20 msec 48 msec 48 msec


----------



## imperio (Jun 30, 2013)

TheLinuxBug said:


> For East Coast find someone who is in Equinix or Latisys in Ashburn, VA as they directly peer Comcast.  One such provider, though I have no real experience with them is: Bitcable who say directly in their Advertisement they peer with Comcast.
> 
> Cheers!


Bitcable peers with latisys who peers with comcast.


----------



## imperio (Jun 30, 2013)

SeriesN said:


> Correct me if I am wrong but I beleive Atlantic metro directly peers with CableVision (I need to look it up though and can't bother from my cell phone).


It seems Atlantic Metro does not peer with comcast nor cablevision.


----------



## SeriesN (Jun 30, 2013)

imperio said:


> It seems Atlantic Metro does not peer with comcast nor cablevision.


I am tired. sorry.


----------



## EarthVPN (Jun 30, 2013)

Peer1 is interesting


----------



## imperio (Jun 30, 2013)

It seems Peer1 is only connected to comcast via peering on Toronto location.

Toronto Peer1 to NY Comcast via Peering

*Router:* Toronto, ON 
*Command:* traceroute 68.86.80.2

1 10ge.xe-11-2-1.tor-151f-cor-1.peer1.net (216.187.114.146) 0.534 ms 0.440 ms 0.406 ms
2 10ge.xe-2-0-0.chi-eqx-dis-1.peer1.net (216.187.114.37) 10.314 ms 10.338 ms 10.263 ms
3 te-0-6-0-3-pe03.350ecermak.il.ibone.comcast.net (173.167.58.9) 10.683 ms 10.907 ms 10.579 ms
4 be-14-cr01.350ecermak.il.ibone.comcast.net (68.86.82.209) 14.229 ms be-13-cr01.350ecermak.il.ibone.comcast.net (68.86.84.249) 13.353 ms be-12-cr01.350ecermak.il.ibone.comcast.net (68.86.84.189) 13.061 ms
5 he-4-2-0-0-cr01.newyork.ny.ibone.comcast.net (68.86.88.137) 30.652 ms * 33.089 ms

 Peer1 NY to Comcast NY via Tinet

*Router:* New York, NY 
*Command:* traceroute 68.86.80.2

1 ae2-205.nyc20.ip4.tinet.net (77.67.70.193) 0.411 ms 0.333 ms 0.348 ms
2 xe-0-0-1.nyc32.ip4.tinet.net (89.149.184.194) 0.779 ms xe-0-1-0.nyc32.ip4.tinet.net (89.149.181.114) 0.810 ms xe-1-0-0.nyc32.ip4.tinet.net (89.149.184.186) 29.044 ms
3 te-1-11-0-5-pe01.111eighthave.ny.ibone.comcast.net (75.149.229.113) 5.992 ms 1.970 ms te-1-11-0-4-pe01.111eighthave.ny.ibone.comcast.net (75.149.229.109) 3.796 ms
4 pos-1-13-0-0-cr01.newyork.ny.ibone.comcast.net (68.86.87.97) 5.301 ms * 2.475 ms


----------



## Tux (Jun 30, 2013)

This is a bit of a impossible request. What application are you running that is heavily latency-sensitive?

Your best general bet is basically finding someone who peers with Comcast and hope for the best, since most (not all) American ISPs peer with them. Sharktech would be one such provider.


----------



## imperio (Jun 30, 2013)

Actually CDN/Game delivery network providers like highwinds have this kind of optimized network mix.So i do not think its impossible.


----------



## Tux (Jun 30, 2013)

imperio said:


> Actually CDN/Game delivery network providers like highwinds have this kind of optimized network mix.So i do not think its impossible.


I raised a question.

What application are you running that is heavily latency-sensitive?


----------



## imperio (Jun 30, 2013)

Gaming.


----------



## D. Strout (Jun 30, 2013)

Off topic, but if you got here because of the thread title, it's worth noting that Hurricane Electric has a direct NY > LAX link, ~60ms across the continent there.


----------



## drmike (Jun 30, 2013)

The peering can or can't matter.  Depends on the provider and the upstreams.

Low latency only exist in small geographic areas where providers are peered or upstreaming to an Internet Exchange.  New York is probably the best example of such a place, with multiple exchanges and good provider tie ins of all sort.  Obviously, some small obscure independent ISP may not have upstreams with as close ties, but the big guys all do.

So putting something on NYIIX or down one level from there, probably would give killer low latency to most folks in NYC metro.   Get outside that area and back to pot luck with increasing number of providers.
 

A place like Peer1 does pretty good.  I am fan of their network and services.  Isn't cheap.  There are other options and they are all costly and upper tier and typically like Peer1 have their own network.

At last check OVH was pretty swell in the US connectivity wise with their private network.

My experience is Time Warner and Comcast often aren't peered or only part of an internet exchange in a metro.  Trying to direct address home consumers like this is a niche/model those cable co's kind of reserved for themselves to sell directly to content providers.  Time Warner and Comcast aren't sane and their networks are ho hum.  Lots of poor routing, Cogent clogs and mystical broken stuff for years on end.

You can't achieve low latency from one geographic point to cover the whole US.  It is far too big and the networks end to end are too fragmented.  Those which aren't fragmented are either expensive or not so great.  

I really like Peer1 and would look at OVH (if you rent servers).  Both networks impressed this year when I tested them again.   Both being Canada-based offerings, they are use to certain Atlantic to Pacific trip times, which are far less than in US typically.  So, both appear to backhaul lots of their own traffic to avoid the US inflation.


----------



## imperio (Jun 30, 2013)

Thanks.One solution is Softlayer.Currently I have four locations of softlayer and all of them are directly peered with comcast for example.I am looking for alternatives.Dedicated/virtualized providers on highwinds/peer1 maybe.Any experience/recommendations ?


----------



## drmike (Jun 30, 2013)

Softlayer indeed has their own network with a bunch of POPs and public peering exchange use.



> The SoftLayer network provides all customers with over 2,000 Gbps of connectivity between our data centers and 16 points of presence (PoPs). We have enabled dozens of 10Gbps network connections across our locations that are provided by leading global network providers, and include multiple public peering links to dozens of additional Internet access networks.


They run a good operation.  You'll pay for it though (will with any company that anyone has recommended). Low latency from everywhere though?

Hmm...

Peering locations:

http://www.softlayer.com/about/network/peering

Seems like a good geo-distribution in the US.  I've seen bad routes in the past during maintenance windows though.  High latency for same "region".


----------



## concerto49 (Jun 30, 2013)

Highwinds has been horrible in LA. It was worse than Cogent / HE in latencies.


----------



## imperio (Jun 30, 2013)

concerto49 said:


> Highwinds has been horrible in LA. It was worse than Cogent / HE in latencies.



Can you please elaborate, source and destination ?


----------



## Wintereise (Jun 30, 2013)

He probably means LAX -> Wherever, since he's in Australia, so all traffic kind of aggregates in LAX/SJC.


----------



## concerto49 (Jun 30, 2013)

imperio said:


> Can you please elaborate, source and destination ?


LA -> Europe. Quite a lot of countries were horrible.



Wintereise said:


> He probably means LAX -> Wherever in the US, since he's in Australia, so all traffic kind of aggregates in LAX/SJC.


No I don't. LAX -> Europe. I haven't seen anything good out of Bandcon / Highwinds in general.


----------



## Wintereise (Jun 30, 2013)

concerto49 said:


> LA -> Europe. Quite a lot of countries were horrible.
> 
> No I don't. LAX -> Europe. I haven't seen anything good out of Bandcon / Highwinds in general.


Post fixed then


----------



## kaniini (Jul 1, 2013)

Steadfast directly peers with Comcast, as far as I know.


----------



## TheLinuxBug (Jul 1, 2013)

imperio said:


> Bitcable peers with latisys who peers with comcast.


Latisys Peers to Equinix which has an incredible amount of bw into Comcast as it is one of Comcast's largest peering points in the north east besides New York.  I am not sure what your looking for but its usually 20ms or less, I use Comcast here and I also have server in Latisys my self, I can max my cable modem and my ping is 20ms and I am not even in a large city.  I bet people in Pennsylvania (north of me) get like 10ms. 

The only other option for what your looking for is to call Comcast and ask them to host directly on iBone (their infrastructure).  Beyond that my suggestion still stands. 

Cheers!


----------



## imperio (Jul 1, 2013)

kaniini said:


> Steadfast directly peers with Comcast, as far as I know.


I couldnt verify.

route-server.newyork.ny.ibone>traceroute 69.162.170.5

 

Type escape sequence to abort.

Tracing the route to lookingglass.nyc.steadfast.net (69.162.170.5)

 

  1 te-1-4-0-6-102-cr01.newyork.ny.ibone.comcast.net (66.208.229.6) 4 msec 4 msec 4 msec

  2 xe-9-1-0.edge1.NewYork2.Level3.net (4.71.186.33) [AS 3356] 0 msec 0 msec 0 msec

  3 vlan52.ebr2.NewYork2.Level3.net (4.69.138.254) [AS 3356] [MPLS: Label 1879 Exp 0] 0 msec 0 msec

    vlan51.ebr1.NewYork2.Level3.net (4.69.138.222) [AS 3356] [MPLS: Label 1178 Exp 0] 0 msec

  4 ae-46-46.ebr2.NewYork1.Level3.net (4.69.201.29) [AS 3356] [MPLS: Label 1886 Exp 0] 0 msec

    ae-48-48.ebr2.NewYork1.Level3.net (4.69.201.49) [AS 3356] [MPLS: Label 1134 Exp 0] 0 msec

    4.69.201.41 [AS 3356] [MPLS: Label 1134 Exp 0] 0 msec

  5 ae-71-71.csw2.NewYork1.Level3.net (4.69.134.70) [AS 3356] [MPLS: Label 1854 Exp 0] 0 msec

    ae-61-61.csw1.NewYork1.Level3.net (4.69.134.66) [AS 3356] [MPLS: Label 1863 Exp 0] 4 msec

    ae-62-62.csw1.NewYork1.Level3.net (4.69.148.34) [AS 3356] [MPLS: Label 1863 Exp 0] 4 msec

  6 ae-3-80.edge4.NewYork1.Level3.net (4.69.155.146) [AS 3356] 16 msec

    ae-4-90.edge4.NewYork1.Level3.net (4.69.155.210) [AS 3356] 0 msec

    ae-1-60.edge4.NewYork1.Level3.net (4.69.155.20) [AS 3356] 0 msec

  7 NOZONE-INC.edge4.NewYork1.Level3.net (4.28.130.154) [AS 3356] 0 msec 20 msec 0 msec

  8 ip26.67-202-117.static.steadfastdns.net (67.202.117.26) [AS 32748] 0 msec 24 msec 4 msec

  9 lookingglass.nyc.steadfast.net (69.162.170.5) [AS 32748] 0 msec 4 msec 0 msec

route-server.newyork.ny.ibone>traceroute 208.100.4.54

 

Type escape sequence to abort.

Tracing the route to lookingglass.chi.steadfast.net (208.100.4.54)

 

  1 te-1-4-0-6-102-cr01.newyork.ny.ibone.comcast.net (66.208.229.6) 8 msec 0 msec 4 msec

  2 he-0-2-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.88.138) 24 msec 24 msec 24 msec

  3 be-15-pe03.350ecermak.il.ibone.comcast.net (68.86.82.214) 20 msec 20 msec 20 msec

  4 as4436.350ecermak.il.ibone.comcast.net (173.167.57.126) 24 msec 24 msec 24 msec

  5 ae1-70g.cr2.ord1.us.nlayer.net (69.31.111.137) [AS 4436] 20 msec 20 msec 20 msec

  6 as32748.ae1-133.cr2.ord1.us.nlayer.net (69.31.110.30) [AS 4436] 20 msec 24 msec 24 msec

  7 ip46.208-100-32.static.steadfastdns.net (208.100.32.46) [AS 32748] 20 msec 20 msec 20 msec

  8 lookingglass.chi.steadfast.net (208.100.4.54) [AS 32748] 20 msec 20 msec 20 msec

route-server.newyork.ny.ibone>


----------

