Kimsufi is fast to a VPS, glacial to home. Why?

[raindog308]

I was whining about my Kimsufi transfer speed in another thread.  

At the moment, I'm seeing 10KB/sec and 6KB/sec in two separate FTP connections, connecting from my home (Portland, OR, USA) .  I just rebooted the kimsufi and there's nothing running on it of note - load average is .07 and there's 1.8GB of free RAM.

I went over to a BuyVM (KVM) in Las Vegas and tried to scp - and I'm get 1.7MB/sec from kimsufi to Vegas.

My home broadband is 50Mbps/10Mbps, and speedtest.net shows I get a little bitter better than that.  If I sftp to the BuyVM VPS, I can get 3MBps without a problem.

 

I realize the VPS has much better connectivity and I'd expect it to do better than a home connection, but...that's a huge difference.  I'd be happy getting 1MB/sec from kimsufi land, or even less...but 16KB/sec is pretty awful.  It seems to me I've gotten much better (like 700KB-1024KB/sec) from kimsufiland in the past, but I didn't keep any stats.

From home I see:



  3    10 ms    11 ms    10 ms  te-0-0-0-6-ur03.troutdale.or.bverton.comcast.net [68.85.150.177] 
  4    10 ms    12 ms    12 ms  ae-4-0-ar03.beaverton.or.bverton.comcast.net [68.87.216.13] 
  5    12 ms    10 ms    12 ms  ae-1-0-ar03.troutdale.or.bverton.comcast.net [68.87.218.162] 
  6    14 ms    18 ms    16 ms  he-2-1-0-0-11-cr01.seattle.wa.ibone.comcast.net [68.86.95.93] 
  7    14 ms    74 ms    14 ms  ix-0-3-1-0.tcore1.00S-Seattle.as6453.net [66.110.64.1] 
  8    53 ms    59 ms    54 ms  if-14-2.tcore1.PDI-PaloAlto.as6453.net [64.86.123.22] 
  9     *       55 ms    75 ms  if-2-2.tcore2.PDI-PaloAlto.as6453.net [66.198.127.2] 
 10    41 ms    40 ms    41 ms  if-5-2.tcore2.SQN-SanJose.as6453.net [64.86.21.1] 
 11   132 ms     *      198 ms  snj-2-6k.ca.us [178.32.135.166] 
 12   141 ms     *        *     lax-1-6k.ca.us [178.32.135.161] 
 13     *        *        *     Request timed out.
 14     *        *        *     Request timed out.
 15   213 ms   211 ms   214 ms  rbx-g2-a9.fr.eu [91.121.131.181] 
 16   296 ms   331 ms     *     rbx-1-6k.fr.eu [91.121.131.13] 
 17   210 ms   210 ms   211 ms  rbx-3-m1.fr.eu [213.251.191.4] 
 18     *        *        *     Request timed out.
 19   225 ms   224 ms     *     ksXXXX.kimsufi.com [213.251.x.x] 
 20   223 ms   226 ms     *     ksXXXX.kimsufi.com [213.251.x.x] 
 21   227 ms   224 ms   225 ms  ksXXXX.kimsufi.com [213.251.x.x] 


From a BuyVM Las Vegas VPS:

 

 2  10.1.1.5 (10.1.1.5)  0.290 ms  0.380 ms  0.325 ms
 3  * * *
 4  * * *
 5  * * *
 6  rbx-g2-a9.fr.eu (91.121.128.167)  141.778 ms  141.444 ms  142.143 ms
 7  rbx-1-6k.fr.eu (91.121.131.13)  143.965 ms *  143.960 ms
 8  rbx-3-m1.fr.eu (213.251.191.4)  144.123 ms  144.155 ms  144.147 ms
 9  ksxxxx.kimsufi.com (213.251.x.x)  142.554 ms  141.459 ms  141.294 ms

I'm using FileZilla for a client...no speed limits enabled.

 

[drmike]

Wonder about Comcast and their notorious clogged common network points.   I've seen similar odd slowdowns in the past with another cable competitor.

Have you tried Kimsufi speed tests and/or other downloadables at the same location?  That's my typical approach to debugging this stuff.

These look like clogged points potentially:

11 132 ms * 198 ms snj-2-6k.ca.us [178.32.135.166]
12 141 ms * * lax-1-6k.ca.us [178.32.135.161]

Try running mtr and see how the numbers compare.

 

[texteditor]

My connection is only 16mb/s so it's a bad test, but I get roughly 1.5-1.6MB/s from my FS-8T in their GRA datacenter as well as my mSP and kimsufi in RBX - it's probably your provider

 

[raindog308]

Meh...now I'm only getting 100KB/sec on the Kimsufi->BuyVM VPS transfer, either ssh or rsync protocol.

 

[Tux]

I get decent transit to OVH here.


I saw the connection went via Tata though, which Comcast typically oversubscribes bandwidth from.

 

[kaniini]

Wonder about Comcast and their notorious clogged common network points.   I've seen similar odd slowdowns in the past with another cable competitor.

Have you tried Kimsufi speed tests and/or other downloadables at the same location?  That's my typical approach to debugging this stuff.

These look like clogged points potentially:

11 132 ms * 198 ms snj-2-6k.ca.us [178.32.135.166]


12 141 ms * * lax-1-6k.ca.us [178.32.135.161]

Try running mtr and see how the numbers compare.

This is actually ICMP exception deprioritization.  OVH runs mostly cat 6504/6509's on their backbone network, with SUP 2T cards, so they want to save CPU for routing exceptions/bgp update load I suspect.

 

[raindog308]

So to summarize:

Wednesday, 6pm Pacific: 1.7MB/sec Kimsufi -> BuyVM Vegas

Wednesay, 6:30pm Pacific through at least 11pm Pacific: 100KB/sec to max 200KB/sec Kimsufi -> BuyVM Vegas

Thursday 7am Pacific: 1.7MB/sec Kimsufi -> BuyVM Vegas

That's taking any Comcast factor out of the picture.

I guess I was right to whine about OVH

 

[manacit]

I get great speeds from my OVH - hitting ~7MB/s from my KS -> CC LA, 11.1MB/s KS -> RN Seattle, etc

 

[TheLinuxBug]

Most likely a lot of that is on Comcast, it seems recently they can't make up their mind of which Euro routes to use because on the East Coast here I have found over the past 2 weeks that my European routes flap 2-3 times a day taking me from bleeding fast transfers of 6-8M/sec to transfers at about 1M/sec or less. My solution to this is to find a VPS to put a vpn or proxy on that has more consistent routes to me and my destination.  I have found that Front Range Hosting's KVM in Denver usually has nice routes into Comcast and to Europe, so if your trying to improve your throughput I can suggest that as an option. 

You have to think of all the people who are on Comcast that are trying to use OVH's network, I bet you their route there is getting over saturated and instead of providing more transit they are QOSing everyone. Using an alternative route (VPN/Proxy) to your destination should relieve this limitation as you will no longer be joining the rest of the people in abusing the already saturated route.

My 2 cents.

Cheers!

 

[raindog308]

I may try the VPN solution you suggest.

But it's pretty crappy to Las Vegas datacenter as well.

 

[drmike]

Using an alternative route (VPN/Proxy) to your destination should relieve this limitation as you will no longer be joining the rest of the people in abusing the already saturated route.

Problem, they QoS everything.   Even VPN/SSH/Proxy is impacted to ugly negative effects.   The approach is fine so long as you are moving remote data between remote datacenter sites.   If you need to download that data to local, there is no fixing their stupid ugly network and QoS stomping.

 

[raindog308]

Problem, they QoS everything.   Even VPN/SSH/Proxy is impacted to ugly negative effects.   The approach is fine so long as you are moving remote data between remote datacenter sites.   If you need to download that data to local, there is no fixing their stupid ugly network and QoS stomping.

I don't have a problem getting multiple 4-5MB/sec downloads in parallel downloading via sftp (to non-standard high ports) from datacenter VPSes in the US.

 

[TheLinuxBug]

QoS is more severely impacted on the most commonly used routes in an area it seems for Comcast.  If you are for example, using ipv6 to connect to your vpn instead of ipv4, you will actually get higher speeds in some cases and better routes.  This can also apply for just using routes that are not saturated in general.  Using resources where they do have high commits may also prove better speeds.  My suggestion is try a few different locations and see what best works for you.  Some of that information can be extrapolated from ping times and the proximity of the (VPN/VPS) host to you.  The lower the latency, the better chance of higher throughput, generally (though not always).

My 2 cents.

Cheers!

 

[drmike]

I don't have a problem getting multiple 4-5MB/sec downloads in parallel downloading via sftp (to non-standard high ports) from datacenter VPSes in the US.

Unsure what causes this limit on single thread transfers in said network.   It's rather common problem with major cable companies in States.  Someone must have sold the major cable co's the same QoS solution.

It gets into hacking configs and non standard stuff, but multithreaded multi file xfer simultaneously is the hack to this.

 

[manacit]

I, too, have had tons of problems with Comcast in general. If your traffic is going over Cogent with Comcast, you're more or less screwed.

 

[Tux]

I, too, have had tons of problems with Comcast in general. If your traffic is going over Cogent with Comcast, you're more or less screwed.

Same here with Charter. Get Cogent (which is so easy to get it's not funny) and you get fucked overway 6 ways from Sunday.