amuck-landowner

wget vs wget

sv01

sv01

Slow but sure
both has similliar result :)

1st script :

download test file, hide output and print speed

2n script :

just run standart wget command
 
CraigA

CraigA

New Member
Verified Provider
Dylan,

  I think the speed difference that you are seeing might just be a matter of variance. As sv01 stated, both produce the same results and any post-processing in the first script is done after the download, which wouldn't affect the actual download speed.
 
Dylan

Dylan

Active Member
That's what I thought too, but I can run both commands over and over and consistently get results like these:

# wget -O /dev/null http://speedtest.wdc01.softlayer.com/downloads/test100.zip 2>&1 | awk '/\/dev\/null/ {speed=$3 $4} END {gsub(/\(|\)/,"",speed); print speed}'

24.8MB/s

# wget -O /dev/null http://speedtest.wdc01.softlayer.com/downloads/test100.zip

2014-02-13 02:57:07 (51.6 MB/s) - `/dev/null' saved [104874307/104874307]

# wget -O /dev/null http://speedtest.wdc01.softlayer.com/downloads/test100.zip 2>&1 | awk '/\/dev\/null/ {speed=$3 $4} END {gsub(/\(|\)/,"",speed); print speed}'

24.1MB/s

# wget -O /dev/null http://speedtest.wdc01.softlayer.com/downloads/test100.zip

2014-02-13 02:57:19 (57.6 MB/s) - `/dev/null' saved [104874307/104874307]

# wget -O /dev/null http://speedtest.wdc01.softlayer.com/downloads/test100.zip 2>&1 | awk '/\/dev\/null/ {speed=$3 $4} END {gsub(/\(|\)/,"",speed); print speed}'

25.0MB/s

# wget -O /dev/null http://speedtest.wdc01.softlayer.com/downloads/test100.zip

2014-02-13 02:57:28 (61.0 MB/s) - `/dev/null' saved [104874307/104874307]

# wget -O /dev/null http://speedtest.wdc01.softlayer.com/downloads/test100.zip 2>&1 | awk '/\/dev\/null/ {speed=$3 $4} END {gsub(/\(|\)/,"",speed); print speed}'

24.8MB/s

# wget -O /dev/null http://speedtest.wdc01.softlayer.com/downloads/test100.zip

2014-02-13 02:57:38 (60.6 MB/s) - `/dev/null' saved [104874307/104874307]

 

I can even use a different file for the test and mix up the order, yet the discrepancy is still consistent:

 


# wget -O /dev/null http://speed.atl.coloat.com/100mb.test

2014-02-13 03:04:01 (68.9 MB/s) - `/dev/null' saved [104857600/104857600]

# wget -O /dev/null http://speed.atl.coloat.com/100mb.test 2>&1 | awk '/\/dev\/null/ {speed=$3 $4} END {gsub(/\(|\)/,"",speed); print speed}'

25.6MB/s


# wget -O /dev/null http://speed.atl.coloat.com/100mb.test 2>&1 | awk '/\/dev\/null/ {speed=$3 $4} END {gsub(/\(|\)/,"",speed); print speed}'

25.2MB/s

# wget -O /dev/null http://speed.atl.coloat.com/100mb.test 2>&1 | awk '/\/dev\/null/ {speed=$3 $4} END {gsub(/\(|\)/,"",speed); print speed}'

24.5MB/s

# wget -O /dev/null http://speed.atl.coloat.com/100mb.test

2014-02-13 03:07:30 (64.5 MB/s) - `/dev/null' saved [104857600/104857600]

# wget -O /dev/null http://speed.atl.coloat.com/100mb.test

2014-02-13 03:07:33 (70.8 MB/s) - `/dev/null' saved [104857600/104857600]


# wget -O /dev/null http://speed.atl.coloat.com/100mb.test

2014-02-13 03:08:25 (64.3 MB/s) - `/dev/null' saved [104857600/104857600]

# wget -O /dev/null http://speed.atl.coloat.com/100mb.test 2>&1 | awk '/\/dev\/null/ {speed=$3 $4} END {gsub(/\(|\)/,"",speed); print speed}'

25.9MB/s

 

 


I feel like a crazy person here, but there's no way all those can just be a coincidence? I've even tried at completely different times of the day -- same thing.
 
Last edited by a moderator:
N

NickM

New Member
Verified Provider
Edit: Nevermind, read the results wrong.
 
Last edited by a moderator:
eva2000

eva2000

Active Member
yeah looks like variance maybe ?

tried on one of my VPS ... look pretty much the same

Code: 
 wget -O /dev/null http://speedtest.wdc01.softlayer.com/downloads/test100.zip 2>&1 | awk '/\/dev\/null/ {speed=$3 $4} END {gsub(/\(|\)/,"",speed); print speed}'
79.3MB/s

 wget -O /dev/null http://speedtest.wdc01.softlayer.com/downloads/test100.zip
--2014-02-13 07:18:15--  http://speedtest.wdc01.softlayer.com/downloads/test100.zip
Resolving speedtest.wdc01.softlayer.com... 208.43.102.250
Connecting to speedtest.wdc01.softlayer.com|208.43.102.250|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104874307 (100M) [application/zip]
Saving to: /dev/null

100%[======================================================================================================================>] 104,874,307 74.6M/s   in 1.3s    

2014-02-13 07:18:17 (74.6 MB/s) - /dev/null saved [104874307/104874307]


 wget -O /dev/null http://speedtest.wdc01.softlayer.com/downloads/test100.zip 2>&1 | awk '/\/dev\/null/ {speed=$3 $4} END {gsub(/\(|\)/,"",speed); print speed}'
83.1MB/s

 wget -O /dev/null http://speedtest.wdc01.softlayer.com/downloads/test100.zip
--2014-02-13 07:18:33--  http://speedtest.wdc01.softlayer.com/downloads/test100.zip
Resolving speedtest.wdc01.softlayer.com... 208.43.102.250
Connecting to speedtest.wdc01.softlayer.com|208.43.102.250|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104874307 (100M) [application/zip]
Saving to: /dev/null

100%[======================================================================================================================>] 104,874,307 86.3M/s   in 1.2s    

2014-02-13 07:18:34 (86.3 MB/s) - /dev/null saved [104874307/104874307]
 
Dylan

Dylan

Active Member
Out of curiosity, what virtualization is that VPS running on? This is 100% reproducible on both of my Xen VPSes (one with CentOS, one with Ubuntu), but it doesn't seem to happen at all on OpenVZ.
 
P

perennate

New Member
Verified Provider
wget will print out two speeds when you run it. The first speed is part of the process bar, and it is the instantaneous speed; after the command terminates (and when awk processes it) this will be the instantaneous speed at the end of the file download (or rather, the average speed over the last second or so of the download). The second speed is the average speed over the entire download. The discrepancy seems to be due to boosting at your ISP, where they boost the initial download and then since download is large it slows down (this increases speed of loading smaller websites, but not downloading very large files).

The instantaneous speed at the end has the advantage that it is not affected by boosting. The average speed over entire download has the advantage of incorporating more statistics.

Edit: I could be wrong, because I don't know awk well, but this would explain your thing

Edit2: actually probably this is wrong, hm
 
Last edited by a moderator:
B

BuyCPanel-Kevin

New Member
Verified Provider
What you could do is snip off little portions of the command until it is exactly the same as the second command or it's speed it similar to the second command and pin point where exactly the speed is coming from
 
sv01

sv01

Slow but sure
that's maybe problem with your provider. send them ticket :)

you alway run


wget -O /dev/null xxxxxx 2>&1 | awk '/\/dev\/null/ {speed=$3 $4} END {gsub(/\(|\)/,"",speed); print speed}'
first. Run wget without additional script/command first then run above.
Run both script in 5/10 or 30 minute interval
 
You must log in or register to reply here.
Top
amuck-landowner