DD Script

[Mun]

I know many of you hate LET now, so I am posting here for feed back as well.

http://lowendtalk.com/discussion/24393/dd-script#latest

I built a bash + crontab script that creates a list of your historical dd as well as one current one.

It can be seen here: Current: http://192.3.139.124/dd.txt Historical:http://192.3.139.124/dd_historical.txt

My fear is that it will be abused and cause people to aimlessly use this to "monitor" there server. Though you may say mine was aimless as well, I will disagree. In any case, I'd like to hear your thoughts if I should make this available as well as a poll.

Mun

 

[yolo]

This

Is

The

Stupidest

Thing

In

The

World

I could not think of anything worse that anybody has made to 'monitor' their VPS

 

[Packety]

hmm, why not?

 

[Mun]

This

Is

The

Stupidest

Thing

In

The

World

I could not think of anything worse that anybody has made to 'monitor' their VPS

Well You only live once, so who gives a fuck, right?

 

[yolo]

Well You only live once, so who gives a fuck, right?

If you want to be banned from every VPS provider yea!

 

[Mun]

I more just hate the fact your username is yolo, I hear your point.

Mun

 

[tonyg]

I like the idea but not the execution. After some time all this data will be unreadable.

Either:

Graph the data

or

Reduce the number of runs each day to maybe every six or eight hours

 

[Mun]

I like the idea but not the execution. After some time all this data will be unreadable.

Either:

Graph the data

or

Reduce the number of runs each day to maybe every six or eight hours

I like the idea but not the execution. After some time all this data will be unreadable.

Either:

Graph the data

or

Reduce the number of runs each day to maybe every six or eight hours

I did actually I think it is on 12?

 

[tonyg]

I thought I saw runs every 2 hours.

 

[Mun]

I thought I saw runs every 2 hours.

You did, I later changed it after I noticed an improvement on the hosts node and saw no need to check every few hours. (plus sorta felt bad)

 

[SkylarM]

You did, I later changed it after I noticed an improvement on the hosts node and saw no need to check every few hours. (plus sorta felt bad)

If everyone started running a DD on a cronjob every x hours I'd honestly setup monitoring to stop and/or limit the tests. That'd ad huge unnecessary disk activity for no real reason.

 

[tchen]

Sweet jesus, 1GB!?

 

[Mun]

Sweet jesus, 1GB!?

Do you have a suggestion on which DD test to use? I can modify the script to use less, as long as people still see it as a viable test.

 

[Flapadar]

You did, I later changed it after I noticed an improvement on the hosts node and saw no need to check every few hours. (plus sorta felt bad)

And so you should. DD shouldn't be run to test write speed unless you're using it as one of several methods to prove a problem to your provider. 

The only time you should even consider releasing that script is if it checks steal time and iowait to see if they're high before running the test. However - that will still make the problem worse! 

 

[Mun]

I think I proved an issue with my provider when they where getting nearly constant I/O speed of less then 20MB/s.

Mun

 

[Flapadar]

I think I proved an issue with my provider when they where getting nearly constant I/O speed of less then 20MB/s.

Mun

Why not just monitor iowait and steal time instead? 

 

[Mun]

Why not just monitor iowait and steal time instead? 

I did those as well.

 

[Flapadar]

I did those as well.

Why do you see DD as a necessary part of the script? 

 

[tchen]

Do you have a suggestion on which DD test to use? I can modify the script to use less, as long as people still see it as a viable test.

I really can't recommend any sustained DD test.  If you're on a full virt like KVM or Xen, just read proc/diskstats.  On OpenVZ, the only thing I'd recommend is a 'single' ioping.  At regular intervals (much like what you're doing now with dd), it still paints a pretty good picture of any IO degradation before it becomes an issue without seriously adding to the problem.

The problem with the 1GB or anything sizeable is that it's going to blow out any cache.  At which point, you're really testing only the raw spindle speed, not anything related to how the node is setup.  To boot, that's going to fluctuate as the IO scheduler interleaves you with the other guy doing the dd test.  

P.S. I choose ioping because it defaults to just 4k.  More indicative of any random seek from your database / apache than sustained dd.

 

[Mun]

IOWAIT is good to determine if your process(es) are/is being held up by your disk / latency, but it isn't a good determiner if that IOWAIT is because of how the program is written, or if it is because the disk is getting hammered.

Mun