DD Script

[tchen]

Big question is there much useful that can be ran from within a container to deduce performance baseline whatsoever?

It kinda depends on what you actually wanted to measure.  Despite our talk about the emulation layers in FS calls, if what you wanted was to measure app-to-metal performance and not raw, it still does that.  It'll just return a large range given

1) Time of day.

2) You're on a shared multi-tenant system.

3) File system/device is cached.

4) IO scheduler is nonlinear.

5) vsyscall may be cached.

I list these factors in descending order of effect.  #1 is diurnal.  #3 is somewhat controllable via the test method and #5 are infinitesimally small compared to #4.  #2-4 have reasonably strong covariances.  Just keep that in mind when parsing the data.  As a preliminary to performance testing

http://msdn.microsoft.com/en-us/library/bb924370.aspx

P.S. the above applies, regardless of virtualization/bare-metal, etc.

 

[Deleted]

Running benchmarks within any Virtualized environments is futile. You will never receive consistent results because of emulation (timers, interrupts), non direct access to hardware, CPU cache thrashing because of the excessive amounts of context switching, emulated instructions, etc.

No matter what 'hardware assisted' VXT stuff you enable in the bios, the above still apply.

 

[drmike]

Well, there needs to be metrics, measurements, etc. of these environments.  Long been an issue and inconsistent/random results.  Top empty node master metrics are equally as flawed, better somewhat, but indicative merely of empty server potential.

Simply, how does an end user really say much of anything when their container is laggy, disk impossibly slow and CPU non existent?   Saying just that falls into accusation territory and suspect to mass lashings from the fanboys of said company.

 

[tonyg]

So according to some people here, VPS benchmarks are meaningless...interesting.

So what do these people do to get a sense of performance?

Maybe they run the actual application?

Well, wouldn't it too be subjected to the same environment as a synthetic benchmark and have periods of highs and lows?

So then why even run a VPS?

 

[manacit]

So according to some people here, VPS benchmarks are meaningless...interesting.

So what do these people do to get a sense of performance?

Maybe they run the actual application?

Well, wouldn't it too be subjected to the same environment as a synthetic benchmark and have periods of highs and lows?

So then why even run a VPS?

Sure, they would be subjected to potential periods of low performance, but a good VPS host's low performance should be adequate. The difference between a host worth using and one that isn't is consistently good performance. 

 

[dcdan]

I would use apache benchmark tool (ab) to benchmark the whole setup. You could do it at certain intervals too, then feed results into mrtg or rrdtool or whatever.

If for some reason I absolutely had to make sure I/O is fast, then I'd probably create a big file (200 GB if possible), and then read a few 4K chunks at random locations of that file every 5 minutes & watch that it does not take over 50-100ms (per chunk) to complete (or 5-10ms per chunk for SSD-based storage).

 

[manacit]

Personally, I don't put anything on a LEB that I actually *need* - I only use a RamNode VPS as staging and the rest of my infrastructure is on dedicated servers.

If I were to host things, I'd simply monitor for things like iowait and steal time, and make sure my pages weren't lowing too slow.

 

[drmike]

That @manacit, a man after my own heart.

That's how I use VPS instances (not just LowEnd*), but all.   I use them as sandboxes for play.   

When things are production level, dedicated servers or colo units (I prefer these).

Even iowait above is poked at as being bad indicator of much in virtual environment   SSDs and caching of SSDs likely further distort/create weirdness with iowait.   Been seeing that a bit here and there over past few years testing things.  

Steal time, what exactly is that and in theory how does one measure?  Not a term I use regularly.

 

[tchen]

Steal time is time the kernel can't account for as user, system or idle.  In a virtual environment, if you have some cpu bandwidth limiter, the kernel will see work being done outside its purview - effectively the hypervisor 'steals' it back.  This becomes the %st.  I've seen it in heavily limited environments like AWS micro instances.  I'm not sure you can even get it under OpenVZ containers, although I haven't looked too closely at it.

The big 1-million dollar question is, is it stealing because the machine is overloaded, or is it something less sinister?  The answer is the later at least in the AWS case.  In order to make sense of %st, you need to look at your actual proc/cpuinfo.  In the EC2 case, the listed CPU clock speeds are way higher than the virtual CPU that's assigned per node.  Because of that, once you reach the allotted work on that vCPU, there's technically still some available work on the real CPU.  The hypervisor needs to reassign the CPU to another VM hence why it appears to be stolen.

The real answer will depend on your provider, but since no one outside of the big-iron bunch ever bother listing what their logical CPU 'core' really is, there's no way to be certain its not overloaded.  The only bright side to it is that since its such a misunderstood term and was horribly called 'steal time', no provider wants to be caught having a consistent %st.  

 

[Deleted]

The only way to know is from the host node itself, not from the containers; that information is not exposed to the containers (since they run in ring3)

The good thing about VPS is the translation layer for block I/O is cached, so you'll always get decent write speed (and latency) compared to dedicated (because of vfs cache on the host node)

What needs to happen is that loadavg needs to have 2 values, your container's 1/5/15, and the host node's last 1/5/15 averages. Then you can tell if it's a piece of shit or not.

 

[Thelen]

If everyone started running a DD on a cronjob every x hours I'd honestly setup monitoring to stop and/or limit the tests. That'd ad huge unnecessary disk activity for no real reason.

+1 there, as a provider 99% of disk IO complaints are FROM dd tests lol.