LiteSpeed vs. OpenLiteSpeed vs. nginx vs. Apache 2.2 vs. Apache 2.4!

[Deleted]

Another useless benchmark.

Apache with MPM Prefork? Are you fucking kidding me? Of course other webservers are going to win, vfork/fork() is expensive!

"The static file was 100 bytes. We used such a small file to avoid saturating the network connection"

100 bytes? Well, litespeed does some mmap() magic on small files, so of course it's going to 'win' .. I could hack thttpd to do the same and have comparable speed with non pipelined connections. 

 

[drmike]

100 bytes? Well, litespeed does some mmap() magic on small files, so of course it's going to 'win' .. I could hack thttpd to do the same and have comparable speed with non pipelined connections. 

Somewhere right now some monitoring company has silly payloads that size or less.  Probably all different non-cacheable data though.

Seriously, real workloads would be appreciated in testing.  Today that's a page with 50 inclusions and 375 different elements.  A big pig pile later we realize throughput of most things is limited to dozens of simultaneous connections per daemon tops.

Years ago when I cared (many years ago) about benchies, I complained about the very same deficiencies.  Troubling to see benchies continuing like this.

 

[George_Fusioned]

Personally I love LiteSpeed! I use it on our Enterprise Hosting servers and it works fantastic with CloudLinux and PHP Selector. Also, I always suggest a customer to try the 15-day LiteSpeed trial before upgrading their hardware when their traffic increases.

Here's a recent case; a customer with a cPanel server running one high-traffic site (~2M pageviews per day) as well as another ~50 low-traffic sites.

Apache 2.2 mod_fcgid, MPM prefork + nginxcp vs LiteSpeed

 

[splitice]

Personally I love LiteSpeed! I use it on our Enterprise Hosting servers and it works fantastic with CloudLinux and PHP Selector. Also, I always suggest a customer to try the 15-day LiteSpeed trial before upgrading their hardware when their traffic increases.


Here's a recent case; a customer with a cPanel server running one high-traffic site (~2M pageviews per day) as well as another ~50 low-traffic sites.


Apache 2.2 mod_fcgid, MPM prefork + nginxcp vs LiteSpeed

Unless that customer was just serving static files (a rare use case) I dont think that graph is correct, PHP / your dynamic web language of choice should be a very large (and constant) part of that graph. Unless something was changed in the process (e.g opcode caching) I would suspect a mistake has been made. Although I am sure removing apache prefork from any workload will result in large improvements.

 

[George_Fusioned]

I'm not sure what you mean when you say it's not correct...

It's the weekly CPU usage graph from a box serving 2M pageviews via Apache until the afternoon of Nov 14th, that switched to LiteSpeed.

Immidiatelly after switching to LiteSpeed, CPU usage went down to minimal levels.

Nothing was changed in the process. I simply installed LiteSpeed, compiled the matching PHP binary and switched from Apache to LiteSpeed (all with default settings).

Customer is running an affiliate network, so pageviews are actually banner views, however banners are not displayed as static files, img src is a dynamic URL (i.e. domain.com/file/get/path/.banners.51adxxxxce124/i/3xx2) to calculate statistics. So there's php+mysql use for each banner view.

 

[splitice]

The usual ratio of CPU usage used in webserving to the CPU used in rendering and processing for a page in PHP is atleast 1:10 (usually much more depending on the complexity and quality of the application). PHP has to do alot more work (parse, connect to mysql, generate a page, include files etc). Therefore, to achieve an improvement in the range that those graphs suggest there are the following possibilities:

a. That the configuration was so horribly wrong in the first place that Apache was using a copious amount of CPU. In which case massive improvements could be gained just from apache alone etc and hence this graph is in-proportionate.

b. Something changed (software, hardware or traffic) during the transition between webservers (e.g php 4->5 or the inclusion of an opcode cache such as APC that was not used before) that drastically altered performance or the scenario.

or

c. The values have been incorrectly recorded. Accidents happen.

Either a, b or c would disqualify this from being a valid comparison.

I am not trying to argue that Apache > LSWS, far the opposite. But if you look at your data, something is obviously wrong with it. Improvements in the range of tenfold from a change in an area (http) that should be at most taking 20% (more likely to be less) of your resources with the rest going to doing the actual work of page generation.

Unless there is something very specific to your clients web load, in which case you probably should mention it. I don't pretend to know that. And I dont mean to be combative, but your graph would lead people to beleive they can get a 20-40x improvement from a PHP workload simply by changing the HTTP layer, which is unrealistic as explained.

 

[eva2000]

splitice, it's not technically just a change in web server but in how PHP is executed and processed

LSW uses LSAPI PHP http://www.litespeedtech.com/products/litespeed-sapi/php

PHP LiteSpeed SAPI has been developed to push PHP performance to next level, getting the most out of LiteSpeed's performance capabilities while keeping the flexibility and security you need from LiteSpeed. In addition to providing efficient communication between LSWS and external applications, PHP LSAPI contains many extra features, some of which are unique to LiteSpeed:

Performance

• LiteSpeed PHP delivers up to 30% better performance than FastCGI PHP and up to 50% better performance than Apache's mod_php.
• LiteSpeed's suEXEC daemon mode forks PHP processes instead of creating new processes. This means more efficiency and allows secure opcode cache use even in shared hosting.

Security

• PHP LSAPI fully supports suEXEC mode, a necessity for shared hosting.
• PHP LSAPI also allows you to jail PHP process with chroot.

Flexibility

• Change PHP configurations via web server configuration or .htacess files.
• Run multiple versions of PHP at once and easily customize which files or virtual hosts use which version.

PHP LSAPI is the best way to run PHP, and it's included in your PHP package. You're missing out if you're using anything else.

(PHP is automatically configured with LSAPI if you compile PHP using our Build PHP utility. You can also download the latest update here.)

For non-cached PHP requests/scalability nothing out there beats LiteSpeed's LSAPI PHP implementation - not even php-fpm. If you bring php-fpm and fastcgi_cache in Nginx then that is a different matter and you'll have to compare it with LiteSpeed Enterprise's LiteSpeed Cache feature for dynamic PHP caching.

For folks who have never played with Litespeed Enterprise paid product, you should check out the free open source OpenLiteSpeed which uses same LSAPI PHP implementation as the LiteSpeed Enterprise and see for yourself how PHP performs i.e. http://centminmod.com/benchmarks_nginx_openlitespeed_cherokee.html

My simple one liner for OpenLiteSpeed install http://openlitespeed.com/threads/centos-git-repository-one-liner-openlitespeed-install-command.61/ and some tweaks you can do at http://vbtechsupport.com/2256/ and http://vbtechsupport.com/2330/. To recompile PHP for your own needs http://vbtechsupport.com/2214/

 

[splitice]

I am aware of LSAPI, its been around since the days that I used to run LSWS on one of my side projects (pre nginx). Yes its faster than php5-fpm in most circumstances but its marginal compared to the processing cost of php. Taking data from fastcgi / LSAPI -> HTTP is a really very small part of the load of most servers. Just include a few files and execute a few mysql queries and you will consume more CPU time.

For the record I have worked on large setups of Apache, LSWS and nginx. LSWS is a nice product, but it will not provide 20-40x the performance of a similarly optimized webserver under 99.99% of circumstances.... even against apache prefork.

 

[eva2000]

Yeah i don't think any of us other than George_Fusioned/his client knows what else makes up that difference seen in the graphs. Although Apache prefork MPM doesn't help heh

LiteSpeed folks published basic PHP test too http://blog.litespeedtech.com/2013/11/15/litespeed-suexec-40x-faster-than-apache-suphp-new-php-benchmarks/ and it falls in line with what I see for LiteSpeed LSAPI PHP compared to Apache mod_php and Nginx's php-fpm for low concurrency tests ~100 users.

Full results at http://www.litespeedtech.com/products/litespeed-web-server/benchmarks/php-hello-world

 

[splitice]

Maybe thats that then, suPHP is so far below everything else that it probably falls under "a".

Its also worth noting that  took a quick look at the configs used between all tests and they are hardly equal. Now I dont know what concurrency mode is used for the test, it isnt mentioned but the number of children available to process requests in PHP should atleast be the same (given they are handled the same way),

But the two files I opened

lsws-4.2.5\httpd_config.xml


PHP_FCGI_CHILDREN=20
php\php-fpm.conf

; The number of child processes to be created when pm is set to 'static' and the
; maximum number of child processes when pm is set to 'dynamic' or 'ondemand'.
; This value sets the limit on the number of simultaneous requests that will be
; served. Equivalent to the ApacheMaxClients directive with mpm_prefork.
; Equivalent to the PHP_FCGI_CHILDREN environment variable in the original PHP
; CGI. The below defaults are based on a server without much resources. Don't
; forget to tweak pm.* to fit your needs.
; Note: Used when pm is set to 'static', 'dynamic' or 'ondemand'
; Note: This value is mandatory.
pm.max_children = 5

 

 

And this is just one example. Not to mention tweaks etc, its entirely possible to make any webserver perform badly. I have seen nginx + fpm do similar req/s on less powerful hardware so I suspect there may be config issues (I haven't investigated).

 

[eva2000]

yeah true that.. i think they're going with out of box defaults for most 

 

[George_Fusioned]

I don't know how Apache was configured before (and if someone had optimized it) as it's a self-managed VPS, but really the only thing I did was login to the server, download the cPanel/WHM LiteSpeed AutoInstaller and run it.

Client opened a ticket regarding high CPU load in his VPS, and as it wasn't node related I offered to install LiteSpeeed on his server for him just to see if it would improve things.

I'm not implying that everyone will see such improvements by switching to LiteSpeed, and of course it's related to the type of traffic each server generates - BUT unlike ab or other stress tests, this is from a real website, with real visitors, showing a real mesurable performance improvement. Unless someone believes I photoshoped those munin graphs...?

Heck my client is super happy, so that's end of story for me

 

[lsmichael]

Howdy Splitice,

I think your absolutely right to be concerned that the Apache was configured incorrectly. However, if I'm a VPS provider and somebody comes to me with a spaghetti VPS mess and I have two options — install LSWS or try to untangle their mess and maybe improve things somewhat — I can absolutely understand why it makes sense for the VPS provider to install LSWS.

Let's say this guy needs an Ultra VPS license. That costs him $20/month. I know that my time is worth at least $70/hour. Even if I can get the VPS figured out in two hours (and that's a big if), he could buy seven months of LiteSpeed for that price. And that doesn't include all the optimization that he gets with LiteSpeed — LSAPI, suEXEC daemon mode, event-driven architecture — benefits that optimization is never going to be able to achieve.

Also (and I think this is the last thing right now), from a very cursory look at the graph, this guy has a problem with traffic spikes. That's where event-driven architecture is really going to help out. Apache deals with spikes poorly, CPU usage increases exponentially because of all the processes being created. This is actually not an uncommon occurrance in very high-traffic situations. (Though I've been looking for a real life graph like this. Thanks, George!)

m

 

[lsmichael]

Somewhere right now some monitoring company has silly payloads that size or less.  Probably all different non-cacheable data though.

Seriously, real workloads would be appreciated in testing.  Today that's a page with 50 inclusions and 375 different elements.  A big pig pile later we realize throughput of most things is limited to dozens of simultaneous connections per daemon tops.

Years ago when I cared (many years ago) about benchies, I complained about the very same deficiencies.  Troubling to see benchies continuing like this.

OK. Looking a little back in the thread.

I definitely understand this request. They are coming (but they take lots and lots of time to make).

 

[lsmichael]

Nice read, but would be good to see a test done from a unbiased source.

Whenever I scrolled over the links and saw they were on the Litespeed site, I already knew the results. I'm not saying I don't believe them, just saying I'd feel more comfortable accepting them as fact if all done by someone with no bias. Litespeed is certainly good, though.

Come to the dark side, MannDude. Coooome to the daaaark side. We have free chocolate, Cheez-Doodles, coconut water. (Not joking.)

 

[scv]

Lay off the caffeine dude, you're triple posting too hard.


Also,

I can second this - I've seen similar performance boosts by swapping Apache out with LiteSpeed on cPanel servers in the past. It makes a huge difference.

 

[TSS - Conor]

Nice speed test but I would never trust one done by one of the software involved in the test... could be bias

This one is half-decent: http://www.whisperdale.net/11-nginx-vs-cherokee-vs-apache-vs-lighttpd.html

 

[iPragmatech]

We have also done benchmark with real application(Wordpress)

http://www.ipragmatech.com/best-webserver-wordpress-hosting-performance

Feel free to send us your feedback

 

[vampireJ]

These are all just speeds- how about the memory consumption and cpu? Because when you have a very busy website on limited resources- cpu and memory might be a bottleneck.

 

[peterw]

Nice speed test but I would never trust one done by one of the software involved in the test... could be bias

This one is half-decent: http://www.whisperdale.net/11-nginx-vs-cherokee-vs-apache-vs-lighttpd.html

3. Lighttpd - While the 10 connections test was somewhat close to Nginx performance at 50 connections, the benchmarks date should be given some credit . And if we take a look at the load difference from 10-50 Simultaneous connections it is only 10%! And that is awesome! While I have never given enough credit to Lighttpd server for it's slow development and release schedule compared to Nginx, this benchmark has opened my eyes and I can say it is still a worthy opponent to Nginx web server.

First test to compare system load. Good work!