# LiteSpeed vs. OpenLiteSpeed vs. nginx vs. Apache 2.2 vs. Apache 2.4!



## lsmichael

The battle you've been waiting for!

We just released a couple of new benchmarks (small static files) for LiteSpeed Enterprise, OpenLiteSpeed, nginx, Apache 2.2 prefork, and Apache 2.4 event MPM. Enterprise and OpenLiteSpeed whipped everyone else, of course.

Summary blog post

Small static file benchmark

Small static file (HTTPS) benchmark

 

These are just the first in a series of tests we're releasing. (Small PHP files should be soon behind.) We really want to hear feedback on these tests. Do you not believe them? Do you have issues with our configurations? What happened with Apache 2.4 with event MPM? What do you think? What kinds of benchmarks do you want to see?

 

Looking forward to it.

 

m


----------



## KuJoe

Any chance of adding Lighttpd to the mix? I foresee it picking up some market share with nginx's recent announcement.


----------



## nunim

KuJoe said:


> Any chance of adding Lighttpd to the mix? I foresee it picking up some market share with nginx's recent announcement.


??? What announcement?   Lighttpd seems to have died off in recent years,  I still using it for some things, mainly serving static files.


----------



## KuJoe

nunim said:


> ??? What announcement?   Lighttpd seems to have died off in recent years,  I still using it for some things, mainly serving static files.


nginx is going to a paid model with their free version removing features. Lighttpd doesn't get updated as often but it's still getting security updates.

EDIT - It looks like they aren't limiting the HTTP features according to their chart so I might be completely off base.


----------



## Dylan

KuJoe said:


> nginx is going to a paid model with their free version removing features.


Nothing's getting removed from the free version. The commercial version adds a few features (like an application delivery controller) that the free version's never had.

http://www.pcworld.com/article/2047200/nginx-web-server-goes-commercial-with-new-release.html

Anyways if OpenLiteSpeed had .htaccess compatibility I'd switch over in a heartbeat.


----------



## texteditor

Yeah the 'Premium' nginx just has a ton of features people had been using for years via modules, 3rd-party daemons, etc. all unified into a more all-in-one enterprise-y product


----------



## budi1413

texteditor said:


> Yeah the 'Premium' nginx just has a ton of features people had been using for years via modules, 3rd-party daemons, etc. all unified into a more all-in-one enterprise-y product


You know what, most enterprise people don't want to use free software because they think free software is not good.


----------



## drmike

I love these comparisons.  Always taken with a side of real sea salt.   

OpenLite, still not putting out a Debian repository ready to use version, right?   Adoption would shoot up some if they did.  

Also, OpenLite, does it have a web-based administration?


----------



## eva2000

drmike, OLS has some admin gui console as LWS

Nice benchmarks !

 

Curious as you're using ApacheBench, which version of ApacheBench was used as i see your logs have ./ab_new ?? i see in the logs using ApacheBench, Version 2.3 <$Revision: 1430300 $> so from Apache 2.4.6 ?

 

I ask as from my limited testing last year with ApacheBench, the version used can make a difference i.e ApacheBench included with Apache 2.4.x is faster than ApacheBench included in Apache 2.2.x for all web servers I tested, LiteSPeed, Apache and even Nginx http://vbtechsupport.com/1835/

 

Also, would be better to test at higher concurrency levels i.e. 250-500 and with Nginx 1.5.6 as well with more average/normal static file sizes

 

Digging into config files, I noticed LSW vs OLS, the inMemBufSize differ with LSW = 120MB and OLS = 60MB ? Would make a difference ?

 

Also seems OLS static file gzip compression level was higher at 6 versus LSW at level 1.

 




Code:


-    <gzipCompressLevel>1</gzipCompressLevel>  
-    <compressibleTypes>text/*,application/x-javascript,application/javascript,application/xml</compressibleTypes>  
+    <gzipCompressLevel>6</gzipCompressLevel>  
+    <compressibleTypes>text/html</compressibleTypes>


----------



## MannDude

Nice read, but would be good to see a test done from a unbiased source.

Whenever I scrolled over the links and saw they were on the Litespeed site, I already knew the results. I'm not saying I don't believe them, just saying I'd feel more comfortable accepting them as fact if all done by someone with no bias. Litespeed is certainly good, though. 



nunim said:


> Lighttpd seems to have died off in recent years,  I still using it for some things, mainly serving static files.


vpsBoard uses Lighttpd, for now at least. May replace it with Nginx in the coming month or so with some planned upgrades, though only because I'm more familiar with Nginx.


----------



## drmike

I am all for non-Apache stuff.. I hate Apache configs. 

Glad to see web configs/status/etc. in these servers.  Shortlisted for experimenting!


----------



## peterw

KuJoe said:


> Any chance of adding Lighttpd to the mix? I foresee it picking up some market share with nginx's recent announcement.


I would like to see the results of lighttpd too.



MannDude said:


> Nice read, but would be good to see a test done from a unbiased source.
> 
> Whenever I scrolled over the links and saw they were on the Litespeed site, I already knew the results. I'm not saying I don't believe them, just saying I'd feel more comfortable accepting them as fact if all done by someone with no bias. Litespeed is certainly good, though.


I stopped reading on



> Highlights
> LSWS Enterprise showed the greatest gains with keep-alive connections enabled:
> 
> HTTP
> 
> *245%* faster than Apache 2.2 with pre-fork MPM
> *533%* faster than Apache 2.4
> *67%* faster than nginx


----------



## fisle

If Litespeed is so much better than the rest, why have the big players been moving on to Nginx?

Needs more unbiased tests. Either way, some benchmarks are better than no benchmarks at all.


----------



## eva2000

benchmarks didn't suprise me much from my own experience and benchmarks, LiteSpeed > Nginx but not by as much as these benchmarks above showed..   probably due to different parameters and fact i use Centmin Mod based Nginx which is slightly tuned better http://centminmod.com/benchmarks_nginx_openlitespeed_cherokee.html



fisle said:


> If Litespeed is so much better than the rest, why have the big players been moving on to Nginx?
> 
> Needs more unbiased tests. Either way, some benchmarks are better than no benchmarks at all.


open source / free is most cited reason + LiteSpeed Enterprise's no Adult content policy (hence why OpenLiteSpeed without such content policy + open source is interesting)


----------



## lsmichael

Dylan said:


> Anyways if OpenLiteSpeed had .htaccess compatibility I'd switch over in a heartbeat.


Oh, man. This is what I get for being busy. Just going through the thread now. Feel free to think I'm an idiot if I repeat somethign that's already been said.

We're debating some sort of .htaccess compatibility for OpenLiteSpeed. It's still a little far away, but we're eyeing. It's wouldn't be as nice as Enterprise (.htaccess compatibility is one of Enterprise's main selling points), but we'd like to put together an option that speaks to people looking for less hassle. Perhaps a function that allows you to check .htaccess files and include them with a restart. If you're running a few web applications, this shouldn't be a huge deal. Just perform a restart when you upgrade your applications. But it would keep people from offering shared hosting on OpenLiteSpeed.

m


----------



## lsmichael

budi1413 said:


> You know what, most enterprise people don't want to use free software because they think free software is not good.


I'm really not sure about this. I mean, there are plenty of big enterprises running free software.

Wait, what am I saying? I mean, Yeah! They should all run LSWS Enterprise!


----------



## lsmichael

> I would like to see the results of lighttpd too.



Sorry, but I don't think this is going to happen. Yes, I know vpsBoard is running lighttpd, but they're just not a big enough competitor. I mean, people earlier in the thread thought they were dead. Configuring the web servers is the biggest time suck in these benchmarks and I don't think it would be worth the time to add lighttpd.

Would love to see the benchmark, if someone else wants to do it, though!

Oops, something just came up. I'll try to be back soon. Thanks for the feedback everyone!


----------



## adly

lsmichael said:


> We're debating some sort of .htaccess compatibility for OpenLiteSpeed. It's still a little far away, but we're eyeing. It's wouldn't be as nice as Enterprise (.htaccess compatibility is one of Enterprise's main selling points), but we'd like to put together an option that speaks to people looking for less hassle. Perhaps a function that allows you to check .htaccess files and include them with a restart. If you're running a few web applications, this shouldn't be a huge deal. Just perform a restart when you upgrade your applications. But it would keep people from offering shared hosting on OpenLiteSpeed.


It seems like an awful lot of effort to avoid giving people what they actually want. I understand you're afraid of canabalising sales of the Enterprise product, but crippling the open one isn't the right way to go about it.


I for one can say I have no interest in using an intentionally crippled product or supporting anyone who releases such a product.


--Adam


----------



## lsmichael

drmike said:


> OpenLite, still not putting out a Debian repository ready to use version, right?   Adoption would shoot up some if they did.


 Not yet. But this nice Italian guy has set up one: http://apt.balocco.name/


----------



## lsmichael

admdly said:


> It seems like an awful lot of effort to avoid giving people what they actually want. I understand you're afraid of canabalising sales of the Enterprise product, but crippling the open one isn't the right way to go about it.
> 
> 
> I for one can say I have no interest in using an intentionally crippled product or supporting anyone who releases such a product.
> 
> 
> --Adam


I have certainly heard this point of view before, and, if that's your view, so be it. We still think there's a market for a product that runs as fast as, if not faster than, nginx, but has some level of .htaccess compatibility and an easy-to-use GUI.

m


----------



## Deleted

Another useless benchmark.

Apache with MPM Prefork? Are you fucking kidding me? Of course other webservers are going to win, vfork/fork() is expensive!

"The static file was 100 bytes. We used such a small file to avoid saturating the network connection"

100 bytes? Well, litespeed does some mmap() magic on small files, so of course it's going to 'win' .. I could hack thttpd to do the same and have comparable speed with non pipelined connections.


----------



## drmike

Monkburger said:


> 100 bytes? Well, litespeed does some mmap() magic on small files, so of course it's going to 'win' .. I could hack thttpd to do the same and have comparable speed with non pipelined connections.


Somewhere right now some monitoring company has silly payloads that size or less.  Probably all different non-cacheable data though.

Seriously, real workloads would be appreciated in testing.  Today that's a page with 50 inclusions and 375 different elements.  A big pig pile later we realize throughput of most things is limited to dozens of simultaneous connections per daemon tops.

Years ago when I cared (many years ago) about benchies, I complained about the very same deficiencies.  Troubling to see benchies continuing like this.


----------



## George_Fusioned

Personally I love LiteSpeed! I use it on our Enterprise Hosting servers and it works fantastic with CloudLinux and PHP Selector. Also, I always suggest a customer to try the 15-day LiteSpeed trial before upgrading their hardware when their traffic increases.

Here's a recent case; a customer with a cPanel server running one high-traffic site (~2M pageviews per day) as well as another ~50 low-traffic sites.

Apache 2.2 mod_fcgid, MPM prefork + nginxcp vs LiteSpeed


----------



## splitice

George_Fusioned said:


> Personally I love LiteSpeed! I use it on our Enterprise Hosting servers and it works fantastic with CloudLinux and PHP Selector. Also, I always suggest a customer to try the 15-day LiteSpeed trial before upgrading their hardware when their traffic increases.
> 
> 
> Here's a recent case; a customer with a cPanel server running one high-traffic site (~2M pageviews per day) as well as another ~50 low-traffic sites.
> 
> 
> Apache 2.2 mod_fcgid, MPM prefork + nginxcp vs LiteSpeed


Unless that customer was just serving static files (a rare use case) I dont think that graph is correct, PHP / your dynamic web language of choice should be a very large (and constant) part of that graph. Unless something was changed in the process (e.g opcode caching) I would suspect a mistake has been made. Although I am sure removing apache prefork from any workload will result in large improvements.


----------



## George_Fusioned

I'm not sure what you mean when you say it's not correct...

It's the weekly CPU usage graph from a box serving 2M pageviews via Apache until the afternoon of Nov 14th, that switched to LiteSpeed.

Immidiatelly after switching to LiteSpeed, CPU usage went down to minimal levels.

Nothing was changed in the process. I simply installed LiteSpeed, compiled the matching PHP binary and switched from Apache to LiteSpeed (all with default settings).

Customer is running an affiliate network, so pageviews are actually banner views, however banners are not displayed as static files, img src is a dynamic URL (i.e. domain.com/file/get/path/.banners.51adxxxxce124/i/3xx2) to calculate statistics. So there's php+mysql use for each banner view.


----------



## splitice

The usual ratio of CPU usage used in webserving to the CPU used in rendering and processing for a page in PHP is atleast 1:10 (usually much more depending on the complexity and quality of the application). PHP has to do alot more work (parse, connect to mysql, generate a page, include files etc). Therefore, to achieve an improvement in the range that those graphs suggest there are the following possibilities:

a. That the configuration was so horribly wrong in the first place that Apache was using a copious amount of CPU. In which case massive improvements could be gained just from apache alone etc and hence this graph is in-proportionate.

b. Something changed (software, hardware or traffic) during the transition between webservers (e.g php 4->5 or the inclusion of an opcode cache such as APC that was not used before) that drastically altered performance or the scenario.

or

c. The values have been incorrectly recorded. Accidents happen.

Either a, b or c would disqualify this from being a valid comparison.

I am not trying to argue that Apache > LSWS, far the opposite. But if you look at your data, something is obviously wrong with it. Improvements in the range of tenfold from a change in an area (http) that should be at most taking 20% (more likely to be less) of your resources with the rest going to doing the actual work of page generation.

Unless there is something very specific to your clients web load, in which case you probably should mention it. I don't pretend to know that. And I dont mean to be combative, but your graph would lead people to beleive they can get a 20-40x improvement from a PHP workload simply by changing the HTTP layer, which is unrealistic as explained.


----------



## eva2000

splitice, it's not technically just a change in web server but in how PHP is executed and processed

LSW uses LSAPI PHP http://www.litespeedtech.com/products/litespeed-sapi/php



> PHP LiteSpeed SAPI has been developed to push PHP performance to next level, getting the most out of LiteSpeed's performance capabilities while keeping the flexibility and security you need from LiteSpeed. In addition to providing efficient communication between LSWS and external applications, PHP LSAPI contains many extra features, some of which are unique to LiteSpeed:
> 
> *Performance*
> 
> 
> LiteSpeed PHP delivers up to 30% better performance than FastCGI PHP and up to 50% better performance than Apache's mod_php.
> LiteSpeed's suEXEC daemon mode forks PHP processes instead of creating new processes. This means more efficiency and allows secure opcode cache use even in shared hosting.
> *Security*
> 
> 
> PHP LSAPI fully supports suEXEC mode, a necessity for shared hosting.
> PHP LSAPI also allows you to jail PHP process with chroot.
> *Flexibility*
> 
> 
> Change PHP configurations via web server configuration or .htacess files.
> Run multiple versions of PHP at once and easily customize which files or virtual hosts use which version.
> *PHP LSAPI is the best way to run PHP, and it's included in your PHP package. You're missing out if you're using anything else.*
> 
> (PHP is automatically configured with LSAPI if you compile PHP using our Build PHP utility. You can also download the latest update here.)


For *non-cached* PHP requests/scalability nothing out there beats LiteSpeed's LSAPI PHP implementation - not even php-fpm. If you bring php-fpm and fastcgi_cache in Nginx then that is a different matter and you'll have to compare it with LiteSpeed Enterprise's LiteSpeed Cache feature for dynamic PHP caching.

For folks who have never played with Litespeed Enterprise paid product, you should check out the *free* open source OpenLiteSpeed which uses same LSAPI PHP implementation as the LiteSpeed Enterprise and see for yourself how PHP performs i.e. http://centminmod.com/benchmarks_nginx_openlitespeed_cherokee.html

My simple one liner for OpenLiteSpeed install http://openlitespeed.com/threads/centos-git-repository-one-liner-openlitespeed-install-command.61/ and some tweaks you can do at http://vbtechsupport.com/2256/ and http://vbtechsupport.com/2330/. To recompile PHP for your own needs http://vbtechsupport.com/2214/


----------



## splitice

I am aware of LSAPI, its been around since the days that I used to run LSWS on one of my side projects (pre nginx). Yes its faster than php5-fpm in most circumstances but its marginal compared to the processing cost of php. Taking data from fastcgi / LSAPI -> HTTP is a really very small part of the load of most servers. Just include a few files and execute a few mysql queries and you will consume more CPU time.

For the record I have worked on large setups of Apache, LSWS and nginx. LSWS is a nice product, but it will not provide 20-40x the performance of a similarly optimized webserver under 99.99% of circumstances.... even against apache prefork.


----------



## eva2000

Yeah i don't think any of us other than George_Fusioned/his client knows what else makes up that difference seen in the graphs. Although Apache prefork MPM doesn't help heh

LiteSpeed folks published basic PHP test too http://blog.litespeedtech.com/2013/11/15/litespeed-suexec-40x-faster-than-apache-suphp-new-php-benchmarks/ and it falls in line with what I see for LiteSpeed LSAPI PHP compared to Apache mod_php and Nginx's php-fpm for low concurrency tests ~100 users.

Full results at http://www.litespeedtech.com/products/litespeed-web-server/benchmarks/php-hello-world


----------



## splitice

Maybe thats that then, suPHP is so far below everything else that it probably falls under "a".

Its also worth noting that  took a quick look at the configs used between all tests and they are hardly equal. Now I dont know what concurrency mode is used for the test, it isnt mentioned but the number of children available to process requests in PHP should atleast be the same (given they are handled the same way),

But the two files I opened

lsws-4.2.5\httpd_config.xml


PHP_FCGI_CHILDREN=20
php\php-fpm.conf




Code:


; The number of child processes to be created when pm is set to 'static' and the
; maximum number of child processes when pm is set to 'dynamic' or 'ondemand'.
; This value sets the limit on the number of simultaneous requests that will be
; served. Equivalent to the ApacheMaxClients directive with mpm_prefork.
; Equivalent to the PHP_FCGI_CHILDREN environment variable in the original PHP
; CGI. The below defaults are based on a server without much resources. Don't
; forget to tweak pm.* to fit your needs.
; Note: Used when pm is set to 'static', 'dynamic' or 'ondemand'
; Note: This value is mandatory.
pm.max_children = 5


 

 

And this is just one example. Not to mention tweaks etc, its entirely possible to make any webserver perform badly. I have seen nginx + fpm do similar req/s on less powerful hardware so I suspect there may be config issues (I haven't investigated).


----------



## eva2000

yeah true that.. i think they're going with out of box defaults for most


----------



## George_Fusioned

I don't know how Apache was configured before (and if someone had optimized it) as it's a self-managed VPS, but really the only thing I did was login to the server, download the cPanel/WHM LiteSpeed AutoInstaller and run it.

Client opened a ticket regarding high CPU load in his VPS, and as it wasn't node related I offered to install LiteSpeeed on his server for him just to see if it would improve things.






I'm not implying that everyone will see such improvements by switching to LiteSpeed, and of course it's related to the type of traffic each server generates - BUT unlike ab or other stress tests, this is from a real website, with real visitors, showing a real mesurable performance improvement. Unless someone believes I photoshoped those munin graphs...?

Heck my client is super happy, so that's end of story for me


----------



## lsmichael

Howdy Splitice,

I think your absolutely right to be concerned that the Apache was configured incorrectly. However, if I'm a VPS provider and somebody comes to me with a spaghetti VPS mess and I have two options — install LSWS or try to untangle their mess and _maybe _improve things somewhat — I can absolutely understand why it makes sense for the VPS provider to install LSWS.

Let's say this guy needs an Ultra VPS license. That costs him $20/month. I know that my time is worth at least $70/hour. Even if I can get the VPS figured out in two hours (and that's a big if), he could buy seven months of LiteSpeed for that price. And that doesn't include all the optimization that he gets with LiteSpeed — LSAPI, suEXEC daemon mode, event-driven architecture — benefits that optimization is never going to be able to achieve.

Also (and I think this is the last thing right now), from a very cursory look at the graph, this guy has a problem with traffic spikes. That's where event-driven architecture is really going to help out. Apache deals with spikes poorly, CPU usage increases exponentially because of all the processes being created. This is actually not an uncommon occurrance in very high-traffic situations. (Though I've been looking for a real life graph like this. Thanks, George!)

m


----------



## lsmichael

drmike said:


> Somewhere right now some monitoring company has silly payloads that size or less.  Probably all different non-cacheable data though.
> 
> Seriously, real workloads would be appreciated in testing.  Today that's a page with 50 inclusions and 375 different elements.  A big pig pile later we realize throughput of most things is limited to dozens of simultaneous connections per daemon tops.
> 
> Years ago when I cared (many years ago) about benchies, I complained about the very same deficiencies.  Troubling to see benchies continuing like this.


OK. Looking a little back in the thread.

I definitely understand this request. They are coming (but they take lots and lots of time to make).


----------



## lsmichael

MannDude said:


> Nice read, but would be good to see a test done from a unbiased source.
> 
> Whenever I scrolled over the links and saw they were on the Litespeed site, I already knew the results. I'm not saying I don't believe them, just saying I'd feel more comfortable accepting them as fact if all done by someone with no bias. Litespeed is certainly good, though.


Come to the dark side, MannDude. Coooome to the daaaark side. We have free chocolate, Cheez-Doodles, coconut water. (Not joking.)


----------



## scv

Lay off the caffeine dude, you're triple posting too hard.


Also,



George_Fusioned said:


>


I can second this - I've seen similar performance boosts by swapping Apache out with LiteSpeed on cPanel servers in the past. It makes a huge difference.


----------



## TSS - Conor

Nice speed test but I would never trust one done by one of the software involved in the test... could be bias 

This one is half-decent: http://www.whisperdale.net/11-nginx-vs-cherokee-vs-apache-vs-lighttpd.html


----------



## iPragmatech

We have also done benchmark with real application(Wordpress)

http://www.ipragmatech.com/best-webserver-wordpress-hosting-performance

Feel free to send us your feedback


----------



## vampireJ

These are all just speeds- how about the memory consumption and cpu? Because when you have a very busy website on limited resources- cpu and memory might be a bottleneck.


----------



## peterw

TSS - Conor said:


> Nice speed test but I would never trust one done by one of the software involved in the test... could be bias
> 
> This one is half-decent: http://www.whisperdale.net/11-nginx-vs-cherokee-vs-apache-vs-lighttpd.html





> 3. *Lighttpd* - While the 10 connections test was somewhat close to Nginx performance at 50 connections, the benchmarks date should be given some credit . *And if we take a look at the load difference from 10-50 **Simultaneous connections it is only 10%! *And that is awesome! While I have never given enough credit to Lighttpd server for it's slow development and release schedule compared to Nginx, this benchmark has opened my eyes and I can say it is still a worthy opponent to Nginx web server.


First test to compare system load. Good work!


----------



## lsmichael

iPragmatech said:


> We have also done benchmark with real application(Wordpress)
> 
> http://www.ipragmatech.com/best-webserver-wordpress-hosting-performance
> 
> Feel free to send us your feedback


Just reposting the feedback I left you on your blog:

Howdy Kapil, 

Thanks for bringing this up again on vpsBoard. I had forgotten to respond.


I believe you are reading your table incorrectly. Your table seems to be a representation of the spread of the times requests took, i.e. this data:


Percentage of the requests served within a certain time (ms)


50% 4414


66% 4488


75% 4543


80% 4578


90% 8705


95% 8948


98% 9054


99% 9118


100% 9252 (longest request)

It has no relation to increasing concurrency, as concurrency was kept the same throughout your test (at 100 connections). I do not think that the x-axis is time, but rather that these requests were ordered by the length of time they took. That is why, on the table, requests only get slower as you move along the x-axis.

What the table does show is that OpenLiteSpeed's slowest requests were slower than the other setups' slowest requests. (This may be where your 15% difference number is from.) However, the table also shows that the majority of requests served by OpenLiteSpeed (about 75%?) were served faster than the other setups. Also, on average, OpenLiteSpeed served requests marginally faster than both other setups, as noted by the average requests per second and median connection times data.

I'm not sure if I'll be able to find someone with time to look at what can be tweaked on your settings, though that might be interesting. I'll definitely see if there's someone here interested in taking that on.

Cheers,

Michael


----------



## lsmichael

vampireJ said:


> These are all just speeds- how about the memory consumption and cpu? Because when you have a very busy website on limited resources- cpu and memory might be a bottleneck.


You're absolutely right. The graphs posted in the thread show resource consumption.


Also, you might like our case studies. We just released two, both showing load and memory usage on real sites before and after switching to LiteSpeed. They definitely support the graphs posted on this thread. That load drop is for real. Event-driven architecture works.


----------

