Benefits and downfalls of running Nginx and Apache side-by-side?

Hey all,

I'm looking into running Nginx and Apache side by side to power vpsBoard. Currently running only Nginx and serving PHP via php5-fpm, though it seems once every day or so that PHP likes to consume all CPU and throw out errors. I'm looking into different options of correcting this as well, but one thing that has been suggested to me more than once was having Apache serve PHP and let Nginx handle the static load. At the time, I hadn't ran a setup like this before so logged into the dev server and setup a basic LNMP stack and did a reverse proxy to let Apache serve PHP requests. It worked, but it's a no-traffic dev server, so no load so not a very good representation of the setup.

So now I'm curious what the benefits and downfalls of this setup would be. On the existing production VM for the WWW server (currently only runs Nginx and PHP) I have the spare resources, so running Apache side-by-side shouldn't be an issue as far as resources go (I don't think so, anyway). Would having Apache in the mix open the door to attacks like Slowloris? Any other things I should be aware about?

Thanks!

Meh, you can what you propose....  Two stacks to maintain.  Big security picture recommended for diversity, but need the knowledge on both and to maintain both.

Why not just double stack Nginx?  Run two instances... Front one filters stupid stuff out, polices common things, etc.   Other one just does the PHP page making stuff..

I've done this for a long time and remember load prior on some real servers compared to post-doubling-up of Nginx.  Night and day. Why?  Tons of crapola traffic.  Malcious stuff intentionally, spiders, etc. I'd start running a police/block layer with some lists and your own custom blocks.

PHP stuff though, needs debugged.   Checked the number of workers?  Might need bumped...  Common issue.

This is what I tend to run with various levels of optimization and security layers on both "stacks". Works fairly well, if one setup does not block something, the other stands in the way.

but one thing that has been suggested to me more than once was having Apache serve PHP and let Nginx handle the static load.

Click to expand...

'Tis what I've been doing for a long time on larger sites.  Nginx > Apache MPM-Worker + PHP-FPM (plus php-apc,memcached, etc)

One idea that comes to mind here is setting up PHP backend in Nginx as-is... but with additional PHP connection pool backend for failover purposes...

Anyone doing such a thing?

PHP hitting 100% CPU and staying there, someone ought to have a script to watch for such and able to graft onto it logic to restart problematic pieces.

Why not just front-end with Varnish caching? Why not look into verbose logging to see what is causing the PHP 100% CPU usage? Curious though as to what the configuration settings are.

pm'd

TruvisT said:

Why not just front-end with Varnish caching? Why not look into verbose logging to see what is causing the PHP 100% CPU usage? Curious though as to what the configuration settings are.

Click to expand...

Agreed.  If you want to improve static file serving, front apache with varnish.

Personally I defiantly agree with all the above, your get little to no performance boast from running Apache + PHP in the backend, but will defiantly get a good performance boost from running Varnish in front of Nginx as a cache. With correct configuration it can load the static files much quicker than Nginx or Apache ever can.

I recently set this up for stephensstory.co.uk which you guys probably seen in the news and happily survived a massive demand on a Quad Core  E3 16GB Ram server with no issues.

Happy to chat through about it more.

TruvisT said:

Why not just front-end with Varnish caching? Why not look into verbose logging to see what is causing the PHP 100% CPU usage? Curious though as to what the configuration settings are.

Click to expand...

Varnish does not handle SSL, that would just add another required service on the stack, something like Pound. 

KS_Phillip said:

Agreed.  If you want to improve static file serving, front apache with varnish.

Click to expand...

 And you can cache dynamic content which is amazing too.

WebSearchingPro said:

Varnish does not handle SSL, that would just add another required service on the stack, something like Pound.

Click to expand...

True. I forgot about that. Generally, you normally do not SSL the entire site, only the login parts for example, which can be pushed directly to nginx.
I've not heard of Pound, I will for sure try that out.

This is a good article which covers the above a backend Nginx/Apache Fronted Varnish and Pound managing the SSL:

http://www.geoffstratton.com/2013/11/web-server-performance-part-ii-varnish-pound/

Something like that will no doubt massively increase performance and remove some load of PHP and defiantly off Nginx.

MannDude said:

So now I'm curious what the benefits and downfalls of this setup would be. On the existing production VM for the WWW server (currently only runs Nginx and PHP) I have the spare resources, so running Apache side-by-side shouldn't be an issue as far as resources go (I don't think so, anyway). Would having Apache in the mix open the door to attacks like Slowloris? Any other things I should be aware about?

Click to expand...

Regarding Slowloris, I'd use mod_qos to kick off any connections that behave like Slowloris.  Here's the configuration I use for mod_qos:


<IfModule mod_qos.c>
# handles connections from up to 100000 different IPs
QS_ClientEntries 100000
# will allow only 50 connections per IP
QS_SrvMaxConnPerIP 50
# maximum number of active TCP connections is limited to 1000
MaxClients 1000
# disables keep-alive when 70% of the TCP connections are occupied:
QS_SrvMaxConnClose 700
# minimum request/response speed (deny slow clients blocking the server, ie. slowloris keeping connections open without requesting anything):
QS_SrvMinDataRate 150 1200
# and limit request header and body (careful, that limits uploads and post requests too):
# LimitRequestFields 30
# QS_LimitRequestBody 102400
</IfModule>

(source)

I've colleagues who suggest mod_antiloris, so that's another option.

If you do it right, Apache and PHP for dynamic content and Nginx for static content would work out quite well.

Why does PHP take up 100% CPU anyway?  Can you `strace` some of the abusive processes, or does FastCGI prevent you from getting any useful information?

Nick

Have you considered using openlitespeed, i host a busy magento site for a client, at first i tried useing apache by it was eating resources to the point that i needed to restart apache 2 or 3 times a day.

Then i tried nginx the resources usage was much better compreed to apache, but got lots of 503 error when the site was busy.

Then i tried openlitespeed which so far has been the only web server i tested that has not throwed 503 error under high traffic and resources usage lower then both apache and nginx

WebSearchingPro said:

Varnish does not handle SSL, that would just add another required service on the stack, something like Pound. 

Click to expand...

Pound, stunnel, etc all fill the bill here.

Razza said:

Have you considered using openlitespeed, i host a busy magento site for a client, at first i tried useing apache by it was eating resources to the point that i needed to restart apache 2 or 3 times a day.

Then i tried nginx the resources usage was much better compreed to apache, but got lots of 503 error when the site was busy.

Then i tried openlitespeed which so far has been the only web server i tested that has not throwed 503 error under high traffic and resources usage lower then both apache and nginx

Click to expand...

It really depends on what you'd refer to as "high traffic". We had the same instance and we just recompiled using Apache + Nigix . Some base settings on the server had to be changed as well. {this was a dedicated client with managed services} Cagefs and a downgrade of php to a lower version to support the cart modules that he was using.

Different instances requires different approach for a solution.

TruvisT said:

Why not just front-end with Varnish caching? Why not look into verbose logging to see what is causing the PHP 100% CPU usage? Curious though as to what the configuration settings are.

Click to expand...

Are you saying Varnish is better than Nginx?

GreenHostBox said:

Are you saying Varnish is better than Nginx?

Click to expand...

Yes/No. Depends on the situation.

()

This website keeps returning HTTP 504 errors on occasion.

@MannDude, are you sure the CPU is the bottleneck when your site spawns PHP processes that your production VM cannot handle?  If it's a virtual machine, you could add more CPU power to it to keep the site alive, couldn't you?

I'd recommend doing an `strace` on some of your PHP processes to see what is taking so long to run.

Here's a Bash one-liner that captures php processes from user user and writes them out to .log files in the current working directory:


x=1; while [ $x = 1 ]; do process=$(pgrep -u user php); if [ $process ]; then x=0; fi; done; strace -tfv -s1000 -o strace.`date | awk '{print $3 $2 $6}'`.log -p $process

Nick

strace data will only log syscalls. it's probably chewing 100% on mprotect or others, or tons of stat()/stat64 calls. You can measure what syscall is taking the most time with the -c flag. Just remember, if php does a mprotect()/mmap() and it returns ENOMEM, PHP /can/ give strange 500 errors. Might want to make sure you're not running out of memory (virtual or otherwise)

PHP /will/ consume a large amount of CPU if, say, a temporary folder (caches or others) have a large amount of files.