PHP - optimization/limiting RAM - low RAM use

[drmike]

Running another out of box clean PHP-FPM instance.   

Having issue where randomly PHP bursts up, smacks VPS RAM ceiling and PHP goes nuts/non responsive.

For those that tinker with PHP and do so on limited RAM servers --- what are the tweaks you recommend to keep PHP within RAM limits and well behaved?

 

[WebSearchingPro]

I'd recommend using a reverse proxy like nginx or varnish in conjunction with a PHP accelerator like xCache. You can also decrease the PHP memory limits in the php.ini file.

 

[zzrok]

Are you using a fixed  or dynamic number of children?  Maybe too many are being spawned in a short period of time and using too much memory.  Also, PHP 5.4 significantly improved memory consumption, so you should run it or 5.5 if you aren't.

 

[sleddog]

/etc/php5/fpm/pool.d/www.conf

pm = ondemand
pm.max_children = 2
 

...tweak max_children to what your memory can safely support.

 

[devonblzx]

Is the problem that too many PHP processes are spawning or the problem that the PHP processes themselves are using too much memory?

 

[drmike]

@devonblzx... the problem is total RAM consumption by PHP.  Have yet to track down the origin/cause.   

Is there a thread/process analysis tool for PHP to isolate the pig request memory users?  (sorry I avoid PHP when I can, but so darn prevalent that I blindly bump into using now and then).

 

[devonblzx]

You should be able to track down some statistics by just looking at 'ps aux | grep php'

I'm assuming you have it setup so php runs as the user and not as nobody?  That way you can start to track down the root cause of the memory usage.  If you have a lot of processes spawned under one user or high memory usage per process under one user (it may be a web application being a memory hog or causing a memory leak).

 

[sleddog]

PHP 101:

- The memory consumption of a PHP process is determined by the app it's running. A process running a "Hello World" script will consume very little memory. A process running Wordpress will consume a helluva lot more.

- If you're serving a lightweight script, you can run a lot more concurrent processes than if you're running a memory-heavy script.

- When running php you should put the server first, and the user second. Use pm.max_children to limit the maximum number of concurrent processes, according to the memory consumption of your php script and the system's available ram.

- If the limit set by pm.max_children is reached, subsequent users (site visitors) will have to wait and watch the browser spinner until a php process is available.

- php-fpm has built in stats which you can enable/setup so you can see how frequently this happens. Using this, you can tweak pm.max_children to ensure that you're making the most of your available memory, or know that you simply need more ram to serve your scripts to the traffic you're getting.

 

[drmike]

You should be able to track down some statistics by just looking at 'ps aux | grep php'

I'm assuming you have it setup so php runs as the user and not as nobody?  That way you can start to track down the root cause of the memory usage.  If you have a lot of processes spawned under one user or high memory usage per process under one user (it may be a web application being a memory hog or causing a memory leak).

Indeed php runs as nobody.  Certainly memory hog is the web application and simultaneous accesses and presumed leaks.  Typical web application scenario

Being able to isolate what is requested/processing when the RAM spikes, hits limit, then self implodes is the overall goal.  I'd like to just quarantine PHP to live on a paupers RAM diet and call it good enough though.  Given that not causing undue queuing and page timeouts.

I'd front end this PHP instance with another Nginx or Varnish instance, but bound to break all sorts of whiz bang features in software stack --- namely with IP origin of the request. Obviously, values get socked elsewhere and accessible, but unsure about existing software and not interested in getting a degree on hacking that to work right.

 

[drmike]

- php-fpm has built in stats which you can enable/setup so you can see how frequently this happens. Using this, you can tweak pm.max_children to ensure that you're making the most of your available memory, or know that you simply need more ram to serve your scripts to the traffic you're getting.

Any idea of where I find this feature - stats?  Sounds like something good to know and look after.

 

[clarity]

Here is a link on how to setup it up.

http://rtcamp.com/tutorials/php/fpm-status-page/

 

[sleddog]

And as @zzrok said, aim for php 5.4. It's much more memory-efficient than 5.3 or earlier.

 

[drmike]

Keep the recommendations and tweaks coming.  Many folks just run PHP as-is and hit batch of similar problems.  So ideally this helps others in the future too

 

[NodeBytes]

I'm running a script that checks the memory usage of nginx and php and then limits their memory usage if they step over... then again, my webserver has 2gb dedicated ram, so that rarely happens.

 

[devonblzx]

I think you should strive to run PHP as the user, rather than nobody.  Not only can it be made more secure, it can allow you to monitor the usage per user to track down where the memory is coming from.  Although, it all depends how you split up your server.  If you have 1000 domains under one user, then it could be pretty hard to track it down that way.

 

[drmike]

Good input folks...

@devonblzx,  will change the user and see.

This is a single site/use instance.   So isolating things *should* be easy and possible.

Made this change:

pm = ondemand
pm.max_children = 2

That has RAM consumption much lower now.   Less than half of where RAM use was before.   No major performance change noticed.   

 

[devonblzx]

Okay, if it is a single site, then it shouldn't be as big of a deal to run php as the user, although it can benefit you as you wouldn't have to assign 777 permissions to be able to write to files/directories.  It can be a bit harder to track down how much each specific script is using, especially considering most scripts are processed* in less than a second.

As a PHP developer, you could always use memory_get_usage() in your script's development phase to isolate portions that use large amounts of memory.

 

[MannDude]

Following some advice here, I lowered the overall usage of RAM that php was using. Every now and then it'd max out ram randomly, despite that it usually stayed idle at 70-110Mb~. I'd have to SSH in and '/etc/init.d/php5-fpm restart' to get PHP working again and RAM usage back to normal.

Anyhow, a chart if those are interested:



I just disabled swapping forcefully again. Swappiness was set to zero but after some maintenance it seemed to have started swapping again randomly. (Like 10MB, not a huge a deal but annoying when there is free memory)

I'm still looking into tweaking it for best stability and performance. I've got excess RAM to play with. Right now only using 49MB out of 256MB. And yes, this is powering vpsBoard.

 

[zzrok]

You shouldn't be so worried about disabling swap completely.  Just set the swappiness low and let the system handle it.  Completely disabling swap can actually reduce system performance because data that is never accessed is forced to be kept in RAM instead of being swapped out to make more room for data that actually gets accessed.

You should be much more worried about understanding your environment (primarily php, web server, database) to determine its memory requirements.  Then you can configure it in such a way that you can be reasonably sure you are getting the most out of it while having a low risk of things blowing up during a surge of page views or similar event.

 

[drmike]

Hehe, hey another case study of PHP tweak improvements....

Well, I run swapless systems.   I see no reason for swap.   Have enough RAM and manage RAM usage proactively.

What I note  from the vpsBoard pic above is 175MB out of 256MB is "cached"?!?!?  Seems rather high and assuming this is "commonly" used files.