Linux Server Crashing Issues

[Nathan]

My dedicated server every day or two will become unresponsive.  I can't get to any of my sites, WHM, or even SSH in via putty when this happens.  I have to request a power cycle and then the server and everything comes back up.

I want to figure out how/why this is happening.  I'm on a CentOS box running whm/panel.  How/where do I look at log files to see what is causing this issue?

[enim]

The logs should hang out in /var/logs.  You'll probably need to ls -lha that to see whats available, then less some of them.  Some google-fu on filenames will be your friend.

 

If you need to know the exact time it goes down, just set up a ping to run from another machine to it.   Do something like 

 

 ping -i 5 [ip address] | while read pong; do echo "$(date): $pong"; done > logfile.txt

 

That will ping your box every five seconds with a timestamp.

 

Then you should be able to do

 

less logfile.txt | grep Request 

 

and find the time.  It'll give you a starting place to look.

Edited December 12, 2013 by enim

[Nathan]

Since posting I found the issues in /var/logs/messages. 

 

Appears PHP code on the webserver is spawning something over and over again and the processes don't die off.  The machine runs out of memory, and tries to invoke the oom_killer which kills off memory hog processes, but does not always succeed in saving the system.

 

Is there some monitoring software I can install to track down what php code is doing this?