Tons of PHP/Apache processes from one user on dedicated server

danswano

Member
Jul 12, 2011
13
0
51
Hello please help me i have tons of processed apache/php processes causing extremely high load

/usr/bin/php

/usr/local/apache/bin/httpd -k start -DSSL

rebuilt php+apache many time with many options/version but nothing helped

tried tuning apache configuration with low and high amount but nothing helped

the user is using a vbulletin forum nothing more.

tried many ssh commands to detect what is the real bad php script is running but couldn't find it, can you help me please :(
 

Infopro

Well-Known Member
May 20, 2003
17,090
518
613
Pennsylvania
cPanel Access Level
Root Administrator
Twitter
Not sure I'd be rebuilding Apache without first taking a closer look at that forum. Does it have any mods installed, like for example a shoutbox or chatroom? These sorts of mods depend on refreshes to stay up to date and out of date scripts of this sort can cause unexpected issues.

If the user states he's got no mods or customizations, test the forum by shutting it off for an hour. Does that solve the load issue? If yes, you'll have to look closer at that forum itself and make sure its up to date and secure. If not it may be being attacked somehow. Servers logs might help there, modsecurity and a good firewall can help a bit as well.

Rebuilding Apache is a bit much at this point, IMHO.

GL.
 

danswano

Member
Jul 12, 2011
13
0
51
There is such mods already but they have been there since months with no issue, the php high load on many files like newthread.php login.php archive/index.php

shutting the forum off solve the problem.

the forum is up-to-date with latest security patch.

what's causing those many processes to start all together and give a high load?

I'm using CGI PHP Handler by the way and i can't use suphp because there is another user using joomla gives white page when using suphp but i tried switching to suphp and didn't help in the vbulletin user.
 

Infopro

Well-Known Member
May 20, 2003
17,090
518
613
Pennsylvania
cPanel Access Level
Root Administrator
Twitter
Owner and permissions on the files for the Joomla site should be checked. Owner should be user, permissions on files 644 directories 755. Sorting that should solve any problems with white pages I think. (you might also need to check the Joomla site's .htaccess )

As for the vB forum, I suggest disabling all mods and check them each, manually, over at vbulletin.org for updates. If all are indeed up to date (and safe to use), enable one at a time and carefully watch for load issues.

A server running SuPHP and ConfigServer Firewall should point you in the right direction via an alert email when that forum starts to run up resources.

That forum has something going on somewhere, tweaking the server might help a bit, but solving the issue whatever it is on the forum would be better.

If you are not the Server Administrator of this server you might want to contact your host for more detailed assistance with this.
 

danswano

Member
Jul 12, 2011
13
0
51
i've used ConfigServer Firewall in the past but it's blocking too many legit users IP's without any reason so i disabled it.
Disabled all the plugins and the problem is still there.
 

danswano

Member
Jul 12, 2011
13
0
51
Enabled the firewall (csf) with medium security settings and disabled all plugins in vbulletin still getting crazy high load on php files, any suggestions?
 

Infopro

Well-Known Member
May 20, 2003
17,090
518
613
Pennsylvania
cPanel Access Level
Root Administrator
Twitter
Yes, set the level to high, for starters. We want someone to get blocked, we want a process to be flagged. Assuming mail is setup proper and CSF emails are getting to you. We want something to happen and the server to email you about it.

In the forums admin > Maintenance > Diagnostics > Suspect File Versions, check this for any issues of files replaced for whatever reason.

Is this server running SuPHP?
 

danswano

Member
Jul 12, 2011
13
0
51
i got this block now
***.117.169.0/24 # lfd: (NETBLOCK) ***.117.169.0/24 has had more than 4 blocks in the last 86400 secs - Wed Jul 13 00:43:16 2011

The ip is on the same range of my server IP, what you think?
 

danswano

Member
Jul 12, 2011
13
0
51
Thanks you for you help really, i'm hopeless now, no one knows what's going on, including datacenter and Vbulletin. :/
 

Infopro

Well-Known Member
May 20, 2003
17,090
518
613
Pennsylvania
cPanel Access Level
Root Administrator
Twitter
I've taken a peek at your posts on vbulletin.com forums and your server details. From what I can tell you've got a lot going on there. vbsphinx, arcade, gameroom, chat and multiple other mods. Nginx, eAccelerator, what appears to be an IRC server as well.

The table for forum cronlog alone is 1561.77 MB in size, this is very big, another called phpkd_vblvb_log that's almost 5,000MB, post table is almost 10,000MB in size. It appears you setup the server using ELS from servermonkeys, or at least used it to edit your my.conf file as well. Not sure I'd trust those settings it made.

You have some errors in your MySQL error log, a snip below.

110712 18:19:01 [ERROR] /usr/sbin/mysqld: Sort aborted
110712 18:21:19 [ERROR] /usr/sbin/mysqld: Sort aborted
110712 18:23:37 [ERROR] /usr/sbin/mysqld: Table './eximstats/sends' is marked as crashed and last (automatic?) repair failed
Going by your earlier posts on vB's forum, it sounds to me like you converted from phpbb to vBulletin, a very large forum. If this forum was operating normally until recently, you might want to go back over your notes and see what changed to make it go wrong.

I'd only be guessing here but it sounds like this forum and server are a real mess. You might want to consider hiring a professional to assist you with all of this to get you back on track.

Hoping for a few forum posts here or at vBulletin to help clear this up is not going to happen, IMHO. There's just too much going on.

Did you lose your webmaster that managed this forum or what? A forum doesn't normally grow to this size without an experienced user running it.

You really need to back up, turn off some things, get back to defaults on others and lose what you can live without. Once you've cleaned things up, then see where you stand.

If this is just a misconfiguration of the server with a large forum on it I'd be very surprised. I suspect if you don't know whats going on there on that server already, you should find out, quick.


Good luck with this.
 

danswano

Member
Jul 12, 2011
13
0
51
Those tables are too old and they are related to disabled mods and some like sphinx and irc are up and running.
It was running blazing fast 3 days ago but suddenly this happened.

Another 25 IP's block on csf from same server IP range too.