TEACH ME how to optimize my dedicated SERVER

[igordr]

Hello,

I am Igor, have 19 years, live in Serbia.
I own dedicated server with about 20 sites on it, but just two of them have traffic - all together 10 000 unique visitors daily.

My server details:

HDD     2x 500 GB SATA
RAM     6 GB DDR2-RAM
CPU     AMD Athlon 64 X2 3400+, Dual-Core
CentOS 6.3

I also use licensed version of WHM/Cpanel, regularly updated.
Have SHH access, root.

My problem: Server load is enormous, taking too long to respond [30+ seconds].

When i run TOP in shh:

top - 14:48:30 up 8 days, 22:35,  1 user,  load average: 80.32, 86.56, 76.90
Tasks: 377 total,  44 running, 330 sleeping,   1 stopped,   2 zombie
Cpu(s): 70.7%us, 21.9%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  7.4%si,  0.0%st
Mem:   5733840k total,  4201196k used,  1532644k free,   159604k buffers
Swap:  2097136k total,   182140k used,  1914996k free,   768072k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4389 mysql     20   0  633m  75m 4484 S 24.3  1.3   0:51.67 mysqld
  617 nobody    20   0 43900  472  264 S  2.0  0.0 288:42.78 perl
 3209 nobody    20   0  138m  32m 4084 R  1.6  0.6   0:03.53 httpd
 4484 nobody    20   0  129m  22m 3736 R  1.6  0.4   0:01.31 httpd
 4523 nobody    20   0  127m  21m 3624 R  1.6  0.4   0:00.92 httpd
 4710 nobody    20   0  119m  11m 3168 S  1.6  0.2   0:00.09 httpd
 2348 nobody    20   0  140m  33m 3976 R  1.4  0.6   0:04.03 httpd
 2825 nobody    20   0  141m  35m 4060 R  1.4  0.6   0:04.07 httpd
 2833 nobody    20   0  140m  34m 3968 R  1.4  0.6   0:02.32 httpd
 3004 nobody    20   0  140m  34m 3848 S  1.4  0.6   0:03.17 httpd
 3277 nobody    20   0  118m  12m 3624 R  1.4  0.2   0:01.54 httpd
 3348 nobody    20   0  142m  36m 3860 R  1.4  0.6   0:03.51 httpd
 3893 nobody    20   0 52904  10m 1344 R  1.4  0.2 384:43.25 perl
 3895 nobody    20   0 54236  11m 1024 S  1.4  0.2   0:07.77 perl
 3897 nobody    20   0 54580  11m 1024 R  1.4  0.2   0:07.71 perl
 3899 nobody    20   0 54232  11m 1024 R  1.4  0.2   0:07.71 perl
 3902 nobody    20   0 54312  11m 1024 R  1.4  0.2   0:07.75 perl

Current my.cnf looks like:

[mysqld]
max_connections = 500
key_buffer = 32M
myisam_sort_buffer_size = 32M
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 2M
table_cache = 4000
thread_cache_size = 286
interactive_timeout = 25
wait_timeout = 7000
connect_timeout = 10
max_allowed_packet = 16M
max_connect_errors = 10
query_cache_limit = 2M
query_cache_size = 24M
query_cache_type = 1
tmp_table_size = 32M
skip-innodb
[mysqld_safe]
open_files_limit = 8192
[mysqldump]
quick
max_allowed_packet = 16M
[myisamchk]
key_buffer = 64M
sort_buffer = 64M
read_buffer = 16M
write_buffer = 16M
[mysqlhotcopy]
interactive-timeout

Default my.cnf which i have changed according to this tutorial
/http://www.firemakerhosting.com/server-management/configuring-optimizing-mysql-for-whm-dedicated-server-vps/ was looking:

[mysqld]
set-variable = max_connections=500
query_cache_limit       = 2M
query_cache_size        = 32M
table_cache = 400
thread_cache_size = 286
log-slow-queries
safe-show-database

After running mysqltuner.pl with current my.cnf settings i get:

root@bravo612 [~]# ./mysqltuner.pl

 >>  MySQLTuner 1.2.0 - Major Hayden <major@mhtx.net>
 >>  Bug reports, feature requests, and downloads at http://mysqltuner.com/
 >>  Run with '--help' for additional options and output filtering

-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.1.63-cll
[OK] Operating on 64-bit architecture

-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB -Federated -InnoDB -ISAM -NDBCluster
[--] Data in MyISAM tables: 172M (Tables: 715)
[!!] Total fragmented tables: 29

-------- Security Recommendations  -------------------------------------------
[OK] All database users have passwords assigned

-------- Performance Metrics -------------------------------------------------
[--] Up for: 9m 29s (32K q [57.963 qps], 990 conn, TX: 64M, RX: 2M)
[--] Reads / Writes: 92% / 8%
[--] Total buffers: 72.0M global + 4.5M per thread (300 max threads)
[OK] Maximum possible memory usage: 1.4G (25% of installed RAM)
[OK] Slow queries: 0% (86/32K)
[OK] Highest usage of available connections: 24% (74/300)
[OK] Key buffer size / total MyISAM indexes: 32.0M/52.8M
[OK] Key buffer hit rate: 99.7% (2M cached / 5K reads)
[OK] Query cache efficiency: 49.1% (12K cached / 25K selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 2% (23 temp sorts / 1K sorts)
[!!] Temporary tables created on disk: 38% (379 on disk / 979 total)
[OK] Thread cache hit rate: 92% (74 created / 990 connections)
[OK] Table cache hit rate: 97% (292 open / 299 opened)
[OK] Open file limit used: 5% (459/8K)
[OK] Table locks acquired immediately: 95% (14K immediate / 14K locks)

-------- Recommendations -----------------------------------------------------
General recommendations:
    Run OPTIMIZE TABLE to defragment tables for better performance
    MySQL started within last 24 hours - recommendations may be inaccurate
    Enable the slow query log to troubleshoot bad queries
    When making adjustments, make tmp_table_size/max_heap_table_size equal
    Reduce your SELECT DISTINCT queries without LIMIT clauses
Variables to adjust:
    tmp_table_size (> 16M)
    max_heap_table_size (> 16M)

Also, i need to mention, i have installed VNC service with GNOME desktop environment on current server, but i don't use it regularly.
Also, i have spotted, when i clean seass_ files server is faster, but just minute or two, after that, it's the same...

If you need any additional details, please ask me.

I won't sleep tonight until my web server start to load faster, so please, reply me, i am ready to teach HOW TO...

Greetings,
Igor

[igordr]

Still no reply. May somebody help me, please?

[prajithp13]

Seems the issue is with apache, so I would suggest you to install nginx or apachebooster, this will reduse the load

[nguylb73]

MySQL usage is high too, one pid using 24% CPU. There is a lot of apache processes, make sure you cache what is static.

[cPanelTristan]

The issue isn't due to any of this. The server is exploited. perl shouldn't be running as the user nobody:

617 nobody 20 0 43900 472 264 S 2.0 0.0 288:42.78 perl
3893 nobody 20 0 52904 10m 1344 R 1.4 0.2 384:43.25 perl
3895 nobody 20 0 54236 11m 1024 S 1.4 0.2 0:07.77 perl
3897 nobody 20 0 54580 11m 1024 R 1.4 0.2 0:07.71 perl
3899 nobody 20 0 54232 11m 1024 R 1.4 0.2 0:07.71 perl
3902 nobody 20 0 54312 11m 1024 R 1.4 0.2 0:07.75 perl

Perl scripts run as the username under SuExec, so perl running as nobody indicates an exploit. Run a cat on each process and and lsof on them:

cat /proc/3902/environ
lsof -p 3902

Here 3902 is the PID number of the last process listed in the above paste. You should do this for each one.