The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Terminate 1000+ accounts. Speed concern

Discussion in 'General Discussion' started by maysoft, Oct 14, 2007.

  1. maysoft

    maysoft Well-Known Member

    Joined:
    Nov 10, 2005
    Messages:
    64
    Likes Received:
    0
    Trophy Points:
    6
    Hello everyone,


    I have a server with 4000+ accounts on it. And I have to terminate about 1500 suspended accounts. I wrote a script that takes care of this, but the problem is that /scripts/killacct is too slow.

    It takes about 20 seconds to terminate one account. It will take almost 10 hours to terminate all accounts. That is not a big deal, but I also noticed that after every termination, httpd restarts and that causes problems to the users who access websites on the server. I would not want to allow this to happen for 10 consecutive hours :)

    Are there any solutions?
     
  2. rustelekom

    rustelekom Well-Known Member
    PartnerNOC

    Joined:
    Nov 13, 2003
    Messages:
    290
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    moscow
    hehe, look as you need do not wait till to time when you have more than 1000 users suspended and do this work on more regular basis :)
    It is impossible speedup this work and not refresh apache, named and other services configs because you can get completely corrupted configs if you try do this work manually.
    What i can recommend is just do this work within few days. Run your scripts for about 100 accounts per day and you will be happy and your users too.
    BTW. Apache restarts gracefully, so deleting user process shouldn't create any big problem for sites visitors.
     
  3. maysoft

    maysoft Well-Known Member

    Joined:
    Nov 10, 2005
    Messages:
    64
    Likes Received:
    0
    Trophy Points:
    6
    Thanks for suggestions :)

    I have just finished coding the part of the script that kills suspended accounts, that is why I had no chance to run it earlier. Suspension part of the script was ready a long time ago and was adding lots of suspended accounts every day it was run. I was just too lazy to do termination part... and so here are the results... :D

    Anyway, I think I will just terminate about 100 accounts at a time per your suggestion.

    Thank you! :)
     
  4. jpetersen

    jpetersen Well-Known Member

    Joined:
    Dec 31, 2006
    Messages:
    113
    Likes Received:
    4
    Trophy Points:
    18
  5. MaraBlue

    MaraBlue Well-Known Member

    Joined:
    May 3, 2005
    Messages:
    335
    Likes Received:
    2
    Trophy Points:
    18
    Location:
    Carmichael, CA
    cPanel Access Level:
    Root Administrator
    I just wanted to say I am blown away at those numbers!

    If it were me, I'd put the list of accounts to term in an array, etc and set the script on a cron to run every 5 minutes or so, deleting one at a time and using graceful restarts.

    Wow...1,500...good luck :)
     
  6. maysoft

    maysoft Well-Known Member

    Joined:
    Nov 10, 2005
    Messages:
    64
    Likes Received:
    0
    Trophy Points:
    6
    I have already got rid of almost 1000. :D
    Ran script every night when the load is mimimal, pausing for 5 minutes after each account removal and completely restarting apache every morning (after script finishes its job).
     
  7. MaraBlue

    MaraBlue Well-Known Member

    Joined:
    May 3, 2005
    Messages:
    335
    Likes Received:
    2
    Trophy Points:
    18
    Location:
    Carmichael, CA
    cPanel Access Level:
    Root Administrator
    Cool, as long as it worked :)
     
  8. jayh38

    jayh38 Well-Known Member

    Joined:
    Mar 3, 2006
    Messages:
    1,215
    Likes Received:
    0
    Trophy Points:
    36
    If all these accounts were paying clients, why not run a billing software that would suspend and terminate automatically after the time frame you set.
     
  9. maysoft

    maysoft Well-Known Member

    Joined:
    Nov 10, 2005
    Messages:
    64
    Likes Received:
    0
    Trophy Points:
    6
    These are on a free webhosting server. Most of those accounts are suspended due to inactivity.

    Now that I am almost finished my script, it will take care about suspending/terminating without my help :)
     
  10. dendis

    dendis Well-Known Member

    Joined:
    Jul 18, 2003
    Messages:
    49
    Likes Received:
    0
    Trophy Points:
    6
    maysoft: can you share this script?
    I have almost identical situation where I need to remove 2000 accounts from server, and apache restart takes 30 seconds each time account is removed .. :(
     
  11. maysoft

    maysoft Well-Known Member

    Joined:
    Nov 10, 2005
    Messages:
    64
    Likes Received:
    0
    Trophy Points:
    6
    This script is a part of my big "server sanitizer" pack; it won't work without my other scripts... somewhat "too complicated" system :)

    Here are some important parts of PHP code. This code will check if the account is suspended for more than 7 days. If it is, then account will be terminated.


    PHP:
    function terminate $username ) {
            
    $s '';
            
    $s .= "********* TERMINATING $username . . .\n";

            
    $handle popen 'echo y | /scripts/killacct ' $username'r' );
            
    sleep );
            while (!
    feof($handle)) {
                
    $s .= fread $handle65536 );
            }
            
    pclose $handle );

            
    $s .= "\n";
            
    $s .= "**************************************\n\n" ;
            return 
    $s;
    }

    function 
    suspend $username ) {
            
    $s '';
            
    $s .= "********* SUSPENDING $username . . .\n";

            
    $handle popen '/scripts/suspendacct ' $username'r' );
            
    sleep );
            while (!
    feof($handle)) {
                
    $s .= fread $handle65536 );
            }
            
    pclose $handle );

            
    $s .= "\n";
            
    $s .= "**************************************\n\n" ;
            return 
    $s;
    }


    ...
    $home '/home/'
    $cpanel_usersdir '/var/cpanel/users';

    while ( 
    false !== ( $file readdir $dirhandle ) ) ) {
        if ( !
    is_file $cpanel_usersdir '/' $file ) ) continue;
        
    $username $file;
        
    $x trim file_get_contents $cpanel_usersdir '/' $file ) ) . "\n"// \n is just to preg_match to work correctly. Lazy me :)
        
    preg_match '/SUSPENDED=(.*)/i'$x$m );
        
    $suspended trim $m ] );
        if ( 
    $suspended == '1' ) {
            
    $suspended true;
            
    $accounts_suspended++;
        } else {
            
    $suspended false;
        }

        
    preg_match '/SUSPENDTIME=(.*)/i'$x$m );
        
    $suspendtime trim $m ] );

        if ( 
    $suspended ) {
            if ( ( 
    date 'U' ) - $suspendtime ) > 86400 ) {
                
    $s = @file_get_contents $home $username '/.canterminate' );

                if ( 
    $s == 'yes' ) {

                    
    $s = @file_get_contents 'http://' $www );
                    
    $s 'This account has been suspended';
       
    ...
                        echo ( 
    str_pad $username10 ) . str_pad $www25 ) );
                        echo ( 
    "Suspended for too long ($suspendtime). TERMINATING...\n" );
                        echo ( 
    terminate $username ) );
                        echo 
    "Sleeping...";
                        
    sleep 300 );
                        echo 
    "\n";
                        
    $accounts_terminated++;
                        continue;

    ...

                }
            }

            echo ( 
    str_pad $username10 ) . str_pad $www25 ) );
            echo ( 
    "SUSPENDED:   $suspendtime     Skipping...\n" );
            continue;
        }

    That's about it :)
     
    #11 maysoft, Oct 21, 2007
    Last edited: Oct 21, 2007
  12. dendis

    dendis Well-Known Member

    Joined:
    Jul 18, 2003
    Messages:
    49
    Likes Received:
    0
    Trophy Points:
    6
    Dmitry, nice script but it still executes /scripts/killacct when terminating account.

    So it does apache restart, and it do not resolve this problem.
    or I am missing something?
     
  13. maysoft

    maysoft Well-Known Member

    Joined:
    Nov 10, 2005
    Messages:
    64
    Likes Received:
    0
    Trophy Points:
    6
    Yes, it still uses killacct.

    I am using apache 2.2 which does not completely go down on restart (there is just a ~5 second delay AFTER apache restarted, but still - no requests discarded, so everyone got served :D). Maybe other versions are not so good, I just do not know... I add a 5 minute pause after killacct so that those delays are infrequent. You can run the script at the evening and stop it in the morning. I did it and no one has complained (and my customers are used to complain A LOT :D).
     
  14. dendis

    dendis Well-Known Member

    Joined:
    Jul 18, 2003
    Messages:
    49
    Likes Received:
    0
    Trophy Points:
    6
    OK thanks for suggestion I will also upgrade to Apache 2.2 :)

    I see you are freehost like me too :)
    What I have found very useful is to run MYSQL on secondary HDD - it increased server performance over 100%. I simply attached new 10k rpm HDD and moved MySQL there. Result - load dropped twice. I can suggest you to do that same.


    Can I ask you, Have you tried upgrading PHP4 to PHP5, have you received a lot complains ? :) I am scared to do that so I would like to know your experience. Also ave you tried to upgrade MYSQL4 to MYSQL5 ?
     
  15. andyf

    andyf Well-Known Member

    Joined:
    Jan 7, 2002
    Messages:
    246
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    UK
    Temporarily link the httpd restart script (sorry, off the top of my head I'm not sure which cpanel uses) to /dev/null while the terminations are in progress (you might want to do the same with rndc). This should speed it up, just remember to undo this afterwards!
     
  16. maysoft

    maysoft Well-Known Member

    Joined:
    Nov 10, 2005
    Messages:
    64
    Likes Received:
    0
    Trophy Points:
    6
    dendis:

    Yes, I run a free hosting service :)

    Thanks for the suggestion. I can actually tell even more - I always have two hard drives; primary hard drive takes MySQL AND all logs, including Apache logs (they also need lots of I/O). Secondary drive is dedicated to /home2/ with all users' home dirs :). This way it has a very nice balance!

    I run PHP5+MySQL5 now on all servers. When I was upgrading to PHP5/MySQL5, I had only 1000 users on a single server, and just several of them complained that the upgrade broke some of their prehistoric scripts. They all were able to fix problems without me investigating further... so I can call that it was a smooth move.

    If I had 2000+ active users on a server, then probably I wouldn't upgrade.

    By the way, do you force ads? If yes, then what do you use for that? :)
     
  17. maysoft

    maysoft Well-Known Member

    Joined:
    Nov 10, 2005
    Messages:
    64
    Likes Received:
    0
    Trophy Points:
    6
    Thanks Andy, I am definitely going to try that on my next big terminate bash :D
     
  18. encryption

    encryption Well-Known Member

    Joined:
    Jun 24, 2005
    Messages:
    72
    Likes Received:
    0
    Trophy Points:
    6
    Is this a cluster ? Thats incredible. Sorry this is off topic but would you be willing to share some server specs with us ? I'm incredibly curious to see what you have running
     
  19. dendis

    dendis Well-Known Member

    Joined:
    Jul 18, 2003
    Messages:
    49
    Likes Received:
    0
    Trophy Points:
    6
    well our server has over 10000 cPanel accounts :) anyway load is around 2-3, its quad Intel(R) Xeon(R) CPU X3220 @ 2.40GHz and 4 GB ram


    Of course around 50% accounts are not active, but that's how free hosting works, we cant force all users to use our hosting so all inactive accounts have to be periodically terminated.

    the biggest problem is Apache restart when account is created or terminated, but I think it will solve once I will upgrade to Apache 2.2
    there was similar problem with MySQL, but I solved it by moving MySQL to secondary SCSI disk.

    I also spent a lot nights by tuning MySQL, Apache and other services while I got cPanel server running fast with 10000 accounts on it :)

    What about forcing ads, I use php.ini's auto_prepend_file, I found that is much better than mod_layout as it fails with some websites using gziped content also I can easy to configure where and how to show ads or not, I also use my custom phishing filter here
     
  20. maysoft

    maysoft Well-Known Member

    Joined:
    Nov 10, 2005
    Messages:
    64
    Likes Received:
    0
    Trophy Points:
    6
    encryption:
    We have several servers, but we have one very special: AMD Sempron 2600, 2GB RAM, and 2x160 gb SATA drives... with 3000+ accounts on it! (that's free hosting). Load average is about 1 :D

    No clusters! :)

    dendis:
    Thanks for your input regarding forced ads! But then... did you switch apache to treat all files as PHP-scripts? Otherwise ads will only show on PHP-generated pages...
     
Loading...

Share This Page