Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

Creating accounts slow on new server

Discussion in 'General Discussion' started by tomsie1000, Jul 9, 2018.

  1. tomsie1000

    tomsie1000 Member

    Joined:
    Jul 9, 2018
    Messages:
    7
    Likes Received:
    1
    Trophy Points:
    1
    Location:
    GB
    cPanel Access Level:
    Root Administrator
    Hello all,

    I work for as a sysadmin at a University and am in the process of building a cPanel instance on CentOS 7 to host students' coursework.

    Since setting up the system, I have noticed creating accounts is seemingly quite slow, and it looks like something is waiting for a response and timing out, but I'm not seeing any errors.. We've done this previously on CentOS 6 and had no issues..

    I've tried via the JSON api, as well as the UI and at the cli - the script always pauses at 'Validating Username......Done' for approx 2 minutes.. This delay also happens when you park a domain for some reason, so I'm feeling like it could be DNS related. DNS is disabled on this server as we have DNS servers already and no need for accounts to manage their own zones.

    Running the 'wwwacct' script directly behind 'time' shows the following:

    Code:
    # time ./wwwacct somesite.com someuser passwrd0101
    +===================================+
    | New Account Info                  |
    +===================================+
    | Domain: somesite.com
    | UserName: someuser
    | PassWord: passwrd0101
    +===================================+
    
    This ok? y
    Checking input data...Validating Username......Done
    ***2m pause here***
    Validating IP......Done
    Validating Contact Email......Done
    ...Done
    Validating system setup......Done
    Checking for database conflicts......Done
    WWWAcct 12.6.0 (c) 2018 cPanel, Inc.......Done
    Running pre creation script (/usr/local/cpanel/scripts/prewwwacct)......Done
    Adding User......Done
    Adding Entries to userdata......Done
    Setting up Mail & Local Domains......Done
    Configuring DNS......Done
    Verifying MX Records and Setting up Databases......Done
    Setting up Proxy Subdomains......Done
    Updating Authentication Databases......Done
    Adding Entries to httpd.conf......Done
    Setting passwords......Done
    Updating the userdata cache......Done
    Setting up Horde database in the background.......Done
    Creating bandwidth datastore......Done
    Updating the dedicated IP address usage cache......Done
    Generating and installing DKIM keys......Done
    Enabling Apache SpamAssassin™......Done
    Sending Account Information......Done
    Running post creation scripts (/usr/local/cpanel/scripts/legacypostwwwacct, /usr/local/cpanel/scripts/postwwwacct, /usr/local/cpanel/scripts/postwwwacctuser)......Done
    Setting up Domain Pointers......Done
    Setting Reseller Privs......Done
    Account Creation Complete!!!......Done
    Account Creation Ok
    Dns Zone check is enabled.
    
    
    +===================================+
    | New Account Info                  |
    +===================================+
    | Domain: somesite.com
    | Ip: xxx.xxx.xxx.xxx (n)
    | HasCgi: y
    | UserName: someuser
    | PassWord: passwrd0101
    | CpanelMod: paper_lantern
    | HomeRoot: /cpanel-sites
    | Quota: 0 bytes
    | NameServer1: xxx.ac.uk
    | NameServer2: xxx.ac.uk
    | NameServer3:
    | NameServer4:
    | Contact Email:
    | Package: default
    | Feature List: default
    | Language: en
    +===================================+
    Custom Account Data Provided: no
    Adding Shell Access (y)
    Successlocaldomains...valiases ...vdomainaliases ...vfilters ...Zone somesite.com has been successfully added
    Reconfiguring Mail Routing:
    LOCAL MAIL EXCHANGER: This server will serve as a primary mail exchanger for somesite.com's mail.:
     This configuration has been manually selected.
    
    nameserver: reload skipped on [serverhostname] (nameserver disabled)
    wwwacct creation finished
    Account Creation Ok
    
    real    2m11.066s
    user    0m0.504s
    sys     0m0.162s
    
    Not a huge issue but it does slow down account creation when we provision new users..

    Should also note that CSF is installed and configured, however disabling it complete makes no difference to these times, so I'm fairly sure it's not a misconfigured firewall.

    Appreciate any tips/thoughts/etc. as I'd like to get that time down a bit if possible..

    Cheers

    Tom
     
  2. cPanelLauren

    cPanelLauren Forums Analyst II
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    3,517
    Likes Received:
    251
    Trophy Points:
    193
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    HI @tomsie1000

    Is the server by chance using an NFS mount somewhere? If you're unsure running something like

    Code:
    cat /etc/fstab
    or

    Code:
     cat /proc/mounts
    Would answer that.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. tomsie1000

    tomsie1000 Member

    Joined:
    Jul 9, 2018
    Messages:
    7
    Likes Received:
    1
    Trophy Points:
    1
    Location:
    GB
    cPanel Access Level:
    Root Administrator
    Hi Lauren, thanks, sorry for the delay..

    I have since rebuilt the server to be absolutely sure it wasn't something I did wrong earlier. The same pause on account creation persists..

    There are no NFS mounts present:

    Code:
    # df -h
    Filesystem      Size  Used Avail Use% Mounted on
    /dev/sda2        37G  5.7G   32G  16% /
    devtmpfs        2.0G     0  2.0G   0% /dev
    tmpfs           2.0G  4.0K  2.0G   1% /dev/shm
    tmpfs           2.0G  9.1M  2.0G   1% /run
    tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
    /dev/sdb1        40G   33M   40G   1% /cpanel-sites
    /dev/sda1       976M  138M  787M  15% /boot
    /dev/loop0      1.6G  2.6M  1.5G   1% /tmp
    tmpfs           396M     0  396M   0% /run/user/1000
    Code:
    # cat /etc/fstab
    UUID=a19043ee-7101-4410-a162-3eaabdecf0e9       /       xfs     defaults,uquota 0       0
    UUID=f2f39603-0723-44c1-9298-a355436cde6b /boot                   ext3    defaults        1 2
    UUID=1365405e-ac9a-443d-bc49-5b4a470ece69 swap                    swap    defaults        0 0
    /dev/sdb1       /cpanel-sites   xfs     defaults,uquota 0       0
    /usr/tmpDSK             /tmp                    ext3    defaults,noauto        0 0
    
     
  4. cPanelLauren

    cPanelLauren Forums Analyst II
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    3,517
    Likes Received:
    251
    Trophy Points:
    193
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Hi @tomsie1000

    Thanks for the response! What about the resolv.conf can you tell me the nameservers listed there? Also, can you tell me is the server NAT routed? If so can you confirm it's a 1:1 configuration?
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. tomsie1000

    tomsie1000 Member

    Joined:
    Jul 9, 2018
    Messages:
    7
    Likes Received:
    1
    Trophy Points:
    1
    Location:
    GB
    cPanel Access Level:
    Root Administrator
    Sure. Just to note that the local BIND instance is disabled as we have no intent to use it

    Here's resolv.conf - these are our DNS servers of course. (I have redacted the actual names and addresses for obvious reasons)

    Code:
    # cat /etc/resolv.conf
    
    search domain.somewhere, sub.domain.somewhere
    nameserver xxx.xxx.98.15
    nameserver xxx.xxx.28.59
    And yes, the server has a dedicated public IP - no NAT is involved.
     
  6. cPanelLauren

    cPanelLauren Forums Analyst II
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    3,517
    Likes Received:
    251
    Trophy Points:
    193
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    HI @tomsie1000


    Are you aware if you're required to use those nameservers? Can you try replacing those with something like google's DNS resolvers?

    Code:
    8.8.8.8
    8.8.4.4
    And let me know if the issue persists?

    Thanks!
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  7. tomsie1000

    tomsie1000 Member

    Joined:
    Jul 9, 2018
    Messages:
    7
    Likes Received:
    1
    Trophy Points:
    1
    Location:
    GB
    cPanel Access Level:
    Root Administrator
    Thanks. It looks like DNS resolution is the problem, but not with the resolver per-se. Under normal circumstances we block outbound DNS queries to any other servers than our internal ones. It looks like during account creation the script is trying to work back the path of the account domain by querying the root servers directly for NS data, specifically not through the configured resolver. This explains why I was also seeing pauses when parking domains.

    Allowing DNS requests to pass the firewall got rid of the pause completely, but I wanted to do some more digging.. See tcpdump output:

    Code:
    16:43:18.117258 IP this.server.com.46707 > d.root-servers.net.domain: 29302 [1au] NS? uk. (31)
    16:43:18.129418 IP this.server.com.52837 > nsa.nic.uk.domain: 40239 [1au] NS? ac.uk. (34)
    16:43:18.125975 IP d.root-servers.net.domain > this.server.com.46707: 29302- 0/8/14 (451)
    16:43:18.169360 IP this.server.com.37810 > d.root-servers.net.domain: 30862 [1au] NS? nl. (31)
    ..etc..
    So, I assume the short answer is "don't block outgoing DNS requests" however I have to ask the question; is there a cPanel option I can set to force it to query DNS using ONLY the configured resolvers or is it hard-coded to go directly?
     
  8. cPanelLauren

    cPanelLauren Forums Analyst II
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    3,517
    Likes Received:
    251
    Trophy Points:
    193
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Hi @tomsie1000

    Unfortunately, because that's hardcoded I don't see a way that would work. I believe it queries the root nameservers when it does this. You might try going to WHM>>Server Configuration>>Tweak Settings and enabling Allow Remote Domains and Allow unregistered domains though I'm not confident that will stop the DNS lookup that occurs.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  9. tomsie1000

    tomsie1000 Member

    Joined:
    Jul 9, 2018
    Messages:
    7
    Likes Received:
    1
    Trophy Points:
    1
    Location:
    GB
    cPanel Access Level:
    Root Administrator
    Thanks @cPanelLauren for the confirmation and your help. I guess I could submit it as a feature request but I'd guess it's not likely to be considered. I'll just make sure that this host can query directly.
     
  10. cPanelLauren

    cPanelLauren Forums Analyst II
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    3,517
    Likes Received:
    251
    Trophy Points:
    193
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Hi @tomsie1000

    You're welcome, just out of curiosity what is the specific configuration you've done to block this? I'd like to try a couple of things on my test environments, no guarantees but I'll try none the less.

    Thanks!
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  11. tomsie1000

    tomsie1000 Member

    Joined:
    Jul 9, 2018
    Messages:
    7
    Likes Received:
    1
    Trophy Points:
    1
    Location:
    GB
    cPanel Access Level:
    Root Administrator
    Our perimeter firewall blocks any DNS requests outbound from clients inside our network. This forces them to use our internal resolvers for recursive lookups, where we run some DNS-RPZ for anti-malware, etc.. Therefore, by default, clients on our network (including servers in our DCs) cannot query the root zone directly, only recursively through our resolvers. This works fine, so long as the applications/clients in question honour their configured resolvers.

    i.e.:

    under normal circumstances, I'd expect to be able to do this:

    Code:
    $ dig @a.root-servers.net uk
    
    ; <<>> DiG 9.9.4-RedHat-9.9.4-61.el7 <<>> @a.root-servers.net uk
    ; (1 server found)
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57965
    ;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 8, ADDITIONAL: 14
    ;; WARNING: recursion requested but not available
    
    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags:; udp: 4096
    ;; QUESTION SECTION:
    ;uk.                            IN      A
    
    ;; AUTHORITY SECTION:
    uk.                     172800  IN      NS      dns1.nic.uk.
    uk.                     172800  IN      NS      dns4.nic.uk.
    uk.                     172800  IN      NS      nsa.nic.uk.
    uk.                     172800  IN      NS      nsd.nic.uk.
    uk.                     172800  IN      NS      nsc.nic.uk.
    ....etc....
    ;; Query time: 11 msec
    ;; SERVER: 198.41.0.4#53(198.41.0.4)
    ;; WHEN: Thu Aug 23 14:46:56 BST 2018
    ;; MSG SIZE  rcvd: 451
    however due to our firewall restrictions you'd see this:

    Code:
    $ dig @a.root-servers.net uk
    
    ; <<>> DiG 9.9.4-RedHat-9.9.4-61.el7 <<>> @a.root-servers.net uk
    ; (1 server found)
    ;; global options: +cmd
    ;; connection timed out; no servers could be reached
    So if the server/client in question is resolving only via its configured resolvers, it would work fine just doing this:

    Code:
    $ dig uk
    
    ; <<>> DiG 9.9.4-RedHat-9.9.4-61.el7 <<>> uk
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 38043
    ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
    
    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags:; udp: 4096
    ;; QUESTION SECTION:
    ;uk.                            IN      A
    
    ;; AUTHORITY SECTION:
    uk.                     10800   IN      SOA     dns1.nic.uk. hostmaster.nic.uk. 1403801968 7200 900 2419200 10800
    
    ;; Query time: 14 msec
    ;; SERVER: my.server.com#53(1.2.3.4)
    ;; WHEN: Thu Aug 23 14:41:50 BST 2018
    ;; MSG SIZE  rcvd: 87
    Honestly I can see the rationale for querying the root servers directly to do this sort of validation. You're asking the authoritative source of information what the current state of play is. It avoids any caching issues. It just slows my particular use case down that's all :P

    To test, you could set up iptables rules on your server to block outbound DNS to any servers but your local resolvers. You should then be able to reproduce the same results I get when doing so.
     
  12. cPanelLauren

    cPanelLauren Forums Analyst II
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    3,517
    Likes Received:
    251
    Trophy Points:
    193
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Hi @tomsie1000

    So specifically the issue occurs when UDP port 53 outbound is blocked you could block TCP requests and still be ok. Otherwise, that's the only workaround to that I'm finding.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  13. tomsie1000

    tomsie1000 Member

    Joined:
    Jul 9, 2018
    Messages:
    7
    Likes Received:
    1
    Trophy Points:
    1
    Location:
    GB
    cPanel Access Level:
    Root Administrator
    Pretty much, yeah. That's what I'm going with anyway!
     
    cPanelLauren likes this.
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice