The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Clustered Nameservers and Problems

Discussion in 'Bind / DNS / Nameserver Issues' started by silversurfer, Jan 10, 2005.

  1. silversurfer

    silversurfer Well-Known Member

    Joined:
    Dec 29, 2002
    Messages:
    274
    Likes Received:
    0
    Trophy Points:
    18
    Dear all,

    Currently one of our server clusters are setup as such:

    - Server 1 to 4
    - All 4 servers are set to syncronize with all the other 3 servers. ie. Server 1 syncronize to Server 2 to 4 and Server 4 will syncronize with Server 1 to 3 etc.

    We are facing an issue where the DNS name.conf keeps getting corrupted. When this happens, Bind cannot be restarted and thus, new entries do not get loaded. Restarting will also fail till you correct the named.conf file to remove the duplicates.

    For some unknown reason, Cpanel / Bind / etc seems to be inserted duplicate entries *sometimes* when new accounts are created in the named.conf. It can happen in 1 of the servers, a couple or none at all.

    Is this an issue with our setup? Or do others have this issue. If you had, what fixes are there if any to fix this once and for all, instead of manually fixing the named.conf all the time?
     
  2. casey

    casey Well-Known Member

    Joined:
    Jan 17, 2003
    Messages:
    2,303
    Likes Received:
    0
    Trophy Points:
    36
    Location:
    If there is trouble, it will find me
    I also see this behavior *occasionally* and have to remove the duplicate entry. Have no idea what causes it.
     
  3. silversurfer

    silversurfer Well-Known Member

    Joined:
    Dec 29, 2002
    Messages:
    274
    Likes Received:
    0
    Trophy Points:
    18
    The problem for us is that it occurs on a daily basis and on 2-3 servers (out of 4) daily.

    My hypothesis is that the user when creating the account, instead of waiting for the operation to complete, pressed refresh on their browser. Or the user use the back button after creating the domain resulting in an attempt again to insert the dns record.

    Either way, I guess how this has to be fixed is that the dns clustering has to:

    1. Place a lock on the named.conf till editing is fully completed, file is saved etc.
    2. Before inserting a new record in named.conf to check that no duplicate entries exist.

    That way, this issue should not happen. Sometimes there are as many as 3 duplicate records in named.conf.
     
  4. silversurfer

    silversurfer Well-Known Member

    Joined:
    Dec 29, 2002
    Messages:
    274
    Likes Received:
    0
    Trophy Points:
    18
  5. Marty

    Marty Well-Known Member

    Joined:
    Oct 10, 2001
    Messages:
    630
    Likes Received:
    1
    Trophy Points:
    18
    I had the duplicate entry problem as well until I arrived at this configuration for clustering:

    Server1 is the primary nameserver

    Server2 is my secondary nameserver

    I have 5 servers configured as follows:

    Server1 <--> Server2 (i.e. changes on Server1 are transferred to Server2 and changes on Server2 are transferred to Server1)

    Server1 <-- Server3

    Server1 <-- Server4

    Server1 <-- Server5

    So changes on Server3, Server4, and Server5 are all transferred to Server1, but changes on Server1 are not transferred back to 3, 4, and 5. As changes from 3, 4,and 5 are transferred to 1, Server1 then transfers them to Server2.

    This cluster has worked well. I get a duplicate once in a great while, but it is usually due to something involving a site transfer gone bad.
     
  6. silversurfer

    silversurfer Well-Known Member

    Joined:
    Dec 29, 2002
    Messages:
    274
    Likes Received:
    0
    Trophy Points:
    18
    I run 4 nameservers so this wouldn't work for us.
     
  7. Marty

    Marty Well-Known Member

    Joined:
    Oct 10, 2001
    Messages:
    630
    Likes Received:
    1
    Trophy Points:
    18
    If I were going to run 4 nameservers on 4 servers numbered 1 - 4, I would run it this way.

    Server 1 <--> Server 2
    Server 2 <--> Server 3
    Server 3 <--> Server 4

    Say that server 5 - 7 are boxes that host sites but do not host nameservers, I would set them up this way:

    Server 1 <-- Server 5
    Server 1 <-- Server 6
    Server 1 <-- Server 7

    Based on the pattern I used above, it should work, but it may be what you are already doing.
     
  8. silversurfer

    silversurfer Well-Known Member

    Joined:
    Dec 29, 2002
    Messages:
    274
    Likes Received:
    0
    Trophy Points:
    18
    Sounds plausible. I give it a shot. Afterall there's nothing to lose since we are having named.conf corruption daily on multiple servers already.
     
  9. casey

    casey Well-Known Member

    Joined:
    Jan 17, 2003
    Messages:
    2,303
    Likes Received:
    0
    Trophy Points:
    36
    Location:
    If there is trouble, it will find me
    I can't figure out which server "Standalone" and "Synchronize" is talking about in WHM. In the above scenario, in WHM on Server1, what do you select for Server3, Server4, and Server5 - Standalone or Synchronize?
     
  10. silversurfer

    silversurfer Well-Known Member

    Joined:
    Dec 29, 2002
    Messages:
    274
    Likes Received:
    0
    Trophy Points:
    18
    It means

    server 3 to 5 set to syncronize with 1 (but 1 isn't set to syncronize with 3 to 5)
    server 2 set to syncronize with 1
    server 1 set to syncronize with 2
     
  11. casey

    casey Well-Known Member

    Joined:
    Jan 17, 2003
    Messages:
    2,303
    Likes Received:
    0
    Trophy Points:
    36
    Location:
    If there is trouble, it will find me
    Aha! :) Thanks. I had them all synchronising with each other, which I guess is pretty unnecessary.
     
  12. Marty

    Marty Well-Known Member

    Joined:
    Oct 10, 2001
    Messages:
    630
    Likes Received:
    1
    Trophy Points:
    18
    I had it that way at first, but when I changed it so that everything syncronized back to Server 1 and Server 1 only syncronized with 2, and all other servers were standalone with 1, my duplications went away.

    Definistions for symbols in previous post:

    1 <--> 2 = 1 is added to 2 as synchronize, and 2 is added to one as syncronize

    1 <-- 2 = 1 is added to 2 as syncronize, and 2 is added to one as standalone.
     
  13. MediaServe

    MediaServe Well-Known Member
    PartnerNOC

    Joined:
    Apr 9, 2004
    Messages:
    126
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Nashville, TN USA
    cPanel Access Level:
    DataCenter Provider
    Twitter:
    I'm fighting this same issue! Daily at least one server has duplicates in named.conf, sometimes up to three servers are affected.

    I like the scenario suggested, but I wonder if anyone has tried more of a round robin approach, where, given 4 servers for example, server 1 updates 2, 2 updates 3, 3 updates 4 and 4 updates 1.

    I just wonder if anyone has taken that approach, and does it help with duplicates? We have several sets of reseller nameservers spread across 6 servers, and I'm not sure whether I should try the idea posted above, or this circular approach.

    Of course, with this method, inserting new servers requires a little more configuration, as you'll have to break the last link (last server updating first) and fit the new server in.
     
  14. silversurfer

    silversurfer Well-Known Member

    Joined:
    Dec 29, 2002
    Messages:
    274
    Likes Received:
    0
    Trophy Points:
    18
    Yes. Since I used Marty's approach, not a single corruption issue. Been over a week.
     
  15. Marty

    Marty Well-Known Member

    Joined:
    Oct 10, 2001
    Messages:
    630
    Likes Received:
    1
    Trophy Points:
    18
    wtl,

    The problem with the scenario you suggest is this.

    If you an account to server 1, that actual addition of the account adds the zone and the conf entry, then when server 4 gets updated, it updates server 1 again which I think causes the duplicate.

    If you use the scenario I outlined above, no server ever gets updated more than once and thus the dupes are eliminated just as silversurfer has seen.

    I am glad it worked for you. It bugged the heck out of me when I first started using clustering.
     
  16. MediaServe

    MediaServe Well-Known Member
    PartnerNOC

    Joined:
    Apr 9, 2004
    Messages:
    126
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Nashville, TN USA
    cPanel Access Level:
    DataCenter Provider
    Twitter:
    What I'm considering is a little different though, it'll look more like this:

    1 -> 2 -> 3 -> 4 -> 1

    There will be no bidirectional updates in my cluster if I take this approach.

    I just wonder if anyone has tried this sort of approach to this problem. I went ahead and implemented the above just now, so if anyone cares I'll come back in a couple days and let you know how it goes.

    I'm mostly worried about the possibility of an endless loop with the above circular setup. At least the graphical display cPanel provides ends properly, i.e if I look at the DNS path on 2, for example, it shows 2 -> 3 -> 4 -> 1, so hopefully it'll work.

    Previously I had all servers updating all other servers, I'm sure that probably caused the problem, with 1 server getting updates from several others at the same time.
     
    #16 MediaServe, Jan 19, 2005
    Last edited: Jan 19, 2005
  17. MediaServe

    MediaServe Well-Known Member
    PartnerNOC

    Joined:
    Apr 9, 2004
    Messages:
    126
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Nashville, TN USA
    cPanel Access Level:
    DataCenter Provider
    Twitter:
    Hmm, good point. I wonder if manually adding a fake zone on 1 will demonstrate whether or not this will be a problem? I'll give it a shot and see what happens.
     
    #17 MediaServe, Jan 19, 2005
    Last edited: Jan 19, 2005
  18. MediaServe

    MediaServe Well-Known Member
    PartnerNOC

    Joined:
    Apr 9, 2004
    Messages:
    126
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Nashville, TN USA
    cPanel Access Level:
    DataCenter Provider
    Twitter:
    Well, I'm sure one test isn't conclusive, but I can add a zone on the first server without it getting duplicated by the last server. I'll leave it as is just for the heck of it and observe what happens over the next several days.
     
  19. Marty

    Marty Well-Known Member

    Joined:
    Oct 10, 2001
    Messages:
    630
    Likes Received:
    1
    Trophy Points:
    18
    wtl,

    If it works, great! You may be right that it was the addition of zones to the same server from several servers "simultaneously" that was the problem. I just did a lot of expermentation that led me to the conclusion that each server should only be setup so that only one server actually updates it no matter where I add a zone. It was a working theory, and only a theory. Can't do much more than theorize since there is no real working documentation for these features. We get features and then go about trying to figure out how they work. :)
     
  20. MediaServe

    MediaServe Well-Known Member
    PartnerNOC

    Joined:
    Apr 9, 2004
    Messages:
    126
    Likes Received:
    0
    Trophy Points:
    16
    Location:
    Nashville, TN USA
    cPanel Access Level:
    DataCenter Provider
    Twitter:
    Figured I'd come back to this thread and report that the circular cluster arrangement I used has been working flawlessly. No duplicate named.conf entries. I have one cluster of five servers, and another cluster of three, and it's working great.
     
Loading...

Share This Page