The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

RRDtool bandwidth graph gaps

Discussion in 'General Discussion' started by retali8, Sep 9, 2010.

  1. retali8

    retali8 Member
    PartnerNOC

    Joined:
    Oct 1, 2008
    Messages:
    7
    Likes Received:
    0
    Trophy Points:
    1
    I meant to also include a screenshot of the bandwidth graphs that we're seeing for this specific user.
     

    Attached Files:

  2. retali8

    retali8 Member
    PartnerNOC

    Joined:
    Oct 1, 2008
    Messages:
    7
    Likes Received:
    0
    Trophy Points:
    1
    We are seeing a very strange issue on several of our servers where a users bandwidth graph will have many gaps and have been unable to figure out what exactly is causing the issue. I've checked the file in /var/cpanel/bandwidth/{username}.hour and see the missing data for specific times of the day as you can see below..

    2010.09.08T08-all=124029557
    2010.09.08T08-http=124029557
    2010.09.08T13-all=2367
    2010.09.08T13-pop3=2367
    2010.09.09T03-all=19130782
    2010.09.09T03-http=19130782
    2010.09.09T04-all=84447074
    2010.09.09T04-http=84447074
    2010.09.09T05-all=102852479
    2010.09.09T05-http=102852479

    Does anyone know why exactly this would be occurring? I've also checked /etc/stats.conf and no blackout hours are configured. I've enabled raw log archiving and they do have HTTP traffic around these times but for some reason it's not being recorded. I've also checked the /usr/local/cpanel/logs/stats_log and I'm not seeing any errors for this user but we are seeing it for other users on the same server and other servers as well.
     
  3. GWade

    GWade Registered

    Joined:
    Oct 2, 2008
    Messages:
    2
    Likes Received:
    0
    Trophy Points:
    1
    Gaps in bandwidth data

    There are a couple of different mechanisms that can cause data not to appear in the bandwidth graphs. One of them is a long-standing issue and the other is more recent.

    RRDtool Artifacts
    The first is a little hard to understand. RRDtool takes an odd approach to storing data which results in unusual results. The data is actually stored as a rate of change of data, not as actual data points. This has caused odd graph artifacts in the past.

    The artifact that sometimes caused "missing bandwidth" has to do with the rate-storage and the fact that the data is stored in "steps". For example, in the data for today, bandwidth is collected in 5 minute steps. So, in effect, all data in a particular 5 minute span is stored as a single number. Additionally, since RRDtool only stores a rate, you need two steps in order to generate a rate.

    In the the traditional usage of RRDtool, if data comes in widely separated bursts (each of which fits in a single step) a rate can never be calculated because there are no two points close together. A rate requires two data points (to calculate a change), so a single data point does not get reported. Moreover, if bandwidth is being detected most of the time, this effect would not show up.

    This issue was actually fixed several months ago by added "synthetic" data points with a value of 0 in the places where we did not get data. Although it seems obvious, this is not how RRDtool is normally used.

    0-Length Remainder Files
    The more recent issue has to do with a change to a module that cPanel depends on for several features: YAML::Syck. The YAML::Syck was changed a few months ago to throw an exception when a 0-length file is read with YAML::Syck. Before the change, the YAML::Syck returned empty data.

    The Bandwidth processing regular reads and writes a YAML file that keeps up with data that we cannot yet write to RRDtool (because we are in the middle of one of the time steps). If the bandwidth processing doesn't have any left-over data, a 0-length remainder file is written. Before the YAML::Syck change, that was perfectly legitimate.

    This issue has been fixed in the upcoming version of cPanel. Unfortunately, if you are using the 11.26 branch and YAML::Syck gets updated, there is a problem.

    Conclusion
    A new version of 11.26 should be released within a few days that has the updated code that solves this problem. If you cannot wait and definitely have the problem, I have attached a patch that should repair this problem (on 11.26.9).

    This can be applied by executing
    patch -p0 < 43327.patch
    in the directory /usr/local/cpanel. (Use the real path to 43327.patch.)

    Hope that helps.
     

    Attached Files:

    #3 GWade, Sep 10, 2010
    Last edited: Sep 13, 2010
Loading...

Share This Page