The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

"argument list too long" how can i check logs then?

Discussion in 'General Discussion' started by Jorel, Nov 19, 2005.

  1. Jorel

    Jorel Well-Known Member

    Joined:
    Aug 15, 2003
    Messages:
    45
    Likes Received:
    0
    Trophy Points:
    6
    I have 1,000 domains on a dual-opteron server so obviously we have some pretty big log files in /usr/local/apache/domlogs/. So when I'm trying to search through them to find out where attacks are originating from, I always getting the "argument list too long" error. How can I get around this?
     
  2. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    It can be quite tricky. One way is to search a subset of the files, e.g. *.com then *.net then *.org, etc. That way you're restricting the matches as you search, but it does make it time consuming, though writing a script would help if you do it a lot.
     
  3. btrieve

    btrieve Well-Known Member
    PartnerNOC

    Joined:
    Mar 20, 2002
    Messages:
    47
    Likes Received:
    0
    Trophy Points:
    6
    Pretty easy workaround, pipe it to xargs, example grepping through thousands of logs:

    find . -name '/usr/local/apache/domlogs/*' |xargs grep "whatever"
     
  4. chirpy

    chirpy Well-Known Member

    Joined:
    Jun 15, 2002
    Messages:
    13,475
    Likes Received:
    20
    Trophy Points:
    38
    Location:
    Go on, have a guess
    find can still fall prey to that issue, though.
     
  5. btrieve

    btrieve Well-Known Member
    PartnerNOC

    Joined:
    Mar 20, 2002
    Messages:
    47
    Likes Received:
    0
    Trophy Points:
    6
    Not when the results are then piped to xargs.

    man xargs

    Also we use this on servers with 1500-2000 log files and it is a non issue in regards to the argument list error. Taking the results from standard input and generating/building command line execution is the sole purpose of xargs.
     
    #5 btrieve, Nov 20, 2005
    Last edited: Nov 20, 2005
  6. astopy

    astopy Well-Known Member

    Joined:
    Apr 3, 2003
    Messages:
    165
    Likes Received:
    0
    Trophy Points:
    16
    cPanel Access Level:
    Root Administrator
    Wouldn't something like: find /usr/local/apache/domlogs/ -exec grep something {} \;
    be better? Rather than piping it all to xargs.
     
  7. Jorel

    Jorel Well-Known Member

    Joined:
    Aug 15, 2003
    Messages:
    45
    Likes Received:
    0
    Trophy Points:
    6
    this worked for me, while
    Code:
    find . -name '/usr/local/apache/domlogs/*' |xargs grep "whatever"
    did not (for whatever reason). chirpy's method would probably work too but i haven't tried it because like he says, it's more of a pain.

    thanks!
     
Loading...

Share This Page