Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

ajax crawler doesn't work

Discussion in 'General Discussion' started by uadm, Aug 15, 2017.

  1. uadm

    uadm Well-Known Member

    Joined:
    May 19, 2003
    Messages:
    85
    Likes Received:
    0
    Trophy Points:
    156
    Hello,

    We have a strange problem. On a local computer we run sitemap generator and gets about 2000 pages in sitemap.xml. When we run the same generator on the exact same site that is installed on the cpanel server we only get about 620 pages in sitemap.xml.

    Is there anything to hange on the cpanel server?

    Thanks
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    38,165
    Likes Received:
    1,371
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello,

    Do you notice any output to /usr/local/apache/logs/error_log when running the application?

    Thank you.
     
  3. uadm

    uadm Well-Known Member

    Joined:
    May 19, 2003
    Messages:
    85
    Likes Received:
    0
    Trophy Points:
    156
    no, nothing out of the oridinery.
     
  4. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    38,165
    Likes Received:
    1,371
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Hello,

    If it were an issue related to Apache blocking the request, then you'd generally see an error message in the Apache error log. You may want to consult with the support team for the application you are using to generate the sitemap.xml file to determine if there are any application logs you can review to determine why it's not fetching all of the pages.

    Thank you.
     
Loading...
Similar Threads - ajax crawler doesn't
  1. josh77
    Replies:
    4
    Views:
    422
  2. arvid3nel
    Replies:
    8
    Views:
    1,277
  3. keat63
    Replies:
    4
    Views:
    609

Share This Page