i have adsence account but google team warn me about 160 failed AdSense crawls detected

  • 1
  • Problem
  • Updated 8 years ago
  • Acknowledged
i have adsence account but google team warn me about
160 failed AdSense crawls detected


failed crawls
detected last week

We noticed that our AdSense crawler is having some trouble crawling pages of your site on yola.com. As a result, your AdSense ads are less targeted and are generating less revenue than they could be.

We detected 160 failed crawl requests last week, caused by a misconfigured robots.txt file that's blocking certain pages of your site from the AdSense crawler. Since these pages already have AdSense code on them, you'll want to allow the AdSense crawler so that the most relevant ads will be displayed.

View crawler errors
You can fix this issue today by editing your robots.txt file and granting the AdSense crawler the ability to access these blocked pages. To do this, find your robots.txt file and add the following two lines of text at the very top:

User-agent: Mediapartners-Google

Earn additional revenue

Fixing this issue will allow our crawler to serve more relevant ads across more pages of your site, provide your users with better-targeted ads and, as a result, help you potentially earn more revenue.

Please note that making this change will not impact your Google search rankings; adding these two lines to your robots.txt file will simply deliver better, more relevant ads to pages with AdSense code already on them. Pages that don't have AdSense ad code will not be affected.
please help me

For more in-depth information, make sure that you read our blog post about this on the Inside AdSense blog.

Yours Sincerely,
The Google AdSense team


  • 1 Post
  • 0 Reply Likes

Posted 8 years ago

  • 1
Photo of Laura Thomas

Laura Thomas

  • 4536 Posts
  • 211 Reply Likes

We do not support the use of robots.txt files. The only thing I can think of is if you have password protection enabled on any of your pages. If you do, I would ensure you don't have Adsense code on your protected pages.