by Many of us find this notice on AdSense dashboard on blogger and WordPress website,
“You have ad crawler errors, which can result in lost revenue”
Here we’ll tell you, how to fix AdSense ad crawler error after going to Action.
Working on Google AdSense, you will one day or another come across this one of the most common errors You have Ad Crawler Errors, Which can result in Lost Revenue. There are many reasons this error can be caused, so here we will talk about all the ways to solve this error. One more thing, this error can occur for any site be it a WordPress site or a Blogger site. The solutions work fine for each of the cases – as long as it is connected with Google AdSense.
Google Adsense Crawler
In this case, you’ve blocked the ads crawler in your robots.txt file. All you need to do is to check whether the ads crawler disallowed in your robots.txt file? If yes, then give access to the ads crawler in your robots.txt file. There is information about a robots.txt file but we do not have a custom one and have never had this issue.
If you’ve modified your site’s robots.txt file to disallow the Googl AdSense crawler from indexing your pages, then Google is not able to serve Google ads on these pages.
To update your robots.txt file to grant Google crawler access to your pages, remove the following two lines of text from your robots.txt file:
User-agent: Mediapartners-Google
Disallow: /
WordPress / Blogger Site Access Issues
Step 1. Click on the Action button as shown in the notification. Alternatively, you may click on Account from the left pane and select Policy Centre
This will show results on the right side.
Here in, under Sites with issues, you will see the website affected page or URL. Click on the result to deep dive in.
Step 2. Now you will see page issues. For instance, in our case, Type of issue: Crawler: Page not found
and there is one affected page with URL
What we have found so far is that Google Crawler doesn’t find this particular page or URL in this website (Crawler: Page not found).
The reason could be any, such as, either page doesn’t exist or never exist, page or URL may be deleted, the URL may be changed, etc.
To check and confirm, you can copy this URL and open it in a new tab of any web browser and paste it there then, hit Enter.
Fix err Google Search Console
Now, the rest of the steps we need to do it on Google Search Console.
Login to Google Search Console
Note: Make sure you have copied that URL. For example: in our case it is example.com/download-office…%E2%80%8B
From the left side pane, click Removals
Click NEW REQUEST button from the right.
New Request page will open on your screen. Follow below steps now:
Select the CLEAR CACHED URL tab
Paste the URL that we have already copied above
Select “Remove this URL only” not the other one
Lastly, click Next to finish the process.
The moment you are done with above all steps. You are free to click on Dismiss option shown in the notification.
As a result, next time when a Google Crawler runs on your site (usually one or two days or in a few days), that URL won’t be found in crawler error.
Issues with Your Hosting
An issue with your site’s name server
Make sure the name server for your domain or subdomain is properly directing the ads crawler to your content. For this purpose, you can contact your hosting providing company immediately to get this problem resolved.
There’s an issue with your site’s server
Sometimes when the ads crawler tries to access site content, the site’s server is unable to respond in time. This can happen because the server is down, slow or overloaded by requests.
We recommend ensuring your site is hosted on a reliable server or by a reliable service provider.
