Blocked URL ...robot.text file

  • 2
  • Question
  • Updated 10 years ago
Hi...I need help again::.

On my Google adsense diagnostics..I have this repeatedly showing:

Blocked URL:
http:/ / sitebuilder. yola. com/ sites/ D5f5/ D212/ D500/ D1d7/ U8a4986c82068116701207d10052125f5/ 8a4986cb2068115e01207d1e40da2630/

Reason Blocked:
Robots.txt File

Last Crawl Attempt:
May 25, 2009

How do I allow the robots permission to crawl my site ads etc..?

Apparently, I have to alter my robots.text file...do I have one? What or where is it? or do the good people at Yola have to do it for me (ask your provider....)

I ́d really like to get this resolved today guys..If it helps ..my site is http://westseas.com which was the domain i bought from Yola .. transferring over from the original synthasite sub-domain.

thanks

Mark
Photo of akasback

akasback

  • 36 Posts
  • 0 Reply Likes
  • confident

Posted 10 years ago

  • 2
Photo of Emmy

Emmy

  • 5892 Posts
  • 299 Reply Likes
Official Response
Hi Mark,

If you get a message saying that your URL is blocked because of a robots.txt file, please ignore this.

Google has simply tried to index the preview URL of one of your pages. A preview URL is what you see in your browser window when you open your site in the Yola sitebuilder and then preview it. Yola prevents Google from crawling these URLs as Google should not index unpublished sites. This message simply serves to indicate that everything is working correctly. This will not interfere with your search engine ranking or Google Webmaster's ability to return data about your site.

If you have further questions please let us know!

Emmy
Photo of RSPublishing

RSPublishing

  • 298 Posts
  • 9 Reply Likes
i have exactly the same problem. my adsense showing several blocked url/robots.txt. also, my site displaying irrelevant ads!!!!! quick question, does the ads display differently in different countries or does the same ads i see displayed, gets seen by everyone visiting my website? How sure are you (Yola), that these restricted/blocked robots are nothing to worry about? does this in any way affect our adsense ads/rankings/etc? Last question, if this is a subdomain, can i still rank well in google? Please check out my website and let me know about any problems i am missing. Its an informative medical website. (Applied for my HON seal of approval from Health on the net code of conduct).
http://www.mesotheliomacancer.yolasit...
Photo of Ruth

Ruth

  • 2819 Posts
  • 135 Reply Likes
Hi Rynaldo,

As you add more content to your site and your site is crawled with more frequency your ads should become more relevant because Google will learn what your site is about. This will take time, and necessitate you updating your content on a regular basis.

People have had very good success with Google using a free subdomain name, but you may wish to purchase a custom domain name as it can look more professional and be easier for your visitors to remember.

Without seeing the exact message that you received about the blocked url/robot.txt there is no way for us to be 100% sure that this is your situation, but I am very confident that this is the same case as Emmy outlined above.

Good luck with your site. Please let us know if there is anything else we can help you with.
Photo of akasback

akasback

  • 36 Posts
  • 0 Reply Likes
So they (Google etc..) will still be able to display the relevant adverts?

Thanks again Emmy

Mark
Photo of Emmy

Emmy

  • 5892 Posts
  • 299 Reply Likes
Yep, the adverts will still display!
Photo of Mark N

Mark N

  • 466 Posts
  • 30 Reply Likes
Glad someone brought this up as I have had the same problem. Phew! That's a relief!
Photo of Gemma

Gemma

  • 2 Posts
  • 0 Reply Likes
Google is crawling my site but when I check the adsense report, it says there is a problem with the robot.txt not allowing adsense to crawl the site and only the server can allow this. Yola, can you permit this or what can I do? Or should I just delete all the google ads out as I am giving them free advertising space on my site with the current arrangement...
Photo of Marije

Marije, Official Rep

  • 4636 Posts
  • 237 Reply Likes
Hi Gemma - Google has simply tried to index the preview URL of one of your pages. A preview URL is what you see in your browser window when you open your site in the Yola site builder and then preview it. Yola prevents Google from crawling these URLs as Google should not index unpublished sites. This message simply serves to indicate that everything is working correctly. Is your site published and are your adverts displaying?
Photo of Gemma

Gemma

  • 2 Posts
  • 0 Reply Likes
Thanks, Marije. My site has been published for some time. It has appeared on the first page of a google search. The ads appear on the site but I have never made 1 cent from them. It could be that no one has clicked on them but according to the adsense report, it tried to crawl the site twice but was blocked and I can only unblock it by requesting the server to do that.

Nevermind, I'm very happy with Yola and my site looks good. Deleted all the ads and now it looks even better :)
Photo of RSPublishing

RSPublishing

  • 298 Posts
  • 9 Reply Likes
i have exactly the same problem. my adsense showing several blocked url/robots.txt. also, my site displaying irrelevant ads!!!!! quick question, does the ads display differently in different countries or does the same ads i see displayed, gets seen by everyone visiting my website? How sure are you (Yola), that these restricted/blocked robots are nothing to worry about? does this in any way affect our adsense ads/rankings/etc? Last question, if this is a subdomain, can i still rank well in google? Please check out my website and let me know about any problems i am missing. Its an informative medical website. (Applied for my HON seal of approval from Health on the net code of conduct).

http://www.mesotheliomacancer.yolasit...