in robots.txt

  • 1
  • Question
  • Updated 5 years ago
  • Answered
how can I EDIT robot.txt , google webmaster tools is showing presence of http://www.mysite.com/sitemap.xml in robots.txt
does it means robots is blocking http://www.mysite.com/sitemap.xm and so it is preventing google bot and bing bot to index updated webpages from the website

http://www.freeastrologyvedic.com
Photo of Akash Tiwari

Akash Tiwari

  • 1 Post
  • 0 Reply Likes

Posted 5 years ago

  • 1
Photo of Sanja

Sanja, Official Rep

  • 10698 Posts
  • 495 Reply Likes
If you get a message saying that your URL is blocked because of a robots.txt file, please ignore this. Google has simply tried to index the preview URL of one of your pages.

A preview URL is what is used when you open your site in the Yola Sitebuilder and then preview it. Yola prevents Google from crawling these URLs as Google should not index unpublished sites. This message simply serves to indicate that everything is working correctly. It will not interfere with your search engine ranking or Google Webmaster's ability to return data about your site.

As long as your site is published and you have submitted the correct, published URL, Google will be able to index your site. If you get a "Page not Found" or 404 error this probably means that there was a problem with your DNS server (the server of your domain name registrar) or that you had made a DNS change around the time that Google tried to index your site. Google makes an attempt approximately every 5 days, so you simply need to wait for the Google spiders to come around again and everything should be fine.

I hope that information helps!