Loading Profile...
URLs restricted by robots.txt ... ?
I'm getting the "URLs restricted by robots.txt" message and just wanted to make sure that this is ok/normal when google can't read these. In my case, what google cant see is a few pdf files, that I saved from the internet. Is this normal? Is there anyway I can get rid of the errors as I like to have a clean bill of health, if you see what I mean?
I’m happy that things are going so well
1
person has this question
I have this question, too!
Tell me when someone answers.
The more people who ask this question, the more it gets noticed.
The more people who ask this question, the more it gets noticed.
-
Inappropriate?Hi Will
All Yola sites have a robots.txt file associated with them to prevent Google from crawling parts of your site that should not be indexed. If you want to see your site's robots.txt file, type your domain name into your browser, followed by /robots.txt. You will then see the URLs that are disallowed. One of these URLs is /resources. However, Google will be able to index the file once you embed it on your page.
However, there is some discussion about how Google handles the indexing of PDF files. Here is a useful forum discussion on the subject: http://www.webmasterworld.com/forum10...
You should not be concerned about this message as it does not indicate a problem with your site.
The company says
this answers the question
2 Comments Add a comment
This answers the question
-
i have exactly the same problem. my adsense showing several blocked url/robots.txt. also, my site displaying irrelevant ads!!!!! quick question, does the ads display differently in different countries or does the same ads i see displayed, gets seen by everyone visiting my website? How sure are you (Yola), that these restricted/blocked robots are nothing to worry about? does this in any way affect our adsense ads/rankings/etc? Last question, if this is a subdomain, can i still rank well in google? Please check out my website and let me know about any problems i am missing. Its an informative medical website. (Applied for my HON seal of approval from Health on the net code of conduct).
http://www.mesotheliomacancer.yolasit... -
Hi Rynaldo,
As you add more content to your site and your site is crawled with more frequency your ads should become more relevant because Google will learn what your site is about. This will take time, and necessitate you updating your content on a regular basis.
People have had very good success with Google using a free subdomain name, but you may wish to purchase a custom domain name as it can look more professional and be easier for your visitors to remember.
Without seeing the exact message that you received about the blocked url/robot.txt there is no way for us to be 100% sure that this is your situation, but I am very confident that this is the same case as outlined above.
Good luck with your site. Please let us know if there is anything else we can help you with -
EMPLOYEE
