robots.txt need changes

  • 1
  • Question
  • Updated 3 years ago
  • Answered
Archived and Closed

This conversation is no longer open for comments or replies and is no longer visible to community members.

I need to add lines to robots.txt based on current google changes how can i do this?

User-Agent: Googlebot
Allow: .js
Allow: .css<br><br>needs to be in there.<br>
Photo of Joe Moore

Joe Moore

  • 44 Posts
  • 1 Reply Like

Posted 4 years ago

  • 1
Photo of Joe Moore

Joe Moore

  • 44 Posts
  • 1 Reply Like
wow 2 days not a single reply what kind of support is this?
Photo of Philip Taylor

Philip Taylor

  • 355 Posts
  • 66 Reply Likes
OK, at the risk of bringing down the wrath of Joe upon me:
Read the "Related Conversations" at the right - in particular the one that says
"Unfortunately it is not possible to access the robots.txt file of your site to make changes to it."
(Edited)
Photo of Gop

Gop

  • 7473 Posts
  • 2115 Reply Likes
That post was made 5 years ago.

It is probably the same but Yola Support should have answered Joe's post before now. That's tardy! Obviously Laura and Stefan are not available but there's a whole department responsible for these queries.
(Edited)
Photo of Joe Moore

Joe Moore

  • 44 Posts
  • 1 Reply Like
Yeah Phillip i did but like he said that was 5 years ago and they have changed allot since.  If we cannot they could atleast reply or make it an fricken option.  Google changes everyday i am an SEO and shit like this is important i have 14 client sites in here i need to change on most.
Photo of Gop

Gop

  • 7473 Posts
  • 2115 Reply Likes
Hello Joe,

This following strategy has been proposed previously but there has never been proof that it works.

Add the specific robots.txt additions to the tracking area of the head of the site. You may then be able to see if it has taken subsequently via webmasters tools. 

Because we can't see the htaccess or the real head of sites, it is possible that Yola have already added this to their over-all. This would need Yola verification.
Photo of Sanja

Sanja

  • 10698 Posts
  • 495 Reply Likes
Hello Joe, 

In the midst of our moderators' absence, we let this fall through the crack and that shouldn't have happened. I sincerely apologize and hope we can do a better job going forward. 

Does this relate to Google Webmaster Search Console which indicates that Googlebot cannot access CSS and JS files?

If so, this seems to be a generic message that is sent out which doesn't necessarily affect your site. There are two articles that you may be interested to read: 

Google Search Console Warnings Issued For Blocking JavaScript & CSS

Google Webmaster Search Console warning: "Googlebot cannot access CSS and JS files" in Google Search Console

In the meantime, if you are seeing blocked resources being listed, could you please share this with us.  If you need to know which resources are blocked, here is how to find them. The blocked resources also show up in the "Blocked Resources" section under "Google Index" in Google Search Console.

Most likely, these are false positives for resources that Yola manages to run your site, and Google has previously said webmasters do not need to worry about these.

If this is related to something different, please do let us know. 

Photo of Gop

Gop

  • 7473 Posts
  • 2115 Reply Likes
"....  let this fall through the crack" ???

Absolute rubbish! It was in the top positions of postings for several days. Where did it fall to? 

Can't accept that, Sanja, but rather that no-one bothered to check the posts or if they did simply ignored it as too hard.

So many times has the VP stated that the forum is monitored continuously by Yola Support people. It wasn't even acknowledged and would show up in that GS category as it does on this forum.

"fallen through the cracks" is an euphemism for dereliction. 
(Edited)
Photo of Joe Moore

Joe Moore

  • 44 Posts
  • 1 Reply Like
yes it is related to that and as some have specified in comments on those posts it may effect some more then others.  Using you Ecommerce is an example as that uses both.  I am a PAYING customer that does not have access to his own ROBOTS.TXT file? 

Is it simply too hard too give all paying subscriptions access to such a file.  I mean hell most of the SEO adjustments require your highest package or upgrade to be able to modify.  In my opinion anyone paying monthly for any of your packages should be eligible to adjust anything on that website or account.

In this case my ecommerce site went from top 10 to top 50 since that notification came out.  All my other sites NOT with Yola are in the same spots they where in.  My home page ranks still it is the products and categories that no longer do.  Java Script and CSS on those pages being blocked would be my first blame.  I have had the same ranks for 10 4 years straight up until that notice.

You are correct it will not effect those who do not use JS or CSS as intense such as basic 5 page sites or a 1 page landing page etc that is simply a sales pitch and nothing fancy.  Or even a basic wordpress site which uses CSS but not so much JS.
Photo of Monique

Monique, VP of Customer Support

  • 6294 Posts
  • 446 Reply Likes
Hi Joe

We are discussing this and will get back to you.  Currently it is not possible to edit your robots.txt file regardless of your account level.  I am not in a position to comment on what would be involved in giving customers access but I will find out and let you know whether this is a feature we could consider offering in future.  We are always investigating ways of improving the SEO friendliness of our sites and the Online Store.  Up till now there hasn't been a significant demand for the ability to edit the robots.txt file but if that changes due to Google's requirements then we would take this into consideration.
Photo of Lyall

Lyall, Official Rep

  • 142 Posts
  • 10 Reply Likes
Hello Joe,

After discussing this matter with our engineers, we can confirm we have taken steps to ensure Googlebot can access JS and CSS on Yola sites. Will you republish the site in question and let us know if you have further trouble? If you do, please confirm the site or sites you are referring to.
Photo of Ryan Bennett

Ryan Bennett

  • 2 Posts
  • 0 Reply Likes
Photo of Jen

Jen

  • 206 Posts
  • 24 Reply Likes
Hello Ryan,

This screenshot shows that Googlebot cannot access some 3rd party resources.  Google has advised that 3rd party resources are not an issue.  You can read more here:

http://www.thesempost.com/google-confirms-third-party-blocked-resources-not-problematic/

I hope that helps!
Photo of Ryan Bennett

Ryan Bennett

  • 2 Posts
  • 0 Reply Likes
YOLA, Have you fixed this issue yet?