Google Shopping Feed being blocked by robots.txt
-
I had created a manual Google Shopping Feed that was working fine, and then someone well meaning put a block in my robots.txt file so Google couldn't read the images folder. because of this, Google now won't accept my feed.
I changed the robots.txt file to allow them to read the images again, but it's been 3 days now and I'm still getting the error saying my products are disallowed because the robots.txt file won't let them scan for images.
Does anyone know how long it will take for Google to see it again?
-
When I manually fetch, the /images folder isn't on the list anymore.
For now I have just reuploaded the images in to a new folder and I'm creating my feed by hand, so that should get around it for now.
Thanks for your help.
-
Did you try manually fetching as Googlebot the robots.txt file via the webmaster tools? If not, do that, then click submit to index. Then do the same thing for some of the images or better if you have a file linking to all images (like an image sitemap). Once that is done, let Google recrawl everything for a few days (even a week) and try again.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Merchant Center product reviews
My googlefu is not good enough to put the words together to find an answer for this, but I am sure there is one out there. I am making a review feed for a client's site to submit their product reviews to google merchant center. But one problem we are having is that the reviews are generalized, like how amazon does it. A sample product would be Whey Protein, this product is available in 4 different sizes and has 4 different GTIN's (one for each size). But the reviews we collect are just for the base product of Whey Protein. What I am trying to figure out, is if I can assign a parent GTIN, then correlate the reviews to that GTIN and then let them populate for the child GTIN's some how.
Paid Search Marketing | | LesleyPaone0 -
Approximate costs of keywords in Google AdWords
Just wondering if anyone has any suggestions about how to go about getting some approximate costs for keywords through Google Adwords to assist my SEO efforts. I don't want to open an AdWords account, just looking for costs. If the pricing is too high then the opened account was a waste of time. So if anyone has any resources to find out this information, that would be great! Thanks in advance!
Paid Search Marketing | | MainstreamMktg0 -
Google Adwords Clicks not registering properly???
Hello Moz World, I am running a Google Adwords campaign utilizing Google Search and Search Partners only. These are traditional text ads. When I look at the analytics through Google Adwords, I am seeing 60 clicks. Great! However, when I look through the keywords in the ad groups, none of my keywords are registering clicks. I also checked search terms under the keyword tab, and no keywords are populating. Where are my ads being clicked, and how can I view information on these clicks. This is a relatively new campaign, and I am still trying to figure out if I am targeting the right keywords. Thanks ahead of time for all the great responses! B/R Will H.
Paid Search Marketing | | MarketingChimp100 -
Any data for Google results page click through rates?
Hi. Trying to find data for click through rate for the Google search results page? If I run Adwords, do I want a PLA listing or a text listing? Or do I work on PLA + Organic or Text + Organic? What percentages do each position get approximately? E-commerce website. Technology sector. Any help?
Paid Search Marketing | | YNWA0 -
How do I print off Google Adwords Certification certificate through Google Partners?
Hey everyone, How do I print off Google Adwords Certification certificate through Google Partners?
Paid Search Marketing | | jhinchcliffe1 -
Google Analytics and Adwords Tracking Codes
A site I help someone with in link building was recently redesigned. I do not have access to the site in any way. Well, after the redesign, no conversions were tracking in Adwords and Analytics ecommerce tracking showed no transactions and no amounts. I found the Analytics code and sent it to their programmer. I have linked the Adwords and Analytics accounts. So now conversions are tracked in Adwords and transactions are tracked in Analytics. However, amounts have vanished and not returned. I'm not technically savvy enough to figure this out. How do I get this Analytics code to track transaction amounts so that they appear in Adwords and in Analytics ecommerce tracking?
Paid Search Marketing | | DanDeceuster0 -
How much does Google adwords cost increase year to year?
I am trying to find about how much the cost of Pay-Per-Click increases year to year? If anyone can point me to some data/study that speaks to this I would appreciate it very much.
Paid Search Marketing | | JerrodDavid0 -
Adwords Product Listing Ads & Google Analytics mis-reporting
I hope you're sitting comfortably, this could be a long one and loaded with questions! Cut to the chase: Why is traffic from google product ads showing as 'organic' traffic in GA? Here's the scenario: Google Shopping I have thousands of products in a feed to google shopping (froogle, google base, google merchant, whatever you like to call it, I'll settle for google shopping for the purpose of this question). The URLs of this feed is tagged with GA tracking data (notably utm_source=GoogleBase&utm_medium=Product-Search), I have also tagged this with internal tracking which comes through in the back-end to assign orders to that specific source. In this case 'GOOGLEBASE'. Adwords Product Listing Ads As you know, a new (ish) feature of adwords pulls in your products from google shopping so that you get a richer ad (image, title, price) and displays this in the 'advert section' of the SERP. Once setting up a few of these, I noticed I was getting a fair amount of traffic for these new ads, taking one example, which resided in a relatively specific ad group (advertising Aviation Snips). Naturally, I wanted to find out which keywords were driving that traffic in order to improve the ads, or kill them if they weren't working. What was interesting is that I can't find anything about that traffic anywhere in adwords or google analytics. 254 clicks to 'aviation snips' must show up somewhere in analytics, if not the keywords, then what about the product? Analytics is showing nothing like that quantity of visits to those product landing pages where you'd expect. It's like ghost traffic. Google Analytics Since experimenting with product listing ads the organic traffic in GA has suddenly shot up, looking at the new keywords they are all queries which when I test them show up product listing ads in the SERP so it's obviously the paid listing ads driving this traffic. Why is google reporting these as organic, rather than paid? I also noticed a keyword appear as * in the PAID segment of analytics. I thought this was my missing aviation snips traffic, but digging into the landing pages for the * keyword, they are many different ones. There's a connection between the * and product listing ads, but what is it? Is the traffic being doubly reported? Back End Meanwhile we've seen an increase for orders tagged in the back-end of GOOGLEBASE which makes sense - google are pulling in my google shopping feed into the paid part of the SERPs and these are generating sales. Here are some of my initial thoughts / theories: 1. When google pulls in google shopping results into the organic part of the SERP, these get reported as ORGANIC in google analytics, even if you've tagged them otherwise. It seems they strip the tags out. This makes it very difficult to know if your google shopping feed is working well, or if you are doing well on standard organic traffic. 2. Google isn't separating out traffic as PAID with their new product listing ads, completely skewing the reports. It makes it look like you've gained great natural organic listings when if fact you are paying. 3. With relation to the missing Aviation Snips data - maybe google is showing a huge variety of products for that adgroup (even though it's specific) and therefore I can't see the traffic to the specific products that you'd expect. This I'm most confused about and wondered if I've missed a trick in setting the product listing ads up? I've attached a couple of screenshots which I hope will help clarify some of this. I can see product listing ads being great if you could get proper data to analyse and improve them. So here are my questions again if anyone can help? How do I see which keywords are driving the product listing ads? How do I see the landing pages for the product listings ads? What is the * keyword coming through in GA? How can you get GA to report product listing ads as paid rather than organic? Thank you so much. If I can gather enough data on this all and work it out I'll try to write up in a blog post to help others. 0rOMM.png GUAE0.png fWPL7.png
Paid Search Marketing | | ewanr0