Question about robots.txt
-
Solved!
-
Just a friendly reminder. Please don't delete your question after it's been answered. It's very likely that someone in the future could have the same question and they would have been able to find the answer if you hadn't deleted the question.
-
Consider deleting all of this:
Disallow: /&limit
Disallow: /?limit
Disallow: /&sort
Disallow: /?sort
Disallow: /?route=checkout/
Disallow: /?route=account/
Disallow: /?route=product/search
Disallow: /?route=affiliate/
Disallow: /?marca
Disallow: /&manufacturer
Disallow: /?manufacturer
Disallow: /?filter
Disallow: /&filter
Disallow: /?order
Disallow: /&order
Disallow: /?price
Disallow: /&price
Disallow: /?filter_tag
Disallow: /&filter_tag
Disallow: /?mode
Disallow: /&mode
Disallow: /?cat
Disallow: /&cat
Disallow: /?product_id
Disallow: /&product_id
Disallow: /?route=affiliate/
Disallow: /*?keywordThose rules are telling Google not to crawl domain.com/EVERYTHING(then the URL parameter). This could be where the issue stems from. If you're worried about URLs with these things ranking, consider implementing canonical tags instead to point to the proper pages
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Canonical question - Would this create some sort of crawler redirection loop?
What happens if a canonical link, links to the url with / but the main url does not have the / For example: rel="canonical" href="https://www.exampleURL.co.uk/"> Main URL - https://www.exampleURL.co.uk (without the /) 301 Redirect https://www.exampleURL.co.uk/ to https://www.exampleURL.co.uk Would this create some sort of crawler redirection loop?
On-Page Optimization | | Evosite10 -
Simple question: I wonder if I'm over internally linking?
Hi, What's you guys' policy on how much to internally link. I do it a lot - whenever it makes sense, but hold off if I just linked to the same page in the last paragraph, for instance. Would like to know your thoughts to see if I'm overdoing it. This is for Ecommerce blog posts, category descriptions, and product descriptions if that matters. Thanks.
On-Page Optimization | | BobGW0 -
Magento Canonical & Default Robots Settings
Hello! I'm working with Magento 1.9 for an eCommerce site with several hundred products. Currently I understand it is best practices to use the Canonical tag, however I also have my default robots set to "Index, Follow". Will this cause an issue having product pages set to index, follow but also having a canonical tag included? What are some best practices regarding Magento default robots & canonical tags? Any help is appreciated.
On-Page Optimization | | BretDarby0 -
A question on cached pages
Hi all. I have been testing different ways of writing the text on the homepage of the website www document-management-solutions co.uk because we are ranking very low for the keyword Document Management Software. I've recently learnt about pages being 'cached' and wanted to just clarify something. The last time the page was cached was on 26th March. I have made changed to the homepage since then and wanted to see how it has affected our ranking. Will I not be able to know if it has had a positive/negative effect until the next time our page is cached? Thanks
On-Page Optimization | | janc0 -
Re="tag" Question
Hello, I own the site I putted rel="tag" in categories and product links , its correct? There are a lot of categories and products , I wont get penalized right? Just making sure to know if Im doing it right , thank you for help me 🙂
On-Page Optimization | | matiw0 -
Title tag questions
General title tag questions how important is it not to change your title tag, I own a prom dress site and I'm always changing them between prom and homecoming' Is it bad to have title tags that are only different by one word across thousands of pages on your site. I have thousands of dresses so each title tag only varies by the style # I have always had title tags that are for example black prom dresses, well i recently discovered that just black dresses is googled 10 times the amount so im debating changing them to black dresses so that the word dresses is closer to the front of the title tag, am I over reacting or is that a good idea or would it be bad to put black dresses, black prom dresses, black homecoming dresses I also put the year in almost every title tag 2012, is that bad, I ranked great for 2012 stuff but could it hurt my homepage domain rankings on major broader keywords
On-Page Optimization | | Dirty0 -
I run a Q&A site, should I limit post questions to 64 characters?
I run a Q&A site (Recessionitis.com), should I limit posts (questions) to 64 characters for SEO purposes?
On-Page Optimization | | 10JQKAs0 -
Page title in SERP question
Has anyone typed in a phrase in Google and seen their listing on the SERP, but the page title on the Google SERP is not what the CMS is set to ? Ie the page title in the SERP is not what is expected? Something related to the company, but not what is set on the CMS… Very odd – has anyone seen something like this before? What could be causing it? Is there a way to change it?
On-Page Optimization | | inhouseninja0