Magento Dynamic Pages Being Indexed
-
Hi there, I have about 50k Moz medium priority errors in my Crawl Diagnostic report.
The bulk of them are classified as "Temporary Redirect" problems. Then if you drill into those further, I can see that the problem urls all kinda are center around:
mysite.com/catalogsearch/result..
mysite.com/wishlist..
mysite.com/catalog..
Is this something I should disallow in my Robstxt file? And if so how specific do I get with it..
Disallow /catalogsearch/result/?q=
Will listing the /catalogsearch be enough to cover anything after it?
thanks
-
Jeff is completely right on this one!
-
Thank you very much Jeff!
-
Yes, Magento does have a way of handling search results that are not always SEO-friendly.
To restrict Google from indexing anything in the /catalogsearch/ directory, I'd add this to your robots.txt file:
User-agent: *
Disallow: /catalogsearch/Here's a Moz blog about the robots.txt file:
http://moz.com/blog/interactive-guide-to-robots-txtAnd also, just in case the "killer" robots ever take over, Google has made sure that their two founders are not able to be indexed by both the T-1000 and T-800 models, with a special killer-robots.txt file:
http://www.google.com/killer-robots.txtHope this helps!
-- Jeff
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Magento Site "Title Missing or Empty"
Hi everyone . . ..bear with me as I am "noob". My moz analysis brought up a few "critical" issues, one of which was a missing or empty title for a page. That page is: http://www.thirdcoastsigns.com/sales/guest/form Which does not even appear to be a working page . . . so I'm at a bit of a loss ow to fix this. I suspect it's a little bit of SEO expertise and a little bit of Magento expertise. Thoughts and suggestions welcome! P.S. Look forward to my NEXT noob question about a bunch of pages I have flagged as "Pages with temporary redirects". Stay tuned!
Moz Pro | | damon12120 -
Duplicate Page Title error for an eCommerce store !!
I currently launched my eCommerce startup hosted in Shopify and linked with MOZ. From my first Crawl Report I am getting 580 Duplicate Page Title i.e. all my Collection page have the same title. I have googled and have been checking the MOZ community but cannot find a fix to it. Some of the URL's are - http://www.onlypetstore.com/collections/all http://www.onlypetstore.com/collections/all?page=10 http://www.onlypetstore.com/collections/all?page=100 http://www.onlypetstore.com/collections/all?page=101 http://www.onlypetstore.com/collections/all?page=102 http://www.onlypetstore.com/collections/all?page=103 I am new to SEO and any suggestions will be a great help to me.
Moz Pro | | OnlyPetStore2 -
On page analysis showing old results
Hi, My on page crawl analysis is showing that I have 38 duplicate content issues. These issues are because we had used tags in our blog. We remove all the tags 10+ day ago but the crawl that was done 5 days ago is still showing the tags as causing a duplicate content issue. Why would this be? We cannot correct is as it is already corrected. How do we get moz to crawl the correctly? Thanks Andrew
Moz Pro | | Studio330 -
History of first page/top3 rankings
Is there a way to see a graph of the history of these two metrics that are displayed on semoz?
Moz Pro | | santiago230 -
Adding canonical still returns duplicate pages
According to SEOmoz, several of my campaigns show that I have duplicate pages (SEOmoz Errors). Upon reading more about how to resolve the issue, I followed SEOmoz's suggestion to add rel='canonical' <links>to each page. After the next SEOmoz crawl, the number of SEOmoz Errors related to duplicate pages remained the same and the number of SEOmoz notices shot up indicating that it recognized that I added rel='canonical'.</links> I'm still puzzled as to why the SEOmoz errors did not go down with respect to duplicate page errors after I added rel='canonical', especially since SEOmoz noticed that I added them. Can anyone explain this to me? Thanks,
Moz Pro | | MOZ2
Scott.0 -
Moztool and on page ranking matching
How does the Moztool compare and filter the search phrases you enter in your campaign? Or more correctly, will it filter out stop words or is it an exact match? For example I enter a phrase to track that say: "book ski trip austria" Identified in Google I see that most users search for just that "book ski trip austria" But in content, I cant write that as that is uncorrect english and I want to maby write something like: "When you book a ski trip to austria you get..." How will this affect my on page SEO report, will it still match and mark a "V" in done or show a an error? Even more interesting is, what happen if you do phrases in different order like "An austrian skip trip will make you feel..."
Moz Pro | | Macaper0 -
Duplicate page content due to Sort By dropdown
Hi there, I have over 150 Duplicate Page Title errors showing up in SEOMoz but on closer inspection these are related to the 'Sort By:' functionality on our ecommerce site that allows customers to sort our products by Price, Alphabetically etc. To give an example: http://www.parklanechampagne.co.uk/park-lane-champagne/special-occasions/easter Is showing as being duplicated by this page: http://www.parklanechampagne.co.uk/park-lane-champagne/special-occasions/easter?productlisting_page=1&sortorder=Price Does anyone know how I can resolve this? Any help greatly appreciated. Kind regards, Jon CDFyp.jpg
Moz Pro | | jonmorse860 -
SEOmoz indicating duplicate page content on one of my campaigns
Hello All, Alright, according to SEOmoz's PRO campaign manager, one of my websites is returning about 2,700 pages that supposedly have duplicate content. I checked a few of them manually and am not seeing where the issue lies. Is anyone else experiencing something similar to this and do you know if it is just a glitch with the crawl? Here are 2 of the pages it is indicating have dup page content: http://www.dieselpowerproducts.com/c-3120-1994-98-59l-12v-dodge-cummins-carbon-fiber-hoods.aspx http://www.dieselpowerproducts.com/c-90-dodge-cummins-94-02-59l-12v24v.aspx Any insight would be greatly appreciated! -Craig
Moz Pro | | ckilgore0