Our Duplicate Content Crawled by SEOMoz Roger, but Not in Google Webmaster Tools
-
Hi Guys,
We're new here and I couldn't find the answer to my question. Here it goes:
We had SEOMoz's Roger Crawl all of our pages and he came up with quite a few erros (Duplicate Content, Duplicate Page Titles, Long URL's).
Per our CTO and using our Google Webmaster Tools, we informed Google not to index those Duplicate Content Pages. For our Long URL Errors, they are redirected to SEF URL's.
What we would like to know is if Roger is able to know that we have instructed Google to not index these pages. My concern is
-
Should we still be concerned if Roger is still crawling those pages and the errors are not showing up in our Webmaster Tools
-
Is there a way we can let Roger know so they don't come up as errors in our SEOMoz Tools?
Thanks so much,
e
-
-
Thanks Kent!
-
In order to answer your questions, we need to understand exactly how you "instructed Google".
If you went into Google Webmaster Tools and changed settings such as parameters of URLs, then only Google will know about those changes. Roger wont know and neither will Bing or any other crawler.
Whenever possible I recommend making the proper adjustments on your site so all crawlers know and understand how to crawl your website.
**1) Should we still be concerned if Roger is still crawling those pages and the errors are not showing up in our Webmaster Tools **
Yes. First of all, Bing + Yahoo = 30% of the search market and they will experience the same trouble as Roger. Second, by informing Google directly you are making changes in a way that make it impossible for tools such as Roger to provide you with actionable data.
2) Is there a way we can let Roger know so they don't come up as errors in our SEOMoz Tools?
Not that I am aware of. I wish Roger made better friends with Googlebot and could learn all the personal settings, but that was not possible the last time I checked.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
My "tag" pages are showing up as duplicate content. Is this harmful?
Hi. I ran a Moz sitecrawl. I see "Yes" under "Duplicate Page Content" for each of my tag pages. Is this harmful? If so, how do I fix it? This is a Wordpress site. Tags are used in both the blog and ecommerce sections of the site. Ecommerce is a very small portion. Thank you. | |
Moz Pro | | dlmilli1 -
Duplicate Content
Hello, I'm managing a site which shows as having duplicate page issues (in the crawl analyser) for 3 pages. Basically the site is offering 3 different options of the same product so depending on which size you select, you are directed to the relevant page. These 3 pages are basically identical apart from a slight difference in copy regarding the size (small, medium, large) Is this likely to be a big issue regarding SEO, and what would the moz community suggest re this? Thank you!
Moz Pro | | wearehappymedia0 -
What's my best strategy for Duplicate Content if only www pages are indexed?
The MOZ crawl report for my site shows duplicate content with both www and non-www pages on the site. (Only the www are indexed by Google, however.) Do I still need to use a 301 redirect - even if the non-www are not indexed? Is rel=canonical less preferable, as usual? Facts: the site is built using asp.net the homepage has multiple versions which use 'meta refresh' tags to point to 'default.asp'. most links already point to www Current Strategy: set the preferred domain to 'www' in Google's Webmaster Tools. set the Wordpress blog (which sits in a /blog subdirectory) with rel="canonical" to point to the www version. Ask programmer to add 301 redirects from the non-www pages to the www pages. Ask programmer to use 301 redirects as opposed to meta refresh tags & point all homepage versions to www.site.org. Does this strategy make the most sense? (Especially considering the non-indexed but existent non-www pages.) Thanks!!
Moz Pro | | kimmiedawn0 -
Used a free WP theme and now Webmaster Tools has shown hundreds of broken links
Hello everyone, We've recently launched two new music instrument listing websites. One in the US http://www.yourinstrument.com One in AU http://www.yourinstrument.com.au While we're waiting for our custom, branded WP to be built, I went and put up one of the free themes with closely matching colors. Clearly, this was a mistake because now my WMT account has spit out around 200 'not found' errors and my Moz account is calling them around 282 'dupe page titles' and 'content'??? 1. Aren't WP outbound links supposed to be 'nofollow' ? / Why is my theme creating strange links? 2. Can I clean all these up with assigning rel=canonical to the original blog post? Do I need to wait for the new theme to be installed and then 301 the random URLs found by Moz and WMT ? weird link examples: http://imgur.com/Cm6pO5H Please help 😕
Moz Pro | | Beetlebum0 -
Google Webmaster Tools and Open Site explorer's links not matching up
My question is why do my GWT's links not match up to the ones on Open Site Explorer. I watched John Mueller's video, and he said that they had problems with the link counts recently. After checking my links today, I can see that the problem is fixed, but my link count differs from Open Site Explorer. On Web Master Tools I have 307 links, but on Open Site Explore I have 42. Has anyone dealt with this problem? thanks. Peter
Moz Pro | | PeterRota0 -
Changing the way SEOmoz Detects Duplicate Content
Hey everyone, I wanted to highlight today's blog post in case you missed it. In short, we're using a different algorithm to detect duplicate pages. http://moz.com/blog/visualizing-duplicate-web-pages If you see a change in your crawl results and you haven't done anything, this is probably why. Here's more information taken directly from the post: 1. Fewer duplicate page errors: a general decrease in the number of reported duplicate page errors. However, it bears pointing out that: **We may still miss some near-duplicates. **Like the current heuristic, only a subset of the near-duplicate pages is reported. **Completely identical pages will still be reported. **Two pages that are completely identical will have the same simhash value, and thus a difference of zero as measured by the simhash heuristic. So, all completely identical pages will still be reported. 2. Speed, speed, speed: The simhash heuristic detects duplicates and near-duplicates approximately 30 times faster than the legacy fingerprints code. This means that soon, no crawl will spend more than a day working its way through post-crawl processing, which will facilitate significantly faster delivery of results for large crawls.
Moz Pro | | KeriMorgret2 -
SEOmoz Keyword Difficulty Tool been down for a few days?
Hi All, I notice the SEO moz keyword difficulty tool has been down for a few days!!! I know from support that they say it is going to be a "while" till it fixed, but some type of estimation on how long it will be will be good. Also in regards to the types of accounts, why do the top accounts have the same limitations as the 79/month tool in regards to the keyword tool reports (50 max and 5 per scan)? I mean this is probably a wider question for the SEOmoz team need to answer. Kind Regards.
Moz Pro | | ColumbusAustralia2 -
Duplicate Page Content and Title - Miva - How to fix?
Hi, I'm new to SEOmoz and just diving into it. I'm feeling a bit overwhelmed. I use Miva Merchant as my storefront interface. SEMOz is returning a bunch of duplicate page content and duplicate page titles and I can't figure out what to do about it. It seems it may have something to do with Miva shortlinks. I click on the dup URL's in SEMOz and it brings me to a dead page. I can't figure out where it's coming from. I know without seeing the actual information it'll probably be tough to help me but any suggestions would be appreciated. I try to fix them and come to a point (after about three hours of getting nowhere) it becomes too frustrating. Thanks!
Moz Pro | | musicforkids
Gary0