Can Google see all the pages that an seomoz crawl picks up?
-
Hi there
My client's site is showing around 90 pages indexed in Google. The seomoz crawl is returning 1934 pages.
Many of the pages in the crawl are duplicates, but there are also pages which are behind the user login.
Is it theoretically correct to say that if a seomoz crawl finds all the pages, then Google has the potential to as well, even if they choose not to index?
Or would Google not see the pages behind the login? And how come seomoz can see the pages?
Many thanks in anticipation!
Wendy
-
Well, that could be your easy solution. Make sure they're all set not to be indexed, then you'll be able to (mostly) ensure Google won't crawl them, and they'll probably disappear from your moz crawl report as well. As far has how moz is finding them to begin with behind your login wall, sorry, I have no idea.
-
The pages behind the login? No not yet - they are a new client, so I am just auditing at the moment to identify what we need to do
Many thanks for your replies!
-
This may be an obvious question, but to you have those pages set to noindex?
-
Hi Marisa
seomoz are crawling unecessary pages, (they return pages ignored by screaming frog for example)
BUT my concern is that if Google can also see them, even if they choose to ignore them my client maybe getting slammed for duplicate issues or the pages behind the login may suddenly appear in the index.
We'll get no index / no follow added, and fix the dupes, but am really interested as to how seomoz sees behind the login
-
Here's the real question: Do you WANT Google to see all these pages, or is SEOmoz crawling unnecessary pages?
-
Great, many thanks Nakul - they are a new client so am waiting on getting access to WMT - will go through with a fine tooth comb! Just seems really weird with regards to the pages behind the login ...
-
Wendy, if SEOMOZ can see it, I am sure Google can see it as well. I would login to your webmaster console and check the index status. Do you have an XML sitemap submitted for your website ? Once you do, you'll have a more accurate read on the number of pages you submitted and how many of them are indexed. The new index status Google introduced last month also lets you see pages Google ignored for multiple reasons.
I hope this helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate Content: Marketing Page / Content Page
So I am getting duplicate content warnings on my website for my pages white paper and webinar video pages. Each white paper / webinar video page is behind a marketing form page that must be filled out. I am getting a lot of warnings that the marketing page and the content page are being picked up as duplicated content. In the past, both the marketing page and the content page were given the same title and url, the body content is not similar. My question: Is the URL / Title similarity enough to set off the duplicate content warnings and would changing one or the other solve the issue?
Moz Pro | | AllMedSeo0 -
Crawl credits how to buy more?
Just wondering if there is a way of increasing, my 2 crawl credits per day limit?
Moz Pro | | aussieseoguy0 -
Can I exclude a sub-domain from SEOMoz campaigns?
We have recently implemented a white label site that is on a sub-domain. The site employs noindex on most of the pages I imagine due to duplicate content concerns on other white label versions of the site. It has led to a spike of over 14 thousand notices on our report. Is there a way to exclude a sub-domain from the SEOMoz scans and reports?
Moz Pro | | TSDigital0 -
Seomoz crawling filtered pages
Hi, I just checked an seo campaign we started last week, so I opened seomoz to see the crawl diagnostics. Lot's of duplicate content & duplicate titles showing up, but that's because Rogerbot is crawling all of the filtered pages as well. How do I exclude these pages from being crawled? /product/brand-x/3969?order=brand&sortorder=ASC
Moz Pro | | nvs.nim
/product/brand-x/3969?order=popular&sortorder=ASC
/product/brand-x/3969?order=popular&sortorder=DESC&page=10
/product/brand-x/3969?order=popular&sortorder=DESC&page=110 -
How does SEOMoz get ranking data? is it automated queries to Google?
Google's terms don't allow automated queries without Google's permission. https://support.google.com/webmasters/bin/answer.py?hl=en&answer=66357 How does SEOMoz get ranking data then? Is it sending automated queries with or without Google's permission? is it violating Google's guideline? Will it affect my site performance as far as rankings?
Moz Pro | | huangyouyou1170 -
Can I change the SEOMoz logo on the report to input one of mine?
Hello everybody! I have an PRO PLUS account, and as I'm working with SEOMoz Campaigns, i'd like to know if I can change the logo of SEOMoz on the Ranking Report, to a different one, on of mine in this case. Thanks!
Moz Pro | | seomasterbrasil0 -
Crawl test. Bot crawled only 200 or so links when it should have crawled thousands
Hi everyone, I just recieved my crawl test report and its only given me 200 or so URL's when my site has thousands, any thoughts?
Moz Pro | | Ev840