Getting your site totally indexed by SEOMOZ
-
Hi guys!
Ijust started using SEOMOZ software and wondered how it could be that my site has over 10.000 pages but in the Pro Dashboard it only indexed about 1500 of them. I've been waiting a few weeks now but the number has been stable ever since. Is there a way to get the whole site indexed by SEOMoz software?
Thanks for your answers!
-
Thanks Abe!
As soon as I can analyse the whole site, including all the links, i will get back at you.
I think you got a good point about the links. I don't think SEOmoz can do anything about that
Greets,
-
Hello,
I'm sorry that some of your pages didn't end up getting crawled. There's a lot of reasons why this could have happened, some of which have to do with our crawler, and others that have to do with a site. Basically, the crawler finds pages by crawling all of the links on the homepage/front page and then crawling all of the pages those links took him to, and so forth, until all of the pages we can find under that directory are crawled. Usually, if a page is linked to from the homepage, it should end up getting crawled. If it doesn't, it's either a sign that those pages aren't as accessible as they could be to search engines, or that there is an issue with our crawler. Some of the time, it's a problem with Javascript - our crawler can't handle Javascript very well right now. Other times, content was created a little bit after we crawled but before we finished processing and putting up the results. That content will be captured in the next crawl usually. It can also be due to meta tags or robots.txt telling rogerbot not to crawl.
On the whole, I'm not sure why some of your pages wouldn't be showing up in the crawl. Can you let me know some examples of pages that should be in the crawl but didn't end up in your campaign? If you could also outline the links going to those pages, I can send all that information over to our engineering team and we can try to diagnose the issue. Thanks a ton!Talk to you soon,
Abe Schmidt-
SEOmoz Helpster
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Site Crawl Stalled and Can't Restart
In my GreenSeed campaign, the site crawl continues to say "in progress." I can't figure out how to stop it or how to restart the site crawl. Can you please help?
Moz Pro | | Winger1 -
Site Crawl 4xx Errors?
Hello! When I check our website's critical crawler issues with Moz Site Crawler, I'm seeing over 1000 pages with a 4xx error. All of the pages that are showing to have a 4xx error appear to be the brand and product pages we have on our website, but with /URL at the end of each permalink. For example, we have a page on our site for a brand called Davinci. The URL is https://kannakart.com/davinci/. In the site crawler, I'm seeing the 4xx for this URL: https://kannakart.com/davinci/URL. Could this be a plugin on our site that is generating these URLs? If they're going to be an issue, I'd like to remove them. However, I'm not sure exactly where to begin. Thanks in advance for the help, -Andrew
Moz Pro | | mostcg0 -
Still Cant Crawl My Site
I've removed all blocks but two from our htaccess. They are for amazonaws.com to block amazon from crawling us. I did a fetch as google in our WM tools on our robots txt with success. SEOMoz crawler here hit's our site and gets a 403. I've looks in our blocked request logs and amazon is the only one in there. What is going on here?
Moz Pro | | martJ0 -
Open Site Explorer CSV reports
Hello, Small question about the Open Site Explorer. on August 24th I've been trying to create several CSV reports which I knew would take a few days. But now, almost a week later when I check Open Site Explorer > Recent CVS reports it tells me it's still saving data. It's been saving data for 4 days now. Am I missing something, is this a bug, do I have to do something or should I wait a little longer? Thanks in advance, Dennis Tappij Gielen Retail&Clicks JFwMt.jpg
Moz Pro | | RetailClicks0 -
Site explore reporting error over week
unable to dispaly anchor text error Doh! Roger is still working out the kinks with the new index and is having issues untangling anchor text data. We're currently showing anchor text data from the previous index, but we will update as soon as we can.
Moz Pro | | 1step2heaven120 -
HTTPS site in Open Site Explorer
I'm looking at a site for which the https URL currently ranks in Google. Using a header checker on the http URL I see that it is being 302 redirected to the https version (I have no control or input on this site). In OSE there's no option to specify an https URL as the http part is pre-populated and uneditable. My question is: does OSE treat the https and http version as the same URL? I'm guessing so as the http URL has a lot of domain authority despite not being the "default" URL.
Moz Pro | | Equatorites0 -
301 redirect in SEOMoz campaigns tool
I did a 301 redirect to another domain and I would like to change the domain name in SeoMoz campaigns tool to continue to track the keywords, is it possible ?
Moz Pro | | mhenriques0 -
Seomoz staff help needed
I have this new client new domain new website, www.ard.uk.com when seo moz crawls the site it says it has 24 million links mozrank of over 7, looks amazing but impossible can someone as seomoz have a look please
Moz Pro | | francesco-2850160