How to crawl the whole domain?
-
Hi,
I have a website an e-commerce website with more than 4.600 products. I expect that Seomoz scan check all url's. I don't know why this doesn't happens.
The Campaign name is Artigos para festa and should scan the whole domain festaexpress.com. But it crels only 100 pages
I even tried to create a new campaign named Festa Express - Root Domain to check if it scans but had the same problem it crawled only 199 pages.
Hope to have a solution.
Thanks,
Eduardo -
Hi Kery, thanks, I just sent to them.
Regards,
Eduardo. -
Hi Eduardo,
I'm sorry you're still having problems. At this point, it'd be best for you to send an email to help@seomoz.org and have our help team look at it for you. They'd be the ones who could give you the most advice for diagnosing this.
Keri
-
Still have the same problem. Isn't that an issue with SEOMoz?
The domain is www.festaexpress.com has no flash and is crawled by google with no issues.Regards,
Eduardo. -
Hi Eduardo.
The way crawlers work is the begin on your home page and "crawl". They look at all the links on your home page and follow each one to the next page, then the next until your whole site has been captured.
Why are only 100 pages being crawled?
Most likely either because your site is not very well linked, or because you don't have a good navigation system, or because your navigation and links are presented in a format such as flash which the crawler cannot read.
Another possibility would be if the crawler is being blocked or hindered by your robots.txt file.
-
Not sure, but you could try Microsoft's IIS tool to spider your site. It is possible that your site has issues that make it difficult to spider, hence why SEOMoz's bot isn't working. You could also try something like Xenu Link Sleuth or HTTrack.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Best Keyword Strategy for Lower-Authority-Ranked Domain
Hi, What should a lower-authority-ranked blog (DA - 21) who is trying to increase general traffic focus on above all when selecting keywords: difficulty, monthly volume, opportunity, or priority? I know that all are important factors to consider, but I fear that without an already consistent following our new content will just get hidden behind more established domains if we select too difficult of keywords. I'd love to hear from someone who's built up a blog audience as to what strategy they used here. Thanks!
Moz Pro | | ascott7770 -
Domain redirection www or without
when i start campain for the first time it says " "Roger has detected a problem: We have detected that the domain www.xxxxxxtour.com and the domain xxxxxxtour.****.com both respond to web requests and do not redirect. Having two "twin" domains that both resolve forces them to battle for SERP positions, making your SEO efforts less effective. We suggest redirecting one, then entering the other here." How to resolve this and how can i redirect xxxxxxtour..com to www.xxxxxxtour.com ? Thank you
Moz Pro | | innofidelity0 -
Mass lookup of Domain Authority
Is there a way to upload 100 sites to somewhere in SEOmoz and get the domain authority of each site? If not, what is the fastest way to get DA for 100 sites? Thanks, Steve
Moz Pro | | sgirardot0 -
SEomoz slow to crawl?
Hello - I am just trying out the trial and it said the next crawl was nov 1st but I see no change in any of the errors since the initial crawl... so just waiting to find out if what I changed was fixed or not. Is this normal ?
Moz Pro | | Bethany_BabyBrowns0 -
Change campaign domain name
I have just changed the domain name on one of my sites, can i change the domain name of the reletive campaigns and keep the data, or do i have to set up a new campaigns?
Moz Pro | | AlanMosley0 -
Can I exclude pages from my Crawl Diagnostics?
Right now my crawl diagnostic information is being skewed because it's including the onsite search from my website. Is there a way to remove certain pages like search from the errors and warnings of the crawl diagnostic? My search pages are coming up as: Long URL Title Element Too Long Missing Meta Description Blocked by meta-robots (Which is how I want it) Rel Canonical Here is what the crawl diagnostic thinks my page URL looks like: website.com/search/gutter%25252525252525252525252525252525252525252525252525252525 252525252525252525252525252525252525252525252525252525252525252 525252525252525252525252525252525252525252525252525252525252525 252525252525252525252525252525252525252525252525252525252525252 52525252525252525252525252525252525252525252525252Bcleaning/ Thank you, Jonathan
Moz Pro | | JonathanGoodman0 -
SEOMoz only crawling 5 pages of my website
Hello, I've added a new website to my SEOmoz campaign tool. It only crawls 5 pages of the site. I know the site has way more pages then this and also has a blog. Google shows at least 1000 results indexed. Am I doing something wrong? Could it be that the site is preventing a proper crawl? Thanks Bill
Moz Pro | | wparlaman0 -
How best to gain Followed Linking Root Domains?
Hi All, Sorry if this question has been asked before. I've had a look through the Q&A sections and I couldn't really see anyone who'd asked the question as direct as this? Basically, I'm using the SEOMoz competitive domain analysis to compare some of our domains and our competitors domains and see where we're falling down. At the minute, we seem to be only scoring around '31', whereas our two closest competitors as 32 and 33 on the overall score. This still seems pretty low to me (maybe it's not) so I'm assuming we can make some good headway. At the minute, the analysis shows that we're beating our competitors hands down with the number of links to our site, these are all relatively good quality and they haven't just been spammed out there to gain links. We are however struggling to get followed linking root domains. As I understand it, these are basically just links from domains not on the same IP that are followed? Since I've seen this, I've looked through the open site explorer and gone through all of our competitors sites to get backlinks from URL's linking to them. I've had fairly good success with this but they only seem to have gone up by 1 - 2 links. Does this only get updated once a month? In addition to this, does anyone have any tips on how to build up followed linking domains? Kind Regards
Moz Pro | | LewisSellers
Lewis0