How do i get the crawler going again?
-
The initial crawl only hit one page. Set up another campaign for another site and it crawled 260 pages.
How can I get the crawler started up again or do I really have to wait a week ?
-
It's easiest for the Moz team to respond to individual questions like this via an email to help@seomoz.org or sending in a ticket through the help hub at seomoz.org/help.
-
Hi Keri,
Yeah, I've followed these steps. htaccess forwards to www if none and my campaign is using www.
The robots.txt is available in browser no problem.
my htaccess had massive country ip blocks as well as amazonaws..
I've removed all blocks that arent china or amazon.
Still waiting on a response from the MOZ team here on how to get the real crawl going as what I found in tools is not the same as the weekly scan.
-
Also, for why we only crawled one page, check out https://seomoz.zendesk.com/entries/409821-Why-Isn-t-My-Site-Being-Crawled-You-re-Not-Crawling-All-My-Pages- for some helpful tips.
-
Whenever you have crawl issues, the SEOMoz help team is the place to go. They're the only ones who can look into your account to see if there's an issue or such. (And they are really nice and quick to respond.)
-
Research Tools > Crawl Test at bottom left of page.
Not sure why it's taken me 2 days to find that but it did.
Thanks. Sorry for the dumb question...
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Meta no index crawler warnings
I've decided that the duplicate content issues on my site weren't worth the effort from the amount of traffic the archive pages on my WordPress site received no I decided to no-index them using Yoast. Now I have 60 meta no-index crawler warnings. Should I just ignore these? It seems I get warnings, either way, I use the site. Does anyone have advice on how to move on with this?
Moz Pro | | Libra_Photographic0 -
New Mozscape index released! Learn just what's been going on.
Hi everyone, I'm very, very pleased to announce that the latest Mozscape index has been released five days ahead of schedule, and it's back up to the quality you expect from Moz! It's no secret that we've struggled this year with our index. I can't overstate how much we appreciate everyone sticking with us—we know how hard this has been. The good news—well, apart from the early release—is that we've worked out the issue, and we expect to finally be back on track moving forward. Director of Engineering Martin York has shared a post to the Moz blog explaining just what's been going so wrong, and outlining our plan for the future. Thank you, from the bottom of our hearts, for being such an incredible community. As for the folks who are seeing Domain Authority and Page Authority drops since the index release, please see Rand's recent post for more.
Moz Pro | | MattRoney7 -
Why did I not get on page analysis
I signed up last Friday for a 30 day trial and was hoping to get the on page analysis so I could get cracking on it. Moz said the crawl was done but I have no info for on page analysis which I find really useful when I have used this before. Any ideas and are any of the MOZ moderators able to get the system to perform one so I can make changes tonight for the MOZ crawl again tomorrow? Thanks Paul
Moz Pro | | ptrobson0 -
Is there any way to get historical API data?
I'm looking for a way of requesting domain authority and some other stats via the API for several previous months. I noticed on here: http://apiwiki.seomoz.org/what-is-the-mozscape-api
Moz Pro | | ReallyEasyCart
It says "SEOmoz PRO members now have access to historical API linking data - read about this great new feature on our blog!" but i could not find this blog post anywhere and maybe I'm misreading this line anyway.0 -
I go crazy!
Hi everybody, a short summary at the start: I have my page (www.nlp-deutschland.de) since a long time positioned good, but not perfect in google. The best ressource is a forum wich brings me about 50.000 content-pages, unique and nice. My most important keyword is "NLP". I never needet to look for backlinks. There where only two pages above me: nlp.de, which is the direct key, and wikipedia. Since about one year my page is suffering on google. Mostly it is because of competitors. I personaly think, they buy a lot of backlinks. I startet Seomoz PRO three months ago and I see, that they have about 5 times as much backlinks from other domains / c-blocks as I have. Round about, I got 100 and they got 500. However, in the link analytics I also see that some pages with fewer backlinks are ranked higher. And I get so many different hints about SEO, that I don´t know anymore which one to follow. I am sure about what I am doing with backlinks, but I am not sure about onpage. Here an example:
Moz Pro | | JulianW
All my internal links are relative. As my key is also includet in the domain itself, it get´s no repetition in the url. One of my competitors, which has not the key in his url, just created a subfolder "/nlp/" to all of his pages and has absolute links in his navigation. This brings him the key to every single link in his navigation, while I have no key in the navigation, because it´s in the URL itself and the navigation is relative. I was told that it would make a lot of sense to change the navigation to absolute links, just to get my key into every url of the navigation. Others say, this is bullshit. Another question is double meta-description and meta-tags. With 50.000 content-pages it would be a huge lot of work to get them all done different (by hand). But it is shown as an ERROR in the SeoMoz-Reports. Are they so much important? However, I just realize I have no clue about onpage-optimization. So what would you suggest are the most important factors onpage? All the best, Julian!0 -
Can I get a list of all links on a given domain?
Sorry, this is actually kind of a tripartite question: I was looking at the Competitive Link Analysis on one my clients' campaigns. Sometime between June and September their total links went up by about 120,000. We have no idea where those links came from (although the numbers would indicate that they're mostly internal). Question 1: In none of the other tools can I figure out how to list these links on a domain level. Is there a way to get a list of all links for our given domain? I've been playing around with the page-by-page and even that doesn't show me everything. For example, I'm looking at OSE for their homepage and it lists 45 links for a page that it claims has 151 total. Question 2: How did it pick those 45 to display out of the 151 possible? If these are only external links, why do half of them come from one of our subdomains? Also... Question 3: If our client hasn't made any major changes recently, why has the number of internal links gone up so dramatically? Thanks.
Moz Pro | | MackenzieFogelson1 -
Why am I getting duplicate content errors on same page?
In the SEOmoz tools I am getting multiple errors for duplicate page content and duplicate page titles for one section on my site. When I check to see which page has the duplicate title/content the url listed is exactly the same. All sections are set up the same, so any ideas on why I would be getting duplication errors in just this one section and why they would say the errors are on the same page (when I only have one copy uploaded on the server)?
Moz Pro | | CIEEwebTeam0 -
How are our competitors getting these inbound linking domains?
I'm currently managing SEO for my company's website, and I'm getting into link building for the first time. As part of the process, I'm using Open Site Explorer to see who's linking into our competitor sites, to get a better sense of what's available to us in our particular avenue of e-commerce. However, I'm finding that our competitors are getting inbound links from high-authority sites pretty far afield from selling jewelry - census.gov, parallels.com, warnerbros.com, and others. I try clicking through to these links, but each link starts a download of a file. I've seen .f4v, .7z, and .apk files listed as inbound links to our competitor. How is this happening? Again, I'm new to link building, so there may be a simple answer here, and if so I apologize for asking. However, this seems really strange to me, and a difficult situation to confront.
Moz Pro | | jozaksut0