How do i get the crawler going again?
-
The initial crawl only hit one page. Set up another campaign for another site and it crawled 260 pages.
How can I get the crawler started up again or do I really have to wait a week ?
-
It's easiest for the Moz team to respond to individual questions like this via an email to help@seomoz.org or sending in a ticket through the help hub at seomoz.org/help.
-
Hi Keri,
Yeah, I've followed these steps. htaccess forwards to www if none and my campaign is using www.
The robots.txt is available in browser no problem.
my htaccess had massive country ip blocks as well as amazonaws..
I've removed all blocks that arent china or amazon.
Still waiting on a response from the MOZ team here on how to get the real crawl going as what I found in tools is not the same as the weekly scan.
-
Also, for why we only crawled one page, check out https://seomoz.zendesk.com/entries/409821-Why-Isn-t-My-Site-Being-Crawled-You-re-Not-Crawling-All-My-Pages- for some helpful tips.
-
Whenever you have crawl issues, the SEOMoz help team is the place to go. They're the only ones who can look into your account to see if there's an issue or such. (And they are really nice and quick to respond.)
-
Research Tools > Crawl Test at bottom left of page.
Not sure why it's taken me 2 days to find that but it did.
Thanks. Sorry for the dumb question...
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is a new website the only way to get better rankings?
Our website is https://www.franklin-bell.com.au. For a while now we've been trying to get a few of our pages on there to get a better rank on google. A local SEO agency told us that some of our pages are not like "the modern standard" for web pages, but our page score for the keywords we want is 90% and higher. Do we really need to completely rebuild the website (it was built in 2014) in order to get better rankings? Or is it sufficient to have the 97% page score and then just get some back links and social media referrals etc. Thanks in advance.
Moz Pro | | franklin-bell0 -
Competitor getting External Links from search.aol.com
Recently, I noticed that one of the competitors I track within my Moz campaign received about 12 new inbound links. As a result, there DA jumped about 10 points. I reviewed these new external links and was surprised to see that they are all "search.aol.com/aol/search?query= ..." with Link Anchor Text that is good for the industry we compete in. Can anyone tell me why these are being counted as "Inbound Links". It just doesn't seem right. Is this some sort of black hat seo tactic?
Moz Pro | | itvisionsinc0 -
Do we get "Removal of "nofollow" from first custom URL on profile" when we cross 200 Moz Points? I have not received it yet, anything I can do?
Though I have only recently subscribed to Moz Pro, I have been using Moz Blog for quite some time. I recently crossed 200 Moz Points. As per Moz Points, it says "Removal of "nofollow" from first custom URL on profile" for crossing 200 points. I still dont see any links from Moz when I am using OSE. Can anyone suggest what i need to do?
Moz Pro | | vinodh-spintadigital2 -
New Mozscape index released! Learn just what's been going on.
Hi everyone, I'm very, very pleased to announce that the latest Mozscape index has been released five days ahead of schedule, and it's back up to the quality you expect from Moz! It's no secret that we've struggled this year with our index. I can't overstate how much we appreciate everyone sticking with us—we know how hard this has been. The good news—well, apart from the early release—is that we've worked out the issue, and we expect to finally be back on track moving forward. Director of Engineering Martin York has shared a post to the Moz blog explaining just what's been going so wrong, and outlining our plan for the future. Thank you, from the bottom of our hearts, for being such an incredible community. As for the folks who are seeing Domain Authority and Page Authority drops since the index release, please see Rand's recent post for more.
Moz Pro | | MattRoney7 -
I go crazy!
Hi everybody, a short summary at the start: I have my page (www.nlp-deutschland.de) since a long time positioned good, but not perfect in google. The best ressource is a forum wich brings me about 50.000 content-pages, unique and nice. My most important keyword is "NLP". I never needet to look for backlinks. There where only two pages above me: nlp.de, which is the direct key, and wikipedia. Since about one year my page is suffering on google. Mostly it is because of competitors. I personaly think, they buy a lot of backlinks. I startet Seomoz PRO three months ago and I see, that they have about 5 times as much backlinks from other domains / c-blocks as I have. Round about, I got 100 and they got 500. However, in the link analytics I also see that some pages with fewer backlinks are ranked higher. And I get so many different hints about SEO, that I don´t know anymore which one to follow. I am sure about what I am doing with backlinks, but I am not sure about onpage. Here an example:
Moz Pro | | JulianW
All my internal links are relative. As my key is also includet in the domain itself, it get´s no repetition in the url. One of my competitors, which has not the key in his url, just created a subfolder "/nlp/" to all of his pages and has absolute links in his navigation. This brings him the key to every single link in his navigation, while I have no key in the navigation, because it´s in the URL itself and the navigation is relative. I was told that it would make a lot of sense to change the navigation to absolute links, just to get my key into every url of the navigation. Others say, this is bullshit. Another question is double meta-description and meta-tags. With 50.000 content-pages it would be a huge lot of work to get them all done different (by hand). But it is shown as an ERROR in the SeoMoz-Reports. Are they so much important? However, I just realize I have no clue about onpage-optimization. So what would you suggest are the most important factors onpage? All the best, Julian!0 -
Getting your site totally indexed by SEOMOZ
Hi guys! Ijust started using SEOMOZ software and wondered how it could be that my site has over 10.000 pages but in the Pro Dashboard it only indexed about 1500 of them. I've been waiting a few weeks now but the number has been stable ever since. Is there a way to get the whole site indexed by SEOMoz software? Thanks for your answers!
Moz Pro | | ssiebn70 -
Why does SEOMoz crawler ignore robots.txt?
The SEOMoz crawler ignores robots.txt It also "indexes" pages marked as noindex. That means it is filling up the reports with things that don't matter. Is there any way to stop it doing that?
Moz Pro | | loopyal0 -
Getting PA & DA off of a list of links
I have a list of links that I want to get PA and DA for each individual link, can this be done in some way other than one at a time? I've heard this can be done with excel and using api but I don't know the specifics of this.. Help would be appreciated
Moz Pro | | Fergclaw2