How to use Crawl Report for a test directory
-
I have a client's new site set up in a folder that is not linked to from the main site. When setting up the Crawl Report, I put in the starting url for the new folder, http://oldsite.com/new/start.php. The Crawl Report came back with a crawl of the current site instead. How do folks run the crawl report to test sites before they are public? Thanks!
-
Mary,
Thanks for reaching out to us! That is a good question! Our crawler can only crawl sites that are indexable and that are public, so if you set the report to run against the starting URL for the new folder, it would try to find that folder if it is public. One thing you could try out is to set your site public right before a crawl date, then immediately close access to it, this is the only way I can think Roger would be able to do it
I am sorry that I can't offer something better, I hope that helps!
Best,
Peter
Moz Help Team.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Spammy Directories Using Title Meta-Tag For Link
I've been noticing a growing amount of spammy directories using my title Meta-Tag to create a link back to my site. Since my title uses competitive keywords, these external links on spammy directories look over optimized and I fear an algo penalty might get triggered. Should I be concerned and what can I do?
Moz Pro | | alrockn0 -
I have a new website and would like to use the keywords my competitors are using and was wondering the best way to take a look using Moz Pro?
The website is www.fapwrap.com - it's a new adult wall decals website. [editor's note: may not be safe for some workplaces] The wall decals we have are obviously a bit more niche than the bigger competitors, but I want to get into the first page to see how it helps us. I'm slowly taking over some kws like "xbox wraps" and "ps4 wraps" but those have low search volume and I feel like "wall decals" will help - though it's highly competitive. Suggestions? I just kinda want to plug in my competition and pick from their list of existing kws..... Ha.
Moz Pro | | JohnnyRoq0 -
MOZ Starter Crawl Not Working
Hello, I just added a new subdomain as one of my campaigns on MOZ. The starter crawl report keeps coming back to me with just one page crawled (it should crawl up to 250 pages). I've deleted and added this subdomain three times and it continues to present me with this problem.I've even waited a week for the full crawl report but that also showed just one page crawled. Does anybody know why this is happening? Thanks!
Moz Pro | | jampaper0 -
Occurrences of Keyword - Report card
Hi WHen I run the report card for a particular keyword - URL its says "Occurrences of Keyword 0" however I there is the keywords in the page at least twice and exact match of keywords x1 ? any help would be appricated. For some reason google is just not indexing any keyword that I have optimized page for either. I have checked and they are all "follow" thanks
Moz Pro | | summer3000 -
Where is the crawl test tool located in this new site?
Hi there, Where is the crawl test tool located in this new moz site? Formerly it was, http://pro.seomoz.org/tools/crawl-test Hope you can help. Thanks:)
Moz Pro | | steveovens1 -
Using Seomoz for Site Evaluation am I up to par ?
Just wanted to see how people using the seomoz bar would rate a four month old site with Domain-Homepage Authority of 27 Mozrank of 5.08 and Moztrust of 5.65 . I've read up on all the factors but just wanted to know if Im up to par on building a great site thats search engine friendly. Inner pagers are on a PA of 20 and around the same mozrank and moztrust levels of +- 5.
Moz Pro | | NikolasNikolaou0 -
Competitor using Black hat SEO
I think my competitor is using Black Hat SEO. They are a training institute but when I use the PRO tool to check their inbound links, many are links to porn sites. Is this illegal? Can i do something about it?
Moz Pro | | Jigsaw1230 -
SEOMoz only crawling 5 pages of my website
Hello, I've added a new website to my SEOmoz campaign tool. It only crawls 5 pages of the site. I know the site has way more pages then this and also has a blog. Google shows at least 1000 results indexed. Am I doing something wrong? Could it be that the site is preventing a proper crawl? Thanks Bill
Moz Pro | | wparlaman0