Only Crawling 1 page?
-
Hi Guys,
Any advice much appreciated on this!
Recently set up a new campaign on my dashboard with just 5 keywords. The domain is brammer.co.uk and a quick Google site:brammer.co.uk shows a good amount of indexed pages.
However - first seomoz tool crawl has only crawled 1 url!!
"Last Crawl Completed: Apr. 12th, 2011 Next Crawl Starts: Apr. 17th, 2011"
Any ideas what's stopping the tool crawl anymore of the site??
Cheers in advance..
J
-
Agreed. I've passed this to the devs.
You've been most helpful today - Thanks for your time, very much appreciated.
J
-
Well I don't think we can hold this against SEOmoz particularly, if something as basic as Xenu can't crawl it and W3C can't view it's source code I think it's somewhat fair to blame the site. Even if Google can see it I would imagine if the matter of fixed you might see a boost regardless.
Google needs to be able to see sites no matter what their state because you or I can do the same, they have the resources to implement that as well. Smaller operations (everything else) have to make do with figuring it out the old fashioned way through the source code.
I think it's the encoding simply because that is the first port of call on the page and it's broken. If it was anything further down we would at least be seeing some page data cropping up.
The only other thing it could be (because I can't find a robots.txt) is something server side, and that's something it's very difficult to establish without direct access.
-
Interesting. Do you think that could be it? Googlebot seems to find it's way around it though. I'd have thought if G could do it then SEOmoz tools would, otherwise I imagined getting an inaccessibility error or similar from moz.
I'll get that changed and see if it makes a difference..
Thanks again for looking at it!
-
Xenu doesn't like it either, only indexes the one page.
Ran a w3c validation check and it threw up the fact there is no character encoding specified, which may well be the whole of the problem.
If you look at the source code that w3c displays you can see it's essentially an empty document.
-
Any ideas? got a report due on the 19th and the next crawl is due on the 17th. Would be great to remove any blockers before then if poss. thanks!
-
Definately. Cheers.
-
Worth a stab.
Probably worthwhile setting that forwarding up in the meantime anyway.
-
Thanks Tom, but no, it's setup as www.brammer.co.uk so it's not that.
-
Well if you've just put in http://brammer.co.uk it may well be falling over because the domain isn't forwarding to http://www.brammer.co.uk
That's my guess. You just need to forward the domain and it should all be sorted.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Site Crawl Stalled and Can't Restart
In my GreenSeed campaign, the site crawl continues to say "in progress." I can't figure out how to stop it or how to restart the site crawl. Can you please help?
Moz Pro | | Winger1 -
WEbsite cannot be crawled
I have received the following message from MOZ on a few of our websites now Our crawler was not able to access the robots.txt file on your site. This often occurs because of a server error from the robots.txt. Although this may have been caused by a temporary outage, we recommend making sure your robots.txt file is accessible and that your network and server are working correctly. Typically errors like this should be investigated and fixed by the site webmaster. I have spoken with our webmaster and they have advised the below: The Robots.txt file is definitely there on all pages and Google is able to crawl for these files. Moz however is having some difficulty with finding the files when there is a particular redirect in place. For example, the page currently redirects from threecounties.co.uk/ to https://www.threecounties.co.uk/ and when this happens, the Moz crawler cannot find the robots.txt on the first URL and this generates the reports you have been receiving. From what I understand, this is a flaw with the Moz software and not something that we could fix form our end. _Going forward, something we could do is remove these rewrite rules to www., but these are useful redirects and removing them would likely have SEO implications. _ Has anyone else had this issue and is there anything we can do to rectify, or should we leave as is?
Moz Pro | | threecounties0 -
How to find page with the link that returns a 404 error indicated in my crawl diagnostics?
Hi Newbie here - I am trying to understand what to do, step by step, after getting my initial reports back from seomoz. The first is regarding the 404 errors shown as high priority to fix, in crawl diagnostics. I reviewed the support info help on the crawl diagnostics page referring to 404 errors, but still did not understand exactly what I am supposed to do...same with the Q&A section when I searched how to fix 404 errors. I just could not understand exactly what anyone was talking about in relation to my 404 issues. It seems I would want to find the page that had the bad link that sent a visitor to a page not found, and then correct the problem by removing the link, or correcting and re-uploading the page being linked to. I saw some suggestions that seemed to indicate that seomoz itself will not let me find the page where the bad link is and that I would need to use some external program to do this. I would think that if seomoz found the bad page, it would also tell me what page the link(s) to the bad page exists on. A number of suggestions were to use a 301 redirect somehow as the solution, but was not clear when to do this versus, just removing the bad link, or repairing the page the link was pointing to. I think therefore my question is how do I find the links that lead to 404 page not founds, and fix the problem. Thanks Galen
Moz Pro | | Tetruss0 -
How long will it take for Page Rank (or Page Authority) to flow via a 301 redirect?
I've recently redeveloped a static site using WordPress and have created 301 redirects for the original urls to the new urls. I know I won't get all the value passed via the 301, but I'm hoping some will. Any idea how long this may take? It's been nearly a month since the changeover so wondering if it would be weeks, months or more?
Moz Pro | | annomd0 -
"Too many on-page links" warning on all of my Wordpress pages
Hey guys, I've got like 120 "Too many on-page links" warnings in my crawl diagnostics section. They're all the pages of my WordPress blog. Is this an acceptable and expected warning for Wordpress or does something need to be better optimized? Thanks.
Moz Pro | | SheffieldMarketing0 -
How do i increae my page authority
Hi i am still finding my way around semoz and still learning what the tools do and i am trying to find out how i can increase my page authority. My competition page authority is 52 while mine is 36 and i would like to learn how i can increase mine to beat my competition. If anyone could give me some step by step instructions on what tools i should use and how i should use them to increase my page authority many thanks
Moz Pro | | ClaireH-1848860 -
Only 1 page is being crawled by SEOmoz for the last 2 crawls
I would like to ask for the possible problem plus solution on one of our campaigns. Only 1 page is being crawled by SEOmoz for the last 2 crawls. Before the last two crawls, SEOmoz crawls numerous pages and we can’t think of a possible reason for this error. For this particular campaign , there are no data --- no errors, warnings and notices. Thanks!
Moz Pro | | TheNorthernOffice790 -
How to crawl the whole domain?
Hi, I have a website an e-commerce website with more than 4.600 products. I expect that Seomoz scan check all url's. I don't know why this doesn't happens. The Campaign name is Artigos para festa and should scan the whole domain festaexpress.com. But it crels only 100 pages I even tried to create a new campaign named Festa Express - Root Domain to check if it scans but had the same problem it crawled only 199 pages. Hope to have a solution. Thanks,
Moz Pro | | EduardoCoen
Eduardo0