Seo moz has only crawled 2 pages of my site. Ive been notified of a 403 error and need an answer as to why my pages are not being crawled?
-
SEO Moz has only crawled 2 pages of my clients site. I have noticed the following.
- A 403 error message
- screaming frog also cannot crawl the site but IIS can.
- Due to the lack of crawling ability, im getting no feed back on my on page optimization rankings or crawl diagnostics summary, so my competitive analysis and optimization is suffering
Anybody have any idea as to what needs to be done to rectify this issue as access to the coding or cms platform is out of my hands.
Thank you
-
Hi Joel,
I have indeed!
I'll take all this information to the development team and hopefully have this issue resolved asap.
Thanks once again to everybody for their input.
Regards,
Andy
-
Hey Andy,
You've gotten a lot of good info here and in your email with Maura. I just wanted to add in here that the crawl trouble you're having (at least with our system) is with the http://www.inoncology.com/ home page itself.
You've got a 301 redirect from the root to the www subdomain, the www then returned a 403 forbidden to our crawler.
It may have something to do with the verification you require, it may be something more obvious on the server level. It's tough to say.
Either way, you'll want to get this addressed to make sure your site is crawlable.
I hope that helps.
Cheers,
Joel. -
Hi Lynn
Thank you for taking a look into this issue, your feedback has given me food for thought.
I am going to forward on your feedback to the powers that be on the web development side of things and see if this is an answer to the problem.
I've just looked at http://www.inoncology.com/home/ and from first review, it seems that this is where the crawlers are breaking down, as it comes to a dead end.
Hopefully this is the reason.
Thank you once again
-
Hi Andy,
Not sure why you are seeing a 403 error (I don't see this in the headers), but you seem to have an issue with your rel=canonical tag on your homepage. It is showing a relative link to "/home" which leads you to this page: http://www.inoncology.com/home/ which is actually showing a 200 OK status, but is obviously not!
Digging into the headers a bit more it seems you might have a circular redirect going on before that, I am seeing /home redirect to /content/internet/pm/inoncology/com_EN which seems to redirect back to /home/.
No 403 in all of those, but maybe the combination of wrong rel canonical tag plus the mixed up redirects is causing the moz crawler and screaming frog to choke?
-
Not at all Jesse. Thank you for taking a look
Brief summary - the pharmaceutical industry has to have a disclaimer on their pages for legal reasons. Forcing a selection/disclaimer is required by law. If you select the first option, it'll take you to the web address intended.
In terms of it being related to the disclaimer, SEO Moz was indexing it with this disclaimer up until a few months ago. There's no robot txt file as you stated, so that also cannot be the reason for this.
hhhhmmm........the mystery continues!!!
-
no robots file so it's not that. I do notice that your landing page has an annoying popup that forces a decision and then redirects you upon choosing it... I'm not 100% sure but that might have something to do with it. I can't seem to get past that and onto the site itself honestly. Everytime I click a selection I get to another page with a popup.
If I were an average internet browser *(which in this instance I am) I would leave your site and not return. Poor user experience hurts ultimately, and that was what I experienced.Not trying to be rude, just trying to be helpful. Sorry if I came off cross.
G'luck!
-
the url is http://www.inoncology.com/
-
Please share the URL of your website so we can check.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved How does Moz compile the "Important pages on your site are returning a 4xx error!" report?
I have over 200 links in this report (mostly from a staging site). I have deleted that staging site and I cannot find the reference to the other links. So my question is, where is Moz finding these links?
Moz Pro | | nomad_blogger0 -
Moz & Xenu Link Sleuth unable to crawl a website (403 error)
It could be that I am missing something really obvious however we are getting the following error when we try to use the Moz tool on a client website. (I have read through a few posts on 403 errors but none that appear to be the same problem as this) Moz Result Title 403 : Error Meta Description 403 Forbidden Meta Robots_Not present/empty_ Meta Refresh_Not present/empty_ Xenu Link Sleuth Result Broken links, ordered by link: error code: 403 (forbidden request), linked from page(s): Thanks in advance!
Moz Pro | | ZaddleMarketing0 -
How to fix the Crawl Diagnostics error and warnings
hi im new to the seo world and i dont know a lot about it , so after my site get crawled i found 1 error and 151 warning and 96 notices , it that bad ?? and plz cam someone explain to me how to fix thos problem , a will be very thankful
Moz Pro | | medlife0 -
Only One page crawled..Need help
I have run a website in Seomoz which have many URLs with it. But when I saw the seomoz report that showing Pages Crawled: 1. Why this is happen my campaign limit is OK Tell me what to do for all page crawling in seomoz report. wV6fMWx
Moz Pro | | lucidsoftech0 -
Crawled pages are missing and showing just 1 page crawled
One of my campaign has got around 8500 pages crawled(seomoz) and reports are shown, but suddenly it is showing 1 page crawled. Why it is happened like this? How can i get back the previous reports?
Moz Pro | | Sulekha0 -
Conflicting Data with Open Site Explorer & SEO Moz Toolbar
Hi there, Why is my SEOmoz toolbar showing me higher linking root domains than open site explorer? For example, for this site: http://thisisouryear.com/, the linking root domains in SEOMOZ show 1,015 and in open site explorer it says it has 35. I have noticed this for other sites as well. Which one is right? And why are they different?
Moz Pro | | qlkasdjfw1 -
Crawling One Page
I set up a profile for a site with many pages, opting for setting up as a root directory. When SEOMoz crawled, they only found one page. Any ideas for why this would be? Thanks!
Moz Pro | | Group160 -
Why are these pages considered duplicate page content?
A recent crawl diagnostic for a client's website had several new duplicate page content errors. The problem is, I'm not sure where the error comes from since the content in the webpage is different from one another. Here's the pages that SEOMOZ reported to have duplicate page content errors: http://www.imaginet.com.ph/wireless-internet-service-providers-term http://www.imaginet.com.ph/antivirus-term http://www.imaginet.com.ph/berkeley-internet-name-domain http://www.imaginet.com.ph/customer-premises-equipment-term The only thing similar that I see is the headline which says "Glossary Terms Used in this Site" - I hope that the one sentence is the reason for the error. Any input is appreciated as I want to find out the best solution for my client's website errors. Thanks!
Moz Pro | | TheNorthernOffice790