Why is the SEOmoz crawler crawling the old version of our website?
-
Hello,
I'm a new SEOmoz member.
On Dec. 2nd, after completely redesigning our website, we migrated to a new hosting company by switching our DNS to the new server.
The vast majority of the URLs have changed and we configured redirects of the old URLs to the new ones. Although, this task is not completed yet.
After the migration, I created an account on SEOmoz to be able to track our progress and find the issues to fix to optimize our SEO.
For some reason, in the SEOmoz reports it is the old URLs that show up. Unless the crawler does not actually crawl the pages and only uses the indexed pages to generate its report, I don't understand how could this possible.
Anyone has a clue?
When will the new URLs be indexed by SEOmoz and the major search engines?
Thanks for your help!
-
Thanks for the answers guys.
@Robert: It is indeed the initial report. Yet I did receive other reports since. Also thanks for suggestion the Screaming Frog tool. The report I get from it will be useful in my discussion with our CRM provider.
@Cyrus: I also used URI Valet. I don't understand, the 302 redirects are on picture links. I'll communicate with our CRM provider about that. Also, I don't understand how any of our old URLs could be reached crawling our website as this is a completely new website. No part was transferred over so the shouldn't be any way for the crawler to see our old URLs.
Thanks again
-
The only way that SEOmoz will crawl your old URLs is if that is what your web servers is delivering to the SEOmoz crawlers. In this case, you might want to make sure all your redirect are in place.
I did a quick check of the URL listed in your public profile using the tool URI valet, and found at least 4 redirects from the homepage, including a 302, which passes no link juice. This is definitely something you want to look into.
Generally the SEOmoz robots will crawl your site once a week, so 99% of the time you will see new infomation contained in your campaign within 7-10 day. If you suspect an error, be sure to contact the help team (help@seomoz.org)
Also, here's my favorite article on site migration. Although you didn't migrate a complete site, many of the principals still apply.
https://seogadget.co.uk/surviving-seo-site-migration/
Best of luck with your SEO!
-
Gestisoft-QC
First, SEOmoz does crawl the site it is not using indexed pages. I am assuming you set this up as a campaign and got the initial report and saw the old urls? If you are looking for links, OSE is different and does take roughly 3 to 7 weeks to show due to the amount of work involved (they are crawling the web as opposed to a site for this).
I ran your site through Screaming Frog and see the 301's, etc. You might try using Screaming Frog to see the Header Responses, etc. http://www.screamingfrog.co.uk/seo-spider/ Screaming Frog has a free download for windows or Mac and that should be sufficient to see what URLs are showing, etc. It will not give you the other info you get from moz.
I cannot see where the moz would be different, but this should at least give you a good cross reference.
If you don't get resolution there, here is the link to helpful info on moz Pro: http://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-only-crawled-one-page
You can also ask for help via help@SEOmoz.org. They are very responsive.
Hope this helps
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Meta no index crawler warnings
I've decided that the duplicate content issues on my site weren't worth the effort from the amount of traffic the archive pages on my WordPress site received no I decided to no-index them using Yoast. Now I have 60 meta no-index crawler warnings. Should I just ignore these? It seems I get warnings, either way, I use the site. Does anyone have advice on how to move on with this?
Moz Pro | | Libra_Photographic0 -
Since July 1, we've had a HUGE jump in errors on our weekly crawl. We don't think anything has changed on our website. Has MOZ changed something that would account for a large leap in duplicate content and duplicate title errors?
Our error report went from 1,900 to 18,000 in one swoop, starting right around the first of July. The errors are duplicate content and duplicate title, as if it does not see our 301 redirects. Any insights?
Moz Pro | | KristyFord0 -
Website not being indexed by Google - does seomoz have a index checker?
i have been to a couple of index checking websites that are saying my website is not being indexed, my developer and seo team have been making a lot of changes recently and I think it is because of that. Also my first byte grade is D. Can I check here on seomoz if my site is being indexed by Google here in the UK?
Moz Pro | | Malx0 -
No follow links also been reported in SEOmoz crawl diagnostics
Hi, Why does SEOmoz reports links which has been marked as 'nofollow'. I am getting 'Overly-Dynamic URL' reports on links which I have designated as nofollow which means Google will discount them. So why does SEOmoz still report them. Thanks.
Moz Pro | | malpani0 -
FF SEOMoz toolbar issue
I sent the email below to help@seomoz.org on the 12th April and haven't heard anything back so thought I'd fire it up here in case someone has a solution for me... "For the past month or so I've had an issue in Firefox where my SEOmoz toolbar appears to not sign me in, or allow me to sign in. All metrics are 0 for all websites/pages and if I click the login button, nothing happens at all. If I click the Quick Links button the drop down menu appears but whichever option from the drop down I choose, nothing happens. I'm funning Firefox 11.0 and have attached a screen grab of the bar as it appears no matter where I browse.
Moz Pro | | DWJames
Do you have any suggestions how I can get it working again? It works fine in Chrome but I primarily use Firefox..." UNRMw.jpg0 -
How do you use SEOmoz
I'm a newbie here. I've got a week left on my free trail. I really like SEOmoz, I might even love it a bit. 🙂 I'm trying to justify the cost. I usually don't spend money on my Internet Marketing. I've done it through finally buying some tools to help me get there. Massive change in the last year. I'm a long ways from getting rich YET though. 🙂 I've gone from $30 a month to $1000 a month in a year. So I think I'm on my way now. I'm trying to figure out why I should continue my subscription and actually start paying for it. If I were a developer I can certainly see why but the cost would so go way up. 🙂 I'm in the lowest level package with only five websites. I have 10 I'm working on. I know that's probably five to many at one time. I'm having a little trouble justifying the cost at this time. At my income level $100 a month is significant. I can see the value of seeing where my competition has backlinks. Sometimes my competition is Cnet. haha Well, I can beat them on a specific page at times. Hmm, can I search the URL list for links? Perhaps i can if I export it. That didn't occur to me. Now that could be of use if I can find my keywords. If their link is on Amazon though, I'm going to have trouble getting a link there. 🙂 Maybe I should work on less competitive links but I made good money last month with one page that was beating them. WAS beating them. How do you use SEOmoz? How do you justify the cost, that is offset the expense. If its not making me money i don't see the value. I can get a lot of this information for free but it sure is easier to get it here. Which saves me time. And the other tool i use isn't that good and not always reliable and slow. I've got a week left on my free trail. I had decided to let it go but now I'm reconsidering. $100 is a lot for me and I need to spend more on SEO outsourcing. I wonder if I wouldn't be better off dropping it for a while and then coming back. This certainly isn't mean to be a critical post at all. I'm looking for reasons to stay and make better use of it. Thanks for the thoughts, Rusty
Moz Pro | | RustyF0 -
An error in the SeoMoz On page note?
Hello folks, Whenever I go the OnPage link in SeoMoz some of my links show a F ranking note. And when I click in one of them to see the detail of the page rank, it shows me as an A ranking note. Do you have seen the same problem? Which note shall I rely on? Thanks!!
Moz Pro | | jgomes0 -
How often does SEOmoz reports get refreshed?
Is there a way I can refresh the reports manually instead of waiting for it to pull the updated data?
Moz Pro | | RBA0