Screaming frog, Xenu, Moz giving wrong results
-
Hello guys and gals,
This is a very odd one, I've a client's website and most of the crawlers I'm using are giving me weird/ wrong results.
For now lets focus on screaming frog, when I crawl the site it will list e.g. meta titles as missing (not all of them though), however going into the site the title is not missing, and Google seems to be indexing the site fine. The robots.txt are not affecting the site (I've also tried changing the user agent).
The other odd thing is SF gives a 200 code but as a status tells me "connection refused" even though it's giving me data.
I'm unable to share the clients site, has any one else seen this very odd issue? And solutions for it?
Many thanks in advanced for any help,
-
Thanks Dan!
-
^ That's customer service!
-
Cheers for the tweet Andy.
Chris, I'd recommend running through this list here in our FAQ -
http://www.screamingfrog.co.uk/seo-spider/faq/#17
In particular, I'd recommend reducing the crawl speed & adjusting the user-agent.
You can also pop it through to our support (http://www.screamingfrog.co.uk/seo-spider/support/) & we can advise. All confidential etc.
Cheers
Dan
-
I have shared this question with Dan from Screaming Frog for you
-Andy
-
Hi Chris
This is hard to answer without an example - do you have a URL or two that you can share with the community?
Thanks so much!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Does Moz Pro include Moz Local
My client has bought about six Moz Local accounts and are pleased with results. We have not yet used your Moz Pro program. The client might be interested in switching to the Moz Pro if those Moz Local accounts can be included into it. Please let me know as soon as possible. Thanks!
Moz Pro | | gallowaywebteam0 -
Moz Pro Tools
Hello I ran into a error while using Moz Pro Tools Crawl Site feature. Stating that my wordpress website could not be crawled. When contacting moz they sent me this screenshot stating the reason for this error is because of the odd ip address highlighted in yellow. Only time I've seen this particular ip-address is during local development. If anyone has any advice on how to fix this or what may have caused this issue. I feel this maybe effecting the site's overall search visibility. ednqnL7
Moz Pro | | willakawillow220 -
My moz campaing not update completely
I have create new account and campaign last week, till now my moz campaign not update like: keywords ranking, competitor websites links etc. Why it is so please update me.
Moz Pro | | Surabhi_Dewra0 -
We need your expertise! Do you have a tool hack or trick you use with Moz Pro?
Hey friends! We're looking for any tricks and tips you have for other people using Pro. Everyone has their own way of doing things. We'd love to see if you have any hacks you've created. You can send over just a screenshot, or a blog post, or anything you've got! Here are a couple examples: [blog post] Did you know you can use OSE's anchor text export to see if you are getting setup for a penguin penalty? This is a blog post about it: http://moz.com/academy/google-penguin-and-you [screenshot] Did you know you can get all your keyword ranking historical data by clicking this one little button? http://i.imgur.com/l4KxqBt.jpg Do you have anything you'd like to add? We're hoping to help spread these to others to help people utilize the tools to their potential. Thanks!
Moz Pro | | jennita2 -
Internal and external links showing in Webmaster Tools but not in MOZ
Hi there I set up a campaign 2 weeks ago but I am registering zero internal or external links for my website. It's a small site with only about 300 external links but they show in Webmaster tools but not in my link report or Open Site Explorer. Do I need to wait until the next OSE crawl or is something wrong with my set up? Thanks in advance! Laurie
Moz Pro | | turnbullholdingsltd0 -
Is there any way to get Moz to generate a regular (e.g. weekly) report of new links to my domain, automatically?
I want to know when new people link to my domain. I do not want to log in once a week and use a complex interface to figure this out, I want it in my inbox every week. Is there a way to get Moz to do this? (I am a PRO user) Cheers, Daniel
Moz Pro | | swombat0 -
Stats from Moz bar and OSE not changed over last three months - new update?
I have noticed for a few of my websites that the SEOMoz stats have not changed over the last three months; PR, MR and even total external links. Is both the tool bar and OSE going through further updates and not yet pulling through accurate results? Any advice on this would be much appreciated.
Moz Pro | | Adido-1053990 -
Why would opensite explorer show a link from baidu result page with baidu as a linking root domain?
I found an old domain that has very high domain authority and one of its top linking root domains is baidu I clicked the link in ose and it took me to a baidu serp????? please explain...im happy to clarify if need be or give you the site in question just let me know what you think. I checked ose for the page authority of the serp page it was 50. would that page be helping the site in question to rank? or am I just dumb for asking this question One more thing I couldnt actually find a link to the page I was looking at in ose on the baidu result page.. thanks
Moz Pro | | duncan2740