Strange 404s in Screaming Frog
-
I just ran a website (Drupal) through screaming frog and the only 404s I found related to web pages which were the same as URLs already used on the website plus the company phone number so... www.company.com/[their phone number] - www.company.com/services[their phone number] - any ideas what might be causing this problem?
-
Hi Luke,
As the guys above replied with, sounds like an a href with a phone number
If you check the 'inlinks' (via the lower window tab), you'll be able to see the source of these errors (the pages they are located). Obviously you can then view the source code & find the exact link, and what might be the issue.
Hope that helps!
Feel free to pop through any further questions directly to our support btw (http://www.screamingfrog.co.uk/seo-spider/support/), I only spotted this via a Google alert.
(We try and reply super quick & will always look into any problems!)
Cheers.
Dan
-
This is typically caused by a link on the page that is not formed correctly.
-
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How Best to Handle Inherited 404s on Purchased Domain
We purchased a domain from another company and migrated our site over to it very successfully. However, we have one artifact of the original domain in that there was a page that was exploited by other sites on the web. This page allowed you to pass any URL to it and redirect to that URL (e.g. http://example.com/go/to/offsite_link.asp?GoURL=http://badactor.com/explicit_content). This page does not exist on our site so the results always go to a 404 on our site. However, we find that crawlers are still attempting to access these invalid pages. We have disavowed as many of the explicit sites as we can, but still some crawlers come looking for those links. We are considering blocking the redirect page in our robots.txt but we are concerned that the links will remain indexed but uncrawlable. What's the best way to pull these pages from search engines and never have them crawled again? UPDATE: Clarifying that what we're trying to do it get search engines to just never try to get to these pages. We feel the fact they're even wasting their time on getting a 404 is what we're trying to avoid. Is there any reason we shouldn't just block these in our robots.txt?
Intermediate & Advanced SEO | | russell_ms1 -
Very strange HTML docs - what should I do with them through site migration?
I've just been looking at a website and it includes numerous web pages with addresses like this. I click on the URL and it takes me to a fully functional web page (not an image) and when I run it through Screaming Frog this comes up as an HTML page. The site has around 150 unique pages and over 450 pages like this one - how should I deal with these pages during an SEO migration (only a few are backlinked to)? I look forward to reading your thoughts. http://www.[companyname].co.uk/property/caravan-sleeps-4/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/blank.png
Intermediate & Advanced SEO | | McTaggart0 -
Should I redirect 404s or should I eliminate them?
Hello! I am now checking a website that has been migrated months ago from osCommerce to Prestashop.
Intermediate & Advanced SEO | | teconsite
While I was checking crawl errors in search console I found a lot of 404s coming from the last website. The urls are mainly 4 types: popup_image.php?pID=125&osCsid=507c27261ba5ca2568f06ce5bad2ebc9 product-friendly-url-pr-125%3FosCsid.... product-friendly-url-p-125%3FosCsid..... products_new.php?page=228 I've have realized that the parameter pId, and the number that comes after pr- and p- is the product Id in the new website, so I think our team will be able to create an script to redirect those. My question is: Is it ok to send several urls to the same url?. I mean, the popup_image.php was not the product page, as its name says it's more like a popup page. We don't have now a pop up page for images, so I was thinking to send that url to the product page. the one with the pr- was product review page the one with the p- was the product page I was thinking on redirecting the 3 of them to the product page? Should I? Or should I just redirect the last one (p-) and eliminate the others from the index? And... the ones with products_new.php?page=228 I was thinking to redirect all to the page 1 of new products. Is it ok? thank you!0 -
Joomla to Wordpress site migration - thousands of 404s
I recently migrated a site from Joomla to Wordpress. In advance I exported the HTML pages from Joomla using Screaming Frog and did 301 redirects on all those pages. However Webmaster Tools is now telling me (a week after putting the redirects in place) that there are >7k 404s. Many of them aren't HTML pages, just index.php files but I didn't think I would have to export these in my Screaming Frog crawl. We have since done a blanket 301 redirect for anything with index.php in it but Webmaster Tools is still picking them up as 404s. So my question is, what should I have done with Screaming Frog re exporting to ensure I captured all pages to redirect and what should I now do to fix the 404s that Webmaster Tools is picking up?
Intermediate & Advanced SEO | | Bua0 -
Why is Google Webmaster Tools reporting a massive increase in 404s?
Several weeks back, we launched a new website, replacing a legacy system moving it to a new server. With the site transition, webroke some of the old URLs, but it didn't seem to be too much concern. We blocked ones I knew should be blocked in robots.txt, 301 redirected as much duplicate data and used canonical tags as far as I could (which is still an ongoing process), and simply returned 404 for any others that should have never really been there. For the last months, I've been monitoring the 404s Google reports in Web Master Tootls (WMT) and while we had a few hundred due to the gradual removal duplicate data, I wasn't too concerned. I've been generating updated sitemaps for Google multiple times a week with any updated URLs. Then WMT started to report a massive increase in 404s, somewhere around 25,000 404s per day (making it impossible for me to keep up). The sitemap.xml has new URL only but it seems that Google still uses the old sitemap from before the launch. The reported sources of 404s (in WMT) don't exist anylonger. They all are coming from the old site. I attached a screenshot showing the drastic increase in 404s. What could possibly cause this problem? wmt-massive-404s.png
Intermediate & Advanced SEO | | sonetseo0 -
Massive drop in organic rankings - but strange issues
Hi guys, We experienced a massive drop in organic rankings on Saturday 29 October, a drop of around 80%.
Intermediate & Advanced SEO | | cashchampion
I have received no notifications in webmaster tools
The site still ranks for its name so we are not banned
The site has been around since 2007
My mozrank and trustrank are very high (higher than sites that now still rank while we don't)
The strange this is the homepage doesn't rank for the main keyword (which it always did), but now a deeper page ranks for the main keyword, albeit on page 4.
Quite a few of our sites were affected, even though they target different industries, different countries, with different hosts, different unique content, different links (for one we havent even started link building and it was getting 250 UV per day, and now 8). I am completely confused by this.
Any advice or direction would be extremely helpful. Thanks so much0 -
Strange Linking Data in Webmaster Tools
I run a site that was a Wordpress blog with Edirectory software for a directory on the back end. I've scrapped the Edirectory and built the entire site on Wordpress. After the site change I'm seeing about 700 404 Not Found crawling errors, which appear to be old Edirectory pages that no longer exist. My understanding is that they'll cycle out eventually. What troubles me is the linking data I'm seeing. In the "Links to My Site" area of Webmaster tools, I'm seeing 4,430 links to the "About" page, another 2,900 to an obscure deleted directory listing page and only 2,050 to the home page. I show 1,700 links to a terms and conditions pdf and other strange data. To summarize, I'm showing huge numbers of links to obscure pages. Any help would be greatly appreciated.
Intermediate & Advanced SEO | | JSOC0 -
Does Google penalize for having a bunch of Error 404s?
If a site removes thousands of pages in one day, without any redirects, is there reason to think Google will penalize the site for this? I have thousands of subcategory index pages. I've figured out a way to reduce the number, but it won't be easy to put in redirects for the ones I'm deleting. They will just disappear. There's no link juice issue. These pages are only linked internally, and indexed in Google. Nobody else links to them. Does anyone think it would be better to remove the pages gradually over time instead of all at once? Thanks!
Intermediate & Advanced SEO | | Interesting.com0