Soft 404s for unpublished & 301'd content
-
Hi,
One site I work with unpublished a lot of thin content. Great idea, right?
These unpublished pages were then 301'd up to the main category page that they previously existed in.
Now Google Webmaster Tools calls them out as soft 404 errors. This seems unexpected since the pages
were 301'd. Here is my question; Is this a serious problem that may affect the site's overall organic results
and if so what should I do about it?
Thanks... Darcy
-
Short answer: create a custom 404 page, not just for these pages, but one that can show for everypage on your site.
A few resources:
https://support.google.com/webmasters/answer/93641?hl=en
Example: http://moz.com/sadfklfadsadfjs
-
Cyrus, thanks for hanging in there with my questions. If I just give back a 404, what am I showing them on the page?
I would think seeing the main questions page would be better than just sitting at the original url and looking at 404 page notice - seems like a bad user experience if Google wants to get all user-experiency about it.
Thanks... Darcy
-
Yes, it's possible, but that could be considered cloaking. I'd say best to return a 404.
-
Hi Cyrus,
Have not experienced a dip, but things have been a little static.
Can you do both... forward the page and give back a 404?
What would you do?
Thanks... Darcy
-
Yes, I would think that at the point Google crawls it and finds it forwarded it would drop it from the index and not waste resources crawling it again unless linked somewhere. I will keep an eye out for links, but don't believe that there are any.
Thanks, Dirk... Darcy
-
In that case, sounds like you should either:
- 404 them if you have evidence these have hurt your rankings/traffic (have you experienced a dip?)
- Ignore them and go about your day
-
Hi Cyrus,
Thanks for the info. These are forum pages where no one ever answered the question, so
there is no helpful info and very little content.
They were forwarded up to the main questions page (one / up the url structure).
The page they were forwarded to is like a questions category page, not specific to the subject of the
forwarded page. These forwarded pages don't get much/any traffic because they never ranked
and we didn't promote them.
If it doesn't hurt overall search on other pages, I'd rather not go to the substantial effort of finding subject-relevant pages to forward to, since no one will ever go to the original url and need to see something super relevant.
Your thoughts? Thanks! Best... Darcy
-
If Fetch like Google is also giving a 301 - I would mark them as solved in WMT & check if they re-appear.
If you click on the i next to the redirect message in Fetch like Google - it shows the type of redirect & the page it's redirecting to. I assume you checked that this is also a 301.I have a similar issue on one of my sites - if a user gets to a non-existing url - the server first tries to find out if the page exists - if it doesn't it's redirected to a 404 page. Although technically it is a 301 - WMT sees them as a soft 404 as the destination page is a "Page not found" type of page (called 404.php) - which (quite ironically) renders a 200 status.
On the destination page - do you mention somewhere a message like "page not found" or is it just a plain category page?
The SEO impact is difficult to assess - Google says these pages are mainly wasting the bot's time as it's indexing pages that do no longer exist, not sure if it is also affecting rankings. As you did the crawl with Screaming Frog, I guess you are also removing all internal links to these redirected pages? If these links disappear, and as the content was thin, I suspect you don't have many external links pointing to them, so the problem should disappear after a while.
rgds,
Dirk
-
If Google thinks the 301 leads to a page that isn't relevant enough, they may flag it as a "soft 404" even though it returns a 301. That's Google's way of saying they think you should 404 these pages instead.
How much will it hurt you? Probably not much, but it's hard to say.
Let's ask these questions:
- How much traffic goes to these pages? If not much, is it okay to 404 them?
- Are there more relevant pages you could redirect these to? (ideally, something with a similar title as the original page?)
- Have you seen much traffic loss overall? If not, it's likely this isn't hurting you.
Hope this helps! Best of luck with your SEO.
-
Okay, that is extra weird. It could be that GWT hasn't update your information since you made the changes. Since everywhere else is telling it's correct -- especially the fetch tool -- then you should wait a few more days and see if it updates.
-
Hi Erica,
I'm saying that the only place it shows a soft 404 is in GWT errors. Screaming Frog, web-sniffer and now Fetch As Google In GWT, all show them as 301 re-directs. I can't re-direct them more than they are. So, is GWT just goofy?
Thanks... Darcy
-
Hi Darcy,
Yeah, if it's still showing as a soft 404, there's still something wrong. I'd try using fetch and render as Google bot and see what happens.
Best of luck!
-
Hi Dirk,
Thanks for the suggestion. As noted above, I put the whole list thru screaming frog and a few thru your suggestion of web-sniffer.net.
95% of the whole list is 301s and 100% of the few put one at a time thru web-sniffer come back as 301s.
My question remains "Is this a serious problem that may affect the site's overall organic results
and if so what should I do about it?"
Thanks... Darcy
-
Hi Erica,
I put the list through screaming frog and 95% of the urls are shown as 301s.
Do you think screaming frog has it right or is there something they wouldn't catch?
Thanks... Darcy
-
Maybe an obvious question but did you check that the url's are indeed properly redirected - checking them with 'Fetch like Google' in WMT or by using a tool like web-sniffer.net?
rgds,
Dirk
-
I'd check to make sure your 301s were done correctly. If they are showing up as soft 404s, they are probably implemented wrong.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Ranking for keyword I don't optimize for & Other oddities
Hi Moz Community! I've been working with a clients website for about a year now. They were hit with the original Panda update because of some spammy links from a shady SEO firm. We've made a decent climb back but not a full recovery. There are some weird things happening that I would love some insight into. 1. Ranking for keywords we don't optimize for: I noticed some low keyword volume for a keyword term that is close to our main term, but is slightly different. We don't optimize for this term at all on our website. We rank third for this term, and actually show site links in the result, which doesn't happen for any of our other pages. 2. Index not found when doing site: search: Other oddity is that when you search site:www.mywebsite.com, I see all the pages within the site except the homepage. Not sure whats going on here, but when I fetch the homepage in GWMT, it returns the homepage. When you query the homepage by itself, it also ranks. Any help would be appreciated! Regards, J
Intermediate & Advanced SEO | | artscienceweb0 -
Hreflang targeted website using the root directory's description & title
Hi there, Recently I applied the href lang tags like so: Unfortunately, the Australian site uses the same description and title as the US site (which was the root directory initially), am i doing something wrong? Would appreciate any response, thanks!
Intermediate & Advanced SEO | | oliverkuchies0 -
Duplicate content when changing a site's URL due to algorithm penalty
Greetings A client was hit by penguin 2.1, my guess is that this was due to linkbuilding using directories. Google webmaster tools has detected about 117 links to the site and they are all from directories. Furthermore, the anchor texts are a bit too "perfect" to be natural, so I guess this two factors have earned the client's site an algorithm penalty (no manual penalty warning has been received in GWT). I have started to clean some of the backlinks, on Oct the 11th. Some of the webmasters I asked complied with my request to eliminate backlinks, some didn´t, I disavowed the links from the later. I saw some improvements on mid october for the most important KW (see graph) but ever since then the rankings have been falling steadily. I'm thinking about giving up on the domain name and just migrating the site to a new URL. So FINALLY MY QUESTION IS: if I migrate this 6-page site to a new URL, should I change the content completely ? I mean, if I just copy paste the content of the curent site into a new URL I will incur in dpolicate content, correct?. Is there some of the content I can copy ? or should I just start from scratch? Cheers hRggeNE
Intermediate & Advanced SEO | | Masoko-T0 -
Can I dissavow links on a 301'd website?
So we are performing link removal for a client on his old website (A), which is being 301 redirected to his new website (B). We have identified toxic links on site A and are removing, once complete we will undo the current 301, confirm a new GWT account for website A, and then submit the disavow report. We would then like to reapply the 301 redirect to site B while we are waiting for Google to process the disavow report, the logic being we can retain some current rankings on site B while waiting for the disavow to process on site A. Has anyone had experience with this method? I foresee some potential issues here but am interested to here from others on this. Thanks!
Intermediate & Advanced SEO | | SEOdub1 -
Does link equity to a page that is 301'd to a new domain pass juice on?
If we build some quality inbound links to certain pages, that are a later date 301'd to another domain, does any equity or juice get transferred across? Or is the inbound link's value wasted? Thanks
Intermediate & Advanced SEO | | bjs20100 -
Removing Content 301 vs 410 question
Hello, I was hoping to get the SEOmoz community’s advice on how to remove content most effectively from a large website. I just read a very thought-provoking thread in which Dr. Pete and Kerry22 answered a question about how to cut content in order to recover from Panda. (http://www.seomoz.org/q/panda-recovery-what-is-the-best-way-to-shrink-your-index-and-make-google-aware). Kerry22 mentioned a process in which 410s would be totally visible to googlebot so that it would easily recognize the removal of content. The conversation implied that it is not just important to remove the content, but also to give google the ability to recrawl that content to indeed confirm the content was removed (as opposed to just recrawling the site and not finding the content anywhere). This really made lots of sense to me and also struck a personal chord… Our website was hit by a later Panda refresh back in March 2012, and ever since then we have been aggressive about cutting content and doing what we can to improve user experience. When we cut pages, though, we used a different approach, doing all of the below steps:
Intermediate & Advanced SEO | | Eric_R
1. We cut the pages
2. We set up permanent 301 redirects for all of them immediately.
3. And at the same time, we would always remove from our site all links pointing to these pages (to make sure users didn’t stumble upon the removed pages. When we cut the content pages, we would either delete them or unpublish them, causing them to 404 or 401, but this is probably a moot point since we gave them 301 redirects every time anyway. We thought we could signal to Google that we removed the content while avoiding generating lots of errors that way… I see that this is basically the exact opposite of Dr. Pete's advice and opposite what Kerry22 used in order to get a recovery, and meanwhile here we are still trying to help our site recover. We've been feeling that our site should no longer be under the shadow of Panda. So here is what I'm wondering, and I'd be very appreciative of advice or answers for the following questions: 1. Is it possible that Google still thinks we have this content on our site, and we continue to suffer from Panda because of this?
Could there be a residual taint caused by the way we removed it, or is it all water under the bridge at this point because Google would have figured out we removed it (albeit not in a preferred way)? 2. If there’s a possibility our former cutting process has caused lasting issues and affected how Google sees us, what can we do now (if anything) to correct the damage we did? Thank you in advance for your help,
Eric1 -
No matter what I do, my website isn't showing up in search results. What's happening?
I've checked for meta-robots, all SEO tags are fixed, reindexed with google-- basically everything and it's not showing up. According to SEOMoz all looks fine, I am making a few fixes, but nothing terribly major. It's a new website, and i know it takes a while, but there is no movement here in a month. Any insights here?
Intermediate & Advanced SEO | | Wabash0 -
Hit by Penguin, Can I move the content from the old site to a new domain and start again with the same content which is high quality
I need some advice please. My website got the unnatural links detected message and was hit by penguin.. hard. Can I move the content from the current domain to a new domain and start again or does the content need to be redone also. I will obviously turn of the old domain once its moved. The other option is to try and identify the bad links and change my anchor profile which is a hit and miss task in my opinion. Would it not be easier just to identify the good links pointing to the old domain and get those changed to point to the new domain with better anchors. thanks Warren
Intermediate & Advanced SEO | | warren0071