Sites with dynamic content - GWT redirects and deletions
-
We have a site that has extremely dynamic content.
Every day they publish around 15 news flashes, each of which is setup as a distinct page with around 500 words. File structure is bluewidget.com/news/long-news-article-name. No timestamp in URL.
After a year, that's a lot of news flashes. The database was getting inefficient (it's managed by a ColdFusion CMS) so we started automatically physically deleting news flashes from the database, which sped things up.
The problem is that Google Webmaster Tools is detecting the freshly deleted pages and reporting large numbers of 404 pages. There are so many 404s that it's hard to see the non-news 404s, and I understand it would be a negative quality indicator to Google having that many missing pages.
We were toying with setting up redirects, but the volume of redirects would be so large that it would slow the site down again to load a large htaccess file for each page.
Because there isn't a datestamp in the URL we couldn't create a mask in the htaccess file automatically redirecting all bluewidget.com/news/yymm* to bluewidget.com/news
These long tail pages do send traffic, but for speed we only want to keep the last month of news flashes at the most.
What would you do to avoid Google thinking its a poorly maintained site?
-
Get someone to look at the database queries in coldfusion. Unless you have tens of millions of flashes it should be able to handle it on even a reasonably modest server for your traffic levels. It doesn't sound like it should be taxing.
However it sounds like your problem is some badly structured queries. The good news though is that this is probably quicker and easier too fix than upgrading hosting, coding new removal behaviour or any other "work-around"
What would you do to avoid Google thinking its a poorly maintained site?
Sorry to sound glib, but the answer is "maintain it better".
-
Well, to be honest Chris, It is a poorly maintained site. I mean deleting the past news is not at all the solution... I guess the right solution is to work on hosting side and enhance the data base or else the problem will be continuous.
If you are going to use too many redirections this will cause another problem and at the same time will slow down your website speed.
Actually 404s are generating because the URL is available in Google index but there is no page on your website available against it and this causes a 404... The easy way is to reduce 404 is to send a removal request to Google from Google webmaster tool.
And for future... instead of removing pages you should try investing on database and better hosting.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Indexed Site A's Content On Site B, Site C etc
Hi All, I have an issue where the content (pages and images) of Site A (www.ericreynolds.photography) are showing up in Google under different domains Site B (www.fastphonerepair.com), Site C (www.quarryhillvet.com), Site D (www.spacasey.com). I believe this happened because I installed an SSL cert on Site A but didn't have the default SSL domain set on the server. You were able to access Site B and any page from Site A and it would pull up properly. I have since fixed that SSL issue and am now doing a 301 redirect from Sites B, C and D to Site A for anything https since Sites B, C, D are not using an SSL cert. My question is, how can I trigger google to re-index all of the sites to remove the wrong listings in the index. I have a screen shot attached so you can see the issue clearer. I have resubmitted my site map but I'm not seeing much of a change in the index for my site. Any help on what I could do would be great. Thanks
Intermediate & Advanced SEO | | cwscontent
Eric TeVM49b.png qPtXvME.png1 -
Large Site - Complete Site URL Change and How to Preserver Organic Rankings/Traffic
Hello Community, What is your experience with site redesign when it comes to preserving the traffic? If a large enterprise website has to go through a site-wide enhancement (resulting in change of all URLs and partial content), what do you expect from Organic rankings and traffic? I assume we will experience a period that Google needs to "re-orientate" itself with the new site, if so, do you have similar experience and tips on how to minimize the traffic loss? Thanks
Intermediate & Advanced SEO | | b.digi0 -
301 Redirecting from Static to Dynamic URLs. I think we messed up
I'm looking for some guidance on an issue I believe we created for ourselves and if we undo what we did. We recently added attributed search to our sites. This of course created a bunch of dynamically generated URLS. For various reasons, it was decided to take some of our existing static URLs and 301 redirect them to their dyanamic counterpart. Ex .../Empire-Paintball-Masks-0Y.aspx now redirects to .../Paintball-Masks-And-Goggles-0Y.aspx?Manufacturer=Empire Many of these stat URLS had top 3 rankings for their associated keywords. Now, we don't rank for anything. I realize that 301 redirecting is the way to go...if you NEED to. My guess is our drop in keyword ranking is directly tied to what we did. I'm looking for an solid argument to be made to my boss as to why we should not have done this and that it, more than likely has resulted in dropped keyword rankings and organic traffic. I welcome any input. Also, if we decided to revert back (remove all 301 redirects and de-index all dynamic URLS), what is the likely hood we can recapture some of this lost organic traffic? Can I disallow indexing in a robot.txt file to remove, say anything with a '?' in the URL? Would the above URL example (which was ranking in the top 3 in SERPs), have a good chance of finding its way back? thanks
Intermediate & Advanced SEO | | Istoresinc1 -
Will using 301 redirects to reduce duplicate content on a massive scale within a domain hurt the site?
We have a site that is suffering a duplicate content problem. To help resolve this we intend to reduce the amount of landing pages within the site. There are a HUGE amount of pages. We have identified the potential to reduce the pages by half at first by combing the top level directories, as we believe they are semantically similar enough that they no longer warrant being seperated.
Intermediate & Advanced SEO | | Silkstream
For instance: Mobile Phones & Mobile Tablets (Its not mobile devices). We want to remove this directory path and 301 these pages to the others, then rewrite the content to include both phones and tablets on the same landing page. Question: Would a massive amount of 301's (over 100,000) cause any harm to the general health of the website? Would it affect the authority? We are also considering just severing them from the site, leaving them indexed but not crawlable from the site, to try and maintain a smooth transition. We dont want traffic to tank. Has anyone performed anything similar? Id be interested to hear all opinions. Thanks!0 -
Microsite as a stand-alone site under one domain and sub-domained under another: duplicate content penalty?
We developed and maintain a microsite (example: www.coolprograms.org) for a non-profit that lives outside their main domain name (www.nonprofit-mainsite.org) and features content related to a particular offering of theirs. They are utilizing a Google Grant to run AdWords campaigns related to awareness. They currently drive traffic from the AdWords campaigns to both the microsite (www.coolprograms.org) and their main site (www.nonprofit-mainsite.org). Google recently announced a change in their policy regarding what domains a Google Grant recipient can send traffic to via AdWords: https://support.google.com/nonprofits/answer/1657899?hl=en. The ads must all resolve to one root domain name (nonprofit-mainsite.org). If we were to subdomain the microsite (example: coolprograms.nonprofit-mainsite.org) and keep serving the same content via the microsite domain (www.coolprograms.org) is there a risk of being penalized for duplicate content? Are there other things we should be considering?
Intermediate & Advanced SEO | | marketing-iq0 -
Best-of-the-web content in steep competition, ecommerce site
Hello, I'm helping my client write a long, comprehensive, best-of-the-web piece of content. It's a boring ecommerce niche, but on the informational side the top 10 competitors for the most linked to topic are all big players with huge domain authority. There's not a lot of links in the industry, should I try to top all the big industries through better content (somehow), pictures, illustrations, slideshows with audio, and by being more thorough than these very good competitors? Or should I go for something that's less linked to (maybe 1/5 as much people linking to it) but easier? or both? We're on a short timeline of 3 and 1/2 months until we need traffic and our budget is not huge
Intermediate & Advanced SEO | | BobGW1 -
Should I robots block site directories with primarily duplicate content?
Our site, CareerBliss.com, primarily offers unique content in the form of company reviews and exclusive salary information. As a means of driving revenue, we also have a lot of job listings in ouir /jobs/ directory, as well as educational resources (/career-tools/education/) in our. The bulk of this information are feeds, which exist on other websites (duplicate). Does it make sense to go ahead and robots block these portions of our site? My thinking is in doing so, it will help reallocate our site authority helping the /salary/ and /company-reviews/ pages rank higher, and this is where most of the people are finding our site via search anyways. ie. http://www.careerbliss.com/jobs/cisco-systems-jobs-812156/ http://www.careerbliss.com/jobs/jobs-near-you/?l=irvine%2c+ca&landing=true http://www.careerbliss.com/career-tools/education/education-teaching-category-5/
Intermediate & Advanced SEO | | CareerBliss0