Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Thoughts about stub pages - 200 & noindex ok, or 404?
-
With large database/template driven websites it is often possible to get a lot of pages with no content on them.
What are the current thoughts regarding these pages with no content, options;
-
Return a 200 header code with noindex meta tag
-
Return a 404 page & header code
-
Something else?
Thanks
-
-
I would agree with all the comments on how to technically deal with the random pages, but it is a losing battle until you get your website database/templates under control. I once had a similar issue and had to work months to get a solution in place as the website would create all kinds of issues like this.
We had to implement a system so that the creation of these pages would be minimized. I think the issue is that you need to make sure that any random page requests, make sure they get a 404 to start with so that the URL does not get indexed to start with.
That said, all the random URLs that are already indexed, I like the 200 options with the noindex meta tag. My reasons: This is because otherwise with the 404s you get all these error messages that are meaningless in GWT. The noindex also gets the page out of the index. I have seen Google retry 404s on one of our sites, crazy. Ever since Google started showing soft 404s for 301s that redirect many pages to a single URL, I only try to use 301s on more of a one to one basis.
Good luck.
-
Ok, a understand better. I have the same problem with a Site un Drupal, I think is better use a robot.txt to block the empty pages.
These because the link juice that the page transfere is minimum and use extra resources from the server.
If you can't block with robots.txt the noindex,follow meta es ok. But if you see in Analytics that some Landing Pages are www.example.com/product/ {} random_text_here es better use a 404 with redirect 301 to Site Map for user experience.
-
Thanks for the info.
For more information, let me try and explain the scenario a little better.
When using a template to generate all product page on a site, often these are designed in a way so that any URLs of the form "www.example.com/product/{something}" will map to a script called "GenerateProductPage.java" likely based on the rule that anything in the /product/ directory will map there (or .asp etc depending on the language being used).
On the site, there are only going to be links to the actual products that are stored in the DB, so for a user there are no issues there.
But Google manages to find all manor of strange URLs and since they are of the form "www.example.com/product/{random_text_here}" then this also will 'try' and generate a product page. Since there is no actual product in the database called 'random_text_here' then this will result in an empty product page with nothing there except the template navigation, footer links and menus etc.
We currently are doing as you mentioned, by "noindex, follow" the pages for the same reasons you listed.
So the question was; is this ok to do? is this bad to do? (if so why). Is there any harm in doing things the current way? Should we be 404'ig the pages (and what value does this have over the other methods?) etc.
Thanks for your input Carlo as it shows your thoughts are along the same lines as ours.
Has anyone else got anything to add to the information provided?
Thanks
-
Hi, mmm, I not really sure that understand why you have invalid pages, options:
- Products without stock
- Is build based in other database
If you have a product name without content is better a meta noindex, follow because transferred link juice.
But like I say I dont know why these products exist. If you have more info I could help more
-
Thanks for the response.
I guess what I was getting at with the question is when websites are built on flexible platforms and can easily create these pages automatically.
For example, if there was flexible URLs in place whereby URLs such as www.example.com/product/{product_name} all mapped to one script which generated a product page.
So www.example.com/product/{invalid_product_name} would also work and essentially show a blank product page.
The question being, how is the best way to handle these for Google and is there any benefit/harm from either of the methods outlined in the original question.
Has anyone else any thoughts on best ways to handle these scenarios?
Thanks
-
If you know that a Page doesn't have content I recomend:
- A page without content have to response 404.
- If the Page return a 404 make a 301 to Site map.
- In the Site Map use meta noindex, follow to transfer the link juice.
- Eventually you need clean these pages because is bad for users and SEO.
Regards
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Search Console Showing 404 errors for product pages not in sitemap?
We have some products with url changes over the past several months. Google is showing these as having 404 errors even though they are not in sitemap (sitemap shows the correct NEW url). Is this expected? Will these errors eventually go away/stop being monitored by Google?
Technical SEO | | woshea0 -
Robots.txt & meta noindex--site still shows up on Google Search
I have set up my robots.txt like this: User-agent: *
Technical SEO | | RoxBrock
Disallow: / and I have this meta tag in my on a Wordpress site, set up with SEO Yoast name="robots" content="noindex,follow"/> I did "Fetch as Google" on my Google Search Console My website is still showing up in the search results and it says this: "A description for this result is not available because of this site's robots.txt" This site has not shown up for years and now it is ranking above my site that I want to rank for this keyword. How do I get Google to ignore this site? This seems really weird and I'm confused how a site with little content, that has not been updated for years can rank higher than a site that is constantly updated and improved.1 -
Home Page Ranking Instead of Service Pages
Hi everyone! I've noticed that many of our clients have pages addressing specific queries related to specific services on their websites, but that the Home Page is increasingly showing as the "ranking" page. For example, a plastic surgeon we work with has a page specifically talking about his breast augmentation procedure for Miami, FL but instead of THAT page showing in the search results, Google is using his home page. Noticing this across the board. Any insights? Should we still be optimizing these specific service pages? Should I be spending time trying to make sure Google ranks the page specifically addressing that query because it SHOULD perform better? Thanks for the help. Confused SEO :/, Ricky Shockley
Technical SEO | | RickyShockley0 -
Is it good to redirect million of pages on a single page?
My site has 10 lakh approx. genuine urls. But due to some unidentified bugs site has created irrelevant urls 10 million approx. Since we don’t know the origin of these non-relevant links, we want to redirect or remove all these urls. Please suggest is it good to redirect such a high number urls to home page or to throw 404 for these pages. Or any other suggestions to solve this issue.
Technical SEO | | vivekrathore0 -
Noindex user profile
I have a social networking site with user- and company profiles. Some profiles have little to no content. One of the users here at moz suggested noindex-ing these profiles. I am still investigating this issue and have some follow up questions: What is the possible gain of no-indexing uninteresting profiles? Especially interested in this since these profiles do bring in long-tail traffic atm. How "irreversable" is introducing a noindex directive? Would everything "return to normal" if I remove te noindex directive? When determining the treshold for having profiles indexed, how should the following items be weighed Sum of number of words on the page (comprised of one or more of the following: full name, city, 0 to N company names, bio, activity) (unique) Profile picture (Nofollowed) Links to user's profiles on social networks or user's own site. Embedded Google Map Thanks!
Technical SEO | | thomasvanderkleij0 -
Best way to handle pages with iframes that I don't want indexed? Noindex in the header?
I am doing a bit of SEO work for a friend, and the situation is the following: The site is a place to discuss articles on the web. When clicking on a link that has been posted, it sends the user to a URL on the main site that is URL.com/article/view. This page has a large iframe that contains the article itself, and a small bar at the top containing the article with various links to get back to the original site. I'd like to make sure that the comment pages (URL.com/article) are indexed instead of all of the URL.com/article/view pages, which won't really do much for SEO. However, all of these pages are indexed. What would be the best approach to make sure the iframe pages aren't indexed? My intuition is to just have a "noindex" in the header of those pages, and just make sure that the conversation pages themselves are properly linked throughout the site, so that they get indexed properly. Does this seem right? Thanks for the help...
Technical SEO | | jim_shook0 -
Page titles in browser not matching WP page title
I have an issue with a few page titles not matching the title I have In WordPress. I have 2 pages, blog & creative gallery, that show the homepage title, which is causing duplicate title errors. This has been going on for 5 weeks, so its not an a crawl issue. Any ideas what could cause this? To clarify, I have the page title set in WP, and I checked "Disable PSP title format on this page/post:"...but this page is still showing the homepage title. Is there an additional title setting for a page in WP?
Technical SEO | | Branden_S0 -
Should i Noindex my privacy policy page?:
Hi, We have a privacy policy page but it can be found at Copyscape and might affect Google Panda content farming. My questions is, should i Noindex my private policy page?:
Technical SEO | | chanel270