Most Painless way of getting Duff Pages out of SE's Index
-
Hi,
I've had a few issues that have been caused by our developers on our website. Basically we have a pretty complex method of automatically generating URL's and web pages on our website, and they have stuffed up the URL's at some point and managed to get 10's of thousands of duff URL's and pages indexed by the search engines.
I've now got to get these pages out of the SE's indexes as painlessly as possible as I think they are causing a Panda penalty.
All these URL's have an addition directory level in them called "home" which should not be there, so I have:
www.mysite.com/home/page123 instead of the correct URL www.mysite.com/page123
All these are totally duff URL's with no links going to them, so I'm gaining nothing by 301 redirects, so I was wondering if there was a more painless less risky way of getting them all out the indexes (IE after the stuff up by our developers in the first place I'm wary of letting them loose on 301 redirects incase they cause another issue!)
Thanks
-
If all /home directories should be removed, and they all correlate directly to the same URL without the /home then the best means to correct the issue would be a single regex expression to redirect the URLs.
Once you add the redirect Google's index should be cleaned up in about 30 days.
-
Add robots.txt to stop Google crawling the subdirectories you don't want shown such as /home
More information is available on http://www.seomoz.org/learn-seo/robotstxt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Specific page does not index
Hi, First question: Working on the indexation of all pages for a specific client, there's one page that refuses to index. Google Search console says there's a robots.txt file, but I can't seem to find any tracks of that in the backend, nor in the code itself. Could someone reach out to me and tell me why this is happening? The page: https://www.brody.be/nl/assistentiewoningen/ Second question: Google is showing another meta description than the one our client gave in in Yoast Premium snippet. Could it be there's another plugin overwriting this description? Or do we have to wait for it to change after a specific period of time? Hope you guys can help
Intermediate & Advanced SEO | | conversal0 -
Why is my website not ranking for it's brand name in SERPs but has been indexed by Google?
The website https://christchurch.crowneplaza.com has been live for a couple of months but is not being found in Google search results - even when searching for it's own brand name 'crowne plaza christchurch.' Google has indexed the site - but we are still not showing - https://www.google.co.nz/search?q=site%3Ahttp%3A%2F%2Fchristchurch.crowneplaza.com&rlz=1C1NHXL_enNZ735NZ735&oq=site%3A&aqs=chrome.0.69i59j69i57j69i58j69i59l2j69i65.896j0j7&sourceid=chrome&ie=UTF-8 Any ideas as to why? I think it may be because their are two versions of the site, http and https, both with their own rel=canonical tags. Could this be the cause? Any help much appreciated.
Intermediate & Advanced SEO | | Timmy30 -
What's more valuable, a Blog or a Forum, and how to integrate?
We want to start a blog or forum (maybe eventually both) and are unsure what is the best way to publish it from an SEO standpoint. If the blog is published on our domain, like domain.com/blog then that obviously helps the site but if the base site is a for-profit business wouldn't it get less credibility, eyeballs, links as opposed to if you started the blog as it's own separate community on a separate domain and then just strategically linked to the for profit site (sponsorship links)? Essentially the question is, if I'm the Lucky Soday Company, do I start a Blog on the Lucky Soda website, or do I start a separate website to grow a soft drink enthusiast community blog / forum? I would guess a blog has more SEO potential than a discussion forum?
Intermediate & Advanced SEO | | MrSem0 -
How will changing my website's page content affect SEO?
Our company is looking to update the content on our existing web pages and I am curious what the best way to roll out these changes are in order to maintain good SEO rankings for certain pages. The infrastructure of the site will not be modified except for maybe adding a couple new pages, but existing domains will stay the same. If the domains are staying the same does it really matter if I just updated 1 page every week or so, versus updating them all at once? Just looking for some insight into how freshening up the content on the back end pages could potentially hurt SEO rankings initially. Thanks!
Intermediate & Advanced SEO | | Bankable1 -
Google's Knowledge Panel
Hi Moz Community. Has anyone noticed a pattern in the websites that Google pulls in to populate knowledge Panels? For example, for a lot of queries Google keeps pulling data from a specific source over and over again, and the data shown in the Knowledge Panel isn't on the target page. Is it possible that Google simply favors some sites over others and no matter what you do, you'll never make it into the Knowledge box? Thanks.
Intermediate & Advanced SEO | | yaelslater0 -
Drop in Indexed pages
Hope everyone is having an Awesome December! I first noticed a drop in my index in the beginnings of November. My site drop in indexed pages from 1400 to 600 in the past 3-4 weeks. I don't know the cause of it, and would like the community to help me figure out why my indexing has dropped. Thank you for taking time out of your schedule to read this.
Intermediate & Advanced SEO | | BSC0 -
Do internal links from non-indexed pages matter?
Hi everybody! Here's my question. After a site migration, a client has seen a big drop in rankings. We're trying to narrow down the issue. It seems that they have lost around 15,000 links following the switch, but these came from pages that were blocked in the robots.txt file. I was wondering if there was any research that has been done on the impact of internal links from no-indexed pages. Would be great to hear your thoughts! Sam
Intermediate & Advanced SEO | | Blink-SEO0 -
Hidden text that's not show in default view, does it hurt my on page optimization?
Hello, I am developing our new site that will create tabs such as "design", "tech specs", "customer ratings" etc. just like http://www.dell.com/us/p/inspiron-15z-5523/pd. My question is, if most of my content is on the 2nd and 3rd tabs, would my content effect my overall SEO if it's not on it's default view? Because, if users don't physically click on the 2nd and 3rd tab, does it mean that since there is no impression of that content, that content will be allocated to half of its SEO juice? Let me know, I would love to know! Shawn
Intermediate & Advanced SEO | | Shawn1241