Test contet/pages indexed by search engines
-
During the web development stages of our Joomla CMS website, we have managed to get our site indexed for totally irrelevant test pages mainly to do with Joomla and some other equally irrelevant test content. How damaging is this to our domain from an SEO prospective and is there something we can do about it?
When we do a site:domain.com search we see hundreds of testpages with test/irrelevant meta tags etc.
-
Search engines regularly recrawl every website and will update their information based on changes you make to your site. It is a natural part of the internet. The "site under construction" information is not harmful, but in the future should be blocked from indexing.
-
Thankfully its only test urls that have been indexed by Google only.
However all 3 major engines have indexed our domain against "Site under construction" page with untitled/incomplete tags.
Is this harmful or will this be overwritten when we launch properly and get our site indexed?
-
When you begin developing a site, you should use the robots.txt file to block all search engine access to the site. This is one of the few times where a robots.txt file is very useful.
With respect to fixing the issue, it depends on whether the URLs will be used on the live site, how long it will be until your site launched, and whether unique URLs such as /testing were used or you are working with the same URLs which will exist on the live site.
If your site is still in testing and it will remain in testing for 30+ days, you could add the noindex tag sitewide. Once all the pages were removed from the index, you can then add the robots.txt file. Be careful not to adjust the robots.txt file prior to the pages being removed as the search engines wont be able to see the noindex tag.
You did not mention which search engine indexed your pages. If you are working with Google and the URLs will not exist on the live site, you could use the Google Removal Tool. This is really overkill and should not be necessary, but if the site owner is paranoid about the test pages causing damage to SEO you can take this approach. Any URL removed in this manner cannot be re-added to the index for 90 days.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What do You Think the Biggest Search Trends will Be in 2020?
Just interested to hear everyone's thoughts. Personally I think that, even though voice search is already pretty big - it's about to get much, much larger and this may even entirely change the infrastructure of the web. What are your thoughts for changes in search 2020?
Algorithm Updates | | effectdigital2 -
Search console
I'm submitting my site map via search console and the site map is waiting for approval. I've been taking this error for two or three months. my site is not getting index. Therefore, I have experienced a decline in many words. Can the new search console solve my error? help please . for this site
Algorithm Updates | | Analizaraclari0 -
Duplicate website pages indexed: Ranking dropped. Does Google checks the duplicate domain association?
Hi all, Our duplicate website which is used for testing new optimisations got indexed and we dropped in rankings. But I am not sure whether this is exact reason as it happened earlier too where I don't find much drop in rankings. Also I got replies in the past that it'll not really impact original website but duplicate website. I think this rule applies to the third party websites. But if our own domain has exact duplicate content; will Google knows that we own the website from any other way we are associated like IP addresses and servers, etc..to find the duplicate website is hosted by us? I wonder how Google treats duplicate content from third party domains and own domains. Thanks
Algorithm Updates | | vtmoz0 -
De-indexed homepage in Google - very confusing.
A website I provide content for has just suffered a de-indexed homepage in Google (not in any of the other search engines) - all the other pages remained indexed as usual. Client asked me what might be the problem and I just couldn't figure it out - no linkbuilding has ever been carried out so clean backlink profile, etc. I just resubmitted it and it's back in its usual place, and has maintained the rankings (and PR) it had before it disappeared a few days ago. I checked WMT and no warnings or issues there. Any idea why this might've happened?
Algorithm Updates | | McTaggart0 -
Member's Badge as Link Building to Homepage or Internal Pages?
Providing members and embeddable badge is a well known link building tactic. Is it better to have the badges from hundreds or even thousands of members link back to the homepage of a website, or a lot of different inner pages? The inner pages would the their individual's profile which sits under a category (such as a service and organisation by location). Member's websites would be related to the content of the website generally. What are the advantages of each? 1. Links to homepage make it easier to rank for competitive keywords on the homepage? If the types of websites were to vary a lot, say a carpet cleaning website and a web designer website, if they all linked to the homepage, would it cause some confusion for the link profile?
Algorithm Updates | | designquotes0 -
Page rank of 2 with zero SEO and a 2 month old domain?
Hello, I helped work on a website for a friend. We used wordpress, a theme from elegant themes and wrote the content over 4 days. Zero back links, no seo, etc. Well, a little on page optimization and that's about it. Oh, we did ONE back link from a review site. The domain was brand new; never registered before. About a week after it started getting indexed, it jump from no page rank to a page rank of 1. About a week later, it jumped to a page rank of 2. Again, zero seo (aside from above stated). The site is: trade lines review dot com A page rank of 2 is nothing to write home about, but given the circumstances, how is this even possible? Thanks you!
Algorithm Updates | | Friedman0 -
Why would Google read different pages to rank for a keyword?
I have noticed a large drop in a number of keywords in the latest rankings report. I have checked the results on the 'Ranking History Graph' and it appears that Google is reading different pages for the specific keyword and therefore, giving large fluctuations in ranking dependant on the page from week to week. Why would this be happening?
Algorithm Updates | | Benjamin3790 -
When Google crawls and indexes a new page does it show up immediately in Google search - "site;"?
We made changes to a site, including the addition of a new page and corresponding link/text changes to existing pages. The changes are not yet showing up in the Google index (“site:”/cache), but, approximately 24 hours after making the changes, The SERP's for this site jumped up. We obtained a new back link about a couple of weeks ago, but it is not yet showing up in OSE, Webmaster Tools, or other tools. Just wondering if you think the Google SERP changes run ahead of what they actually show us in site: or cache updates. Has Google made a significant SERP “adjustment” recently? Thanks.
Algorithm Updates | | richpalpine0