Homepage/Root domain de-indexed by Google
-
This morning I discovered that the homepage/root domain of our company site, http://www.collegeplus.org/, has been de-indexed by Google and Bing. Out IT dept. is claiming it's our fault because we changed the meta title on our homepage. But they will not give me access to GWT to see if there's any issues.
I believe the issue lies within our robots.txt file - http://www.collegeplus.org/robots.txt
I also don't believe we're suffering a penalty because all of our tier 2 pages are still indexed when any type of branded search is performed. We don't do things that can get a site de-indexed like this.
Any ideas on what the issue may be? Or at least something to convince our IT dept. that simply changing a meta title won't get your homepage totally de-indexed? Thanks.
-
When I was in a similar situation where I didn't have the best of relations with the development company, I used Pole Position's free Code Monitor (https://polepositionweb.com/roi/codemonitor/index.php) to check the robots.txt files of the live site and any development sites/subdomains on a daily basis. I'd get an email if anything had changed, so I could go to the dev company right away and try to mitigate any problems.
-
Hi Keri. Thank you for the info, I wasn't aware of the view only option. I'll send this post to our IT Director. Appreciate your help! Have a great weekend.
-
So sorry to hear about the battles going on. I've seen some of those, and they're no fun.
One thing that may be of help: last month Google rolled out new user access to GWT, including a way to let view without changing any settings (Barry Schwartz writes about it at http://www.seroundtable.com/google-webmaster-tools-users-14838.html). Is there a chance IT would let your team have a read-only view if you let them know it was now available?
-
Hi Dan. Greatly appreciate your response and insights. I think you've completely identified the issue(s). Basically from a technical SEO perspective our site is a trainwreck hit by a nuclear bomb. The battle between IT and my marketing department rages on, making it really difficult to get anything fixed. There's some politics at play that won't get solved here
Anyway, many thanks for your help on this. We'll try again tomorrow.
-
Hi David
First off (and I know I'm preaching to the choir here) but that's completely silly they won't let you look at WMT!! Seriously?! You're not going to BREAK anything just by looking!!
Arggg...
OK... now that we got that out. Let me give you some ideas.
- The homepage is missing from the sitemap - http://www.collegeplus.org/googlesitemap
- Also, shouldn't the sitemap end in .xml - as in /googlesitemap.xml ?
- The worst is I think what you point out from robots.txt - **Disallow: /.php$* Isn't this asking it to block all pages with the file extension .php??? IF so... your homepage does load with the php extension - http://www.collegeplus.org/index.php
- In general, Google's preferred method of keeping pages out of the index is with a meta robots noindex tag - as opposed to the robots.txt
- ALSO - look at this site search - **over 27,000 pages indexed for /**events?state - i'd say not good!\
- You're not using any canonical tags
- The homepage is NOT indexed in Bing either.
- The robots.txt file does look more messed up the more I look at it - for example they're blocking a forums subfolder, yet none exists on the site. It sits on a subdomain, and is still in the index as you can see here
So there's a lot going on here, and anything could be contributing to the deindexation of your homepage. But I'm <sarcasm>pretty sure</sarcasm> its not your title tags.
Hope that helps get you in the right direction. Either way you've got some on-site stuff to clean up.
-Dan
PS - Meant to say, on a happier note, it was nice to meet you at LinkLove Boston
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
301 redirected all my pages to my new domain, now I have a problem with Google Search Console
Hi guys! I bought a new domain name and redirected all my URLs from the old domain to the new one. Everything worked perfectly but now I have a little problem. I want to use the option 'Address Change' in google search console. Step 1 Works (Select new website in the list) Step 2 Works (Confirm that the 301 are working) Step 3 Asks me to Verify the old domain (huh!?) in order to complete the request. Obviously that doesn't work because my 301s WORKS! So if I try to verify the old website by putting a google file in the root of my domain Google tries to access it and it automatically redirects to the new domain. I must be missing something lol help!
Technical SEO | | benoit_20180 -
Does Google add parameters to the URL parameters in webmaster tools/
I am seeing new parameters added (and sometimes removed) from the URL Parameter tool. Is there anything that would add parameters to the tool? Or does it have to be someone internally? FYI - They always have no date in the configured column, no effect set, and crawl is set to Let Google decide.
Technical SEO | | merch_zzounds0 -
Getting Google to index a large PDF file
Hello! We have a 100+ MB PDF with multiple pages that we want Google to fully index on our server/website. First of all, is it even possible for Google to index a PDF file of this size? It's been up on our server for a few days, and my colleague did a Googlebot fetch via Webmaster Tools, but it still hasn't happened yet. My theories as to why this may not work: A) We have no actual link(s) to the pdf anywhere on our website. B) This PDF is approx 130 MB and very slow to load. I added some compression to it, but that only got it down to 105 MB. Any tips or suggestions on getting this thing indexed in Google would be appreciated. Thanks!
Technical SEO | | BBEXNinja0 -
Does Google Parse The Anchor Text while Indexing
Hey moz fanz, I'm here to ask a bit technical and open-minding question.
Technical SEO | | atakala
In the Google's paper http://infolab.stanford.edu/~backrub/google.html
They say they parse the page into hits which is basically word occurences.
But I want to know that they also do the same thing while keeping the anchor text database.
I mean do they parse the anchor text or keep it as it is .
For example, let's say my anchor text is "real car games".
When they indexing my link with anchor text, do they parse my anchor text as hits like
"real" distinct hits
"car" distinct hits
"games" distinct hits.
OR do they just use it as it is. As "real car games"0 -
Linking root domains and youtube
All of my competitors have high linking root domains from youtube and our isn't showing up although we have 1.5 million views to youtube. I tried adding our URL to the videos but it hasn't recognized as a linking root domain. What should I do?? There's a ton of SEO juice here I want to tap into! watch?v=GTXFRTY4CCA&list=UUOcfF9LAHKedNSyk-gk5xDw&index=28
Technical SEO | | tonymartin0 -
Will Google index a site with white text? Will it give it bad ratings?
Will google not rank a site bc pretty much all the copy is white (and the background is all white)? Here's the site in question: https://www.dropbox.com/s/6w24f6h5p0zaxhg/Garrison_PLAY.vs2-static.pdf https://www.dropbox.com/sh/fwudppvwy2khpau/t43NozpG3E/Garrison_PLAY.vs3.jpg thanks--if you need me to clarify more let me know TM Humphries LocalSearched.com
Technical SEO | | CloudGuys0 -
301 forward of index to root
Hi, In my crawl diagnotics, I received an error for duplicate content: 1. www.website.com 2. www.website.com/ 3. www.website.com/index.html Which code do I have to add to my htaccess to avoid this?
Technical SEO | | wellnesswooz0 -
Domain restructure, sitemaps and indexing
I've got a handcoded site with around 1500 unique articles and a handcoded sitemap. Very old school. The url structure is a bit of a mess, so to make things easier for a developer who'll be making the site database-driven, I thought I'd recategorise the content. Same content, but with new url structure (I thought I'd juice up the urls for SEO purposes while I was at it) To this end, I took categories like: /body/amazing-big-shoes/
Technical SEO | | magdaknight
/style/red-boots/
/technology/cyber-boots/ And rehoused all the content like so, doing it all manually with ftp: /boots/amazing-boots/
/boots/red-boots/
/boots/cyber-boots/ I placed 301 redirects in the .htaccess file like so: redirect 301 /body/amazing-boots/ http://www.site.co.uk/boots/amazing-boots/ (not doing redirects for each article, just for categories which seemed to make the articles redirect nicely.) Then I went into sitemap.xml and manually overwrote all the entries to reflect the new url structure, but keeping the old dates of the original entries, like so: <url><loc>http://www.site.co.uk/boots/amazing-boots/index.php</loc>
<lastmod>2008-07-08</lastmod>
<changefreq>monthly</changefreq>
<priority>0.5</priority></url> And resubmitted the sitemap to Google Webmasters. This was done 4 days ago. Webmaster said that the 1400 of 1500 articles indexed had dropped to 860, and today it's climbed to 939. Did I adopt correct procedure? Am I going about things the right way? Given a little time, can I expect Google to re-index the new pages nicely? I appreciate I've made a lot of changes in one fell swoop which could be a bit of a no-no... ? PS Apologies if this question appears twice on Q&A - hopefully I haven't double-posted0