I'm noticing that URL that were once indexed by Google are suddenly getting dropped without any error messages in Webmasters Tools, has anyone seen issues like this before?
-
I'm noticing that URLs that were once indexed by Google are suddenly getting dropped without any error messages in Webmasters Tools, has anyone seen issues like this before?
Here's an example:
http://www.thefader.com/2017/01/11/the-carter-documentary-lil-wayne-black-lives-matter -
I tried your exact search and it's still not coming up for me.
I also used the GWT fetching and rendering tool on some other links that were once indexed and then removed- and what I noticed is that they would get reindexed for short time, and then will get removed from the index again.
We are submitting a sitemap through GWT and no errors are getting returned.
We must be having an issue with Google crawling our site, but I haven't been able to find anyone who has had a similar experience- do you know of any other things I can try to fix this issue. The main problem is that it's intermittent and there is no consistency as to why some of our links are indexed and some aren't.
-
With the site search, are you including "site:" in the search query before the URL, like in my example? You shouldn't be getting this thread in a site search if you're doing it right.
If the pages are indexed, then you just need to get them ranking in a better position. In which case, you should form an SEO strategy.
For sites that are no longer indexed, try fetching and rendering the URL in Google's Search Console to see if there's any issue. You can also request Google submit the URL to index through this tool.
If you haven't already, it would also be beneficial for you to submit your sitemap through Google's Search Console to improve Google's crawling of your site. It might just be that Google is having a hard time crawling your site, and that's why you're dropping links.
-
Neither one is showing as indexed for me, actually this messageboard thread is what I'm getting in my SERP.
I am noticing this randomly with a lot of our URLs, with no error messages in Webmasters Tools, any suggestions on how to fix this?
-
If you do a site search for the URL (just search in Google: "site:http://www.thefader.com/2017/01/09/lil-wayne-best-rapper-alive-rap-millennial-grows-up-essay" - without quotation marks), you can see if it's indexed in Google.
The first URL you gave was indexed for me. But the second one doesn't appear to be.
-
Hi Ria,
How are you determining that this URL has been indexed? When I search our page title, keywords that should return this result, or even the direct URL- this page doesn't show in any of the search results.
Here's another example that doesn't seem to be indexed:
http://www.thefader.com/2017/01/09/lil-wayne-best-rapper-alive-rap-millennial-grows-up-essayAm I missing something obvious?
-
I'm seeing that URL as being indexed in Google. Do you have any other examples?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why Aren't My Images Being Indexed?
Hi, One of my clients submitted an image sitemap with 465 images. It was submitted on July 20 2017 to Google Search Console. None of the submitted images have been indexed. I'm wondering why? Here's the image sitemap: http://www.tagible.com/images_sitemap.xml We do use a CDN for the images, and the images are hosted on a subdomain of the client's site: ex. https://photos.tagible.com/images/Les_Invalides_Court_Of_Honor.jpg Thanks in advance! Cheers,
Intermediate & Advanced SEO | | SEOdub
Julian0 -
Completely redesigned webmaster - set up new site in Google Webmaster Tools, or keep existing??
Hi - our company just completely redesigned our website and went from a static HTML site to a PHP based site, so every single URL has changed (around 1500 pages). I put the same verification code into the new site and re-verified but now Google is listing tons and tons of 404's. Some of them are really old pages that haven't existing in a long time, it would literally be impossible to create all the redirects for the 404s it's pulling. Question - when completely changing a site like this, should I have created a whole new Search Console? Or did I do the right thing by using the existing one?
Intermediate & Advanced SEO | | Jenny10 -
Do Google webmaster tool and other backlinks analysis tool ignore the disavow data ?
Hello, Lots of site i have disavow so if i download backlinks of my site from google webmaster so google will ignore the disavow data and give me backlinks other than disavow data? Same if i use backlink tools like moz or semrush or ahref etc for checking backlinks of my site or competitor site so will this tool ignore the disavow data? If such tools not aware of disavow then it is worthless to check competitor links? Thanks! dev
Intermediate & Advanced SEO | | devdan0 -
Best way to permanently remove URLs from the Google index?
We have several subdomains we use for testing applications. Even if we block with robots.txt, these subdomains still appear to get indexed (though they show as blocked by robots.txt. I've claimed these subdomains and requested permanent removal, but it appears that after a certain time period (6 months)? Google will re-index (and mark them as blocked by robots.txt). What is the best way to permanently remove these from the index? We can't use login to block because our clients want to be able to view these applications without needing to login. What is the next best solution?
Intermediate & Advanced SEO | | nicole.healthline0 -
Does Google index url with hashtags?
We are setting up some Jquery tabs in a page that will produce the same url with hashtags. For example: index.php#aboutus, index.php#ourguarantee, etc. We don't want that content to be crawled as we'd like to prevent duplicate content. Does Google normally crawl such urls or does it just ignore them? Thanks in advance.
Intermediate & Advanced SEO | | seoppc20120 -
How to determine URL Parameters in Google Webmaster
Hi there! I have a new website with so many duplicate meta titles and descriptions because of its expanded features from the e-commerce shopping cart that I am using like mobile website, product sorting, etc. Aside from canonical, is it advisable to use the URL parameters from Google webmaster tools to disallow crawling of mobile website and other parameters like, "parent", "catalogsetview", "pcsid", "pg" "mode". I appreciate and advise. 🙂 Thanks!
Intermediate & Advanced SEO | | paumer800 -
Is there any negative SEO effect of having comma's in URL's?
Hello, I have a client who has a large ecommerce website. Some category names have been created with comma's in - which has meant that their software has automatically generated URL's with comma's in for every page that comes beneath the category in the site hierarchy. eg. 1 : http://shop.deliaonline.com/store/music,-dvd-and-games/dvds-and-blu_rays/ eg. 2 : http://shop.deliaonline.com/store/music,-dvd-and-games/dvds-and-blu_rays/action-and-adventure/ etc... I know that URL's with comma's in look a bit ugly! But is there 'any' SEO reason why URL's with comma's in are any less effective? Kind Regs, RB
Intermediate & Advanced SEO | | RichBestSEO0 -
URL Error or Penguin Penalty?
I am currently having a major panic as our website www.uksoccershop.com has been largely dropped from Google. We have not made any changes recently and I am not sure why this is happening, but having heard all sorts of horror stories of penguin update, I am fearing the worst. If you google "uksoccershop" you will see that the homepage does not rank. We previously ranked in the top 3 for "football shirts" but now we don't, although on page 2, 3 and 4 you will see one of our category pages ranking (this didn't used to happen). Some rankings are intact, but many have disappeared completely and in some cases been replaced by other pages on our site. I should point out our existing rankings have been consistently there for 5-6 years until today. I logged into webmaster tools and thankfully there is no warning message from Google about spam, etc, but what we do have is 35,000 URL errors for pages which are accessible. An example of this is: | URL: | http://www.uksoccershop.com/categories/5_295_327.html | | Error details In Sitemaps Linked from Last crawled: 6/20/12First detected: 6/15/12Googlebot couldn't access the contents of this URL because the server had an internal error when trying to process the request. These errors tend to be with the server itself, not with the request. Is it possible this is the cause of the issue (we are not currently sure why the URL's are being blocked) and if so, how severe is it and how recoverable?If that is unlikely to cause the issue, what would you recommend our next move is?All help is REALLY REALLY appreciated 🙂
Intermediate & Advanced SEO | | ukss19840