Our website is not being indexed
-
We have an issue with a site that we can't get to the bottom of. This site: (URL removed) is not being properly indexed.
When we do a search for (URL removed) in google.com.au. The site appears as the 4th listing with the following title and description:
(Title removed)
A description for this result is not available because of this site's robots.txt – learn more.
We have checked the site's robots.txt and can see its been now implemented correctly: (URL removed)
About a week ago, we also went into Webmaster Tools and submitted a request for Google to recrawl our site.
We are unsure what the issue is that is causing the site to not be properly indexed and how to resolve it. Any assistance on this topic would be most appreciated!
-
Antonio is absolutely correct - in the 10th line of your homepage's code is a no-index tag.
To elaborate more on his second point, the reason the page: _kaya3976.com.au/wp-admin/admin-ajax.php _is the only one being shown, is because Google sees that there's a website, but can't actually index any visitor-intended pages - and is therefore showing your WP Login page. You definitely want that to be resolved ASAP, but I would make absolutely sure with whomever's been developing the site that there isn't a reason it's de-indexed currently.
-
Hi Gavo:
- First of all, you have a noindex tag in the section of your site.
- Your site it's made in Parallax or something similar, so, the sections are not really pages at all, just a part of the homepage. To index all the sections, this ones should have an unique URL (not with the # prefix, maybe a web developer can help you with this)
-
The issue is that you still have the meta tag name='robots' content='noindex,follow' /> on your pages.
Now that search engines can crawl your site, they are able to pick up on this tag, and thus will not index your pages. Once you remove that tag, it should be indexed, especially upon requesting indexation through search console.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Same URL-Structure & the same number of URLs indexed on two different websites - can it lead to a Google penalty?
Hey guys. I've got a question about the url structure on two different websites with a similar topic (bith are job search websites). Although we are going to publish different content (texts) on these two websites and they will differ visually, the url structure (except for the domain name) remains exactly the same, as does the number of indexed landingpages on both pages. For example, www.yyy.com/jobs/mobile-developer & www.zzz.com/jobs/mobile-developer. In your opinion, can this lead to a Google penalty? Thanks in advance!
Intermediate & Advanced SEO | | vde130 -
Please have a look at my website. I am stuck here.
Here might be the reason. I had loads of unnecessary content so I given them the noindex tag. I tried to change the robot.txt file but that shouldn't be a problem in SEO. First my site had a country specific domain and then a year later I changed it to .Com, as to target globally (Mainly US). My site is ranking well in that specific country (never been close to page 1) on page 3 almost every time. It's not ranking in other countries, despite the fact that I've not targeted it to any specific country since the domain was changed. A month ago, I deleted 404 pages and all the thin content which was indexed in the SERP and also deleted the duplicated contents and as well as the copied contents. Meanwhile I've also tried changing the headings in some of the products articles as they were causing the duplicate heading issue. I've recently switched my hosting from the UK based server to the Us based server because the last hosting has bad downtime. So far until now nothing seems to be working in my favor. I'm just tired of resolving issues and in return finding a zero result. This is my devil site: 10stuffs.com plz check it out and tell me why my site is not ranking at all and what sould I do.
Intermediate & Advanced SEO | | anshu14320 -
Only a fraction of the sitemap get indexed
I have a large international website. The content is subdivided in 80 countries, with largely the same content all in English. The URL structure is: https://www.baumewatches.com/XX/page (where XX is the country code)
Intermediate & Advanced SEO | | Lvet
Language annotations hreflang seem to be set up properly In the Google Search Console I registered: https://www.baumewatches.com the 80 instances of https://www.baumewatches.com/XX in order to geo target the directories for each country I have declared a single global sitemap for https://www.baumewatches.com (https://www.baumewatches.com/sitemap_index.xml structured in a hierarchical way) The problem is that the site has been online already for more than 8 months and only 15% of the sitemap URLs have been indexed, with no signs of new indexations in the last 3 months. I cannot think about a solution for this.0 -
Moving html site to wordpress and 301 redirect from index.htm to index.php or just www.example.com
I found page duplicate content when using Moz crawl tool, see below. http://www.example.com
Intermediate & Advanced SEO | | gozmoz
Page Authority 40
Linking Root Domains 31
External Link Count 138
Internal Link Count 18
Status Code 200
1 duplicate http://www.example.com/index.htm
Page Authority 19
Linking Root Domains 1
External Link Count 0
Internal Link Count 15
Status Code 200
1 duplicate I have recently transfered my old html site to wordpress.
To keep the urls the same I am using a plugin which appends .htm at the end of each page. My old site home page was index.htm. I have created index.htm in wordpress as well but now there is a conflict of duplicate content. I am using latest post as my home page which is index.php Question 1.
Should I also use redirect 301 im htaccess file to transfer index.htm page authority (19) to www.example.com If yes, do I use
Redirect 301 /index.htm http://www.example.com/index.php
or
Redirect 301 /index.htm http://www.example.com Question 2
Should I change my "Home" menu link to http://www.example.com instead of http://www.example.com/index.htm that would fix the duplicate content, as indx.htm does not exist anymore. Is there a better option? Thanks0 -
How to rank my website in Google UK?
Hi guys, I own a London based rubbish removal company, but don't have enough jobs. I know for sure that some of my competitors get most of their jobs trough Google searches. I also have a website, but don't receive calls from it at all. Can you please tell me how to rank my website on keywords like: "rubbish removal london", "waste clearance london", "junk collection london" and other similar keywords? I know that for person like me (without much experience in online marketing) will be difficult task to optimize the website, but at least - I need some advices from where to start. I'm also thinking to hire an SEO but not sure where to find a trusted company. Most importantly I have no idea how much should pay to expect good results? What is too much and what is too low? I will appreciate all advices.
Intermediate & Advanced SEO | | gorubbishgo0 -
How do the Quoras of this world index their content?
I am helping a client index lots and lots of pages, more than one million pages. They can be seen as questions on Quora. In the Quora case, users are often looking for the answer on a specific question, nothing else. On Quora there is a structure setup on the homepage to let the spiders in. But I think mostly it is done with a lot of sitemaps and internal linking in relevancy terms and nothing else... Correct? Or am I missing something? I am going to index about a million question and answers, just like Quora. Now I have a hard time dealing with structuring these questions without just doing it for the search engines. Because nobody cares about structuring these questions. The user is interested in related questions and/or popular questions, so I want to structure them in that way too. This way every question page will be in the sitemap, but not all questions will have links from other question pages linking to them. These questions are super longtail and the idea is that when somebody searches this exact question we can supply them with the answer (onpage will be perfectly optimised for people searching this question). Competition is super low because it is all unique user generated content. I think best is just to put them in sitemaps and use an internal linking algorithm to make the popular and related questions rank better. I could even make sure every question has at least one other page linking to it, thoughts? Moz, do you think when publishing one million pages with quality Q/A pages, this strategy is enough to index them and to rank for the question searches? Or do I need to design a structure around it so it will all be crawled and each question will also receive at least one link from a "category" page.
Intermediate & Advanced SEO | | freek270 -
How ot optimise a website for competitive keywords?
Hi guys, I hope to find some good answers to my questions, because here are some of the best SEO's in the world. I'm doing SEO as a hobby for a few years and had some very good results before the latest Google updates. Now I'm not able to rank any website for competitive keywords. The last project I started is this website (man and van hire company targeting London market).
Intermediate & Advanced SEO | | nasi_bg
The problem is that I can't rank even in Top 100 in Google UK for the main keywords like: "man and van london" , "man and van service london" ,"london man & van"...
The site has over 1k good backlinks (according to Ahrefs), unique content, titles and descriptions but still can't rank well. Am i missing something? Few years back that was more than enough to rank well in Google.
I will be very grateful to hear your suggestions and opinions.0 -
Old pages STILL indexed...
Our new website has been live for around 3 months and the URL structure has completely changed. We weren't able to dynamically create 301 redirects for over 5,000 of our products because of how different the URL's were so we've been redirecting them as and when. 3 months on and we're still getting hundreds of 404 errors daily in our Webmaster Tools account. I've checked the server logs and it looks like Bing Bot still seems to want to crawl our old /product/ URL's. Also, if I perform a "site:example.co.uk/product" on Google or Bing - lots of results are still returned, indicating the both still haven't dropped them from their index. Should I ignore the 404 errors and continue to wait for them to drop off or should I just block /product/ in my robots.txt? After 3 months I'd have thought they'd have naturally dropped off by now! I'm half-debating this: User-agent: *
Intermediate & Advanced SEO | | LiamMcArthur
Disallow: /some-directory-for-all/* User-agent: Bingbot
User-agent: MSNBot
Disallow: /product/ Sitemap: http://www.example.co.uk/sitemap.xml0