Historic issue with incomplete indexing
-
Hi there
We run quite a big site in the UK in the commercial real-estate space.
Historically we have always had a challenge getting our "primary" landing pages indexed, which are location based property result pages.
e.g. https://realla.co/to-rent/commercial-property/oxford
For example, for the "towns" category we have 8,549 submitted in our xml sitemap, with only 3,171 indexed. This is a general issue across all our sitemaps. 120k submitted, 80k indexed. Our pages are linked through breadcrumbs, and nearby links.
In the new search console these pages are reported as "crawled - currently not indexed"
These all sit under the folder:
site:https://realla.co/to-rent/commercial-property/*
site:https://realla.co/to-rent/office/*
We have done extensive work to optimise performance, including AMP pages.
Each location page has many details pages for individual properties e.g.
https://realla.co/to-rent/details/0ffbbd0a1a1147edb8847c5ce6179509
One action we have remaining is to nest the details under the locations pages, which may help. These details pages are indexed fully.
Any feedback much appreciated
-
Hi Ian,
The details URL should ideally have keywords in it, getting property name in details page URL would be of great help, like : https://realla.co/to-rent/details/Office-to-let-John-Eccles-House-Robert Robinson-Avenue-Oxford-Science-Park-Oxford-OX4-4GP
About the category (locations in your case), you are submitting too many of them, your URL structure needs to re-structured, there is work to be done there and sitemap updated according to that. For example:
https://realla.co/to-rent/commercial-property/
can be changed to
https://realla.co/commercial-property-to-rent/
I hope this helps, let me know if you have further queries.
Regards,
Vijay
-
Thanks for your reply
We are just about to nest the "details" pages under the results path e.g. /to-rent/commercial-property/newbury/details/1294321739712973129 etc so it sits under the right location.
I think this is in line with your recommendation.
We have alot of individual sitemap files, should these be consolidated?
-
Hi Ian,
I have analyzed the website in detail, the problem seems to be that you are not giving any differentiation to search engine bots between important category/sub-category(in your case different locations) pages compared to product pages (in your case property details page). The location pages URL structure and their sitemap submission strategy can be re-worked to get the desired results.
Another scope of improvement is in URL structure for property details page **For example, **
https://realla.co/to-rent/details/0ffbbd0a1a1147edb8847c5ce6179509 should be https://realla.co/to-rent/details/Office-to-let-John-Eccles-House-Robert Robinson-Avenue-Oxford-Science-Park-Oxford-OX4-4GP
Your site structure is huge, and it must be getting dynamic links generated or removed, you need to be careful with the site structure and how often to submit sitemap.
I hope this helps. Let me know if you have further queries, I will be happy to help.
Regards,
Vijay
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to effectively de-index in Magento site?
We have thousands of Missing Description issues but most of them are account/login pages. i.s. /customer/account/ etc... We tried to de-index them through the Configuration using the instructions here - https://docs.magento.com/user-guide/marketing/search-engine-robots.html But they're still appearing as issues in the Site Crawl. Even without the site crawl issue, we don't really want these to appear in the SERPs. Does anybody know how to properly de-index these login pages in Magento? Thank you!
Technical SEO | | LASClients0 -
Help Center/Knowledgebase effects on SEO: Is it worth my time fixing technical issues on no-indexed subdomain pages?
We're a SaaS company and have a pretty extensive help center resource on a subdomain (help.domain.com). This has been set up and managed over a few years by someone with no knowledge of SEO, meaning technical things like 404 links, bad redirects and http/https mixes have not been paid attention to. Every page on this subdomain is set to NOT be indexed in search engines, but we do sometimes link to help pages from indexable posts on the main domain. After spending time fixing problems on our main website, our site audits now flag almost solely errors and issues on these non-indexable help center pages every week. So my question is: is it worth my time fixing technical issues on a help center subdomain that has all its pages non-indexable in search engines? I don't manage this section of the site, and so getting fixes done is a laborious process that requires going through someone else - something I'd rather only do if necessary.
Technical SEO | | mglover19880 -
URL Indexing with Keyword
Hi, My webpage url is indexed in Google but don't show when searching the Main Keyword. How can i index it with keyword. It should show on any SERP when the keyword is searched. Any suggestions.
Technical SEO | | green.h1 -
Fake 404 Issue
Hello, I just had a problem my site started showing up 404 issues for all my wordpress pages and post but visually the page was loading with content but yet all pages and software including google WMT was showing the 404 issue. I never found the issue but was able to move the site into a new hosting and restore from a backup and it work. Google found the issue on Jan 27th and they remove all the pages with 404 from the index and I lost most of my top ranking I have since fix the issue and was wondering if google would restore my ranking in such a case? Regards, M
Technical SEO | | thewebguy30 -
Carwling and indexing problems
hi, i have noticed since my site was upgraded that google is taking a long time to publish my articles. before the upgrade google would publish the article straight away, but now it takes an average of around 4 days. the article i am talking about at the moment is here http://www.in2town.co.uk/celebrities-in-the-news/stuart-hall-has-his-prison-sentence-for-sex-crimes-doubled-to-30-months now i have a blog here on blogger and the article was picked up within six mins http://showbizgossipandnews.blogspot.co.uk/2013/07/stuart-hall-has-his-prison-sentence-for.html so i am just wondering what the problem is and what i need to solve this my problem is, my site is mostly a news site so it is no good to me if google is publishing new stories every four days, any help would be great.
Technical SEO | | ClaireH-1848860 -
Bad Duplicate content issue
Hi, for grappa.com I have about 2700 warnings of duplicate page content. My CMS generates long url like: http://www.grappa.com/deu/news.php/categoria=latest_news/idsottocat=5 and http://www.grappa.com/deu/news.php/categoria%3Dlatest_news/idsottocat%3D5 (this is a duplicated content). What's the best solution to fix this problem? Do I have to set up a 301 redirect for all the duplicated pages or insert the rel=canonical or rel=prev,next ? It's complicated becouse it's a multilingual site, and it's my first time dealing with this stuff. Thanks in advance.
Technical SEO | | nico860 -
Odd Google Indexing Issue
I have encountered something odd with Google indexing. According to the Google cache my site was last updated on April 6. I had been making a series of changes on April 7th and none of them show up in the cached version of the site (naturally). Then, on the 8th, my rankings seem to have dropped about 6 places and the main SERP is showing a text that isn't even on the Web site. The cached version has the correct page title from the page that was indexed on the 6th. How do I learn where Google is picking this up from? There is a clean page title tag on my Web site. I've checked the server, etc to see what's going on. The text isn't completely unrelated, but it definitely impacted my ranking. Does Google ever have these hiccups when indexing?
Technical SEO | | VERBInteractive0 -
Re-direct issues
Forgive me for the novice question. But I was recently looking at open site explorer and was checking out my site www.visualawards.com , I know we have a re-direct to www.visualawards.com/home.php . After checking both URL's, I found that I have links pointing to both. Is this bad, am I diluting the links? If yes, which one should I point the future ones to, and is there anyway to recover the current links already? Thanks again for your help!!!
Technical SEO | | RENDEV0