How to detect where Google gets indexed URL's
-
Google index some kind of way some links that create duplicate content. We doesn't understand how these are created so we would like detect where Google robots find these links.
We tried:
- Moz Crawl Diagnostics but it shows 0 as Internal Link Count for these kind of links.
- Find some information from Google Analytics, that maybe there is trace (site content - all content) from visitors side. There wan't.
- We tried to find some information in Webmaster Tools under Internal link and HTML Improvements but didn't find any trace.
- Tried some search commands. Is there maybe some good one to search.
- TO search URL's form code with https://search.nerdydata.com.
-
It really isn't possible for an outsider to know why your website is generating those URLs in error; you would have to talk to your developer about that.
As far as canonicals, if your problem is page.com is getting duplicated by added parameters: page.com/?id=1, page.com/?id=2, page.com/?id=3, etc. as long as you have the canonical on page.com, all of the parameter pages will have the correct canonical on them as well. (But you are right, you should track down the source; your developer will know.)
-
Thanks you for your answer but yes I know that these are generated by our site. But problem is that I can use canonical tag for these that are indexed right now but later new ones will be created someway. Problem root isn't that we doesn't know how to use canonical, it's how to get to know where these URL's are find/indexed/detected by Google.
These kind of URL's have been there for months so we can't just hope that somehow these will be droped. We need to find some kind of solution and detect real problem.
-
If you found those URLs by doing a site: search, then those parameters are being generated by your site. (I am surprised that Google is even indexing them; I assume that pretty soon all but one will be dropped.) Here is an article that explains more about those types of duplicate pages: http://moz.com/blog/which-page-is-canonical
You can fix this by using a canonical tag on your homepage with the version that doesn't have the parameter.
-
Our front page has almost 50 duplicate versions. These are shown when we do site:oursite.com, there are /et?id=xx, /et?productId=xx, etc. In URL xx are different numbers.
-
Where are you seeing these duplicate content links? Does Webmaster Tools say that they are duplicate content? Or does this show up in your Moz crawl? What do these URLs look like?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Whats (Other) traffic in Google Analytics?
When I look through all our clients, a few are receiving the majority of their traffic from (other). [ Acquisition > Channels > (Other) ]. The only option in (other) is "website" or "offline", whatever that may be. And even weirder, the avg session duration is 0:00. Any idea what this may be?
Reporting & Analytics | | W2GITeam0 -
Google is not indexing all URLs
My website have company and events profile from 200 countries. So it does have lots of URL. Earlier in August 2014, Google used to crawl 90% of URLs we submit. Thing goes wrong when we shifted from http to https. We lost traffic. But we are gaining it slowly. Main concern is that, It still does not indexed all submitted URLs. It have crawled merely 8% of all URLs submitted. site address is businessvibes.com Any help would be appreciated.
Reporting & Analytics | | irteam0 -
Parenthesis in URL?
For some reason, we have some URLs on our site with parentheses in them such as somesite.com/used-this(that)What will parenthesis do to the ranking of those pages?
Reporting & Analytics | | CFSSEO0 -
No Data in Custom Report set to 'Hit' Scope
Hi Guys, Been having a problem recently with a custom report I have set up... I want to find out number of sessions, bounce rate, session duration etc for different dimensions on my site - store area, store name, product type etc but I cannot seem to get the data to filter through to the report I have set up when 'Session' scope is selected. If I set it to 'hit' then I do get the data but this will only record the first instance of a dimension being triggered (from what I can gather) rather than all dimensions that might be triggered during a complete session. Has anyone experienced similar problems? Thanks, Dan
Reporting & Analytics | | SEOBirmingham810 -
Landing page URL appearing as keyword
Hi Mozers, I've recently experienced the URLs of my key landing pages coming up as keywords. This has been on the rise since early July (when it was relatively insignificant) to the current position (see image below) where they make up the majority of my top keywords. Drilling down into a bit more detail, this seems to be almost exclusively Desktop traffic but in terms of Technology there are no clear standouts (seems to be mostly Windows OS and Chrome). Has anyone else been experiencing this?
Reporting & Analytics | | mopland0 -
Have we been penalised by Google?
Hello guys, Looking for some help here hope someone will be able to point us to the right direction, we have been scratching our heads for months! Basically our Google rankings and organic traffic from Google have dropped signficantly in the last 12 months. At one point we ranked as high as page 3 with the term "holiday lettings". Now we are on page 16, and sometimes not found at all... This has been the case for months now. Curiously, Bing and Yahoo have been much more generous to us, we have been and still consistently rank at No. 2 with the same term. We are in the vacation rental industry - our website is www.alphaholidaylettings.com and most of the sites that rank above us are of much less relevancy, quality and authority. We just could not understand why we would rank so poorly on Google (both US and UK). Organic traffic has also dropped by more than 50% steadily in the last 12 months. One thing we thought might have affected our site is the quality of content. Most content/listings on our site are user generated, and it is not uncommon for users to copy and paste details from an existing listing that they have on another site. Having said that, the same would apply to our competitors too. In this case, might Panda or Penguin have had any negative effects on our seo? We have been adding unique content to the site in the form of a blog and "Holiday Feature" articles. Could the actual design/structure/coding of our pages contribute to our poor rankings too? Any opinions or feedback is much appreciated! Thanks guys, Tom
Reporting & Analytics | | forgottenlife0 -
Why is Google Analytics showing index.php after every page URL?
Hi, My client's site has GA tracking code gathering correct data on the site, but the pages are listed in GA as having /index.php at the end of every URL, although this does not appear when you visit the site pages. Even if there is a redirect happening for site visitors, shouldn't GA be showing the pages as their redirect destination, i.e. the URL that visitors actually see? Could this discrepancy be adversely affecting my search performance? Example page: http://freshstarttax.com/innocent-spouse/ shows up in GA as http://freshstarttax.com/innocent-spouse/index.php thanks
Reporting & Analytics | | JMagary0 -
Setting up Google Analytics for Subsites
I currently have one main .com site and am planning on launching geo-location subsites .co.uk, .com.au, .ru, etc... Traffic will flow between both sites and some of the content on the subsites will be duplicate and therefore include a canonical tag to the main site. I want to set up GA to capture who is going to the subsites and vice versa and correctly capture crossover traffic. Any advice on implementing advanced analytics directly (or links to sources that will direct me the right direction for this project)
Reporting & Analytics | | theLotter0