Getting pages that load dynamically into the SE's
-
SEO'ers,
Am dealing with an issue I cannot figure out the best way to handle. Working on a website that shows the definitions of words which are loaded dynamically from an open source. Source such as: wiktionary.org
When you visit a particular page to see the definition of the word, say; www.example.com/dictionary/example/ the definition is there. However, how can we get all the definition pages to get indexed in search engines? The WordPress sitemap plugin is not picking up these pages to be added automatically - guess because it's dynamic - but when using a sitemap crawler pages are detected.
Can anybody give advice on how to go about getting the 200k+ pages indexed in the SE's? If it helps, here's a reference site that seems to load it's definitions dynamically and has succeeded in getting its pages indexed: http://www.encyclo.nl/begrip/sample
-
I see what you mean there - thanks for sharing your expertise and views on this issue. Much appreciated
-
The only way I'd let those pages be indexed is if they had unique content on them AND/OR provided value in other ways besides just providing the Wiki definition. There are many possibilities for doing this, none of them scalable in an automated fashion, IMHO.
You could take the top 20% of those pages (based on traffic, conversions, revenue...) and really customize them by adding your own definitions and elaborating on the origin of the word, etc... Beyond that you'd probably see a decline in ROI.
-
Everett, yes that's correct. I will go ahead and follow up on what you said. I do still wonder what the best way would be to go about getting it indexed - if I wanted to do that in the future. If you could shed some light on how to go about that, I'd really appreciate it. Thanks so much in advance!
-
It appears that your definitions are coming from wiktionary.org and are therefore duplicate content. If you were providing your own definitions I would say keep the pages indexable, but in this case I would recommend adding a noindex, follow robots meta tag to the html header of those pages.
-
Hi Everett, I've been looking at the index for word definitions and there's so many pages that are very similar to each other. It's worth giving it a shot I think. If you can provide feedback please do. Here's the domain: http://freewordfinder.com. The dictionary is an addition to users who'd like to see what a word means after they've found a word from random letters. You can do a search at the top to see the results, then click through to the definition of the word. Thanks in advance
-
Ron,
We could probably tell you how to get those pages indexed, but then we'd have to tell you how to get them removed from the index when Google sees them all as duplicate content with no added value. My advice is to keep them unindexed, but if you really want them to be indexed tell us the domain and I'll have a look at how it's working and provide some feedback.
-
Hi Keri, did you think that the site might get penalized because it would in essence be duplicate content from another site? Even though the source is linked from the page? Please let me know your thoughts when you can
-
No they currently do not have additional information on them. They are simply better organized on my pages compared to the 3rd party. The unique information is what drives visitors to the site and from those pages it links to the definitions just in case they're interested understanding the meaning of a word. Does that help?
-
Do the individual pages with the definitions have additional information on them, or are they just from a third party, with other parts of the site having the unique information?
-
Hi Keri, thanks for your response. Well, I see what you're saying. The pages that show the definition pulled from the 3rd party are actually supplementary to the solution the site provides (core value). Shouldn't that make a difference?
-
I've got a question back for you that's more of a meta question. Why would the search engines want to index your pages? If all the page is doing is grabbing information from another source, your site isn't offering any additional value to the users, and the search engine algos aren't going to see the point in sending you visitors.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Should Hreflang x-default be on every page of every country for an International company?
UPDATED 4/29/2019 4:33 PM I had made to many copy and pastes. Product pages are corrected Upon researching the hreflang x-default tag, I am getting some muddy results for implementation on an international company site older results say just homepage or the country selector but…. My Question/Direction going forward for the International Site I am working on: I believe I can to put x-default all the pages of every country and point it to the default language page for areas that are not covered with our current sites. Is this correct? From my internet reading, the x-default on every page is not truly necessary for Google but it will be valid implemented. My current site setup example:
International SEO | | gravymatt-se
https://www.bluewidgets.com Redirects to https://www.bluewidgets.com/us/en (functions as US/Global) Example Countries w/ code Site:- 4 countries/directories US/Global, France, Spain Would the code sample below be correct? https://www.bluewidgets.com/us/en/ (functions as US/Global) US/Global Country Homepage - https://www.bluewidgets.com/us/en/ US/Global Country Product Page(s) This would be for all products - https://www.bluewidgets.com/us/en/whizzer-5001/ http://www.bluewidgets.com/us/en (functions for France) France Country Homepage - https://www.bluewidgets.com/fr/fr/ France Country Product Page(s) This would be for all products- https://www.bluewidgets.com/es/es/whizzer-5001 http://www.bluewidgets.com/us/en (functions as Spain) Spain Country Homepage - https://www.bluewidgets.com/es/es/ Spain Country Product Page(s) This would be for all products - https://www.bluewidgets.com/es/es/whizzer-5001 Thanks for the spot check Gravy0 -
Geo-Targeting separate TLD's where both are .com domains
Hi I have a client who owns two separate TLDs for the same brand (for the sake of this post, we'll call the two sites www.site-a.com and www.site-b.com). For site www.site-a.com the website has been around for a while and is their primary site for their US operations which is their heartland, is well established in the SERPS and is where they make most of their money. As they looked to expand to the UK, they then created www.site-b.com and added the UK as a subfolder (so www.site-b.com/uk) and geo-targeted it towards the UK in Webmaster tools . The site has recently launched but they now find that, when a customer searches for their brand in the UK, they find www.site-a.com in position 1 (which, given it's tailored for a primary US audience, has a significantly lower conversion rate for UK traffic) and www.site-b.com in position 2. However, the client doesn't want to specifically geo target www.site-a.com to the USA as they feel it might affect where they appear for other international markets aside from the UK. So the question is, how can they, with the existing infrastructure, help remove www.site-a.com from the UK SERPs without adversely affecting their rank elsewhere? Hope this makes sense and thanks in advance for your help. James
International SEO | | jimmygs19820 -
Trying To Use Parent Company's Content In Another Country
Hi, I'm hoping someone can help me out here but this is what I am dealing with: Say John Smith Companies sells Widgets across the United States. They have also formed a company called "Widgets of Canada" in an effort to sell their Blue Widgets only in Canada and I am in responsible for their website. Recently, John Smith Companies completely redesigned their website and it now has a really slick look and is loaded with great widgets content. I would like to take their site and re-purpose it for use in Canada. However, I am concerned about duplicate content. I would be converting all the widget specifications from imperial to metric units, changing the title and description elements and also using a much different folders/ paths. Is this enough to avoid any issues with similar page content? Is there anything I can do with hreflang? Thanks
International SEO | | DohenyDrones0 -
In the U.S., how can I stop the European version of my site from outranking the U.S. version?
I've got a site with two versions – a U.S. version and a European version. Users are directed to the appropriate version through a landing page that asks where they're located; both sites are on the same domain, except one is .com/us and the other is .com/eu. My issue is that for some keywords, the European version is outranking the U.S. version in Google's U.S. SERPs. Not only that, but when Google displays sitelinks in the U.S. SERPs, it's a combination of pages on the European site and the U.S. site. Does anyone know how I can stop the European site from outranking the U.S. site in the U.S.? Or how I can get Google to only display sitelinks for pages on the U.S. site in the U.S. SERPs? Thanks in advance for any light you can shed on this topic!
International SEO | | matt-145670 -
Recent Google Link Scheme Updated ? What's Your Reaction against Link Building, Link Exchanging ?
Many Bloggers and Webmasters are upset over this !
International SEO | | Esaky
Recent Google Link Scheme Updated ? What's Your Reaction against Link Building, Link Exchanging ? https://support.google.com/webmasters/answer/66356?hl=en What will you Do, if we are good at traffic to our blog and advertiser link will be no-follow - will they accept it ! and guest post also. They need a do-follow link back to their blog or website they hired for !0 -
Recent Google Link Scheme Updated ? What's Your Reaction against Link Building, Link Exchanging ?
Many Bloggers and Webmasters are upset over this !
International SEO | | Esaky
Recent Google Link Scheme Updated ? What's Your Reaction against Link Building, Link Exchanging ? https://support.google.com/webmasters/answer/66356?hl=en What will you Do, if we are good at traffic to our blog and advertiser link will be no-follow - will they accept it ! and guest post also. They need a do-follow link back to their blog or website they hired for !0 -
Getting A Sub Domain To Out-Rank The Main Domain
Hi, We have a prospective client who currently have a sub domain setup for each language, they all have the same content as the main domain. The problem is that the main domain is written in English (but not UK English), and they want the UK sub domain to outrank it (it's the other way round at the moment). Effectively, there are duplicate content issues here and as a result it looks like Google have chosen to keep the main domain (as it has more authority) and lower the UK sub results in its rankings. Is there a feature in webmaster tools where you can target subdomains to a location (I know you can do this with a main domain). Additionally, any other tips for the above would be greatly appreciated. Thanks in advance,
International SEO | | jasarrow0 -
International (foreign language) URL's best practices
I'm curious if there is a benefit or best practice with regards to using the localized language on international sites (with specific ccTLDs). For example, should my french site (site.fr) use the french language as keywords within the URLs or should they be in english? e.g. www.site.fr/nourriture vs. www.site.fr/food Is that considered best practice for SEO (or just for brand perception those markets?). Is there a tangible loss in SEO if we do not use the correct language for those URLs and just stick with English around the world? I recall seeing a Matt Cutts video on the topic and he said that google does support i18n URL's but other SE's might not support them as gracefully but he didn't come down with a hard recommendation to go with i18n URL's or just English. Would love a strong ruling in favor one direction based on best practices.
International SEO | | mongillo0