Getting pages that load dynamically into the SE's
-
SEO'ers,
Am dealing with an issue I cannot figure out the best way to handle. Working on a website that shows the definitions of words which are loaded dynamically from an open source. Source such as: wiktionary.org
When you visit a particular page to see the definition of the word, say; www.example.com/dictionary/example/ the definition is there. However, how can we get all the definition pages to get indexed in search engines? The WordPress sitemap plugin is not picking up these pages to be added automatically - guess because it's dynamic - but when using a sitemap crawler pages are detected.
Can anybody give advice on how to go about getting the 200k+ pages indexed in the SE's? If it helps, here's a reference site that seems to load it's definitions dynamically and has succeeded in getting its pages indexed: http://www.encyclo.nl/begrip/sample
-
I see what you mean there - thanks for sharing your expertise and views on this issue. Much appreciated
-
The only way I'd let those pages be indexed is if they had unique content on them AND/OR provided value in other ways besides just providing the Wiki definition. There are many possibilities for doing this, none of them scalable in an automated fashion, IMHO.
You could take the top 20% of those pages (based on traffic, conversions, revenue...) and really customize them by adding your own definitions and elaborating on the origin of the word, etc... Beyond that you'd probably see a decline in ROI.
-
Everett, yes that's correct. I will go ahead and follow up on what you said. I do still wonder what the best way would be to go about getting it indexed - if I wanted to do that in the future. If you could shed some light on how to go about that, I'd really appreciate it. Thanks so much in advance!
-
It appears that your definitions are coming from wiktionary.org and are therefore duplicate content. If you were providing your own definitions I would say keep the pages indexable, but in this case I would recommend adding a noindex, follow robots meta tag to the html header of those pages.
-
Hi Everett, I've been looking at the index for word definitions and there's so many pages that are very similar to each other. It's worth giving it a shot I think. If you can provide feedback please do. Here's the domain: http://freewordfinder.com. The dictionary is an addition to users who'd like to see what a word means after they've found a word from random letters. You can do a search at the top to see the results, then click through to the definition of the word. Thanks in advance
-
Ron,
We could probably tell you how to get those pages indexed, but then we'd have to tell you how to get them removed from the index when Google sees them all as duplicate content with no added value. My advice is to keep them unindexed, but if you really want them to be indexed tell us the domain and I'll have a look at how it's working and provide some feedback.
-
Hi Keri, did you think that the site might get penalized because it would in essence be duplicate content from another site? Even though the source is linked from the page? Please let me know your thoughts when you can
-
No they currently do not have additional information on them. They are simply better organized on my pages compared to the 3rd party. The unique information is what drives visitors to the site and from those pages it links to the definitions just in case they're interested understanding the meaning of a word. Does that help?
-
Do the individual pages with the definitions have additional information on them, or are they just from a third party, with other parts of the site having the unique information?
-
Hi Keri, thanks for your response. Well, I see what you're saying. The pages that show the definition pulled from the 3rd party are actually supplementary to the solution the site provides (core value). Shouldn't that make a difference?
-
I've got a question back for you that's more of a meta question. Why would the search engines want to index your pages? If all the page is doing is grabbing information from another source, your site isn't offering any additional value to the users, and the search engine algos aren't going to see the point in sending you visitors.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Important pages are being 302 redirected, then 301 redirected to support language versions. Is this affecting negatively the linking juice distribution of our domain?
Hi mozzers, Prior to my arrival, in order to support and better serve the international locations and offering multiple language versions of the same content the company decided to restructure its URLs focused on locale urls. We went from
International SEO | | Ty1986
https://example.com/subfolder to https://example.com/us/en-us/new-subfolder (US)
https://example.com/ca/en-us/new-subfolder (CAN)
https://example.com/ca/fr-ca/new-subfolder (CAN)
https://example.com/de/en-us/new-subfolder (Ger)
https://example.com/de/de-de/new-subfolder (Ger) This had implications on redirecting old URLs to new ones. All important URLs such as https://example.com/subfolder were
302 redirected to https://example.com/us/en-us/subfolder and then 301 redirected to the final URL. According to the devs: If you change the translation to the page or locale, then a 302 needs to happen so you see the same version of the page in German or French, then a 301 redirect happens from the legacy URL to the new version. If the 302 redirect was skipped, then you would only be able to one version/language of that page.
For instance:
http://example.com/subfolder/state/city --> 301 redirect to {LEGACY URL]
https://example.com/subfolder/state/city --> 302 redirect to
https://example.com/en-us/subfolder/state/city --> 301 redirect to
https://example.com/us/en-us/new-subfolder/city-state [NEW URL] I am wondering if these 302s are hurting our link juice distribution or that is completely fine since they all end up as a 301 redirect? Thanks.1 -
Ecommerce Product Page Optimization & International SEO
Hello, I'm working on our website SEO optimization. We have a thousands of products pages with different structures for the languages (arg) and very depth folder path .com/[folder]/[folder]/[folder]/product1.hmtl So now I have the happiness of working on the optimization of the website with themajor risk of impacting all current ranking. But anyway, here are a few questions I have on the way. Part 1 - International URL Our websites target people per country and languages. We do not have shops per countries (not enough resources_) but we try to get at least website per languages. What could be the best option?_ Url Parameters +hreflang So we save one folder less and the proper setup. But I'm just scared it's gonna be too messy for Google URL:.com/product1**?lang=fr** Product page:link rel="alternate" hreflang="en" href=".com/product1" / Language folder + hreflang one folder more but clearer structure URL:.com**/fr/**product1 **Product **page:****link rel="alternate" hreflang="en" href=".com/product1" / Part 2 - Product URL Our website is structure per categories so the product comes after. However, I've seen a lot of websites recently removing the categories to save folders space. What should be the most efficient option? Category folder It's obviously a good practice but this + the language folder makes already 2 folders URL:.com/categoryA/product1-{targetedKW} {targetedKW} = cheap product, best price or else All in url I've never done it but it somehow makes sense URL:.com/categoryA-product1-{targetedKW} Part 3 - Keyword stuffing As I'd like to get most of it automatically done, what could be the best places to add a few KW. **Markups:**All the ones we can **Meta Descriptions:**optimize one for Google + one for twitter + one for facebook Longer to do but then from google shopping and other automatic links, we could have the perfect or, at least, best description possible **All other option:**Reuse our product name + {targetter KW1 KW2 ...} Product description_ex: content_ Buttons (click to buy)ex: button title="Buy product_name cheap" alt="Purchase product_name"Buy Product name/button Images:same than above Meta:Titles and meta description Hn
International SEO | | omnyex0 -
Geolocation issue: Google not displaying the correct url in the SERP's
Hello, Im running a multi-country domain with this structure: domain.com/ar/
International SEO | | EstebanCervi
domain.com/mx/
domain.com/cl/
etc I also have: domain.com/int/ for x-default
domain.com/category/ does a 301 redirect through IP geo-location to the correspondent url, example if your IP is from Mexico, then you got redirected to domain.com/mx/category/ hreflang is correct. webmaster tool geo-location is correct. Example of the issue Im facing right now: When users from Chile do a keyword search in Google Chile, the domain ranks well but the URL that appears in the SERP is the /mx/ version, or the /int/ version or any other country version. Other times is the /cl/ version. The same happens for all the users / countries / keywords. I need to understand what Im doing wrong, because Google is not displaying in the SERP's the correct URL version for the country of the user who is doing the search. Thank you so much! I will appreciate your ideas. PS: I think I should try to change the 301 to a 302 redirect, or completely remove those redirects. Any ideas? Suggestions? Thanks!0 -
How to get EU visitors to .co.uk and US visitors to .com
Hi all Re: www.explanar.com and www.explanar.co.uk We have developed a website for the US and UK, with a shop on each. Each site has a network of countries they can export to. US sells and ships to US, South America, Caribbean and so on UK sells and ships to UK, Europe, and Africa, etc Currently, visitors from Germany (.de) searching for phrases "Explanar", or "golf swing trainer" get the .com (US) site. They cannot buy from this site, so we have added a message to tell them to go to the UK site (.co.uk, but what I really want is for the European visitors to find the UK site only. .com is the original URL, but with new content
International SEO | | Crumpled_Dog
.co.uk has just been set up In the long run, we will set up .de URLs with German language, and other countries will follow, but not all. Any thoughts would be great. Thanks
Scott0 -
Dynamic Google search snippet text based on user's language
On Google search results page, I want to show search snippet text (of my webpage) in Hindi language if user is user is using Google in Hindi language. If user chose another language on Google search page, my snippet text should be shown in that language. Is this possible? How?
International SEO | | Avinashmb0 -
What language to use for URL's for Russian language?
Hi, Our site is in English, Spanish, Danish and Russian - the URL's are individual to the language they are in, but of course, Russian contains some strange characters so I decided not to use them in the URL's Any advice on how to create the URL's for russian language pages? thanks
International SEO | | bjs20100 -
International (foreign language) URL's best practices
I'm curious if there is a benefit or best practice with regards to using the localized language on international sites (with specific ccTLDs). For example, should my french site (site.fr) use the french language as keywords within the URLs or should they be in english? e.g. www.site.fr/nourriture vs. www.site.fr/food Is that considered best practice for SEO (or just for brand perception those markets?). Is there a tangible loss in SEO if we do not use the correct language for those URLs and just stick with English around the world? I recall seeing a Matt Cutts video on the topic and he said that google does support i18n URL's but other SE's might not support them as gracefully but he didn't come down with a hard recommendation to go with i18n URL's or just English. Would love a strong ruling in favor one direction based on best practices.
International SEO | | mongillo0 -
Risks of Migrating tld's to sub folders
Hi Guys, I am thinking of migrating our .co.nz and our .co.uk websites into sub folders on our .com website (eg: .com/uk and .com/nz). Do you think this is a risky strategy in regards to our performance in the localised search engines or should the centralisation of all these websites and their link authority into the .com help us move up the rankings? We are thinking of doing this in the next week, we have some really good rankings for the local googles, however we also have plenty of phrases sitting just on page 2 and I was hoping this might help boost them onto page 1? Has anyone else had experience migrating tld sites to sub folders on a .com and if so what was your experience of the impact on search rankings in the local googles and the timeframe that these changes took to have an effect? Did you have any negative results?
International SEO | | ConradC0