Getting pages that load dynamically into the SE's
-
SEO'ers,
Am dealing with an issue I cannot figure out the best way to handle. Working on a website that shows the definitions of words which are loaded dynamically from an open source. Source such as: wiktionary.org
When you visit a particular page to see the definition of the word, say; www.example.com/dictionary/example/ the definition is there. However, how can we get all the definition pages to get indexed in search engines? The WordPress sitemap plugin is not picking up these pages to be added automatically - guess because it's dynamic - but when using a sitemap crawler pages are detected.
Can anybody give advice on how to go about getting the 200k+ pages indexed in the SE's? If it helps, here's a reference site that seems to load it's definitions dynamically and has succeeded in getting its pages indexed: http://www.encyclo.nl/begrip/sample
-
I see what you mean there - thanks for sharing your expertise and views on this issue. Much appreciated
-
The only way I'd let those pages be indexed is if they had unique content on them AND/OR provided value in other ways besides just providing the Wiki definition. There are many possibilities for doing this, none of them scalable in an automated fashion, IMHO.
You could take the top 20% of those pages (based on traffic, conversions, revenue...) and really customize them by adding your own definitions and elaborating on the origin of the word, etc... Beyond that you'd probably see a decline in ROI.
-
Everett, yes that's correct. I will go ahead and follow up on what you said. I do still wonder what the best way would be to go about getting it indexed - if I wanted to do that in the future. If you could shed some light on how to go about that, I'd really appreciate it. Thanks so much in advance!
-
It appears that your definitions are coming from wiktionary.org and are therefore duplicate content. If you were providing your own definitions I would say keep the pages indexable, but in this case I would recommend adding a noindex, follow robots meta tag to the html header of those pages.
-
Hi Everett, I've been looking at the index for word definitions and there's so many pages that are very similar to each other. It's worth giving it a shot I think. If you can provide feedback please do. Here's the domain: http://freewordfinder.com. The dictionary is an addition to users who'd like to see what a word means after they've found a word from random letters. You can do a search at the top to see the results, then click through to the definition of the word. Thanks in advance
-
Ron,
We could probably tell you how to get those pages indexed, but then we'd have to tell you how to get them removed from the index when Google sees them all as duplicate content with no added value. My advice is to keep them unindexed, but if you really want them to be indexed tell us the domain and I'll have a look at how it's working and provide some feedback.
-
Hi Keri, did you think that the site might get penalized because it would in essence be duplicate content from another site? Even though the source is linked from the page? Please let me know your thoughts when you can
-
No they currently do not have additional information on them. They are simply better organized on my pages compared to the 3rd party. The unique information is what drives visitors to the site and from those pages it links to the definitions just in case they're interested understanding the meaning of a word. Does that help?
-
Do the individual pages with the definitions have additional information on them, or are they just from a third party, with other parts of the site having the unique information?
-
Hi Keri, thanks for your response. Well, I see what you're saying. The pages that show the definition pulled from the 3rd party are actually supplementary to the solution the site provides (core value). Shouldn't that make a difference?
-
I've got a question back for you that's more of a meta question. Why would the search engines want to index your pages? If all the page is doing is grabbing information from another source, your site isn't offering any additional value to the users, and the search engine algos aren't going to see the point in sending you visitors.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What's the difference between 'en-gb' and 'en-uk; when choosing Search engines in campaign set up?
Hi What's the difference search engine wise and which one should I choose, i presume GB since covers entire British landmass whereas UK excludes Ireland according to political definition, is it the same according to Google (& other engines) ? All Best Dan
International SEO | | Dan-Lawrence0 -
Pages with Duplicate Page Title
Blog - FDM Group has duplicate page title for all blog posts. We also have multiple localized versions of pages, so the titles are seen as duplicate. Possible resolutions? Thanks in advance.
International SEO | | fdmgroup0 -
Specific page URL in a multi-language environment
I've read a lot of great posts on this forum about how to go about deciding the best URL structure for each language that your site will support, so thank you to everyone that has provided input on that. I now have a question that I haven't really found answers/opinions on. When providing a page translation, should my content URL reflect that of the country I'm targeting or always remain the same across all sites? Below is an example using the "About Us" page. www.example.com/about-us/
International SEO | | Matchbox
www.example.com/es-mx/about-us/ -- OR -- www.example.com/about-us
www.example.com/es-mx/sobre-nosotros Thank you in advance for your help. Cheers!0 -
Redirect the main site to keyword-rich subfolder / specific page for SEO
Hi, I have two questions. Question 1: is it worthwhile to redirect the main site to keyword-rich subfolder / specific page for SEO? For example, my company's webpage is www.example.com. Would it make sense to redirect (301) the main site to address www.example.com/service-one-in-certain-city ? I am asking this as I have learned that it is important for SEO to have keywords in the URL, and I was thinking that we could do this and include the most important keywords to the subfolder / specific URL. What are the pros and cons of this? Should I create folders or pages just the sake of keywords? Question 2: Most companies have their main URL shown as www.example.com when you access their domain. However, some multi-language sites show e.g. www.example.com/en or www.example.com/en/main when you type the domain to your web browser to access the site. I understand that this is a common practice to use subdomains or folders to separate different language versions. My question is regarding subfolders. Is it better to have only the subfolder shown (www.example.com/en) or should I also include the specific page's URL after the subfolder with keywords (www.example.com/en/main or www.example.com/en/service-one-in-certain-city)? I don't really understand why some companies show only the subfolder of a specific language page and some the page's URL after the subfolder. Thanks in advance, Sam
International SEO | | Awaraman1 -
International Landing Page Strategy
Hello, I'm looking for some insight in an area that I don't have much experience in - hoping the community can help! We are a healthcare staffing company serving clients in the U.S. (www.bartonassociates.com). We are interested in attracting clients in Australia and New Zealand. I'm wondering if anyone as experience with best practices for doing so (both from an SEO and PPC perspective). Would it be best to purchase .au and .nz domains for these landing pages and link back to our US site for more information (or even recreate a modified version of our US site for .au and .nz). My concern here is duplicate content issues, among other things. Or, would it be better to create Australia and New Zealand focused landing pages on our US site and drive PPC there? My concern here is that we would never get organic traffic from Australia and New Zealand to our US site, in light of the competition. Also, the messaging would be a bit mixed if targeting all three countries. Our core term is "locums" and "locum tenens". Greatly appreciate any insight from you guys. Thanks, Jason
International SEO | | ba_seomoz0 -
Should product-pages with different currencies have different URLs?
Here is a question that should be of interest for small online merchants selling internationally in multiple currencies. When, based on geolocation, a product-page is served with different currencies, should a product-page have a different URL for each currency? Thanks.
International SEO | | AdrienOLeary0 -
Can I point some rel alternate pages to a 404?
Hi everyone, I'm just setting up a series international websites and need to use rel="alternate" to make sure Google indexes the right thing and doesn't hit us with duplicate content. The problem is that rel="alternate" is page specific, and our international websites aren't exact copies of the main UK website. We've taken out the ecommerce module and a few blog categories because they aren't relevant. Can I just blanket implement rel="alternate" and let it sometimes point to a 404 on the alternate websites? Or is Google going to find that a bit weird? Thanks,
International SEO | | OptiBacUK
James0 -
Getting A Sub Domain To Out-Rank The Main Domain
Hi, We have a prospective client who currently have a sub domain setup for each language, they all have the same content as the main domain. The problem is that the main domain is written in English (but not UK English), and they want the UK sub domain to outrank it (it's the other way round at the moment). Effectively, there are duplicate content issues here and as a result it looks like Google have chosen to keep the main domain (as it has more authority) and lower the UK sub results in its rankings. Is there a feature in webmaster tools where you can target subdomains to a location (I know you can do this with a main domain). Additionally, any other tips for the above would be greatly appreciated. Thanks in advance,
International SEO | | jasarrow0