SEO for an exponentially growing site?
-
Hey Mozers!
I was having a quick chat with a friend the other day on doing SEO for a site that grows in page size at an exponential rate and was just wondering how you would go about optimizing it?
The example that we used would be a site that allowed users to upload videos and then have people vote on two videos against each other. So, if there are 100 uploaded videos and each of them are pared up with the other 99 to create a unique voting/battle page which has it's own unique URL, the site can get very large, VERY quickly. Meaning if just one more video is uploaded there would be
How exactly would you go about optimizing the site?
My biggest area of confusion would be generating sitemaps. I'm aware of best practices with large sitemaps (i.e. having a sitemap of sitemaps, not going over 50k in entries per sitemap etc..) But, how would you go about creating the sitemaps for this website if it's growing at an exponential rate, if at all?
If you have any other questions feel free to ask and I'll clarify it.
Thanks!
**TL;DR How would you optimize a site that grows at an exponential rate? **
-
Sorry, I wasn't very clear in what I initially responded back with.The code that I use doesn't create a new sitemap but rather just adds an additional node to the current one whenever a new page is added, so yes I am using the type of sitemap code that you are referring to.
BUT, since the site grows at an exponential rate it means that for every one video uploaded it will be matched up with each and every other single video to create a unique URL for each combination.
As an example if there are 10,000 videos currently on site which means there are 2 sitemaps right now and ONE more video is uploaded that means that 10,000 NEW unique urls/pages will be created thus creating 2 additional sitemaps resulting in a total of 4 sitemaps because of one new video being uploaded.
Let me know if I need to clarify anything.
-
Hello Jordan,
I am not sure what your using, but generally just because a video was uploaded to a site doesn't mean that it creates a new sitemap, for example in WordPress if you upload a video, it will get added to an existing sitemap. So when a search engine such as Google crawls it again, it will see the updated sitemap. So a automatic sitemap should work fine for you, unless your using a system that generates a new sitemap for everything uploaded, I have never heard of that, but I guess it's possible. If that's the case I am not sure at all. Very sorry I couldn't be of more assistance to you.
Have a great day and a happy holidays.
MB
-
Automatically generating sitemaps wasn't my concern it was more if you have 2 sitemaps worth of videos and then a user uploads just one more video that will end up creating an entire additional sitemap just because one more video was uploaded.
Thoughts?
-
Just as a note I would use dynamic meta tags that pull from titles, descriptions, for your meta info for each page so you don't run into duplicate meta issues. As for the sitemaps, that's a great question, I know there are many automated systems you can plugin to your site to auto generate sitemaps. I wish I knew more about them. I personally use WordPress for just about everything now, because it's easy to use and it has 1000's of useful plugins.
Have a great night.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Entities and SEO
How do you find the correct words (entities) to explain an entity ? The words (entities) that go together within a sentence seem to be based on the specific corpus (the keyword you want to rank ) and when they are million of results it seems impossible to find what word / entity is going to explain the entity / concept I want to explain. It seems that I got a better chance at the lottery 🙂 a
Intermediate & Advanced SEO | | seoanalytics
Do you have any advice or software that could parse and find the words that co-occure the most often out of millions of results !! Thank you,0 -
When Mobile and Desktop sites have the same page URLs, how should I handle the 'View Desktop Site' link on a mobile site to ensure a smooth crawl?
We're about to roll out a mobile site. The mobile and desktop URLs are the same. User Agent determines whether you see the desktop or mobile version of the site. At the bottom of the page is a 'View Desktop Site' link that will present the desktop version of the site to mobile user agents when clicked. I'm concerned that when the mobile crawler crawls our site it will crawl both our entire mobile site, then click 'View Desktop Site' and crawl our entire desktop site as well. Since mobile and desktop URLs are the same, the mobile crawler will end up crawling both mobile and desktop versions of each URL. Any tips on what we can do to make sure the mobile crawler either doesn't access the desktop site, or that we can let it know what is the mobile version of the page? We could simply not show the 'View Desktop Site' to the mobile crawler, but I'm interested to hear if others have encountered this issue and have any other recommended ways for handling it. Thanks!
Intermediate & Advanced SEO | | merch_zzounds0 -
Linking from a corporate site to a brand site.
Is there an SEO impact to a large corporation linking from a corporate and/or a divisional site to a specific brand site with it's own top level domain? We would like to keep the traffic coming, but not if it will be seen as a black hat tactic. My guess is that Google will be smart enough to see that the corporation owns the brand and at least not penalize us, but I am wondering if anyone else has this experience? Google Analytics is calling it self-referral.
Intermediate & Advanced SEO | | mrbobland0 -
SEO and former site
Hi, my client had a site built and hosted with Avvo but we now shut it down and are using a new server. My concern is that Avvo's internal link structure is causing SEO issues. For example, his site will list for "San Diego Criminal Defense Attorney", but is then removed for no reason. Far worse, while he had the AVVO site, it would never rank at all on Google. He's got great content, and no spammy links. This is the site: www.thesandiegocriminallawyer.com. Any thoughts of what I could do to disavow the AVVO pages that Google still has indexed? Does it matter? Or, is it simply a function of time? Thank you for your help.
Intermediate & Advanced SEO | | mrodriguez14400 -
SEO Priorities for Ecommerce Sites
Hello All! What is the best way to rank SEO tasks by PRIORITY for Ecommerce sites to improve?? It can be quite overwhelming with all the types of projects/tasks needed to improve organic rankings... How would you rank the most CRITICAL tasks to spend the MOST TIME on to the tasks you spend less on. Appreciate your input in advance 🙂 Thank you! Mark
Intermediate & Advanced SEO | | wickerparadise0 -
It appears that Googlebot Mobile will look for mobile redirects from the desktop site, but still use the SEO from the desktop site.
Is the above statement correct? I've read that its better to have different SEO titles & descriptions for mobile sites as users search differently on mobile devices. I've also read it's good to link build, keep text content on mobile sites etc to get the mobile site to rank. If I choose to not have titles & descriptions on my mobile site will Google just rank our desktop version & then redirect a user on a mobile device to our mobile site or should I be adding in titles & descriptions into the mobile site? Thanks so much for any help!
Intermediate & Advanced SEO | | DCochrane0 -
Best way to SEO crowdsourcing site
What is the best way to SEO a crowdsourcing site? The websites content is entirely propagated by the user
Intermediate & Advanced SEO | | StreetwiseReports0 -
SEO Reco?
We're looking for a recommendation for a very good SEO agency that has experience with link building (white hat only). Any suggestions?
Intermediate & Advanced SEO | | BruceMillard0