Magento Core_URL_Rewrite Problems
-
Hi Everyone,
We are currently caught between a rock and a hard place with Magento and are wondering if anyone else had similar problems and could share their advice.
Our Core_URL_Rewrite now containt 1.3 million records for an account that has 12000 products on 4 different store views. This has ballooned past the point that we are no longer able to reindex our URL Management.
The option that is being suggested to us is to truncate the table and start over, though this will essentially kill our SEO for those pages.(Which as there are duplicates, I can only imagine how much they are going to be penalized by it)
Would anyone have any advice other than truncating and starting over?
Any advice would be greatly appreciated.
Thanks!
-
Hi,
I found the exact problem you are facing with a solution on this link
http://magento.stackexchange.com/questions/17553/magento-core-url-rewrite-table-excessively-large
There are patch codes available on this link, however do read this reply on this page
Bugs in earlier (and possibly current) versions of Magento is one. Another is there's logic in this table that tries to track changes to the URL key value so that 301/302 rewrites are setup for old products. Because of this, and complicating things, truncating the table and regenerating may make existing URL rewrites go away, and this will have an unknown effect on your search engine listing (not necessity bad, just hard to predict).
My general advice to clients who ask is
-
Leave the giant growing table as is if you don't have a good handle on your URL/SEO situation
-
Until the table size starts being a problem (generating site maps, for example). When that happens, get a handle on your URL/SEO situation.
-
Once you have a handle on your URL/SEO situation, backup the table, then truncate the table and regenerate. Address any URL/SEO problems caused by the truncating.
-
Automate step 3
Trying to fix this on the Magento code level is admirable, but you'll be swimming upstream. Sometimes it's better to accept that "That's just Magento being Magento", and to solve the problem with and external process.
I hope this helps, if you have further questions, then post a response, I will be happy to answer.
Regards,
Vijay
-
-
I'm not sure the answers previously presented are related to the issues you're having. Having worked with Magento for a long time, this can be an issue that occurs over and over again.
To answer your initial question, truncating your core_url_rewrite table will remove all of these URLs, but it'll only delay the problem until it reoccurs again in the future (unless you've had a problem in the past which has been rectified). You're also correct in that any rewrites in the system previously there will disappear, so you'll probably end up with a lot of crawl issues appearing in Search Console.
Your best move would be to find out why you have so many URLs in there in the first place. Do you have a huge product catalog with multiple stores? Or is this something to do with an issue in your Magento version or some setup issues. The most common time this usually occurs is if two products get added to your site with the same URL Key. Every time the reindex process runs, your core_url_rewrite table will grow. You could check this by looking at the number of rows in the table, reindexing the site and if it grows further, then it's likely to be the problem. The quickest way to fix this is to ensure all URL key are unique.
There's also an article here about duplicate keys - https://firebearstudio.com/blog/magento-url-reindex-core_url_rewrite-duplicates-patch.html - this should hopefully clear the issue.
I hope this helps! If it doesn't solve the problem, then sending over a little more information around the number of stores, catalog site and the split between system generated URL rewrites and custom URL rewrites would be great so we can try to help further!
Thanks,
Lewis -
This is an issue to to set-up. When you set up multiple ecommerce websites on Magento as 'Stores', then all SKUs will load on other domains. if they were set-up as 'Websites' then this would alleviate the issue. However, with Stores you are able to share shopping carts (i.e. Add a product from website A and checkout on website B).
What I did was turn off the XML cron jobs and set-up cross-domain canonicals. Also make sure your session IDs (/?SID=) are working properly. Not sure if this solves the technical issues, but should help clear up dupe content.
-
Is it creating a new url for each option (size, color, etc) as well as what page it shows up on or other various sort orders (by price, by size, etc.) and session id's that you could exclude? Are you sure they are truly duplicates?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Magento missing SEO fields?
Hi Guys, Have a client who's blog is combined with there e-commerce site which is on Magento 2 but there are no SEO fields to add meta title, description. It only has the ability to add h1 tag see: https://cl.ly/3c6897aa5132 We want to add this ability to add meta data like this: https://d.pr/i/8GdbDc Does anyone know how to do this? Cheers.
Intermediate & Advanced SEO | | brandonegroup0 -
Is a rootserver that has previously been used for porn and spam a problem for SEO?
As we found out via Alexa we are hosting our website on a server that has been used under the same IP heavily mostly for pornsites. Will this become a problem for our webproject? By the way: We are not in the porn-sector. We are an NGO with an alternative social media project for a better world. Thanks for your help.
Intermediate & Advanced SEO | | RWW0 -
Magento Store Using Z-Blocks - Impact on SEO?
Hi Guys, I have a question relating to Z-Blocks in Magento. Our Magento store uses a lot of Z-Blocks, these are bits of content that are switched off and on depending on a customer’s user group. This allows us to target different offers and content to new customers (not logged in) and existing customers (logged in). Does anyone have any experience in how this impacts SEO? Thanks in advance!
Intermediate & Advanced SEO | | CarlWint0 -
Is it a problem if a URL has too many backslashes in its address?
The ecommerce platform of the site that I am working on generates URLs that contain ID Codes for each different product category, color variations, styles, etc. An example of a URL for a specific product includes: www.example.com/women/denim-jeans/py/c/109/np/108/p/3834.html Is it a problem for search engine crawlers if the URL address has so many backslashes in its address? Appreciate your feedback.
Intermediate & Advanced SEO | | SEO_Promenade0 -
Magento SEO firm
I'm looking for an SEO company that has substantial experience with the Magento shopping cart system. I've gone thru MOZ.com's Recommended List but I'm unsure of who specializes in Magento. Thanks.
Intermediate & Advanced SEO | | UncleXYZ0 -
Any problems with two sites by same owner targeting same keyword search?
I have a site, let's call it ExcellentFreeWidgets.com. There is a page on the site that is very popular and we'll call the page title, "Big Blue Widget." That page is currently #1 for the search "big blue widget." This week, I was able to buy the exact match domain for that page, we'll call it BigBlueWidget.com. I want to build a site on BigBlueWidget.com to better capitalize on that search "big blue widget," which is huge. The content would not be the same wording at all, but it would be the same subject. It would probably be a five page or so website, all about Big Blue Widgets: what they are, where to get them, etc. The sites will not reciprocally link to each other. New new site, BigBlueWidgets.com, would link to the existing site, ExcellentFreeWidgets.com. The new site and the current page will compete for position in the SERPs. Here are my questions to you experts: 1. Will Google care at all that the same entity owns both sites, or will just just rank for the term as they normally would. 2. I am not sure I'll run Adsense on the new site or not. I will be pointing a link back my ExcellentWidgets.com site from a button that says, "Get an Excellent Widget." But if I do run Adsense on it, does Google Adsense care that the same entity has a site and another site's page that are competing for the same term that both have Adsense add on them? Note: I do not want to start a new entity for the new site (I'm in CA and LLC's are $800/year) as it's probably not worth all that hassle and money. Thank you so much. I hope the that obfuscating the real domain names did not confuse the issue too much.
Intermediate & Advanced SEO | | bizzer0 -
Duplicate Title - Magento Products / Kunena Forum - Nofollow vs. Follow
Hello Mozzers! This is my first post so hopefully I can explain this in a way that is clear! I just signed up for my PRO membership and wanted to finally start hacking away at this companies site that I work for. I ran my report and noticed my Crawl Diagnostics: 715 Dupliate Page Titles I noticed that a lot those were coming from the customer login in magento. So I added the: User-agent: *Disallow:/customer/ as mentioned in this link: http://stackoverflow.com/questions/10742052/duplicate-content-issues-for-login-page Any suggestions on that? Also, We have a Kunena forum on our site and have noticed all of the nofollow links. People say I should change them to follow links, but other people say leave them or I will get penalized. Any advice would be greatly appreciated!
Intermediate & Advanced SEO | | NRMWEBWORKS0 -
We will be switching our shopping cart platform from volusion to magento and really cautious / nervous about our rankings / seo stuff. any advice for anyone that has migrated stores, etc. these urls are years old, etc.
shopping cart platform switch and SEO. What do you suggest? What's the best way to ensure we keep rankings.
Intermediate & Advanced SEO | | PaulDylan0