Magento Core_URL_Rewrite Problems
-
Hi Everyone,
We are currently caught between a rock and a hard place with Magento and are wondering if anyone else had similar problems and could share their advice.
Our Core_URL_Rewrite now containt 1.3 million records for an account that has 12000 products on 4 different store views. This has ballooned past the point that we are no longer able to reindex our URL Management.
The option that is being suggested to us is to truncate the table and start over, though this will essentially kill our SEO for those pages.(Which as there are duplicates, I can only imagine how much they are going to be penalized by it)
Would anyone have any advice other than truncating and starting over?
Any advice would be greatly appreciated.
Thanks!
-
Hi,
I found the exact problem you are facing with a solution on this link
http://magento.stackexchange.com/questions/17553/magento-core-url-rewrite-table-excessively-large
There are patch codes available on this link, however do read this reply on this page
Bugs in earlier (and possibly current) versions of Magento is one. Another is there's logic in this table that tries to track changes to the URL key value so that 301/302 rewrites are setup for old products. Because of this, and complicating things, truncating the table and regenerating may make existing URL rewrites go away, and this will have an unknown effect on your search engine listing (not necessity bad, just hard to predict).
My general advice to clients who ask is
-
Leave the giant growing table as is if you don't have a good handle on your URL/SEO situation
-
Until the table size starts being a problem (generating site maps, for example). When that happens, get a handle on your URL/SEO situation.
-
Once you have a handle on your URL/SEO situation, backup the table, then truncate the table and regenerate. Address any URL/SEO problems caused by the truncating.
-
Automate step 3
Trying to fix this on the Magento code level is admirable, but you'll be swimming upstream. Sometimes it's better to accept that "That's just Magento being Magento", and to solve the problem with and external process.
I hope this helps, if you have further questions, then post a response, I will be happy to answer.
Regards,
Vijay
-
-
I'm not sure the answers previously presented are related to the issues you're having. Having worked with Magento for a long time, this can be an issue that occurs over and over again.
To answer your initial question, truncating your core_url_rewrite table will remove all of these URLs, but it'll only delay the problem until it reoccurs again in the future (unless you've had a problem in the past which has been rectified). You're also correct in that any rewrites in the system previously there will disappear, so you'll probably end up with a lot of crawl issues appearing in Search Console.
Your best move would be to find out why you have so many URLs in there in the first place. Do you have a huge product catalog with multiple stores? Or is this something to do with an issue in your Magento version or some setup issues. The most common time this usually occurs is if two products get added to your site with the same URL Key. Every time the reindex process runs, your core_url_rewrite table will grow. You could check this by looking at the number of rows in the table, reindexing the site and if it grows further, then it's likely to be the problem. The quickest way to fix this is to ensure all URL key are unique.
There's also an article here about duplicate keys - https://firebearstudio.com/blog/magento-url-reindex-core_url_rewrite-duplicates-patch.html - this should hopefully clear the issue.
I hope this helps! If it doesn't solve the problem, then sending over a little more information around the number of stores, catalog site and the split between system generated URL rewrites and custom URL rewrites would be great so we can try to help further!
Thanks,
Lewis -
This is an issue to to set-up. When you set up multiple ecommerce websites on Magento as 'Stores', then all SKUs will load on other domains. if they were set-up as 'Websites' then this would alleviate the issue. However, with Stores you are able to share shopping carts (i.e. Add a product from website A and checkout on website B).
What I did was turn off the XML cron jobs and set-up cross-domain canonicals. Also make sure your session IDs (/?SID=) are working properly. Not sure if this solves the technical issues, but should help clear up dupe content.
-
Is it creating a new url for each option (size, color, etc) as well as what page it shows up on or other various sort orders (by price, by size, etc.) and session id's that you could exclude? Are you sure they are truly duplicates?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SPA (angularJS) Ranking Problem
Hello dear experts; I working on SPA website with More than 20,000 indexed pages by Google there is a problem that I involved with: 1- Some pages had a good ranking on google.com but from a week ago, ranks one by one, dropped and homepage Replaced. for example, URL: https://cafegardesh.com/tours-dubai removes and https://cafegardesh.com replaced. Google still see the page when I use Fetch as Google tool. I can't understand what happens Is there anyone can find my site problems?
Intermediate & Advanced SEO | | cafegardesh0 -
Weird indexing problem - Can it be solved?
Hi Been building and optimising sites for 15 years and this is one of the hardest problems I ever came across. So any help would be very much appreciated. Here we go: For some mysterious reason this URL http://weekend.visitsweden.com/no/ has been indexed as http://weekend.visitsweden.com even if we tried all we can to correct it. The problem is that since the latter points to the first URL with a 301 it refuses to get any page rank. Also it does not get visible in Google at all. Just a recap of what we have tried so far: Add site to webmaster tools Add proper sitemap.xml Add 301 redirect to the correct URL An easy way to locate the problem is to search for the main content of the site. As you can see it returns the wrong URL and the correct URL does not even get listed. Again, any help is very much appreciated. Kind regards Fredrik
Intermediate & Advanced SEO | | Resultify0 -
Xml sitemap only shows up sometimes (magento)
Hi Moz community, I'm using Magento platform. I can generate a sitemap using their xml generator, but it will only pull up sometimes in web explorers, the rest of the time it will show a 404 page. GWT also tells me that I get a 404 error when testing the sitemap, but sometimes it will acknowledge that it's there. Anyone had this problem before or know how to help. sitemap= www.ice.com/sitemap.xml Let me know what other information I can provide to help. Thanks!
Intermediate & Advanced SEO | | IceIcebaby0 -
Negative seo problem
Hello, Someone attacked our website with negative SEO and our website fell drastically. If i use bing webmaster tools link explorer i see dozens and dozens spam links but if open a link i don't find this link in that website. Is it possible that someone added the links and then remove thouse links when we got hit? Or is it possible to hide the links that they don't show up on webpage but they are still there? How can i use Google disavow links tool if there is no links in thouse websites but Bing link explorer shows spam domains pointing to our website. BR, T
Intermediate & Advanced SEO | | otsinguturundus0 -
Is it a problem if a URL has too many backslashes in its address?
The ecommerce platform of the site that I am working on generates URLs that contain ID Codes for each different product category, color variations, styles, etc. An example of a URL for a specific product includes: www.example.com/women/denim-jeans/py/c/109/np/108/p/3834.html Is it a problem for search engine crawlers if the URL address has so many backslashes in its address? Appreciate your feedback.
Intermediate & Advanced SEO | | SEO_Promenade0 -
Potential problems with my site
Dear Mozzers I hope you can help me with the following problems: My site is up and running for a year now and may be there has been problem with the homepage, because it ranks on first page for a competitive keyword on Google.com and Google.com.au only, however in other countries it just shows up as internal page and does not rank well. google.com/Google.com.au: homepage ranks top 10 (example.com)
Intermediate & Advanced SEO | | SteveTran2013
Other countries (.co.uk, .ca..ect) an internal page shows up, example.com/internalpage.html - shows on page 3-4. I can not find the homepage of example.com anywhere around top 1000. Can you please tell me what are the potential problems. Thank you very much. BR/Tran0 -
Title tags with >70 characters but most important words at start. Is this really a problem?
Is there in fact any kind of negative impact having title tags longer than 70 characters, as long as I place the most important keywords at the start and make sure that title still is compelling when cut somewhere around 70 characters? Are the additional words after the 70 characters limit just ignored? May additional words dillute the strength of the first words or may they even be helpful ? Any experience or any studies you know about impact of longer title tags? Or any statement from google about it?
Intermediate & Advanced SEO | | lcourse0 -
SEO value of Articles, Magento vs. Wordpress?
Our e-commerce site is running on the Magento platform while the blog for the site is integrated and is on the Wordpress platform. The blog is not on a separate subdomain, so it is on www.website.con/blog. What I wonder is how Google treats information on a Wordpress blog compared to pages created in the Magento CMS pages. Would a high quality content article posted as a blog post on the Wordpress have a lower SEO value than a page on the Magento? Thanks.
Intermediate & Advanced SEO | | ikomorin0