A/B Tests: How To Verify Difference In Average Order Value?
-
Hi there! When the data from an A/B test shows a difference in AOV between the variants, how do you determine if this difference is statistically probable or not? Best Regards, Martin
-
Thank you, David.
-
Perhaps this will get your on the right track, from http://flintanalytics.com/how-to-determine-if-you-should-advertise-higher-priced-goods-to-mac-users/
Average Order Value Statistical Significance
To do this you need to download the transaction data for both variants, because you need to be able to determine the variance in order value. You can download this data by going to conversions > ecommerce > transactions in Google Analytics. Once there, Google only lets you export 500 rows of transactions at one time, so to keep it simple I ensured my date ranges were under 500 transactions.
Once you get the order values for each set of customers in Excel you need to perform a t-test so that you can compare the means of the two groups. This t-test will give you the p-value that you will use to determine if the difference in average order values between the groups is significant. Yout formula in excel should look like the following:
=t.test(array of group 1, array of group 2,two-tailed distribution, two-sample equal variance homoscedastic)
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Different statistics in Google Analytics
Hi! A client of mine is having some trouble on his Google Analytics overview. When he checks the yearly report with number of visitors, new visitors, returning, ... he gets statistics that doesn't match mine. Also, he gets a notation from Google that no hits were fired, although the script is still there. On the other hand, I (logged in with another account) am seeing all statistics correctly. FYI: no filters were installed, nor segments, ... Why does the client see different and incorrect statistics? There should be a setting or something I'm overseeing. Hope you guys can help.
Reporting & Analytics | | conversal0 -
How can I distinguish new visitors from existing (customers) in Google Analytics to attain an avg # of new visitor traffic per day/week?
i do marketing for a business software site where we have hundreds of clients and each account has on avg 100 users. I am having a very challenging time to attempting to figure out the real number of unique traffic that our site receives. **(what's creating the issue is that we have thousands of user accounts where our users log-in via our site to access our app/platform). Would love help with this! Christian
Reporting & Analytics | | Sundance_Kidd0 -
Changing UX for different types of visitors
If we want to change the UX for different types of visitors (ex: direct, google PPC, google organic, bing organic, etc.) what is the best way to do it? With Universal Analytics, where can we find the source and medium?
Reporting & Analytics | | AMHC0 -
Strange Spike in Direct / None traffic
Over the past week or so, my client's Australian personal training website has experienced a dramatic spike in Google Analytics sessions (see attached screenshot). All the visits are coming from various states in the US and via the "Direct / None" source. All the visits are less than 1 second in duration so I'm assuming it's coming from some sort of automated bots. I'm worried for a couple of reasons: A) Could somebody be deliberately spamming the site to adversely affect our rankings? B) How do I get rid of this traffic from our analytics reports? 7kwsJnB
Reporting & Analytics | | Dave_Eddy0 -
Index.php and /
Hello, We have a php system and in the MOZ error report our index.php shows up as a duplicate for / (home page). I instituted a rel canonical on the index.php because the / gets better rank than the other. This said, the error report through MOZ still shows them as duplicates. Should I be using a 301 instead? Please help! Also, I would love a good technical SEO book (for bridging the gap between SEO and programmer) if someone can recommend one? Thanks in advance!
Reporting & Analytics | | lfrazer0 -
Why seomoz shows me "missing meta discription" on this plugin: http://villasdiani.com/wp-content/plugins/dopbsp/frontend-ajax.php ? Should I edit? how?? IIs it posible??
Good day to all! I am very confuse about results on seomoz Crawl Diagnostics Summary, especially with 6 Crawl Warnings Found. It says title too short: http://villasdiani.com/category/mombasa/, http://villasdiani.com/category/watamu/ , http://villasdiani.com/sitemap/ why would Google punish me for this??? why should I make longer title for sitemap? or Watamu? It is the name of the place - Watamu or Mombasa. It is very confusing for me. I have very big mess with the website and it is not ranking:-( what I have done:-( and is it possible to meta description for this plugin: http://villasdiani.com/wp-content/plugins/dopbsp/frontend-ajax.php how?? I even do not know where is it.
Reporting & Analytics | | VillasDiani0 -
SERPS different based on location of search even with non-personalization
Hello Mozzers, Our agency's website, www.kenta.ro - ranked for a long time at #1 for "ann arbor seo" and similar keywords. For the past several (4-5) months we've been sitting around #5. My guess was that this was Google playing around with the results but I'm not sure why we have been at this position for such a long time. I have a vpn that I use for checking rankings overseas and if I connect to a server in Chicago, LA, Ontario, etc we show up as #1 - only when you search for "ann arbr seo" in our area do we get a lower ranking. All rank checking programs including seomoz show us at the #1 position because of this. What this means for us is that all of the traffic we target with this keyword sees the poor result, while the rest of the world sees the great result (should they search for it). How can we ensure that our target market finds us at #1 like the rest of the world does? Thank you in advance. weabi.png
Reporting & Analytics | | kentaro-2569290 -
Seeing / and no / in Google Analytics
When I look at my Google Analytics code, I see that the same content will show up twice, once with a / at the end of the URL, and once without. My site is at http://www.skypeenglishclasses.com. Does anyone know why this would happen? Is it duplicate content?
Reporting & Analytics | | Paulguy0