AB+ Test Calculator by CXL

One test calculator to answer all your pre and post test analysis questions:

Test analysis
  • Does the test variant beat the original?
  • Does the test have the needed sample size?
  • Does the test have the needed duration?
  • What is the monetary ROI of the test variant?
Pre-test analysis
  • What is the minimum detectable effect I can test for?
  • How long should I test for considering the effect?
  • What's the sample size for a test on this page?
  • Does confidence level affect my test duration?

Pre-Test Analysis

Test Analysis

Pre-Test Analysis

Test Analysis

  • Does the test variant beat the original?
  • Does the test have the needed sample size?
  • Does the test have the needed duration?
  • What is the monetary ROI of the test variant?
  • What is the minimum detectable effect I can test for?
  • How long should I test for considering the effect?
  • What's the sample size for a test on this page?
  • Does confidence level affect my test duration?

To explore calculator without real test data: click here to add dummy test data

1 How long has the test been running?

Test duration

Additional days needed
---
2 Test data

Control

Conversion rate
---

Variation 1

Conversion rate
---
Lift
---
Extra monthly transactions
---
Monthly monetary contribution
---
P-value
---
×
+ Add Variant
3 Test statistic details and sample size and duration calculators (unlock from test data to use as stand alone tools)

Sample size calculator

%
%
%
Required sample size per variant .
---

Duration calculator

%
%
How long in total to run the test
---

Monthly monetary contribution based on data above

Minimum detectable effect (MDE)

Week MDE
1 ---
2 ---
3 ---
4 ---
5 ---
6 ---
4 Results

Results

Significance calculation method:
Bayesian
Z Test
Confidence interval

Verdict:

Confidence interval

Verdict:

Revenue Impact (6 months)
Expected revenue forecast
Control:
Variant 1:
Probability of variant 1 winning:
Expected risk*:
*Average expected deviation between control and variant performances based on 95% confidence interval. Smaller the better.