Learn

A/B testing, learned by playing.

Learn what the numbers in an A/B test actually mean — plain English and pictures first, then the real calculations, done by hand. Each lesson takes one idea and gives you something to play with. Start at lesson one and watch a test lie to you.

The path · from zero
01
A/B testing basics: the statistics, without the formulas Interactive
Start here. What a test is, what a conversion rate really tells you, and the one fact that makes the whole thing need statistics. No formulas.
Start →
02
Why one test can lie to you Interactive
Run two identical variants and watch one "win" by pure luck. This is where the noise from Lesson 01 starts costing you money.
Start →
03
Significance and the p-value Interactive
What a p-value actually means, in plain English. See it as a shaded tail you can slide, not a formula to memorize.
Start →
04
Calculate a p-value by hand Drills
The nitty gritty. Four numbers, three formulas, a z-table — one worked example, then endless generated problems with checked answers. Bring a calculator.
Start →
Not written yet · four lessons live today
05
Power and sample size Next up
Why a weak test can miss a real winner, and how to work out how much traffic you actually need.
not yet
06
The winner's curse
Why the wins that squeak past the bar are almost always smaller than they first look.
not yet
07
Peeking, and why it manufactures winners
Checking daily and stopping on a good day quietly turns a 5% error rate into roughly 1 in 4. See it happen.
not yet
08
Bayesian vs frequentist
Two languages for the same data. What each one actually tells you, and why the choice is not what saves a small store.
not yet
09
SRM: is your platform even honest?
Sample ratio mismatch, tracking loss, and the one check that tells you whether to trust any of the above.
not yet

These five are not written yet. Until they land, the same ideas are covered in the blog and inside the tools themselves: the winner's curse in Reality Check, peeking in this guide, sample size in this one, and SRM in the Platform Validator.