← Home

Survival Curves

SRV
03 — Time-to-Conversion Analysis
// why measure time, not just rate?

A conversion rate is a photo finish that ignores the race. Two variants can end at the same 2.1% CVR and still be very different businesses: in one, buyers commit within two days; in the other they hesitate for nine. Faster conversion means a shorter consideration cycle, earlier cash, and less time for a competitor or an abandoned tab to steal the sale. Whether people convert is half the answer. When they convert is the other half.

The method comes from medicine. Survival analysis was built for drug trials, where "did the patient survive" means nothing without "for how long". Kaplan and Meier published the technique in 1958 and it became one of the most cited statistics papers ever written, because it handles the observations every other method throws away: the ones cut short by time. An ecommerce test window is exactly that situation. A visitor who hasn't bought yet is not a no. They are a not-yet, and standard CVR math cannot tell the difference.

Enter your cohort sizes and cumulative conversions per day, and this tool draws the Kaplan-Meier curves, runs the log-rank test, and gives you the median time to convert and the hazard ratio. If your variant converts the same number of people but gets them there faster, this is the only chart that will show it.

"Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise."

JOHN TUKEY — statistician, Bell Labs & Princeton

01 — Experiment Setup

Enter the total number of unique visitors assigned to each arm for the duration of the experiment. Then enter cumulative conversions per day below — how many of that original cohort had converted by each day.

02 — Cumulative Conversions by Day
Day Control — cumulative conversions Variant — cumulative conversions
03 — Survival Curves (non-conversion rate over time)

Each curve shows what fraction of the original cohort has not yet converted. A curve that drops faster means visitors are converting sooner. Separation between curves indicates the variant is changing conversion speed — even if final rates converge.

Control (A)
Variant (B)
04 — Statistical Results
Enter cohort sizes and at least 3 days of data to see results.
Conversion Speed Score
—
No data
Analysis Checklist
○Sufficient data
○Log-rank significance
○Curve separation
○Monotone separation
○HR excludes 1.0
Median Time-to-Convert
Control (A)
—
days
Variant (B)
—
days

Day by which 50% of converters have converted. Only shown if the survival curve crosses 0.5.

// the math — every calculation this tool runs, in full

Kaplan-Meier survival estimator

For each day, events d = new conversions that day, and n = users still "at risk" (haven't converted yet). Survival probability multiplies down step by step:

S(t) = Π over days i ≤ t of ( 1 − dᵢ / nᵢ )  ·  S(0) = 1

S(t) reads as "the probability a visitor has not yet converted by day t". The curve only drops on days with conversions, which is why it's a step function. Cumulative inputs are converted to daily events internally (dᵢ = cum(i) − cum(i−1)), and the at-risk pool shrinks as users convert.

Log-rank test

For each day with any conversions, we ask: if both arms converted at the same underlying rate, how many of that day's d events would we expect from arm A given its share of the at-risk pool?

E_A = d · n_A / (n_A + n_B)  ·  V = n_A·n_B·d·(n−d) / (n²·(n−1))
χ² = ( Σ(O_A − E_A) )² / ΣV  ·  df = 1

Summing observed-minus-expected over all days gives a single test of whether the two curves differ, using every day's information rather than just the endpoint. This is the standard test used in clinical trials.

Hazard ratio

HR = (O_B / E_B) / (O_A / E_A)  ·  95% CI: exp( ln HR ± 1.96·√(1/O_A + 1/O_B) )

HR > 1 means the variant converts faster at any given moment; HR = 1.15 reads as "a not-yet-converted variant visitor is 15% more likely to convert right now than a control visitor". The CI uses the standard log-scale approximation. Interpretation assumes proportional hazards — if the curves cross, the tool warns you and the single HR number becomes unreliable.

Median time-to-convert

The first day where S(t) crosses 0.5 among eventual converters, linearly interpolated between the two surrounding days. If the curve never reaches 0.5 inside the observation window, the median is reported as "> observation window" rather than guessed.