Landing hero: feature vs. outcome headline
runningControl leads with the feature list. The challenger leads with the outcome and uses a higher-contrast CTA. Rewritten at the proxy, so each variant is its own route.
- Key
- landing-hero
- Mode
- rewrite on /demo/landing
- Primary goal
- signup_click
- Traffic allocation
- 100% of eligible visitors
- Targeting
- Everyone
- Split
- control 50% / outcome 50%
- Started
- Oct 3, 2026
- Ended
- n/a
Results
1,400 of 1,400 visitors are synthetic (seeded or simulated), not real traffic.
Not significant
The difference could plausibly be noise at α = 0.05. Keep running until the planned sample size, or accept there's no detectable effect.
| Variant | Visitors | Conversions | Rate | 95% CI (Wilson) | Uplift vs control | p-value |
|---|---|---|---|---|---|---|
| Feature headlinecontrol | 702 | 29 | 4.13% | 2.89% – 5.87% | baseline | |
| Outcome headline + orange CTAoutcome | 698 | 34 | 4.87% | 3.51% – 6.73% | +17.9% | 0.5042 |
Visitors are unique exposed visitors. Conversions are unique visitors who were exposed and then fired signup_click. p-value: two-sided, pooled two-proportion z-test against the control. To detect a 20% relative change from the current control rate you need about 9,437 visitors per variant.
Sample size calculator
Two-sided test, α = 0.05, power = 0.8. Same formula as Evan Miller's calculator.
9,437 visitors per variant
Detects a change from 4.10% to 4.92%. 18,874 visitors in total for two variants.
Simulate traffic
sandbox onlyGenerates synthetic visitors, bucketed with the same hash as real traffic. Each one converts with the true rate you set for its variant, so you can watch how long it takes for a real difference to become significant.