A/B Testing for Email Revenue

The typical A/B test: two subject lines, winner picked by open rate after 4 hours, sent to the full list. This approach has three problems.

Open rates are unreliable because of Apple MPP. The test window is too short for statistical significance. And open rate does not correlate with revenue.

A subject line that gets more opens but fewer clicks loses you money. You need to test the right things and measure by the right metric.

Content blocks within emails. Hero image, product grid layout, social proof format. These affect click-through and revenue directly. Typical lift: 10-30%.

CTA copy and placement. Button text, color, position (above fold vs below fold). Typical lift: 5-20%.

Send timing. Time of day, day of week, delay between flow emails. Typical lift: 5-15%.

Subject lines. Yes, still test them, but measure by click rate, not open rate. Typical lift: 2-8%.

Full flow variants. Different number of emails, different spacing, different branch logic. This is the highest-effort test but produces the biggest compound gains.

Minimum sample size: 1,000 recipients per variant. Below this, your results are noise.

Confidence threshold: 95%. Most ESPs show a "winner" at 80% confidence. That is a coin flip with better odds. Wait for 95%.

Test duration: at least 7 days for campaigns (to account for day-of-week variation), at least 14 days for flows (to accumulate enough recipients).

One variable at a time. If you change the subject line and the hero image, you do not know which caused the result. Isolate variables.

Configure your A/B tests to report revenue per recipient, not open or click rate. A variant that gets 3% fewer clicks but 12% more revenue is the winner.

Track revenue over a 7-day window from the email send. Same-day revenue misses delayed purchases.

If your ESP does not support revenue-based A/B testing natively, track it manually. Export recipient lists for each variant and match against purchase data.

Run one test per flow per month. Across five flows, that is five tests per month, 60 per year. Each test compounds. A 10% lift in five flows across 12 months multiplies.

Document every test: date, hypothesis, variants, sample size, confidence level, result, revenue impact. This becomes your playbook.

Review results quarterly. Look for patterns. If UGC consistently beats studio photos, apply that across all emails, not one flow.