What is correlation vs causation in email performance?
Still have a question, spotted an error, or have a better explanation or a source we should cite?
Correlation means two things move together. Causation means one thing directly caused the other. The distinction sounds academic but it comes up constantly in email analysis, and getting it wrong leads to optimizing for the wrong things.
Common correlations that aren't causation
Open rate and revenue. Campaigns with higher open rates often produce more revenue. But higher open rates don't cause the revenue. Both are usually caused by a third factor: a more engaged audience segment or a more relevant offer. Sending to a cold, unengaged list won't produce revenue even if you somehow forced open rates up.
Subject line length and performance. Short subject lines sometimes correlate with higher open rates. But the relationship isn't consistent across all senders and audiences. Shorter lines don't cause opens. The correlation, when it exists, is partly because shorter lines tend to be clearer and more direct. The clarity causes the improvement, not the character count.
Send day and open rate. Tuesday gets cited as the best day for email. The data that produced that figure came from averages across many senders and industries. Your audience may behave completely differently. Their engagement pattern and your specific correlation may be stronger on Thursday or Sunday.
How to distinguish them
The reliable way to establish causation in email is a controlled A/B test where you change exactly one variable. If open rate goes up after changing the subject line, and everything else (audience, send time, content) stayed the same, you have reasonable evidence that the subject line caused the change. Without that control, you have a correlation that might or might not be meaningful.
The practical rule: treat correlations as hypotheses, not conclusions. Test them before making permanent program changes based on them.
Contributors
Who worked on this answer
Every name links to their profile. Every company links to their site. Real people, real accountability.
