What’s A/B vs multivariate testing?

You want to improve your click rate, but you're not sure whether to change the subject line, the CTA button text, or the hero image. You could run three campaigns back-to-back. Or you might be tempted to test all three at once. The right choice depends on your list size and how much certainty you actually need from the results.

An A/B test sends two variants to equal segments of your list, changing exactly one element between them. Subject line A versus subject line B. One variable, one answer. Because you're isolating a single change, results are easy to read: whichever version drove more clicks won. You need a large enough segment to reach statistical significance, but the test itself works on a list of a few thousand subscribers and most ESPs support it out of the box.

Multivariate testing (MVT) changes multiple variables at once and measures every combination. If you're testing two subject lines, two CTA phrases, and two hero images, you've got eight combinations to evaluate. That requires a much bigger list (usually tens of thousands per combination) and more time to gather results. The payoff is that you learn how variables interact, not just which version of one element wins in isolation. Klaviyo, HubSpot, and most enterprise ESPs support MVT natively; most small-to-mid-sized platforms only support standard A/B.

And for For most senders, A/B testing is the right starting point. Test your subject line in one campaign, your CTA in the next, your send time after that. You'll build a reliable picture of what moves your click rate without needing a massive list. Save MVT for when you've already optimized the obvious variables and want to understand interaction effects. Pick one element you've always assumed works in your highest-volume campaign, write a contrasting version, and run your first A/B test this week. Then read up on how to interpret statistical significance before you call a winner.