What tools help with A/B testing?
Still have a question, spotted an error, or have a better explanation or a source we should cite?
You've decided to run an A/B test on your next campaign. Great. Now, which tool do you actually use? The honest answer is that it depends on where you are in your testing journey, and most senders start with something they already have access to.
Start with your ESP's built-in testing features. They cover the most common use cases without any extra cost or complexity. Mailchimp, Klaviyo, HubSpot, and Salesforce Marketing Cloud all include native A/B testing. You can split-test subject lines, sender names, send times, and in many cases entire content blocks. The winner gets sent automatically once enough data comes in. That's genuinely enough for most senders most of the time.
Use a statistical significance calculator when your ESP doesn't do the math for you. This matters more than people think. A result that looks like a win after 200 opens might completely flip after 2,000. Evan Miller's sample size calculator (evansimplified.com) is free and widely trusted. Optimizely also offers a free stats engine. These tools tell you whether your result is a real signal or just noise. (If you're not sure what statistical significance means in the context of email, it's worth reading up on running a statistically valid test before you start pulling conclusions.)
Graduate to dedicated platforms when your needs outgrow your ESP. Tools like Optimizely and VWO are built for teams running complex, multi-variable experiments across channels, not just email. They're powerful, but they come with a learning curve and a price tag. Most email-only testing programs don't need them.
Track what happens after the click with analytics. Your ESP tells you who opened and who clicked. Google Analytics (with UTM parameters on your links) tells you what those clicks actually converted into. That downstream data is what separates a real test from a vanity metric exercise.
The practical path is this. Use your ESP's built-in tools to test the things that matter most, like subject lines, CTAs, and send times. Layer in a significance calculator so you know when a result is actually meaningful. Add Google Analytics so you're measuring revenue, not just opens. That stack costs nothing extra and handles 90% of what most teams need.
If you're unsure whether your current setup is giving you reliable test results, feel free to ask us. We're happy to take a look.
Contributors
Who worked on this answer
Every name links to their profile. Every company links to their site. Real people, real accountability.
