How to compare seed test results over time?
Still have a question, spotted an error, or have a better explanation or a source we should cite?
You've run seed tests for three weeks and you've got data. One week, 88 percent of emails landed in the inbox. Next week, 84 percent. This week, 90 percent. Did you improve? Are you sliding backward? Is this just noise? You can't tell unless you're measuring the same thing the same way each time.
Seed testing only works as a comparison tool if your methodology is locked down. That means same seed list composition every single time. Same sending time of day. Same content type (promotional, transactional, newsletter, whatever). Same day of the week, ideally. Same IP. Same sending domain. If you change variables between tests, you're not comparing apples to apples.
Here's what you actually track: inbox percentage by provider. That's your headline metric. Gmail inbox placement this week. Outlook inbox placement this week. Yahoo inbox placement this week. Also track spam percentage by provider and "missing or other" percentage. These three add to 100 percent.
Why percentages and not raw counts? Because your seed list size might shift slightly. Percentage removes that noise. You want to see if placement rate is improving, not whether you tested with 47 seeds or 50 seeds.
Set a testing cadence and stick to it. Weekly is ideal. Bi-weekly works too. Monthly is probably too long because you won't catch trends fast enough. Every seven days, same day, same time, send the same seed list and record results.
Chart it. Seriously. Put your inbox percentages in a spreadsheet. Plot Gmail inbox percent on one line, Outlook on another, Yahoo on another. Look at the slopes. Are they going up? Down? Flat? A flat line for two weeks then a jump up means something you did is working. A downward trend means you've got a new problem.
What counts as real improvement versus noise? If you're bouncing between 84 and 88 percent, that's normal fluctuation. Gmail's algorithm shifts day to day. But if you were bouncing between 84 and 88 percent for three weeks, then jump to 92 percent and stay there, something changed. You recovered.
Document what you changed between tests. If you tightened your list hygiene on Monday and your inbox percentage jumped on Friday's test, write that down. You've just found a fix that works for your sending pattern. Do it again next quarter and measure the same way.
Provider-specific patterns matter too. If Gmail stays at 90 percent but Outlook drops from 85 to 70 percent, you don't have a general reputation problem. You've got an Outlook-specific issue. That's valuable information. It tells you to look at your SPF record alignment for Outlook, or your complaint rate with Microsoft specifically.
Start your tracking now. What were your inbox percentages on your last three tests? That's your baseline. Next month, you'll compare against it.
Contributors
Who worked on this answer
Every name links to their profile. Every company links to their site. Real people, real accountability.
