You're staring at your cold email campaign and wondering if you should test 5 subject lines or 50. Should you test different opening hooks? Different closes? Different value props? You want to optimize, but you're also tired of waiting for data that never quite feels conclusive.

Here's what actually works: test fewer variants than you think, but test them more rigorously.

The Variant Testing Mistake Everyone Makes

Most people test too many things at once. They'll send 10 different subject lines, 5 different opening paragraphs, and 3 different calls-to-action across 2,000 emails, then look at the data and have no idea what actually moved the needle.

You need sample sizes. Real ones. Not 50 emails per variant. Not 100. You need 300-500 emails per variant to have any statistical confidence that one version is actually better than another. The difference between a 15% reply rate and an 18% reply rate feels big until you realize it could just be noise.

So the math is simple: if you're sending 3,000 emails a month, you can test 6-10 variants reliably. If you're sending 10,000 emails a month, you can test 20-30 variants. More emails equals more testing room, but most people don't have that volume early on.

What You Should Actually Test (In This Order)

Don't test everything at once. Test in order of impact, with the highest-leverage changes first.

1. Subject Line (Always Start Here)

Your subject line determines if the email gets opened. If it doesn't open, nothing else matters. Test 2-3 subject line variants against your control, keeping everything else identical.

Run this for 500-800 emails per variant before deciding. That's roughly 2 weeks of sending at moderate volume.

Here are two real subject lines that moved replies for B2B service businesses:

Quick question about [Company Name]'s [specific thing they do]

This one works because it's curiosity-based without being click-baity. People open it because you're asking about their actual business, not promising them something vague.

Follow up: [Your name]'s email from last week

This works on email 2 or 3 in a sequence because it creates the impression that they're missing a conversation they were already part of. It doesn't work cold on email 1, but it works well on follow-ups.

Once you have a winner on subject lines, hold it steady and move to the next test. Don't keep toggling it.

2. First Line / Hook (500 emails per variant)

The first line is what determines if they read past the subject. Test your control (whatever you're using now) against 1-2 alternatives. Three options here:

Most people do a combination. The key is that it has to be true. Your opening line should actually get replies, which means it can't be generic. It needs to be specific to the person or company you're emailing.

3. Email Length and Structure (Only Test Once You Have Traction)

Short vs. long emails is a conversation that wastes more time than it's worth early on. Once you have a subject line and opening that's working, then you can test whether your email should be 60 words or 150 words. Test one structural change at a time - either make it shorter, or make it more narrative, but don't do both.

4. Call-to-Action (Only If Your Reply Rate Is Already Above 20%)

If your reply rate is below 20%, your CTA isn't the problem. Your positioning or opening is. Once you're above 20%, you can test whether "Quick call this week?" converts better than "Does this resonate?" or "Worth a 15-min chat?"

Most people test CTAs way too early and waste data on a variable that doesn't matter yet.

The Testing Timeline That Actually Works

Here's the actual timeline for running variants at 5,000 emails per month:

This takes discipline. You'll want to test three things at once. Don't. Lock in your winners as you go, and test one variable at a time. It's slower to describe, but it's faster in practice because your data is clean.

The Math on Sample Size

If you're sending 300 emails per variant and your baseline reply rate is 15%, here's what you need to know:

Get to 400-500 emails per variant before you trust the data. At 500 emails with a 15% baseline:

You need at least 70-100 data points (replies, not emails sent) before you can be confident a variant is actually better.

What Not to Test

Don't test:

Every one of these tests will give you inconclusive data that wastes your time. Focus on what matters: subject line, then opening, then structure. That's it for the first 90 days.

The Reality: Testing Gets Better With Volume

Here's the hard truth: if you're sending 1,000 emails a month, testing is mostly guesswork. You don't have enough volume to isolate variables cleanly. You need at least 3,000-5,000 emails a month to run reliable tests. Before that, just pick a template that works and stick with it. Focus on booking meetings at scale first, then optimize.

Once you're sending 5,000+, testing starts to matter because you have enough data to trust what you're seeing. Below that, you're wasting cycles on variables instead of focusing on fundamentals.

Related Guides