A/B testing your WhatsApp campaigns without fooling yourself
Testing two messages is easy. Knowing whether one actually won is harder: small lists produce noise that looks like a result. Here is a method that is honest about sample size.
4 min read
In short: change one thing, send each version to an equal slice of the same audience, measureorders (or clicks if orders are rare), wait long enough, and treat a small difference on a small list as noise. Wameed supports A/B variants and can pick the winner automatically; you decide what is being tested and how big the slices must be to mean anything.
Most test “winners” on small lists are luck. The discipline below keeps you from rolling out a change that did nothing.
What to test
One variable per test. The common ones:
- The opening line — a question versus a statement.
- The offer — SAR 20 off versus 10% off.
- The call to action — “اطلب الآن” versus “شاهد العرض”.
- The image — product versus lifestyle.
- The send time — afternoon versus evening.
- The length — two lines versus five.
Do not test three things at once; if one version wins you will not know why.
How a test runs
- Form one audience of consenting customers.
- Write version A and version B differing in one thing.
- Send each to an equal, random slice of the audience — say 20% each.
- Wait long enough for most responses — not minutes; at least a day for orders.
- Compare on your chosen measure.
- Send the winner to the remaining 60%.
- Write down the lesson.
Wameed automates the slicing and, if you allow it, the selection of the winner.
Choose the right measure
- Orders or revenue attributed to the message — the real goal.
- Link clicks — a good proxy when orders are too few.
- Replies — useful for conversational offers.
- Not read rate alone: a message can be read and ignored. Reads also tell you more about the subject line and the sender name than about the offer.
The sample-size problem
Suppose each version goes to 200 people.
- Version A: 12 orders (6.0%).
- Version B: 16 orders (8.0%).
B looks 33% better. But a difference of four orders on 200 people each is well within what chance produces. If you ran the same message twice you might see a gap this large.Do not call a winner yet.
Rules of thumb:
- A few hundred per version is a minimum for a difference of several points to start to mean something.
- The smaller the expected difference, the bigger the audience you need.
- If your list is small,run the test across several sends and add the results rather than trusting one.
- Treat a difference of two or three conversions as noise.
If a result is unclear, say so, and use the version you prefer on grounds of brand and clarity.
Avoid the traps
- Peeking and stopping early when your favourite is ahead.
- Unequal slices (the best customers in A).
- Testing on a tiny list and treating the result as a law.
- Different send times that confuse the result.
- Changing the offer and the wording together.
- Never repeating a surprising result.
What good learning looks like
A short log:
| Date | Audience | Changed | A | B | Result | Decision |
|---|---|---|---|---|---|---|
| 5 Mar | Lapsed 90d (800) | Opening line | Question | Statement | 9.1% vs 8.6%, noise | Keep question |
| 2 Apr | Repeat buyers (1,200) | Offer | SAR 20 | 10% | 7.9% vs 6.2%, retest | Retest |
| 30 Apr | Repeat buyers (1,200) | Offer | SAR 20 | 10% | 8.4% vs 6.0% | SAR 20 |
A log tells you which lessons repeat.
Ideas worth testing in Saudi Arabia
- A fixed amount versus a percentage for baskets of different sizes.
- Arabic-only versus bilingual copy.
- A named sender (“سارة من متجر X”) versus the brand.
- Evening after Iftar versus afternoon during Ramadan. Seequiet hours.
- A deadline (“حتى الخميس”) versus none.
Remember the costs
Each test message costs what Meta charges. A test on 400 people costs little; a mistake sent to 20,000 costs a lot. That is the strongest argument for testing first. Seemessage pricing.
Frequently asked questions
Does Wameed pick the winner automatically?
It supports A/B variants and automatic selection of the winner. Decide the measure and the slice size yourself.
How long should I wait?
Long enough for most responses. For orders, at least a day; for clicks, a few hours.
Can I test on a tiny list?
You can, but expect noise. Combine several sends before trusting a result.
What should I test first?
The offer and the opening line usually move results most.
- #اختبار A/B
- #اختبار حملات واتساب
- #تحسين الحملات
- #اختبار الرسائل
- #A/B testing
- #WhatsApp campaign test
- #message variants
Share this article
Ask about your own shop
Thirty minutes on your products, your tax setup and your hardware — not a slide deck.
Keep reading
4 min read
WhatsApp Business services: the twelve things a business can run on WhatsApp
When a vendor says “WhatsApp Business solutions”, what do they mean? Here is a plain map of what a business can actually run on WhatsApp, which ones matter first, and what each needs.
4 min read
WhatsApp for companies: moving from staff phones to one business account in thirty days
In many companies, customer relationships live on employees’ personal WhatsApp. When they leave, the history leaves with them. A thirty-day plan moves it all to one business account without losing a customer.
4 min read
Click-to-WhatsApp adverts: send the ad’s traffic into a conversation that sells
A click-to-WhatsApp advert opens a chat instead of a web page. Used well, it turns attention into a conversation within seconds. Used badly, it dumps a stranger into an empty inbox. The difference is the first reply.

