Compare versions of a campaign and send the winning version to the rest of your list.
A/B Testing lets you compare different versions of an SMS campaign to see which one performs better. Send each version to a portion of your audience, wait for results, and automatically deliver the winning version to everyone else.Looking to test email campaigns instead? See A/B Testing for Email Campaigns.
You’ll create 2 to 5 versions of your campaign. Each version has its own text message editor and phone preview.Version A is your control.To add more versions:
Add Version B, C, D, or E.
Give each version a Version Name for internal tracking.
Write each version’s message in its editor. The phone preview updates as you edit.
Attach that version’s coupon, image, and contact card as needed.
Set link shortening, the opt-out footer, and whether to include a contact card.
Send a test to your phone.
Use Show version differences to compare versions.
Each version can have its own coupon. The coupon merge tag must be in that version’s message. See Add a Coupon Code.
You can duplicate, rename, or remove versions at any time. You need at least two versions to continue.
Select one or more segments to send your test to. You can also exclude a segment if needed.Recipients shows an SMS credit range from the cheapest version to the most expensive version. Actual credit use depends on which version each contact receives and how many message segments that version uses.The audience is frozen when you send the test. Changes to your segments after sending won’t affect who receives the campaign.
Winning MetricChoose how Privy will determine the winner:
Click Rate — The version with the highest percentage of clicks wins.
Order Rate — The version with the highest percentage of orders wins.
Click Rate is the default winning metric.Test DurationSet how long Privy will collect data before choosing a winner. You can specify hours or days.The test duration starts after all test-cohort sends are complete.Recipient AllocationSet what percentage of your audience receives each version during the test. You can allocate from 1% to 100% per version.The allocations are approximate, especially on smaller lists.If you allocate less than 100% total, the remaining recipients will receive the winning version after the test completes. If you allocate 100%, there will be no one left to send the winner to.
Smaller allocations make it harder to identify a clear winner. Privy will show guidance when your allocation is too small to produce reliable results.
Quiet HoursQuiet Hours are 8:00 PM to 9:00 AM in the recipient’s time zone. Tests that overlap Quiet Hours pause and resume when Quiet Hours end. See Quiet Hours for the same rule on ordinary SMS sends.
The Review tab shows each version’s preview, an overview of your test, and an easy way to understand differences among versions.You cannot launch while Campaign Issues has blocking items. Launch is typically blocked when text registration or configuration is incomplete, credits are insufficient, or a version’s content is invalid.When you’re ready:
Click Send to start the test immediately.
Click Schedule to pick a future date and time for the test to start.
After the test starts, the test content, audience, allocations, metric, and duration are frozen. You can cancel an in-flight test, but messages already sent cannot be recalled.
Click Rate = clicks for that version ÷ texts delivered for that version
Order Rate = orders for that version ÷ texts delivered for that version
The version with the highest rate wins. A larger allocation does not automatically win — only the rate matters.If two or more versions tie for that highest rate, Version A (the control) wins when it is one of the tied versions. If Version A is not among the tied versions, Privy picks one of the tied versions at random.If no version has any engagement data, Version A wins.The decision uses data collected through the test deadline. Later activity may change the live report but won’t change which version won.After the winner is chosen, Privy sends that version to the remaining recipients (if any are left outside the test allocation).
At the top of the report, you’ll see an A/B Test Overview card. This card compares each version on the winning metric you selected and marks the Winner.The status shows whether your test is Live, Completed, or Canceled:
Live tests display Time Remaining and Recipients Tested So Far.
Completed tests show Ended and Recipients Tested.
Use Select Version to scope the rest of the report to a specific version. When you select a version, Overview and Contact Activity show data for that version only.
Later clicks and orders may change the live report numbers, but they do not change the winner. The winner is determined by data collected through the test deadline. See How the winner is chosen for details.
