A/B Testing in Privy is for users interested in running split tests to understand the impact of certain designs, forms, triggers, discounts or targeting changes on the conversation rate and sign up volume of the form.
To get started, click the A/B Tests navigation on top of your Privy dashboard.
When you click the “New Experiment” button, you will be prompted to name the experiment, and choose an existing campaign that you want to test against. This name you choose is not visible outside your Privy account, and will not appear to your site visitors, so we suggest using a name that will help you identify what you are testing.
Now you can decide on how you want to configure the experiment. You can choose between a specific schedule, i.e end the experiment after 30 days, or you can set the experiment based on the number of participants, i.e end the experiment after 50,000 visitors participate in your experiment. You can think of A/B Test participants as equivalent to your total site visitors.
Next, it’s time to name the original variant. The variant name is also only visible in your account, and simply helps you keep track of what you’re changing. So, in our example, the original campaign is a flyout style opt-in form, so we name the original variant “flyout”.
The Traffic Ratio setting lets you determine what percent of the test traffic you want to point to this variant. Leaving the traffic ratio to 1 will divide the traffic evenly across all your variants. Once you update the traffic ratio of one variant, the rest of the traffic percentages will update automatically.
Now it’s time to create a variant. Click the add variant card at the bottom to name your next variant. In our example, we’re calling it “bar” since we want to change the display type to a bar, thus testing the effectiveness of a bar campaign as opposed to the original flyout campaign.
Editing a variant within an experiment lets you edit the form fields, design, coupon, triggers, targeting, automation rules or emails section of the original campaign.
When you create a variation, you will start with an exact replica of the original campaign so that you are in control of what varies from variation to variation from the original. As with any campaign you can open the designer if you just want to test small changes like copy, or button color.
In this example we deleted the flyout design from the variant, and then add a new display, selecting a bar display type.
Next you’ll be brought back the experiment summary page, and need to click the “Start Test” button to begin the A/B test.
Once live, you’ll see an overview number of completed participants, the conversion rate from each variant, as well as the over impact of one variant over the others.
You can also choose to view the results based on people who have viewed your campaign variations versus people who have participated.
Once the test has completed, you will also see the winner clearly identified based on the campaign or variant with the highest conversion rate.