Maximize Your Impact With A/B Tests.
Successful annual giving teams use many of the same techniques used by direct marketing campaigns. Running A/B tests on your appeals helps you dial in which pitches work for your audience. When you know what works, you can more efficiently reach your audience. Adopting this direct marketing technique helps you maximize the donors and dollars you receive for your donor appeals.
What Are A/B Tests?
An A/B test compares two pitches on a small sample audience to discover which pitch has a better response rate. Once you know the messaging that works, you can use it with your larger audience. This type of testing helps you spend the bulk of your budget on appeals that are proven to be effective.
Limit Your Variables.
A/B tests only work if you limit yourself to one variable. Annual giving teams have control over the timing of their ask, the list of recipients, the message’s channel, and the content of the message itself. Start simply and choose one area to test. You can run separate tests later to optimize your approach in different areas. Let’s see how that works for messaging.
How to A/B Test Your Messaging.
Start your experiment by identifying which of your appeals has historically performed the best – this will serve as your control group. Next, you’ll change the messaging of this control appeal in a way that you think will cause it to perform better. This will be your test appeal. Remember, list, timing, and channel should remain consistent – only the messaging should be different in this test. Now it’s time to run your experiment.
Create a Test Group
You’ll want to be careful about the size of the audience you choose to test your appeals on. Too big and you’re not optimizing the number of constituents who are sent the highest performing appeal. Too small and your test results won’t be meaningful.
To choose your test group, grab a random subsection of your total list. You should aim for a list size that will get you a minimum of 30 responses for each one of your appeals.
Tip: you won’t get 30 responses for each test group just by sending out 60 appeals. Use your average response rate to predict how many of each appeal you need to send out in order to receive 30 donations back.
Ensure Meaningful Test Results.
Once you’ve got enough responses from your two groups, how do you draw the correct conclusions from your results?
If one appeal is performing better than the other, do a quick check to measure just how much better it’s performing. In order to be significant, the highest performing appeal should have at least 10% more responses. If it does, you know this is the appeal you’ll want to run on your entire group. If there’s a less than 10% difference between your appeal results, the changes between your test and control aren’t large enough to indicate significant results.
Try out some new techniques. Learn how to Energize Your Annual Giving Program with a Person-To-Person Fundraising Strategy