Three things to know about A/B Testing

January 25, 2013

Yes, I know that some of our organizations really don’t have a very big list, or the time and energy to be doing A/B testing. But, before you rule it out, consider these three things about A/B testing:

1. It’s a great way to make your communications decisions

A/B testing could also be described as: the best way to settle your internal arguments. Testing allows you to see what leads your constituents to respond. What gets people to click? What gets people to RSVP to an event? It removes the guess work of “what do we think people will do?” with real information.

2. You can only test one thing at a time

You don’t have to test everything at once. As a matter of fact, you can only effectively test one variable at a time. Examples of things you could test:

  • The length of your online donation page — do more people complete when there are fewer fields?
  • Should you include people’s names in your subject lines, or in the first line of your emails?
  • Should the donate link in your header say “Give” or “Donate” or “Support Us?” Does the color matter?
  • Are people more likely to give if your fundraising mailing includes an insert?

Effective testing only alters one variable per experience. That makes it easier to do, and is the only way to definitively answers the question.

3.  Testing is an ongoing process

Your goal is to be refining what you do on an ongoing basis. You will never find the right tool or strategy that will work for eternity. Some people find that discouraging, but I encourage you to adopt a growth mindset about it. This is a chance to keep growing and getting better at what you do.

For example, if you are successful in a year and increase your web audience by 100%, you’re dealing with a whole new set of people who are different.

If you want to keep your organization growing, you need to adjust how you’re communicating — that’s how this works.

Bottom line: we test because we want to keep paying attention. You want to be in conversation with the audience you have, not the audience you think you have, or that you had a year ago. Testing can tell you whether you’re becoming more effective at communicating, and ultimately, at reaching your communications goals and strategic goals.

Learn more about A/B testing:

 

Posted in , , , ,

My Latest Tweets