Feature Friday: A/B Testing in Professional Edition

Are your emails campaigns falling short either in open, click or click-through rates? Perhaps you’re at the end of your creative limits, trying every trick in the book to get subscribers to interact with your emails. No matter how hard you try to please your subscribers, you can’t win. It’s time to reverse your order of thinking: Give subscribers the content they want. How to do this? Test different from addresses, subject lines and creative to see which ones subscribers engage with most. If you’re a Professional Edition user, our A/B testing feature is one of the best in the industry and just received a facelift along with the rest of the platform.

In a few steps, we’re going to walk through a typical subject line A/B test in Professional Edition.

Create A/B Test Definition

Under the Tasks menu, select View A/B Test Definitions. In the top right corner, click Add A/B Test Definition. Enter a distinctive name for the test, as well as a description. From the next dropdown menu, decide the maximum time the test should run. Depending on your list size, this time may vary. Since this is a subject line test, the most appropriate metric off of which to base our winning email is unique open rate – the action we’re primarily trying to get subscribers to take with this A/B test is opening it.

Setting up A/B testing in Professional Edition

Next, decide how you want a winner to be determined and when the platform should stop collecting data for the test. Select the list you want to use, and a segmentation and/or suppression list if you so desire. Drag the bar to the percent of the list you want to test. This is a simple split test with two samples, so 20 percent is a good metric. However, you can test up to 10 different samples.

Setting up A/B testing in Professional Edition

Next, check the box marked Subject line. Select the template you want to send, and enter the different subject lines. Save the A/B Test Definition.

Choose what part of your email you want to test.

Deploy A/B Samples

Once you’ve set up the A/B Test Definitions, starting the actual test is simple. In fact, it’s done in the same way as deploying a regular campaign. However, choose A/B Test instead of Normal on the Deploy Campaign screen.

Choose A/B test on the Deploy Campaign screen.

Choose the A/B Test Definition you created from the next dropdown menu. Then follow all the other steps you normally take when deploying a campaign. On the following screen, you’ll see the two subject lines and you’ll be able to confirm the send.

View A/B tests before sending

You have three options to determine the winner of an A/B test:

  • Automatically, and the platform sends the winning email to the rest of the list
  • Automatically, but the platform sends an email and lets you decide what to do next
  • Manually. The platform will send you an email at the end of the test and you decide what to do next

If you choose one of the latter options, once you deploy the A/B campaign, you’ll be responsible for reviewing the results and choosing which email version to send to the rest of your list.

So you’ve deployed you’re A/B campaign and you have the results in front of you. Now what? Maybe nothing happened. Perhaps your open rate doesn’t change, no matter how many different subject line tests you send. This can be frustrating, but don’t worry, help is on the way. Our Strategy team has helped clients think up unique testing strategies that move the needle for them. Just shoot us an email – we’d be happy to help you, too!

Joy Ugi
Digital Marketing Coordinator, WhatCounts
Twitter: @ugigirl

More to explore...

The Agile Marketer

In the World of Marketing It Is Common Practice to Spend Hours, Days, or Even Weeks Planning and Implementing a Single Campaign It is also

Read More »

Ready to See WhatCounts in Action?