Perfect Your Multivariate Tests

1. Benefits of Multivariate Testing2. 4 Rules for Running a Test3. 5 Simple Tests You Can Run Now4. Recommended Follow-Ups

Benefits of Multivariate Testing

• Learn about your users• Master a new type of

messaging• Perfect recurring

campaigns, such as onboarding campaigns

• Decide between multiple ideas

• Test out “tried and true” marketing techniques, and see if they really work on your particular audience

4 Rules To Follow

1. Run your test on a large number of users. This makes it less likely that your results will be swayed by outliers.

2. Randomly sort users into different test groups. This helps ensure that differences in reception can be attributed to messaging, and not underlying characteristics of the groups.

3. Know what element you’re trying to test beforehand. Are you trying to test something specific, in which case there would be only a small difference between your copies, or something more general?

4. Decide how long to run your test before beginning it, and don’t end early. Some testers are tempted to end the test as soon as they see the results they want – this will lead to biased findings.

5 Simple Tests You Can Run Now

1. Message behavior, for in-app messages

2. Sound vs. no sound for push notifications

3. Punctuation and phrasing

4. Emojis5. Custom Android titles

Recommended Follow-Ups

1. Change your strategy based on test results

2. Improve future tests3. Run a follow-up test

using a different messaging channel

4. Run a follow-up test on a different segment of users

success@appboy.com

Check out more resources at appboy.com/academy