Open-Rate Controlled Experiment in E-Mail Marketing Campaigns
Open-Rate Controlled Experiment in E-Mail Marketing Campaigns

Summary/Abstract: Purpose – The main purpose of this paper is to test the controlled experiment (A/B split) methodology in B2C oriented e-mail marketing campaigns.Design/Methodology/Approach – E-mail marketing techniques have been a substantial part of e-marketing methodology since the early Internet days of the mid-1990s. From the very beginning of Internet utilization for business purposes, e-mail was one of the most widely used communication techniques in B2B and B2C markets alike. Due to high volumes of spamming and progression of online communication clutter, some practitioners began to question the usability of e-mail as a marketing communication channel, while others embarked on working on improving the message itself. Efforts were invested into improving message quality, as well as into better understanding user expectations. One of the most commonly used techniques to test specific e-mail message elements is the controlled experiment. Findings and implications – This paper explores several types of controlled experiments in a specific Croatian B2C market. Tests were run to determine subscriber behavior towards several newsletter components, including sending time, sending day, sender’s name, and subject line. Open and click rates for tested campaigns, and several other metrics were investigated using MailChimp software. An N − 1 two-proportion test using an adjusted Wald confidence interval around the difference in the proportions was used for comparing the open-rate measure in the controlled experiments between subjects.Limitation – Controlled experiments (A/B split tests) showed a lot of potential as a way of measuring behavior and preferences of subscribers, although several apparent limitations (the data-set scope, comparability issues) indicated a clear need for standardization on a managerial and scientific level.Originality – This paper provides an up-to-date e-mail marketing effectiveness literature review, describes and tests the methodology and metrics for e-mail campaigns measurement, and suggests several important guidelines for further research.

• Details
• Contents