What I Learned from Sending 25,000+ Automated Emails

Love it or hate it, email is still one of the most effective communication tools ever created. Email automation has helped companies market their products or ideas, sell their services and better serve their customers for years. Over the past year we have sent over 25,000 automated emails, and I wanted to find out what was working and what we needed to fix.

Just like every other tool, email only works if you use it appropriately. Blindly blasting your message to a user who never asked to hear from you is a sure-fire way to cast a negative reputation on your brand.

We use automated emails to move leads through the buyer’s journey by delivering content we believe they will be interested in because of actions they have previously taken. Instead of just setting up one workflow, we created four separate workflows that targeted each of our four personas. Then we created workflows for each stage of the buyer’s journey: awareness, consideration, and decision. Users moved to the next workflow after they meet a certain lead score based on user interaction.

After letting these workflows run for a little over a year, it was time to dig into the data to learn what was working, what wasn’t working and what could we use in future projects based on the new knowledge we’ve gained.

Here are the high-level numbers.

• Total Emails Sent: 25,597
• Average Open Rate: 27.88%
• Average CTR: 4.42%
• Average CTOR 11.99%

Just to make sure we are all on the same page I am going to give a definition of open rate, CTR, and CTOR.

• Open Rate: This is the percentage of emails that were opened by the users.
• CTR: This stands for “click-through-rate” and is the percentage of emails that had a link inside of them clicked on.
• CTOR: This stands for “Click to Open Rate.” CTOR measures the relevancy and context of an email by taking the number of unique clicks divided by the number of unique opens. (Source) This is an important metric because it gives insight into how compelling your email copy was.

As a data-driven marketing agency, it’s important for us to know why something is working and what is a realistic benchmark. To give a little more context on how our overall numbers compare, I’ve added some industry averages.

Marketing Industry Averages:

• Open rate: 17.81%
• CTR: 1.92%
• CTOR: Mean: 10.7%
  Median: 7.6%
  Top Quartile: 25.0%
  Bottom Quartile: 1.9%

Want to know your industry averages? Check this post out.

So, if you compare how our emails were doing against industry averages, we did pretty well. Opens, clicks, and CTOR performed above average. While this was encouraging, I wanted to know more. What caused some emails to perform great and others to flop? What can we do to make our emails deliver even more value? What type of content do our personas really want to see?

For me, the industry numbers are a great benchmark to start with, but the truth is, I want to compete against myself. I am not trying to be “industry average.” My goal is to make sure that we are delivering value to our customers and subscribers. This means we need to dig into the data. 

Subject Length

Subject length is often a point of discussion in email case studies. According to Sendgrid, 7 words is the most common subject line word length (14.0% of them), 3 words (1.6% of them) have the highest engagement rates (21.2%, compared to 17.2% overall and 15.8% for 7-word lines).

While our data doesn’t link up perfectly, we did find out that shorter subject links did perform better than longer ones.

• Emails with 6 words in the subject line had an average open rate of 53.82%.
• Emails with 4 words came in second with 50.8%.
• Surprisingly 14 words produced an open rate of 40.646%.

On average the emails with shorter subject lines performed better than those with longer titles in regards to open-rate.

Shorter subject lines also produced a better CTR.

• Emails with 6 words in the title averaged a 15.85% CTR.

As you might expect, there was a direct correlation between open rates and CTR across all the emails.

Subject Style

So, we now know that shorter titles are better, but that is just part of the equation. What I wanted to know is how the style of the title impacted engagement rates. Our emails had three distinct styles.

• Lists: Subjects that started with a #
• Personalized: Subjects that used personalize tokens
• Contextualized: Subjects written around email content

# Emails:

• Average Open Rate of 23.35%
• Average CTR of 4.133%
• Average CTOR of 20.00%

Personalized:

• Average Open Rate of 30.11%
• Average CTR of 3.81%
• Average CTOR of 9.14%

Contextualized:

• Average Open Rate of 29.00%
• Average CTR of 4.76%
• Average CTOR of 12.53%

While the data is pretty close across these subject lines, a few things do stand out. The personalized emails had the highest open rate, but the lowest CTR and CTOR. So while we may have gotten the user to open the email, they took less action than the other two types.

List emails delivered the best overall CTOR. While this segment opened fewer emails on average, when they did open the emails, they also engaged with the content.

Contextualized emails had optimal performance. The open rate was only 1.11% off of the personalized emails, but the CTR was better than the other two segments and the CTOR was at a very impressive 12.53%.

Based on what we’ve learned so far, short subject lines that are contextualized should perform the best. With this data alone, I am sure we could boost our results, but we aren’t done yet. There is still more data for us to weed through and pull knowledge from.

Segments

We have 4 segments that we market to: Business Owners (EE), Marketing Managers (MM), SEO (SS) and Web Designers (WW). Most were sent to more than one persona and existed in two or more workflows.

Most of our clients and business prospects are EE or MM, while the SS and WW personas are members of our lists. What I wanted to see was which persona group emails performed the best and which group needs improvement.

While not all groups received the same number of emails, there is enough here to gain some insight.

Our generic emails, ones that we sent to all persona groups, outperformed all segmented emails in open rates and were the second in CTR and 4^th in CTOR. This was an eye-opener for me. While some segmented emails performed better in other metrics, we clearly need to look at the content we are sending to our segmented users. Knowing what we know now, I can benchmark the segmented emails against the generic emails and make an educated decision on how to optimize.

Our core persona groups EE & MM had pretty decent numbers, but again, we can use the generic emails to help us better optimize those segments as well.

The [EE & MM & WW] group has engagement metrics that can’t be ignored. These emails will provide deeper insight into how we need to structure email content in order to drive engagement.

Breaking out the data by segments helped me see what user group performed the best for our business. This is why segmentation is important. By breaking up your list, you can find out which groups of like users have similar needs and interests. Then you can optimize your message to deliver value beyond their expectations.

Content Type

Similar to email subject type, I wanted to investigate how different kinds of content performed with our users. As you saw in the previous section, we have four different personas. While they all have some interest in digital marketing, what I wanted to know was what type of content was of most interest.

Our content is specifically about digital marketing, and we have a few key areas that we promote. There were also emails that focused on offers, subscriptions, sales and affiliate offers.

Clear loser, web hosting. I got it. Nobody wants to talk about that…

What was surprising was how well the social media content is performing. Don’t get me wrong, we believe in the power of social media, but our expertise is in SEO, Web Development, Inbound and Content Marketing.

I was glad to see that the numbers in our core areas were also healthy, but with social’s 11.98% CTR, there is clearly something there. This report uncovers what our users want. It would be easy right now to just say, “The people want more social media content, so let’s give it to them.” But that would be short-sighted.  Yes, social content performed the best, but now we need to look at the email titles, context, persona and general make-up of the email before we go all-in on social media.

As with most data-backed projects, you can keep drilling down further and further, but the key is to know where and when to stop. The reason most never take action is that they get stuck in the data. It can get so overwhelming that many organizations do nothing. But for me, inaction is not an option. I set out to optimize our automated emails and provide more value to those who have trusted us with their inbox.

So What’s Next

Here is what I learned:

• Emails with 6 words in the subject line had an average open rate of 53.82%.
• Emails with 4 words came in second with 50.8%.
• Surprisingly 14 words produced an open rate of 40.646%.
• Emails with 6 words in the title averaged a 15.85% CTR.
• Contextualized emails had optimal performance. The open rate was only 1.11% off of the personalized emails, but the CTR was better than the other two segments and the CTOR was at a very impressive 12.53%.
• Our generic emails outperformed the segmented emails in most categories.
• Emails about social media lead to the highest engagement rates among our users.

While this case study is not something every business can grab stats from and put them to use, I hope you grasp the bigger picture. My goal was to show the power of drilling into your data and finding some actionable insights that will make your work better.

Email is just one of many touch points for our agency. It would be easy for me to look at the top-level number, decide that we are doing good enough and focus somewhere else. Settling for good enough will never deliver outstanding results. It’s my goal to be outstanding. I want our email subscribers to stay engaged and continue to read our content even if they never become “customers.” I want our customers and prospects to know that we don’t settle for good enough; that we are always looking for ways to improve our content and messaging.

After looking at the more than 25,000 emails, I now have clear and actionable takeaways. I know what was working, what areas need improvement and where to look in my database to find examples of outstanding work I can build off.

While I did use Tableau for some of this analysis, most of the work was done with Excel, Google Sheets, Google Cloud Tools (Data Prep, Big Query) and Data Studio. These are mostly free tools that nearly all of us can access. I want to encourage you to dig into your data and find ways to improve your emails, content and more.