Pre-send email optimizations

I led the design of Pre-Send Optimization, an AI capability within Mailchimp that proactively surfaces performance-driving recommendations before an email is sent. The feature helps marketers act on data-driven insights directly in their email creation flow.

RESULTS

  • 10x adoption within two weeks of launch
  • 51.3% repeat usage
  • 10% lift in click through rate
  • 120K users applied at least one optimization
  • 74K users applied multiple optimizations
  • 20K+ users became repeat users
  • 293K campaigns used in-editor AI assistance
ROLE
Lead Product Designer
DURATION
2 weeks (concept → launch)
TEAM
1 PM, 4 Engineers, Data Science

CUSTOMER PROBLEM

Optimize before it's too late

Marketers often struggle to know what will perform best before sending a campaign. The core problems are:

  • Mailchimp's tools like Content Optimizer offered valuable insights but only after send, when it was too late to improve results.
  • Friction switching between analytics tools and the editor when trying to implement optimizations
Original experience: Marketers previously only saw analytics after sending, leading to missed opportunities and trust issues with optimization tools.

HYPOTHESIS

If we surface tailored recommendations before send, marketers will be more likley to apply optimizations and as a result, improve their campaign performance.

ENTRY POINTS

Exploring the best time to surface optimizations

We knew the optimizations had to be accessible in the email builder. But where was the ideal place in the email builder that was relevant and not disruptive?

In order to determine the best place to surface optimizations, I first mapped all natural touch points where users paused their work or transitioned tasks such as saving progress, previewing content, or sending test emails. There were natural trade offs between visibility and disruption:

Reasoning about possible optimization entry points
When an block is selected in the email that has an associated optimization, the block panel will show the optimization recommendation.
Seperate optimization panel for recommendations across the entire email

ENTRY POINT RESEARCH

To validate these entry point directions, I conducted usability testing with nine Mailchimp customers. When testing the the entry points of the send button, preview button and optimize button, most users preferred the optimization button since it was easy to locate and clearly communicated its purpose.

Finalizing the designs

TRADE OFFS

Although the Optimization button tested best, the Email Builder team was hesitant to add another action to the toolbar. We compromised by using the existing "Optimize" tab on the left as the main entry point. Since very few users typically click this tab, I added a notification dot to improve discoverability.

Although we wanted to show the optimizations directly in the panel so that a user could view the optimizations and the email at the same time, it wasn't in scope for our quick response. As a result, I designed a modal triggered when a user pressed the tab. This wasn't the interaction a user would expect, but it was the best compromise we could make.

Final modal design triggered when a user presses the optimize tab

TECHNICAL CONSIDERATIONS

When do we check for optimizations on the backend?

Now that we had the entry point nailed down we needed to figure out when to notify the user about available optimizations.

Continuous background scans would have added latency and unnecessary compute costs, so I guided the team toward a leaner approach:

  • Recheck only on save, page reload, or when a user clicks the optimization tab.
  • We decided to use a pink notification dot without a number to signal new recommendations, avoiding false expectations if counts weren't up to date. The early designs used a numbered notification dot to make the feature more compelling, but we decided it was unnecessary and would be confusing.

This struck a balance between performance efficiency and user awareness, keeping the experience responsive without increasing load time.

Notification dot on the optimize tab entry point

Results

  • 10x adoption within two weeks of launch
  • 51.3% repeat usage
  • 10% lift in click through rate
  • 120K users applied at least one optimization
  • 74K users applied multiple optimizations
  • 20K+ users became repeat users
  • 293K campaigns used in-editor AI assistance

CONTINUOUS LEARNING

Post-launch iteration

POST-RELEASE RESEARCH

After launch, I ran eight follow-up interviews (30 minutes each) with marketers who had recently interacted with Pre-Send Optimization.

Users loved getting recommendations before send but wanted clearer context about why each appeared and how it affected performance which is a learning that we apply to all of our AI capabilities.

OPTIMIZATIONS VS. ERRORS

I created a framework that was used to guide designers on how to determine if something was an optimization or an error. This framework was used to guide designers on how to surface errors in the email builder at the appropriate time.