Account Engagement (Pardot) A/B Vs. Multivariate Testing (+ AI)
- May 23, 2023
- Pardot A/B Testing, Salesforce Ben | The Drip
A/B testing enables you to test one change between two versions of your marketing assets. In the context of Account Engagement (Pardot), A/B testing is for list emails*. The benefits are that it’s quick to set up and can work even with a small sample size (of course, the greater the sample size, the more significant your results will be).
On the other hand, multivariate testing compares multiple changes between versions of your marketing assets. In the context of Account Engagement (Pardot), A/B testing is for landing pages. Multivariate tests take a little longer to set up and require a larger sample size.
*Note: A/B testing is available for emails sent via the classic email builder.
This guide will walk you through how to decide between the two types of tests, how to set up A/B testing, and how to use AI innovations (i.e ChatGPT) to get up and running faster.
Account Engagement (Pardot) Test Preparation
Experimenting with marketing assets requires you to do preparation to ensure that you can track what you’re testing, and draw sensible conclusions from the results.
You can use the pyramid below, a hierarchy to prioritize testing factors that can deliver the greatest results:
- Value proposition: “What’s in it for me?” (where the “me” is your buyer).
- Layout: Are you highlighting the right points, or is your layout busy with competing messaging?
- Credibility: Are you building the buyer’s confidence, for example, thought leadership, expert quotes.
- Social proof: Do you provide answers for when the buyer asks: “What did people similar to me choose, and were they happy with that decision?”
Next, follow these five steps:
- Determine your goal. Is it to increase clicks? Improve landing page conversion rates?
- Create a hypothesis. Select a factor from the hierarchy above. If you make a change, what would you guess happens?
- Create alternatives: Change something about your current assets (copy and implement those on the copied asset).
- Confirm your measurement: Ensure your tools are set up properly to record the metric you’ve chosen.
- Execute your test: Test alternatives with live traffic, using the right method (A/B test or multivariate)
Source: Marketing Automation Unleashed
Account Engagement (Pardot) A/B vs Multivariate Testing
| A/B Testing | Multivariate Testing |
|---|---|
| Compare one change | Compare multiple changes |
| Quick Setup | Longer Setup |
| Requires smaller sample size | Requires larger sample size |
| For testing emails in Pardot | For testing landing pages in Pardot |
Set Up Account Engagement (Pardot) A/B Testing
With Account Engagement (Pardot) A/B testing, you can test the following factors (remember that it’s best to test one at a time!):
- Email body content
- Subject line
- ‘From’ (sender) name
- Reply ‘to’ address
1. Navigate to Account Engagement Email → A/B Tests.
2. Create a new list email. Select the Enable A/B Testing checkbox.
3. Select an email template. A copy of your existing content will automatically be created for version B.
4. On the building tab, you will be able to switch between versions A and B.
5. On the sending tab, you will be able to change the “from” (sender) name, “reply-to address”, and/or subject line between versions A and B.
6. Continue scrolling down, and you’ll find the section to configure how your A/B test should be run. Set how long you want the test to run with the sample size before a winner is selected (any duration from 1 hour to 30 days), the criteria you want the winner selection to be based on (opens or clicks), and the % of your send list you’d like to use for testing.
7. Once the email is sent, Account Engagement (Pardot) will send the email to the sample group, then record the results based on your selected criteria. When the experiment time frame is up, the winning version will be sent to the remainder of the send list.
By following this hack, you can also perform A/B testing with Engagement Studio:
Leveraging ChatGPT in A/B Testing
Marketers have already been tapping into the power of GPT to write email and landing page copy, marketing plans, and advice for data breaches.
Einstein GPT brings the power of GPT into the Salesforce interface. We saw a marketing use case during the release, where a marketer wants to create a landing page for an upcoming event, asks Einstein GPT to create a landing page template and campaign messaging – which they then ask to be made shorter.
While this is impressive, there are ways you can use ChatGPT on the fly to get the tedious task of amending copy done even faster.
- Head to ChatGPT, and set up an account.
- You will arrive on the screen with suggestions, and a text box below. Ask ChatGPT to make your copy more formal/exciting/funny… whatever you choose!
I asked Chat GPT to generate different versions of one of our SFB Course blurbs. The first is a formal version, the second exciting, and the third funny. This is a quick way to generate your content to meet different tones.
Summary
This guide explained how to decide between the two types of tests (A/B testing and multivariate testing) and what’s required for each. You should now know how to prepare a plan for testing (so that you’re not just testing blindly).
Setting up an A/B test in Account Engagement (Pardot) is actually straightforward once you know what you want to test. Finally, take the legwork out of creating variations in copy with AI innovations, i.e ChatGPT.
This Pardot article written by:
Salesforce Ben | The Drip
Lucy Mazalon is the Head Editor & Operations Director at Salesforceben.com, Founder of THE DRIP and Salesforce Marketing Champion 2020.
Original Pardot Article: https://www.salesforceben.com/the-drip/account-engagement-pardot-a-b-vs-multivariate-testing/
Find more great Pardot articles at www.salesforceben.com/the-drip/
Pardot Experts Blog
We have categorized all the different Pardot articles by topics.
Pardot Topic Categories
- Account Based Marketing (ABM) (8)
- Business Units (13)
- ChatGPT / AI (3)
- Completion Actions (5)
- Connectors (10)
- Custom Redirects (4)
- Demand Generation (8)
- Dynamic Content (5)
- Einstein Features (11)
- Email Delivery (18)
- Email Open Rates (3)
- Pardot A/B Testing (2)
- Email Mailability (16)
- Do Not Email (1)
- Double Opt-in (2)
- Opt Out / Unsubscribe (14)
- Email Preferences Page (6)
- Engagement Studio (14)
- Industries (1)
- Non Profit (1)
- Landing Pages (10)
- Lead Generation (1)
- Lead Management (12)
- Lead Routing (2)
- Lead Scoring (15)
- Leads (3)
- Marketing Analytics – B2BMA (9)
- Marketing Automation (1)
- Marketing Cloud (2)
- Marketing Cloud Account Engagement (4)
- New Pardot Features (6)
- Opportunities (2)
- Optimization (3)
- Pardot Admin (62)
- Duplicates (1)
- Marketing Ops (1)
- Pardot Alerts (1)
- Pardot API (2)
- Pardot Automations (3)
- Pardot Careers (12)
- Pardot Certifications (4)
- Pardot Consulting (1)
- Pardot Cookies (3)
- Pardot Custom Objects (3)
- Pardot Email Builder (7)
- Pardot Email Templates (9)
- HML (6)
- Pardot Events (16)
- Pardot External Actions (1)
- Pardot External Activities (4)
- Pardot Forms (27)
- Form Handlers (7)
- Pardot Integrations (21)
- Data Cloud (1)
- Slack (1)
- Pardot Lead Grading (5)
- Pardot Lead Source (2)
- Pardot Lightning (1)
- Pardot Migration (1)
- Pardot Nurture / Drip Campaigns (1)
- Pardot Personalization (3)
- Pardot Profiles (1)
- Pardot Releases (19)
- Pardot Sandboxes (2)
- Pardot Segmentation (4)
- Pardot Strategy (7)
- Pardot Sync (2)
- Pardot Sync Errors (1)
- Pardot Tracker Domains (5)
- Pardot Training (3)
- Pardot Vs Other MAPs (4)
- Pardot Website Tracking (2)
- Reporting (22)
- Salesforce and Pardot (29)
- Marketing Data Sharing (2)
- Pardot Users (3)
- Salesforce Automation (4)
- Salesforce Flows (1)
- Salesforce Campaigns (20)
- Salesforce CRM (3)
- Record Types (1)
- Salesforce Engage (3)
- Salesforce Queues (2)
- Security and Privacy (1)
- Tags (3)
- The Authors (483)
- Cheshire Impact (9)
- Greenkey Digital (47)
- Invado Solutions (37)
- Jenna Molby (9)
- Marcloud Consulting (6)
- Nebula Consulting (53)
- Pardot Geeks (38)
- Salesforce Ben | The Drip (235)
- SalesLabX (2)
- Slalom (2)
- Unfettered Marketing (45)
- Uncategorized (1)
- Website Tracking (2)
- Website Search (1)
More Pardot Articles
See all posts
This Pardot article written by:
Salesforce Ben | The Drip
Lucy Mazalon is the Head Editor & Operations Director at Salesforceben.com, Founder of THE DRIP and Salesforce Marketing Champion 2020.
Original Pardot Article: https://www.salesforceben.com/the-drip/account-engagement-pardot-a-b-vs-multivariate-testing/
Find more great Pardot articles at www.salesforceben.com/the-drip/