Summarize this articleHere’s what you need to know:
- Experimentation analytics are crucial for turning data into actionable insights, driving business growth, and improving user experience.
- The 3 stages of experimentation – ideation, execution, and analysis – all rely on data and analytics to inform decision-making.
- Asking the right questions is key to gathering useful data. Consider user perspectives and segment data for deeper understanding.
- Track relevant events and engagement metrics to measure the impact of your experiments and avoid misinterpreting results.
- Challenge your assumptions! Use secondary metrics to uncover hidden insights and learn from both positive and negative results.
Today, it’s not enough to be versed in analytics. You need to understand how to use analytics to solve the specific business problems you own. Those of us in the digital world may all be using the same tools—Google Analytics, or Adobe, or any other stack—but we’re using them in very specific ways for very different purposes. The nuances are critical.
SEO teams are likely hyper-focused on keywords and bounce rates, while advertising teams are looking at impressions, costs, and click-through-rates. Content teams pay attention to session metrics and pageviews, and the web development team is keeping its eye on page speed and performance.
These teams are pulling different insights out of the same collection of data, creating fractured subcultures of data analysis, and experimentation teams are not exempt. If you are leading experimentation on your organization’s site or in your product, you need to adopt a particular lens for data analysis to answer the types of questions that an experimentation role will ask of you.
In this post, I’m going to walk you through the three stages of experimentation from ideation through execution and into analysis, and highlight how analytics and data can support your experience optimization process at every turn.
This includes:
- How to use analytics data to generate and validate an evidence-backed hypothesis,
- How to measure experiments to get insights in addition to wins and losses, and
- Ultimately, how to push your experimentation program forward.
Note: The term “Optimization Analytics” is fairly broad. Both qualitative and quantitative analysis are necessary in your optimization work, however, while digital analytics is only one component of a well-rounded optimization process, I’m going to focus here for the bulk of this post.
Experiment ideation
First things first, let’s level set on what an analytics platform really is and what it’s capable of. To a lot of people, their analytics platform is a golden goose—a treasure trove of data that holds the secret to more traffic, more revenue, and more success.
In reality, your analytics platform is like the magic 8-ball toy you had as a kid. It won’t do anything valuable or interesting until you ask it a meaningful question. If you wander aimlessly through analytics looking for an insight, you’ll likely end up exhausted and discouraged.
Instead, take a step back and consider the problems you are trying to solve. You may have a hunch, or a list of questions or ideas about your site, product, or users. Once you have those questions, use your analytics platform to either confirm or reject your hunch.
If the data backs up your initial hunch, this may indicate a meaningful insight and an impactful test idea. If it doesn’t, you may have uncovered something new to dig into, or you may have successfully closed the book on a dead-end idea.
If you find this part challenging, you’re not alone. It’s the reason companies like Google have rolled out the Insights portion of Google Analytics—to try to jumpstart the actual insights flowing out of the data. While still fairly rudimentary, these artificial intelligence-driven insights are a step in the right direction in prompting us to ask the right questions of our data.
My tips for asking the right questions are to:
- Put yourself in your users’ shoes,
- Track events, and
- Segment segment segment
Put yourself in your users’ shoes
One of the best ways to identify a strong experiment idea is to put yourself in your users’ shoes, especially if you are new to your role or business (you’ll have a fresh perspective). Browse your website or app as if you’re a new user and look for points of friction or opportunity.
Perhaps you’re at an eCommerce company and you find that shipping costs aren’t available on your cart page; that may be a potential point of frustration for users. Now, you have a good set of questions to ask of your analytics platform:
- How many users pogo-stick between your cart and the point of your checkout that contains the shipping information?
- How many people exit from there?
- How many convert?
You’re on your way to identifying a new test idea.
An Example
One of our clients is an apartment rental company. Their site allows people to browse apartment buildings in their area and find units for rent. If someone finds an apartment they like, they simply click the “Phone” or “Send Message” buttons to contact the property to set up a viewing or start an application. This lead is the primary metric we are optimizing.
Our first step when partnering with this company was to browse their site and put ourselves in their users’ shoes. This surfaced a hunch: Certain tabs at the top of the apartment building pages contained valuable content that we felt was more likely to induce a conversion than others.
This gave us a question to ask of analytics: Do visitors who have visited certain tabs convert at a higher rate than others? We used Google Analytics data to confirm our hunch and, sure enough, the “Reviews” tab was the most likely to result in a lead from a visitor. (Note: The photos tab is open by default on page load.)
Our hypothesis was simple: Ordering the tabs from left to right in order of likelihood to result in a conversion will result in higher performing tabs receiving more traffic, ultimately increasing conversions. The result? A +4.93% increase in leads.
Track relevant events in your analytics platform
If you’ve been testing for a while, this one may seem like a no-brainer. But I am constantly surprised at how many organizations still under-leverage event-tracking. If you’re building an experimentation program from the ground up, or if you have a redesign or new rollout coming up soon, take some time to map out the important actions and events you may want to track across your site.
Build a framework to organize and tag these events using Google Tag Manager or Dynamic Tag Manager for ease of execution. Engagement with your navigation or footer, clicks to calls-to-action, scroll depths, video plays, etc. are valuable actions that may help you uncover key insights based on these user behaviors.
You may be thinking, “that’s what heatmaps are for.” Heatmaps will show you what users are clicking on, but they won’t enable you to segment based on those actions and cross-reference conversion rate.
Imagine you have a product page featuring a video. A heatmap shows you that many of your users are clicking the video. In fact, it’s one of highest engaged elements on your page. The logical next step is to put more videos on your site, right?
Hold on. There’s an assumption built into that conclusion: That video views are correlated to conversions. If you are tracking video engagement in your analytics platform, you can segment users who watched the video and those who didn’t and see who actually converted. You may find, to your surprise, that those that don’t watch the video are more likely to convert. In this case, you have a counter-intuitive test idea; remove the video and see what happens to your conversion rate.
You may find event tracking somewhat daunting from a time and resource perspective. If that’s the case, you should consider tools like Heap or the auto-event tracking capabilities of Google Tag Manager to automate some of the work and enable event data collection automatically.
Segment segment segment
If they aren’t already, segments should be your best friend. Canned reports in analytics platforms will only tell you so much. Segments can unlock a whole new realm of untapped insights and custom segments are often a faster route to answering a question and providing the most granularity.
One of the most valuable ways you can use segments is to gain a better understanding of the actual users visiting pages on your site.
Analytics platforms often interchangeably report on sessions, pageviews, unique pageviews, users, etc. in pre-built reports—all of which have different definitions. Many A/B testing tools, however, operate on a “unique visitor” level. Building segments based on users will enable you to more accurately estimate things like audience size, conversion rate, and test duration using historical data from your analytics platform.
Mastering Micro-Segmentation for Personalization
The ultimate step-by-step guide for creating and targeting micro-segments to reach your most valuable users today
An Example
To do this in Google Analytics, for example, build a custom segment that utilizes the “conditions” section of the “advanced options” tab and set the condition to filter users (not sessions) that have visited a specific page—say your homepage.
You can then apply that segment to any other report to analyze the behavior of users that have interacted with the homepage at some point in their journey. Adding this segment to the audience view will show you your expected audience size. Adding it to your conversion views will show you how many conversions you can expect for an experiment targeting the homepage.
Another very helpful component is the sequences tab, which can enable you to build segments of users that follow a specific path and compare that journey to estimate audience size or conversion rate, etc. The custom segment feature is a treasure when it comes to detailed analysis.
Experiment execution
Using analytics to identify an experiment idea is only half the battle. Knowing what to track and how to measure the performance of that idea-turned-hypothesis is critical. To evaluate success and to gather insights and learning along the way, you need to ensure you’re tracking the right metrics.
There’s a lot of information out there on this topic, and I’m going to break from the herd on a few best practices. But the ultimate takeaway is this: Your experiments are only as good as the metrics you are tracking, and the thought you put into these metrics.
Define your metrics up front
In an experiment, you are likely tracking both primary metrics and secondary metrics.
Primary metrics should be consistent across many if not all of your experiments. If yours is an eCommerce company, your primary metric might be “Revenue per Visitor (RPV)” or “Transactions.” If your company is powered by leads, it may be “Leads” or “Sales Qualified Leads;” in SaaS, it may be “Product Trials.”
Your primary metric should be closely tied to your business’ bottom line and it should be very difficult to game. That is, you shouldn’t be able to easily influence your primary metric with a simple button color test.
Secondary metrics are often more experiment-specific and will change depending on what you’re testing. You may utilize a few guardrail metrics—like “Visits” or interactions with each step of your funnel—to suss out any noise in your primary metric. You may also track engagement metrics like clicks to a CTA, or visits to a specific page likely to be impacted by the changes you’re making.
Secondary metrics exist as both a sanity check on your primary metric and a learning or storytelling opportunity when your experiment concludes.
Track more secondary metrics than you think you should
Your primary metric is your guiding light for go/no-go roll-out decisions, but it can’t tell the entire story on its own. You need secondary metrics to help explain why your primary metric went up or down.
This is especially important when an experiment has a negative impact on your primary metric. It’s never easy to present negative numbers to your boss, but it becomes a lot easier if you have supporting metrics that tell the story of why and what next.
Secondary metrics can often point to how to turn the observed negative impact into a follow-up experiment because they paint a clearer picture of why users behaved the way they did.
An Experiment Example
Another partner of ours is an eCommerce company. One of their landing pages was a very high-performer; we wanted to test the same experience on the homepage to see if we could replicate this success. The variation featured a hero section that matched the look and feel of that successful landing page. This change, however, resulted in a -10.6% decrease in transactions.
If we had only been tracking a single primary metric, in this case, transactions, we would have closed this experiment as a “loser” and moved on to another idea without understanding why transactions decreased. Luckily, we were tracking secondary metrics.
Among the secondary metrics being tracked were clicks to a “How it Works” video embedded in both the control and the variation. As it turned out, clicks to that link went up +153.6% in the variation! A surprising result and one that got our attention: What if video content wasn’t as helpful as we thought? What if it was actually distracting users and hurting conversion rate on the site? We set out to answer that very question with a follow-up experiment on the company’s product pages.
These pages featured a prominent video link in the hero, similar to what we had tested on the homepage. We hypothesized that by reducing its prominence, and therefore user engagement with the video, and replacing that visual real estate with a static value proposition, we could lift conversion rate on these pages. Sure enough, the experience with the less prominent video resulted in a -29.4% decrease in video views, and a 33% increase in transactions.
A note.
The prevailing wisdom in experimentation and statistics is that the more metrics you track, the less reliable your results are, primarily due to the multiple comparison problem.
The reality is that, while this is true in conventional statistics, most experimentation platforms have adapted their statistical models to control for this. This means that your primary metric is considered in a near vacuum and will not be impacted by additional metrics you’re tracking for insight and learning purposes.
I also want to point out that, in most cases, our experiments are not a life or death situation. The statistics developed to support A/B testing have a deep-rooted history in clinical trials where stakes are very high and the margin of error has to be razor thin.
Unless you’re experimenting on something like a radical change to your core business strategy or pricing model, it’s unlikely that your changes require this level of mathematical scrutiny. The benefit of learning from the movement in your additional metrics likely outweighs any negative impact they may have to the reliability of your final outcome.
Integrate your testing tool with your analytics tool
As much as I’ve talked about making sure you define the right metrics up front, the reality is that you won’t always get that right.
Often times, our assumptions about how users will interact with an experience are wildly different from reality. Sometimes, you’ll see extreme swings in one direction or the other that are far off from your expected result and none of your secondary or “guard rail” metrics illuminate the “why.”
This is why integrating your experimentation platform with your analytics platform is so important. And if you follow my earlier advice and set up your event tracking first, you’ll be in a better position to turn a confusing result into an insight.
For example, you may find that a change in your experiment caused traffic to surge to an unexpected page that you didn’t think to track ahead of time. When your testing platform is integrated with your analytics platform, you’ll be able to utilize all the data already being collected and compare it across your variations to explain these unexpected results.
Analytics and Experiment Analysis
Results analysis is an essential piece of a sustainable experimentation program. Again, analytics allow you to evaluate the success of the experiment given your goals. You should be able to demonstrate the impact of your efforts and uncover new insights to generate a continuous stream of experiment ideas.
Determining Success
Of course, you need to determine whether your experiments perform better (or worse) than baseline. How you do that will depend heavily on your specific environment and the metrics you identified pre-experiment.
In a perfect world, if you saw your primary metric go up, you’d roll out the variation as the new experience; and if you saw it go down, you’d stick with the control. We don’t have the luxury of a perfect world, that’s why we’re optimizing!
In the real world—especially if yours is a larger organization or a longer conversion funnel—it can be difficult to achieve material, measurable impact on your primary metric at every stage of the funnel.
It’s not uncommon to find yourself in a situation where your primary metric looks flat, but you see good movement around your secondary metrics. In some of these cases, you may want to move forward with the variation for reasons beyond solely a lift in your primary metric. Experiments that are neutral (even losers) can still be winners.
For example, imagine you’re at an eCommerce company and you just ran an experiment on the homepage introducing a new featured products section. You find that transactions are flat at the end of the experiment, but engagement with the featured products is high and traffic to these specific product pages is improving as well.
You have succeeded in introducing a new promotional area on the homepage that doesn’t hurt transactions and creates an opportunity to feature products that may be high value to the business for other reasons. You also have a new promotional vehicle to experiment with. Perhaps other iterations of this area will lift conversion rates with the right featured products or messaging.
Demonstrating Impact
Projecting the impact of your experiments is key to getting and maintaining stakeholder buy-in, proving the value of your program, and ensuring your initiatives retain the budget they deserve.
You’re likely responsible for certain KPIs. You need to show that your experiments are having an impact on those KPIs and on your business’ bottom line. The goal of projecting the impact of your experiments is to prove the Return on Investment (ROI) of your initiatives and experiments, as well as demonstrating longer-term value and impact.
The common approach to these projections is to take the observed lift of your experiment and project that lift out over a period, commonly 12 months, to show how much additional revenue or how many additional conversions, leads, or transactions your experiment could potentially produce if it were to become the new baseline on the day the experiment was stopped.
Let’s explore a simple example to clarify that concept. Let’s say you run an experiment with one variation and a control for four weeks. Your variation results in a 5% lift in transactions at 95% confidence. Awesome! But now you need to show what that observed 5% lift means to the business in a more material way.
Over those four weeks, you received 200,000 total visitors to your experiment: 100,000 in the control and 100,000 in the variation. 9,500 people converted in the control, and 10,000 people converted in your variation (your 5% lift). You can take two approaches to project the impact of this experiment:
- Use the experiment data alone, or
- Use experiment data combined with historical data
In the first method, you are basing your projection solely on the data observed during the experiment. You aren’t considering any long-term seasonality or potential external factors outside of the experiment’s duration.
Using this method, you would project the traffic volume observed during your experiment over a full 52 weeks:
Then, you would calculate the difference between the Control conversion rate and the Variant conversion rate on those annual visitors:
That 13,000 is the projected estimated additional conversions you can expect based on the observed 5% lift during the experiment.
If you know your average order value is, say, $100, then you can take this one step further to calculate estimated additional annual revenue:
Using historical data is another approach, which may be more accurate for your business if you have significant seasonality that the experiment didn’t observe. In this approach, you would pull the previous 12 months of traffic and conversions to the page or pages your experiment targeted, and use those numbers in the same “observed lift” calculations above.
Instead of 2,600,000 annual users and 247,000 conversions, you may find that the page actually had 3,200,000 visitors over the last 12 months, and 270,000 conversions due to some higher traffic periods that weren’t accounted for during the experiment.
None of these calculations are perfect, which is why it’s important to label them as projections and estimations, but they’ll help you show a more tangible result from your work in numbers that may appeal more to those higher up in the organization.
Uncovering New Insights
In the end, uncovering new insights is what it’s all about. Of course, a primary goal of marketing experimentation is to improve revenue and lift conversion rates, but insights are the key to true, long-term success.
“Insights-driven businesses bring insight, not just data, into every decision, and they know exactly how to use them for greatest advantage across the entire customer life cycle. For these firms, digital insights and what they do with them are their secret weapons to disrupt your market and steal your customers.”
— Brian Hopkins, Ted Schadler, and James McCormick, Authors of Forrester’s report, The Insights-Driven Business
The tactics and suggestions outlined above will help you lay a foundation to start uncovering these insights, but it’s important to remember that your tools can’t do this for you. You will only uncover core insights if you put the effort into asking the right questions and collecting the right data to answer those questions.
If you do step into the realm of optimization analytics, you can join the ranks of insights-driven businesses that are at the forefront of their industries or disrupting others.
Continue reading
<!DOCTYPE html> <!--[if IE 6]><html id="ie6" lang="en-US"> <![endif]--> <!--[if IE 7]><html id="ie7" lang="en-US"> <![endif]--> <!--[if IE 8]><html id="ie8" lang="en-US"> <![endif]--> <!--[if !(IE 6) | !(IE 7) | !(IE 8) ]><!--><html lang="en-US"> <!--<![endif]--><head><meta name="msvalidate.01" content="1033997EC383621B874EB0392F261984"/><link rel="preconnect" href="//cdn.dynamicyield.com"><link rel="preconnect" href="//st.dynamicyield.com"><link rel="preconnect" href="//rcom.dynamicyield.com"><link rel="dns-prefetch" href="//cdn.dynamicyield.com"><link rel="dns-prefetch" href="//st.dynamicyield.com"><link rel="dns-prefetch" href="//rcom.dynamicyield.com"> <script data-cfasync="false">window.dataLayer = window.dataLayer || [];
window.gtag = function() { dataLayer.push(arguments); }
window.gtag('consent', 'default', {
ad_storage: 'denied',
ad_user_data: 'denied',
ad_personalization: 'denied',
analytics_storage: 'denied',
functionality_storage: 'denied',
personalization_storage: 'denied',
security_storage: 'granted', // Only this is granted by default
});</script> <script data-cfasync="false">(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
'//www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','GTM-MCQHF8');</script> <script data-cfasync="false" async>// Init the adobeDataLayer array
var adobeDataLayer = adobeDataLayer || [];
// Only when the API is ready
function adobeWaitForDY() {
if (typeof(window.DY) !== 'undefined' && typeof(window.DY.AdDetection) !== 'undefined' &&
typeof($dy) !== 'undefined' && typeof($dy(window)) !== 'undefined' && typeof($dy.renderSmartTagSlider) !== 'undefined') {
var zi_data = localStorage.getItem('ZoomInfoData'),
ada_page_info = {
"pageName": "The role of optimization analytics in experimentation",
"siteName": "Mastercard Dynamic Yield",
"pageTitle": "en_US :optimization-analytics-in-experimentation",
"pageURL": "https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/",
"country": "US",
"language": "en",
"dyID": DY.dyid,
"userStatus": "Guest"
};
// Detect the hierarchy and
// site sections based on the URL slugs
var current_pathname = window.location.pathname.replace(/^\/|\/$/g, '');
if(current_pathname) {
var current_sitesections = current_pathname.split('/'),
current_hierarchy = current_sitesections.map(function(segment) {
return segment.split(/[-_]/).map(function(word) {
return word.charAt(0).toUpperCase() + word.slice(1);
})
.join(' ');
});
// Site section 1
if(current_sitesections[0]) {
ada_page_info.siteSection1 = current_sitesections[0];
}
// Site section 2
if(current_sitesections[1]) {
ada_page_info.siteSection2 = current_sitesections[1];
}
// Hierarchy
ada_page_info.hierarchy = current_hierarchy.join('|');
}
// Push the ZI params as well
if(zi_data) {
zi_data = JSON.parse(zi_data);
if(zi_data.name) {
ada_page_info.ziCompany = zi_data.name;
}
if(zi_data.domain) {
ada_page_info.ziDomain = zi_data.domain;
}
if(zi_data.employeesrange) {
ada_page_info.ziEmployees = zi_data.employeesrange;
}
if(zi_data.industry) {
ada_page_info.ziIndustry = zi_data.industry;
}
}
// Decode HTML prior to dispatching the event
Object.keys(ada_page_info).forEach(function(key) {
ada_page_info[key] = $('<textarea/>').html(ada_page_info[key]).text();
});
adobeDataLayer.push({
"event": "pageLoad",
"pageInfo": ada_page_info
});
} else {
setTimeout(adobeWaitForDY, 150);
}
}
adobeWaitForDY();</script> <script type="text/javascript">function OptanonWrapper() {
console.log('OneTrust Loaded');
DY_API_update_consent_mode();
OneTrust.OnConsentChanged(DY_API_update_consent_mode);
}
function DY_API_update_consent_mode() {
var groups = OnetrustActiveGroups.split(',');
DY.userActiveConsent = {accepted: groups.includes('C0004')};
DYO.ActiveConsent.updateConsentAcceptedStatus(groups.includes('C0004'));
if(groups.includes('C0004')) {
console.log('OneTrust_Consent_Is_Granted');
DY.API('event',{ name: 'OneTrust_Consent_Is_Granted', properties: {}});
}
console.log('OneTrust - DY API is ' + (groups.includes('C0004') ? 'Enabled' : 'Disabled'));
}
window.addEventListener('OTConsentApplied', function() {
var groups = OnetrustActiveGroups.split(',');
if(groups.includes('C0004')) {
console.log('OneTrust_Initial_Consent_Granted');
DY.API('event',{ name: 'OneTrust_Initial_Consent_Granted', properties: {}});
}
});</script> <script type="text/javascript">/*WebFontConfig = {
google: { families: [ 'Shadows+Into+Light::latin', 'Handlee::latin', 'Roboto:400,100,300,500:latin' ] }
};
(function() {
var wf = document.createElement('script');
wf.src = ('https:' == document.location.protocol ? 'https' : 'http') +
'://ajax.googleapis.com/ajax/libs/webfont/1/webfont.js';
wf.type = 'text/javascript';
wf.async = 'true';
var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(wf, s);
})(); */
console.log(
'%cLet’s personalize experiences together.\n%cDynamic Yield is hiring smart engineers like you to help solve hard problems.\n%cSee our jobs page: https://www.dynamicyield.com/careers',
'font-size:2.2em;line-height:1.4em;margin-top:20px;color:rgb(93, 56, 185);',
'font-size:1.1em;line-height:1.6em;color:rgb(0, 0, 0);',
'font-size:1.1em;line-height:1.6em;margin-bottom:20px;color:rgb(0, 0, 0);'
);
function waitForElementToDisplay(selector, time, functionName) {
if (document.getElementById(selector) != null) {
functionName();
return;
} else {
setTimeout(function () {
waitForElementToDisplay(selector, time, functionName);
}, time);
}
}
function dcWaitForJQuery(time, functionName) {
if (window.jQuery || window.n2) {
functionName();
return;
} else {
setTimeout(function () {
dcWaitForJQuery(time, functionName);
}, time);
}
}</script> <meta name="msvalidate.01" content="451B75B70520F4C76D67F116C9EFFE5C"/><meta property="twitter:account_id" content="4503599629852947"/><meta name="p:domain_verify" content="f8112f57c09778e892850075a87dc1cf"/><meta name="google-site-verification" content="9FbmfdC4cM7JG8sgYh6w9HhFlxvEHewltexeUVzuSfE"/><meta charset="UTF-8"/><link rel="pingback" href="https://www.dynamicyield.com/xmlrpc.php"/> <!--[if lt IE 9]> <script src="https://www.dynamicyield.com/wp-content/themes/Divi/js/html5.js" type="text/javascript"></script> <![endif]--> <script type="text/javascript">document.documentElement.className = 'js';</script> <script type="text/javascript">var getLangCode = 'en';</script> <link rel="prefetch" href="/wp-content/themes/Divi/css/custom-style.css" as="style" crossorigin="anonymous"><meta name='robots' content='index, follow, max-image-preview:large, max-snippet:-1, max-video-preview:-1' /><link rel="alternate" hreflang="en" href="https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/" /><link rel="alternate" hreflang="x-default" href="https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/" /> <script data-cfasync="false" data-pagespeed-no-defer>var gtm4wp_datalayer_name = "dataLayer";
var dataLayer = dataLayer || [];</script> <title>The role of optimization analytics in experimentation — Mastercard Dynamic Yield</title><link rel="preload" as="font" href="https://www.dynamicyield.com/wp-content/themes/Divi/fonts/CircularXXTT/CircularXXWeb-Bold.woff2" crossorigin/><link rel="preload" as="font" href="https://www.dynamicyield.com/wp-content/themes/Divi/fonts/CircularXXTT/CircularXXWeb-Medium.woff2" crossorigin/><link rel="preload" as="font" href="https://www.dynamicyield.com/wp-content/themes/Divi/fonts/CircularXXTT/CircularXXWeb-Book.woff2" crossorigin/><meta name="description" content="An in-depth look at how optimization analytics and data can support marketers at every turn, ultimately pushing experimentation programs forward." /><link rel="canonical" href="https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/" /><meta property="og:locale" content="en_US" /><meta property="og:type" content="article" /><meta property="og:title" content="The role of optimization analytics in experimentation — Mastercard Dynamic Yield" /><meta property="og:description" content="An in-depth look at how optimization analytics and data can support marketers at every turn, ultimately pushing experimentation programs forward." /><meta property="og:url" content="https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/" /><meta property="og:site_name" content="Mastercard Dynamic Yield" /><meta property="article:publisher" content="https://www.facebook.com/DynamicYield" /><meta property="article:modified_time" content="2024-01-31T06:31:53+00:00" /><meta property="og:image" content="https://www.dynamicyield.com/wp-content/uploads/2019/05/The-role-of-optimization-analytics-in-experimentation-FB.jpg" /><meta property="og:image:width" content="1200" /><meta property="og:image:height" content="628" /><meta property="og:image:type" content="image/jpeg" /><meta name="twitter:card" content="summary_large_image" /><meta name="twitter:site" content="@DynamicYield" /><meta name="twitter:label1" content="Est. reading time" /><meta name="twitter:data1" content="18 minutes" /><meta name="twitter:label2" content="Written by" /><meta name="twitter:data2" content="James Flory" /> <script type="application/ld+json" class="yoast-schema-graph">{"@context":"https://schema.org","@graph":[{"@type":"WebPage","@id":"https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/","url":"https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/","name":"The role of optimization analytics in experimentation — Mastercard Dynamic Yield","isPartOf":{"@id":"https://www.dynamicyield.com/#website"},"primaryImageOfPage":{"@id":"https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/#primaryimage"},"image":{"@id":"https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/#primaryimage"},"thumbnailUrl":"https://www.dynamicyield.com/wp-content/uploads/2019/05/The-role-of-optimization-analytics-in-experimentation-Blog.jpg","datePublished":"2019-05-14T20:20:19+00:00","dateModified":"2024-01-31T06:31:53+00:00","description":"An in-depth look at how optimization analytics and data can support marketers at every turn, ultimately pushing experimentation programs forward.","breadcrumb":{"@id":"https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/#primaryimage","url":"https://www.dynamicyield.com/wp-content/uploads/2019/05/The-role-of-optimization-analytics-in-experimentation-Blog.jpg","contentUrl":"https://www.dynamicyield.com/wp-content/uploads/2019/05/The-role-of-optimization-analytics-in-experimentation-Blog.jpg","width":795,"height":320,"caption":"optimization analytics"},{"@type":"BreadcrumbList","@id":"https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://www.dynamicyield.com/"},{"@type":"ListItem","position":2,"name":"Lessons","item":"https://www.dynamicyield.com/lesson/"},{"@type":"ListItem","position":3,"name":"The role of optimization analytics in experimentation"}]},{"@type":"WebSite","@id":"https://www.dynamicyield.com/#website","url":"https://www.dynamicyield.com/","name":"Mastercard Dynamic Yield","description":"","publisher":{"@id":"https://www.dynamicyield.com/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://www.dynamicyield.com/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https://www.dynamicyield.com/#organization","name":"Mastercard Dynamic Yield","url":"https://www.dynamicyield.com/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https://www.dynamicyield.com/#/schema/logo/image/","url":"https://www.dynamicyield.com/wp-content/uploads/2019/10/DY-Hero_Logo-Black@x2.png","contentUrl":"https://www.dynamicyield.com/wp-content/uploads/2019/10/DY-Hero_Logo-Black@x2.png","width":458,"height":458,"caption":"Mastercard Dynamic Yield"},"image":{"@id":"https://www.dynamicyield.com/#/schema/logo/image/"},"sameAs":["https://www.facebook.com/DynamicYield","https://x.com/DynamicYield","https://www.instagram.com/dynamic.yield/","https://www.linkedin.com/company/dynamic-yield","http://www.pinterest.com/dynamicyield/","https://www.youtube.com/channel/UCPWEcR8lQRadVKYW8H4sndQ","https://en.wikipedia.org/wiki/Dynamic_Yield"]}]}</script> <link rel='dns-prefetch' href='//pages.dynamicyield.com' /><link rel="alternate" type="application/rss+xml" title="Mastercard Dynamic Yield » Feed" href="https://www.dynamicyield.com/feed/" /><link rel="alternate" type="application/rss+xml" title="Mastercard Dynamic Yield » Comments Feed" href="https://www.dynamicyield.com/comments/feed/" /><link rel="alternate" type="application/rss+xml" title="Mastercard Dynamic Yield » The role of optimization analytics in experimentation Comments Feed" href="https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/feed/" /><meta content="Divi v." name="generator"/><link rel='stylesheet' id='sbi_styles-css' href='https://www.dynamicyield.com/wp-content/plugins/instagram-feed/css/sbi-styles.min.css?ver=6.2.6' type='text/css' media='all' /><style id='wp-emoji-styles-inline-css' type='text/css'>img.wp-smiley, img.emoji {
display: inline !important;
border: none !important;
box-shadow: none !important;
height: 1em !important;
width: 1em !important;
margin: 0 0.07em !important;
vertical-align: -0.1em !important;
background: none !important;
padding: 0 !important;
}</style><link rel='stylesheet' id='wp-block-library-css' href='https://www.dynamicyield.com/wp-includes/css/dist/block-library/style.min.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><style id='classic-theme-styles-inline-css' type='text/css'>/*! This file is auto-generated */
.wp-block-button__link{color:#fff;background-color:#32373c;border-radius:9999px;box-shadow:none;text-decoration:none;padding:calc(.667em + 2px) calc(1.333em + 2px);font-size:1.125em}.wp-block-file__button{background:#32373c;color:#fff;text-decoration:none}</style><style id='global-styles-inline-css' type='text/css'>body{--wp--preset--color--black: #000000;--wp--preset--color--cyan-bluish-gray: #abb8c3;--wp--preset--color--white: #ffffff;--wp--preset--color--pale-pink: #f78da7;--wp--preset--color--vivid-red: #cf2e2e;--wp--preset--color--luminous-vivid-orange: #ff6900;--wp--preset--color--luminous-vivid-amber: #fcb900;--wp--preset--color--light-green-cyan: #7bdcb5;--wp--preset--color--vivid-green-cyan: #00d084;--wp--preset--color--pale-cyan-blue: #8ed1fc;--wp--preset--color--vivid-cyan-blue: #0693e3;--wp--preset--color--vivid-purple: #9b51e0;--wp--preset--gradient--vivid-cyan-blue-to-vivid-purple: linear-gradient(135deg,rgba(6,147,227,1) 0%,rgb(155,81,224) 100%);--wp--preset--gradient--light-green-cyan-to-vivid-green-cyan: linear-gradient(135deg,rgb(122,220,180) 0%,rgb(0,208,130) 100%);--wp--preset--gradient--luminous-vivid-amber-to-luminous-vivid-orange: linear-gradient(135deg,rgba(252,185,0,1) 0%,rgba(255,105,0,1) 100%);--wp--preset--gradient--luminous-vivid-orange-to-vivid-red: linear-gradient(135deg,rgba(255,105,0,1) 0%,rgb(207,46,46) 100%);--wp--preset--gradient--very-light-gray-to-cyan-bluish-gray: linear-gradient(135deg,rgb(238,238,238) 0%,rgb(169,184,195) 100%);--wp--preset--gradient--cool-to-warm-spectrum: linear-gradient(135deg,rgb(74,234,220) 0%,rgb(151,120,209) 20%,rgb(207,42,186) 40%,rgb(238,44,130) 60%,rgb(251,105,98) 80%,rgb(254,248,76) 100%);--wp--preset--gradient--blush-light-purple: linear-gradient(135deg,rgb(255,206,236) 0%,rgb(152,150,240) 100%);--wp--preset--gradient--blush-bordeaux: linear-gradient(135deg,rgb(254,205,165) 0%,rgb(254,45,45) 50%,rgb(107,0,62) 100%);--wp--preset--gradient--luminous-dusk: linear-gradient(135deg,rgb(255,203,112) 0%,rgb(199,81,192) 50%,rgb(65,88,208) 100%);--wp--preset--gradient--pale-ocean: linear-gradient(135deg,rgb(255,245,203) 0%,rgb(182,227,212) 50%,rgb(51,167,181) 100%);--wp--preset--gradient--electric-grass: linear-gradient(135deg,rgb(202,248,128) 0%,rgb(113,206,126) 100%);--wp--preset--gradient--midnight: linear-gradient(135deg,rgb(2,3,129) 0%,rgb(40,116,252) 100%);--wp--preset--font-size--small: 13px;--wp--preset--font-size--medium: 20px;--wp--preset--font-size--large: 36px;--wp--preset--font-size--x-large: 42px;--wp--preset--spacing--20: 0.44rem;--wp--preset--spacing--30: 0.67rem;--wp--preset--spacing--40: 1rem;--wp--preset--spacing--50: 1.5rem;--wp--preset--spacing--60: 2.25rem;--wp--preset--spacing--70: 3.38rem;--wp--preset--spacing--80: 5.06rem;--wp--preset--shadow--natural: 6px 6px 9px rgba(0, 0, 0, 0.2);--wp--preset--shadow--deep: 12px 12px 50px rgba(0, 0, 0, 0.4);--wp--preset--shadow--sharp: 6px 6px 0px rgba(0, 0, 0, 0.2);--wp--preset--shadow--outlined: 6px 6px 0px -3px rgba(255, 255, 255, 1), 6px 6px rgba(0, 0, 0, 1);--wp--preset--shadow--crisp: 6px 6px 0px rgba(0, 0, 0, 1);}:where(.is-layout-flex){gap: 0.5em;}:where(.is-layout-grid){gap: 0.5em;}body .is-layout-flow > .alignleft{float: left;margin-inline-start: 0;margin-inline-end: 2em;}body .is-layout-flow > .alignright{float: right;margin-inline-start: 2em;margin-inline-end: 0;}body .is-layout-flow > .aligncenter{margin-left: auto !important;margin-right: auto !important;}body .is-layout-constrained > .alignleft{float: left;margin-inline-start: 0;margin-inline-end: 2em;}body .is-layout-constrained > .alignright{float: right;margin-inline-start: 2em;margin-inline-end: 0;}body .is-layout-constrained > .aligncenter{margin-left: auto !important;margin-right: auto !important;}body .is-layout-constrained > :where(:not(.alignleft):not(.alignright):not(.alignfull)){max-width: var(--wp--style--global--content-size);margin-left: auto !important;margin-right: auto !important;}body .is-layout-constrained > .alignwide{max-width: var(--wp--style--global--wide-size);}body .is-layout-flex{display: flex;}body .is-layout-flex{flex-wrap: wrap;align-items: center;}body .is-layout-flex > *{margin: 0;}body .is-layout-grid{display: grid;}body .is-layout-grid > *{margin: 0;}:where(.wp-block-columns.is-layout-flex){gap: 2em;}:where(.wp-block-columns.is-layout-grid){gap: 2em;}:where(.wp-block-post-template.is-layout-flex){gap: 1.25em;}:where(.wp-block-post-template.is-layout-grid){gap: 1.25em;}.has-black-color{color: var(--wp--preset--color--black) !important;}.has-cyan-bluish-gray-color{color: var(--wp--preset--color--cyan-bluish-gray) !important;}.has-white-color{color: var(--wp--preset--color--white) !important;}.has-pale-pink-color{color: var(--wp--preset--color--pale-pink) !important;}.has-vivid-red-color{color: var(--wp--preset--color--vivid-red) !important;}.has-luminous-vivid-orange-color{color: var(--wp--preset--color--luminous-vivid-orange) !important;}.has-luminous-vivid-amber-color{color: var(--wp--preset--color--luminous-vivid-amber) !important;}.has-light-green-cyan-color{color: var(--wp--preset--color--light-green-cyan) !important;}.has-vivid-green-cyan-color{color: var(--wp--preset--color--vivid-green-cyan) !important;}.has-pale-cyan-blue-color{color: var(--wp--preset--color--pale-cyan-blue) !important;}.has-vivid-cyan-blue-color{color: var(--wp--preset--color--vivid-cyan-blue) !important;}.has-vivid-purple-color{color: var(--wp--preset--color--vivid-purple) !important;}.has-black-background-color{background-color: var(--wp--preset--color--black) !important;}.has-cyan-bluish-gray-background-color{background-color: var(--wp--preset--color--cyan-bluish-gray) !important;}.has-white-background-color{background-color: var(--wp--preset--color--white) !important;}.has-pale-pink-background-color{background-color: var(--wp--preset--color--pale-pink) !important;}.has-vivid-red-background-color{background-color: var(--wp--preset--color--vivid-red) !important;}.has-luminous-vivid-orange-background-color{background-color: var(--wp--preset--color--luminous-vivid-orange) !important;}.has-luminous-vivid-amber-background-color{background-color: var(--wp--preset--color--luminous-vivid-amber) !important;}.has-light-green-cyan-background-color{background-color: var(--wp--preset--color--light-green-cyan) !important;}.has-vivid-green-cyan-background-color{background-color: var(--wp--preset--color--vivid-green-cyan) !important;}.has-pale-cyan-blue-background-color{background-color: var(--wp--preset--color--pale-cyan-blue) !important;}.has-vivid-cyan-blue-background-color{background-color: var(--wp--preset--color--vivid-cyan-blue) !important;}.has-vivid-purple-background-color{background-color: var(--wp--preset--color--vivid-purple) !important;}.has-black-border-color{border-color: var(--wp--preset--color--black) !important;}.has-cyan-bluish-gray-border-color{border-color: var(--wp--preset--color--cyan-bluish-gray) !important;}.has-white-border-color{border-color: var(--wp--preset--color--white) !important;}.has-pale-pink-border-color{border-color: var(--wp--preset--color--pale-pink) !important;}.has-vivid-red-border-color{border-color: var(--wp--preset--color--vivid-red) !important;}.has-luminous-vivid-orange-border-color{border-color: var(--wp--preset--color--luminous-vivid-orange) !important;}.has-luminous-vivid-amber-border-color{border-color: var(--wp--preset--color--luminous-vivid-amber) !important;}.has-light-green-cyan-border-color{border-color: var(--wp--preset--color--light-green-cyan) !important;}.has-vivid-green-cyan-border-color{border-color: var(--wp--preset--color--vivid-green-cyan) !important;}.has-pale-cyan-blue-border-color{border-color: var(--wp--preset--color--pale-cyan-blue) !important;}.has-vivid-cyan-blue-border-color{border-color: var(--wp--preset--color--vivid-cyan-blue) !important;}.has-vivid-purple-border-color{border-color: var(--wp--preset--color--vivid-purple) !important;}.has-vivid-cyan-blue-to-vivid-purple-gradient-background{background: var(--wp--preset--gradient--vivid-cyan-blue-to-vivid-purple) !important;}.has-light-green-cyan-to-vivid-green-cyan-gradient-background{background: var(--wp--preset--gradient--light-green-cyan-to-vivid-green-cyan) !important;}.has-luminous-vivid-amber-to-luminous-vivid-orange-gradient-background{background: var(--wp--preset--gradient--luminous-vivid-amber-to-luminous-vivid-orange) !important;}.has-luminous-vivid-orange-to-vivid-red-gradient-background{background: var(--wp--preset--gradient--luminous-vivid-orange-to-vivid-red) !important;}.has-very-light-gray-to-cyan-bluish-gray-gradient-background{background: var(--wp--preset--gradient--very-light-gray-to-cyan-bluish-gray) !important;}.has-cool-to-warm-spectrum-gradient-background{background: var(--wp--preset--gradient--cool-to-warm-spectrum) !important;}.has-blush-light-purple-gradient-background{background: var(--wp--preset--gradient--blush-light-purple) !important;}.has-blush-bordeaux-gradient-background{background: var(--wp--preset--gradient--blush-bordeaux) !important;}.has-luminous-dusk-gradient-background{background: var(--wp--preset--gradient--luminous-dusk) !important;}.has-pale-ocean-gradient-background{background: var(--wp--preset--gradient--pale-ocean) !important;}.has-electric-grass-gradient-background{background: var(--wp--preset--gradient--electric-grass) !important;}.has-midnight-gradient-background{background: var(--wp--preset--gradient--midnight) !important;}.has-small-font-size{font-size: var(--wp--preset--font-size--small) !important;}.has-medium-font-size{font-size: var(--wp--preset--font-size--medium) !important;}.has-large-font-size{font-size: var(--wp--preset--font-size--large) !important;}.has-x-large-font-size{font-size: var(--wp--preset--font-size--x-large) !important;}
.wp-block-navigation a:where(:not(.wp-element-button)){color: inherit;}
:where(.wp-block-post-template.is-layout-flex){gap: 1.25em;}:where(.wp-block-post-template.is-layout-grid){gap: 1.25em;}
:where(.wp-block-columns.is-layout-flex){gap: 2em;}:where(.wp-block-columns.is-layout-grid){gap: 2em;}
.wp-block-pullquote{font-size: 1.5em;line-height: 1.6;}</style><link rel='stylesheet' id='rfw-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_7c036e0987ef80b703d7eb1abf922895.css?ver=2026020733' type='text/css' media='all' /><link rel='stylesheet' id='divi-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_ab3666a686d3dac03fd3032e5757637e.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='ctt-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_f1d1b2f761cf3768f427cdc5c4c2544b.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='inline-tweet-sharer-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_daa32ca85e114a0c6e5d1432c89237f3.css?ver=2.6.4' type='text/css' media='all' /><link rel='stylesheet' id='et_lb_modules-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_c0102af27938567cb82cc02e4da3a673.css?ver=2.4' type='text/css' media='all' /><link rel='stylesheet' id='et-shortcodes-css-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_40cb404848bc2e556105c8aea77f29aa.css?ver=3.0' type='text/css' media='all' /><link rel='stylesheet' id='et-shortcodes-responsive-css-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_3a30a77569bc4544b9465acbee8e5f97.css?ver=3.0' type='text/css' media='all' /><link rel='stylesheet' id='wp-featherlight-css' href='https://www.dynamicyield.com/wp-content/plugins/wp-featherlight/css/wp-featherlight.min.css?ver=1.3.4' type='text/css' media='all' /><link rel='stylesheet' id='baskerville-font-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_8c23179f030d02e0065d2054eab34279.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='markformc-font-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_0eb51906eca648ca4559d0a03b898d83.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='mackinac-font-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_9c8fec2609ebc9764a6e16471d9451ae.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='button-pulse-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_4cabc8dd29eaa9af7bf2df62f1c40af9.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='fontawesome-css' href='https://www.dynamicyield.com/wp-content/themes/Divi/custom/font-awesome/css/font-awesome.min.css' type='text/css' media='all' /><link rel='stylesheet' id='bootstrap-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_d375e60add46d8de26a78728b517660f.css' type='text/css' media='all' /><link rel='stylesheet' id='slick-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_13b1b6672b8cfb0d9ae7f899f1c42875.css' type='text/css' media='all' /><link rel='stylesheet' id='slick-theme-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_f9faba678c4d6dcfdde69e5b11b37a2e.css' type='text/css' media='all' /><link rel='stylesheet' id='custom-theme-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_57f41281576414370c141cd00d2a80a9.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='custom-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_417039b6d54561fe1e8c94ee67cd902c.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='header-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_d75ad7543415df427913a5fe91606386.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='footer-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_25400e1cfd7099cc557083769c928231.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='video_tabs_module-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_f1109a8bc8c46ec9f2205cc8603c132f.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='mkto-reset-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_d117c8ecde800f5dccd137c16bab71af.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='learning-center-general-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_73dd3b78cf6e21c28be39db6740774b1.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='article-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_e1d850b2d8d6fd6331ad74bd35ddde51.css?ver=1772264034' type='text/css' media='all' /><link rel='stylesheet' id='customers-shortcode-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_c6bf61e9c8beb560d64d119f58757b8b.css?ver=26.27' type='text/css' media='all' /> <script type="text/javascript" src="https://www.dynamicyield.com/wp-includes/js/jquery/jquery.min.js?ver=3.7.1" id="jquery-core-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-includes/js/jquery/jquery-migrate.min.js?ver=3.4.1" id="jquery-migrate-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_dbdab0f35217f6041eb7e2028dc94ba4.js?ver=2.6.4" id="inline-tweet-sharer-js-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/themes/Divi/js/blazy.min.js" id="blazy-js"></script> <script type="text/javascript" src="//pages.dynamicyield.com/js/forms2/js/forms2.min.js?ver=21.85" id="mkto-form-script-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_db6be3b57dba139e789fec1a78186c8e.js?ver=21.85" id="learning-center-js-js"></script> <link rel="https://api.w.org/" href="https://www.dynamicyield.com/wp-json/" /><link rel="EditURI" type="application/rsd+xml" title="RSD" href="https://www.dynamicyield.com/xmlrpc.php?rsd" /><link rel='shortlink' href='https://www.dynamicyield.com/?p=97849' /><link rel="alternate" type="application/json+oembed" href="https://www.dynamicyield.com/wp-json/oembed/1.0/embed?url=https%3A%2F%2Fwww.dynamicyield.com%2Flesson%2Foptimization-analytics-in-experimentation%2F" /><link rel="alternate" type="text/xml+oembed" href="https://www.dynamicyield.com/wp-json/oembed/1.0/embed?url=https%3A%2F%2Fwww.dynamicyield.com%2Flesson%2Foptimization-analytics-in-experimentation%2F&format=xml" /><meta name="generator" content="WPML ver:4.6.8 stt:1,4,3,28,2;" /> <script type="text/javascript">(function(url){
if(/(?:Chrome\/26\.0\.1410\.63 Safari\/537\.31|WordfenceTestMonBot)/.test(navigator.userAgent)){ return; }
var addEvent = function(evt, handler) {
if (window.addEventListener) {
document.addEventListener(evt, handler, false);
} else if (window.attachEvent) {
document.attachEvent('on' + evt, handler);
}
};
var removeEvent = function(evt, handler) {
if (window.removeEventListener) {
document.removeEventListener(evt, handler, false);
} else if (window.detachEvent) {
document.detachEvent('on' + evt, handler);
}
};
var evts = 'contextmenu dblclick drag dragend dragenter dragleave dragover dragstart drop keydown keypress keyup mousedown mousemove mouseout mouseover mouseup mousewheel scroll'.split(' ');
var logHuman = function() {
if (window.wfLogHumanRan) { return; }
window.wfLogHumanRan = true;
var wfscr = document.createElement('script');
wfscr.type = 'text/javascript';
wfscr.async = true;
wfscr.src = url + '&r=' + Math.random();
(document.getElementsByTagName('head')[0]||document.getElementsByTagName('body')[0]).appendChild(wfscr);
for (var i = 0; i < evts.length; i++) {
removeEvent(evts[i], logHuman);
}
};
for (var i = 0; i < evts.length; i++) {
addEvent(evts[i], logHuman);
}
})('//www.dynamicyield.com/?wordfence_lh=1&hid=CE0F4250670A2836BB27C6B712A19994');</script>
<script data-cfasync="false" data-pagespeed-no-defer>var dataLayer_content = {"visitorLoginState":"logged-out","visitorType":"visitor-logged-out","pagePostType":"lesson","pagePostType2":"single-lesson","pagePostAuthor":"James Flory","pagePostDate":"May 14, 2019","pagePostDateYear":"2019","pagePostDateMonth":"05","pagePostDateDay":"14","pagePostDateDayName":"Tuesday","pagePostDateHour":"16","pagePostDateMinute":"20","pagePostDateIso":"2019-05-14T16:20:19-04:00","pagePostDateUnix":1557850819};
dataLayer.push( dataLayer_content );</script> <script type="text/javascript">console.warn && console.warn("[GTM4WP] Google Tag Manager container code placement set to OFF !!!");
console.warn && console.warn("[GTM4WP] Data layer codes are active but GTM container must be loaded using custom coding !!!");</script> <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=3.0, user-scalable=1" /><link rel="icon" type="image/png" href="/favicon-16x16.png" sizes="16x16"><link rel="icon" type="image/png" href="/favicon-32x32.png" sizes="32x32"><link rel="icon" type="image/png" href="/favicon-96x96.png" sizes="96x96"><link rel="icon" type="image/png" href="/favicon-180x180.png" sizes="180x180"><link rel="icon" type="image/png" href="/favicon-192x192.png" sizes="192x192"><link rel="icon" type="image/png" href="/favicon-194x194.png" sizes="194x194"><link rel="shortcut icon" href="/favicon.ico"> <script type="text/javascript" data-noptimize>function dy_bind_adobe_mkto_events(form, form_id, form_name) {
// Init the adobeDataLayer object
window.adobeDataLayer = window.adobeDataLayer || [];
var focus_reported = false;
var form_completed = false;
var form_started_reported = false;
// Monitor input interactions - first focus (form start)
form.getFormElem().find('input, select').on('focus', function() {
// Only if it hasn't been reported yet
if(! focus_reported) {
focus_reported = true;
form_started_reported = true;
adobeDataLayer.push({
"event": "form_start",
"trackAction": {
"formName": form_name,
"formID": form_id,
"zoomInfo": localStorage.ZoomInfoData !== undefined,
}
});
}
});
// Detect dropoff if the form hasn't been completed
window.addEventListener("beforeunload", function(event) {
if (form_completed) return;
var filledLabels = [];
var count = 0;
form.getFormElem().find('input, select').each(function() {
var value = this.value && this.value.trim();
if (value) {
var label = form.getFormElem().find("label[for='" + this.id + "']").text().trim();
if (label) {
filledLabels.push(label);
count++;
}
}
});
if (count > 0) {
adobeDataLayer.push({
"event": "form_dropoff",
"formInfo": {
"formName": form_name,
"formID": form_id,
"zoomInfo": localStorage.ZoomInfoData !== undefined,
"completedFields": {
"labels": filledLabels.join("|"),
"count": count.toString()
}
}
});
}
});
// Hook into successful submission
form.onSuccess(function(values, followUpUrl) {
if(form_started_reported) {
// Report form completed
form_completed = true;
// Send an Adobe Analytics DataLayer event
adobeDataLayer.push({
"event": "form_complete",
"formInfo": {
"formName": form_name,
"formID": form_id,
"zoomInfo": localStorage.ZoomInfoData !== undefined,
"successAction": {
"action": "redirect",
"redirectUrl": followUpUrl
}
}
});
}
});
}
function dy_bind_adobe_cta_event(cta, cta_type, cta_section) {
var cta_name = cta.text().trim(),
cta_url,
page_url = window.location.href;
// const domainName = window.location.origin;
if (cta.attr('href') !== undefined) {
// cta_url = domainName + cta.attr('href');
cta_url = cta.attr('href');
} else {
cta_url = page_url;
}
adobeDataLayer.push({
"event": "ctaClick",
"clickInfo": {
"ctaName": cta_name,
"ctaType": cta_type,
"destinationURL": cta_url,
"ctaLocation": page_url,
"ctaSection": cta_section
}
});
}</script> <link rel="shortcut icon" href="https://www.dynamicyield.com/wp-content/uploads/2018/11/favicon-194x194.png" /><link rel="amphtml" href="https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/amp/"></head><body class="lesson-template-default single single-lesson postid-97849 wp-featherlight-captions et_fixed_nav osx et_right_sidebar safari et_includes_sidebar elementor-default default_header learning-center"><div id="page-container"><header id="main-header" class="et_nav_text_color_dark"
style="background: #fff;"><div class="container-fluid clearfix"><div class="logo-combined"> <a href="https://www.dynamicyield.com/learn"
title="XP Squared"><svg xmlns="http://www.w3.org/2000/svg" width="112.004" height="53.42" viewBox="0 0 112.004 53.42"> <path d="M8968.651,4606.571a49.057,49.057,0,0,0-6.159,6.344q-1.263,1.485-2.782,3.339t-3.079,3.71q-1.559,1.856-2.969,3.6t-2.522,3.079q.3.445,1.261,1.781t2.227,3.19q1.26,1.856,2.745,3.97t2.856,4.044q1.373,1.931,2.486,3.488t1.633,2.375q1.484,2.153,2.708,3.858a25.747,25.747,0,0,0,2.411,2.931,12.808,12.808,0,0,0,2.449,2,7.254,7.254,0,0,0,2.819,1v.742h-24.041v-.742q1.039-.147,2.226-.333a12.155,12.155,0,0,0,2.227-.557,4.522,4.522,0,0,0,1.706-1.039,2.31,2.31,0,0,0,.668-1.707,6.3,6.3,0,0,0-.965-2.783q-.964-1.743-2.226-3.673-1.708-2.672-3.9-5.825t-5.009-7.086q-1.188,1.411-2.6,3.116t-2.819,3.451q-1.411,1.744-2.708,3.376t-2.189,2.894a32.577,32.577,0,0,0-1.854,2.968,5.867,5.867,0,0,0-.816,2.82,2.706,2.706,0,0,0,.668,1.818,5.342,5.342,0,0,0,1.707,1.3,10.711,10.711,0,0,0,2.226.816,21.722,21.722,0,0,0,2.226.445v.742H8919.6v-.742a18.531,18.531,0,0,0,6.158-3.228,45.326,45.326,0,0,0,6.53-6.2q1.112-1.261,2.745-3.228t3.413-4.081q1.783-2.115,3.414-4.118t2.745-3.339q-.445-.593-2.04-2.856t-3.6-5.083q-2-2.819-3.97-5.6t-3.153-4.415q-1.559-2.152-2.782-3.784a26.076,26.076,0,0,0-2.375-2.782,12.747,12.747,0,0,0-2.449-1.93,13.428,13.428,0,0,0-3-1.3v-.742h24.189v.742q-.965.075-2.227.3a14.111,14.111,0,0,0-2.3.593,5.268,5.268,0,0,0-1.78,1.039,2.194,2.194,0,0,0-.743,1.707,5.789,5.789,0,0,0,1.039,2.746q1.039,1.708,2.374,3.636l3.785,5.454q1.928,2.782,4.675,6.715,2.449-2.818,5.009-6.159t4.488-5.787a31.259,31.259,0,0,0,1.93-2.968,6.278,6.278,0,0,0,1.039-2.968,2.5,2.5,0,0,0-.705-1.781,6.393,6.393,0,0,0-1.707-1.262,10.075,10.075,0,0,0-2.188-.816q-1.188-.3-2.152-.445v-.742h19.367v.742A15.309,15.309,0,0,0,8968.651,4606.571Z" transform="translate(-8919.604 -4602.453)"/> <path d="M9422.068,4621.77a14.179,14.179,0,0,1-2.338,3.932,14.748,14.748,0,0,1-3.376,2.931,20.786,20.786,0,0,1-3.934,1.967,24.367,24.367,0,0,1-4.451,1.113q-2.376.373-4.787.557t-4.711.186h-4.008v6.975q0,1.263.037,2.338t.038,2.337q.072,2.376.148,4.081a5.411,5.411,0,0,0,.815,2.82,6.25,6.25,0,0,0,2.3,1.929,10.813,10.813,0,0,0,4.75,1.113v.742h-23.745v-.742a20.734,20.734,0,0,0,3.561-.89,5.8,5.8,0,0,0,2.672-1.781,3.388,3.388,0,0,0,.594-1.336,14.222,14.222,0,0,0,.334-2q.111-1.113.186-2.375t.074-2.449c0-.692.011-1.632.037-2.82s.048-2.485.074-3.9.037-2.856.037-4.341v-18.773q0-1.334-.037-2.708c-.026-.915-.074-1.781-.148-2.6a13.947,13.947,0,0,0-.371-2.226,3.157,3.157,0,0,0-.705-1.447,5.554,5.554,0,0,0-2.856-1.6q-1.745-.407-3.748-.705v-.742h1.781q1.409,0,3.3-.037t4.117-.037q2.227,0,4.415-.037t4.119-.037h3.339a69.43,69.43,0,0,1,8.682.482,19.691,19.691,0,0,1,7.123,2.337,15.1,15.1,0,0,1,5.417,5.046,13.712,13.712,0,0,1,2.152,7.717A13.3,13.3,0,0,1,9422.068,4621.77Zm-9.312-12.986a13.487,13.487,0,0,0-9.535-6.159,28.427,28.427,0,0,0-4.155-.3,20,20,0,0,0-2.412.111q-.854.112-1.372.186a5.343,5.343,0,0,1-.186.779,4.946,4.946,0,0,0-.187.853q-.073.521-.147,1.447t-.149,2.634q-.075,2.6-.111,6.307t-.037,7.272v9.5q3.042,0,5.715-.186a18.593,18.593,0,0,0,4.971-1,13.885,13.885,0,0,0,7.271-5.379,14.741,14.741,0,0,0,2.523-8.422A13.63,13.63,0,0,0,9412.756,4608.784Z" transform="translate(-9329.39 -4601.215)"/> <g transform="translate(98.35)"> <path d="M9839.271,4616.067a10.2,10.2,0,0,1,3.648-3.89,26.827,26.827,0,0,0,3.232-2.56,3.789,3.789,0,0,0,1.127-2.635,2.928,2.928,0,0,0-.646-1.939,2.637,2.637,0,0,0-4.094.456,6.113,6.113,0,0,0-.4,2.256h-3.512a8.885,8.885,0,0,1,.85-3.789q1.447-2.75,5.139-2.75a6.52,6.52,0,0,1,4.643,1.616,5.584,5.584,0,0,1,1.727,4.277,5.783,5.783,0,0,1-1.221,3.624,12.091,12.091,0,0,1-2.627,2.345l-1.447,1.026q-1.359.964-1.859,1.395a4.076,4.076,0,0,0-.844,1h8.023v3.181h-12.584A8.616,8.616,0,0,1,9839.271,4616.067Z" transform="translate(-9838.422 -4601.215)"/> </g> </svg></a> <a href="https://www.dynamicyield.com/"
title="Dynamic Yield"><svg xmlns="http://www.w3.org/2000/svg" width="112" height="14.15" viewBox="0 0 112 14.15"> <g> <path d="M8919.6,5180.139v-10.982h1.729v4.505a2.662,2.662,0,0,1,2.321-1.107c2.214,0,3.412,1.684,3.412,3.853,0,2.23-1.32,3.914-3.473,3.914a2.538,2.538,0,0,1-2.276-1.168v.986Zm3.7-6.021a2.035,2.035,0,0,0-1.987,2.305,2.071,2.071,0,0,0,1.987,2.336c1.153,0,1.986-.88,1.986-2.336S8924.473,5174.118,8923.3,5174.118Z" transform="translate(-8919.604 -5168.976)"/> <path d="M8990.422,5213.118l1.774-3.868-3.154-6.507h1.987l2.124,4.672,1.986-4.672h1.866l-4.7,10.375Z" transform="translate(-8981.609 -5198.966)"/> <path d="M9105.4,5178.79c0,.5.031,1.046.061,1.35h-1.926a5.038,5.038,0,0,1-.076-.864,2.34,2.34,0,0,1-2.094,1.046,3.926,3.926,0,0,1-.061-7.827,2.238,2.238,0,0,1,2.109.925v-4.263h1.986Zm-3.793-.272a1.9,1.9,0,0,0,1.836-2.14,1.849,1.849,0,0,0-1.836-2.078,2.127,2.127,0,0,0,0,4.217Z" transform="translate(-9078.662 -5168.976)"/> <path d="M9178.639,5212.441l1.76-3.884-3.17-6.522h2.26l1.987,4.369,1.835-4.369h2.154l-4.687,10.406Z" transform="translate(-9149.652 -5198.333)"/> <path d="M9264.135,5207.852h-2.019v-7.462h1.958v.925a2.476,2.476,0,0,1,2.153-1.123c1.851,0,2.7,1.32,2.7,2.958v4.7h-2.019V5203.5a1.312,1.312,0,0,0-1.38-1.486,1.4,1.4,0,0,0-1.395,1.547Z" transform="translate(-9225.454 -5196.688)"/> <path d="M9339.31,5203.306l1.835-.272c.425-.061.562-.274.562-.531,0-.531-.409-.971-1.259-.971a1.325,1.325,0,0,0-1.427,1.213l-1.79-.379a2.969,2.969,0,0,1,3.2-2.458c2.366,0,3.247,1.335,3.247,2.836v3.671a8.454,8.454,0,0,0,.09,1.183h-1.851a5.105,5.105,0,0,1-.076-.9,2.494,2.494,0,0,1-2.2,1.107,2.357,2.357,0,0,1-2.563-2.246A2.3,2.3,0,0,1,9339.31,5203.306Zm2.4,1.274v-.334l-1.685.258a.942.942,0,0,0-.925.94.876.876,0,0,0,.971.864A1.516,1.516,0,0,0,9341.706,5204.581Z" transform="translate(-9292.393 -5196.435)"/> <path d="M9414.173,5207.726v-7.463h1.927v.91a2.565,2.565,0,0,1,2.185-1.123,2.308,2.308,0,0,1,2.214,1.244,2.76,2.76,0,0,1,5.066,1.592v4.839h-1.956V5203.3a1.245,1.245,0,0,0-1.32-1.411,1.372,1.372,0,0,0-1.381,1.472v4.369h-2V5203.3a1.252,1.252,0,0,0-1.319-1.411,1.366,1.366,0,0,0-1.4,1.472v4.369Z" transform="translate(-9361.234 -5196.561)"/> <path d="M9535.157,5167.457a1.244,1.244,0,1,1-1.243,1.259A1.243,1.243,0,0,1,9535.157,5167.457Zm-1,11.165v-7.463h2.018v7.463Z" transform="translate(-9468.158 -5167.457)"/> <path d="M9568.379,5203.868a1.921,1.921,0,0,0,1.91,2.108,1.707,1.707,0,0,0,1.715-1.259l1.775.592a3.466,3.466,0,0,1-3.49,2.518,3.861,3.861,0,0,1-3.928-3.959,3.821,3.821,0,0,1,3.867-3.959,3.42,3.42,0,0,1,3.506,2.519l-1.807.607a1.626,1.626,0,0,0-1.652-1.26A1.9,1.9,0,0,0,9568.379,5203.868Z" transform="translate(-9497.133 -5196.435)"/> <path d="M9669.8,5212.441l1.761-3.884-3.171-6.522h2.26l1.987,4.369,1.836-4.369h2.153l-4.687,10.406Z" transform="translate(-9588.244 -5198.333)"/> <path d="M9752.258,5167.457a1.244,1.244,0,1,1-1.244,1.259A1.244,1.244,0,0,1,9752.258,5167.457Zm-1,11.165v-7.463h2.018v7.463Z" transform="translate(-9662.02 -5167.457)"/> <path d="M9790.711,5205.46a3.36,3.36,0,0,1-3.4,2.366,3.966,3.966,0,0,1-.182-7.918c2.3,0,3.687,1.472,3.687,3.868,0,.289-.031.592-.031.622h-5.34a1.818,1.818,0,0,0,1.881,1.7,1.647,1.647,0,0,0,1.7-1.137Zm-1.881-2.442a1.5,1.5,0,0,0-1.668-1.472,1.615,1.615,0,0,0-1.669,1.472Z" transform="translate(-9690.994 -5196.435)"/> <path d="M9863.529,5180.139v-10.982h2.016v10.982Z" transform="translate(-9762.492 -5168.976)"/> <path d="M9901.386,5178.79c0,.5.03,1.046.061,1.35h-1.927a5.163,5.163,0,0,1-.075-.864,2.341,2.341,0,0,1-2.094,1.046,3.926,3.926,0,0,1-.061-7.827,2.234,2.234,0,0,1,2.108.925v-4.263h1.987Zm-3.793-.272a1.9,1.9,0,0,0,1.836-2.14,1.85,1.85,0,0,0-1.836-2.078,2.127,2.127,0,0,0,0,4.217Z" transform="translate(-9789.442 -5168.976)"/> </g> </svg></a></div><div id="et-top-navigation" class="cd-morph-dropdown"><nav id="top-menu-nav" class="main-nav"><ul id="top-menu" class="nav"></li></li></li></li></li></li></li></li></li></li></li></li></li><li class="top-link menu-item menu-item-type-post_type menu-item-object-page menu-item-home" data-content="discover-dynamic-yield" title=""> <a href="https://www.dynamicyield.com/">Discover Dynamic Yield</a></li><li class="has-dropdown learning_center menu-has-cta top-link-use-cases menu-item menu-item-type-custom menu-item-object-custom menu-item-has-children" data-content="learning-center" title=""> <a href="">Learning Center</a></li><li class="contact top-link walker-last mobile-only menu-item menu-item-type-post_type menu-item-object-page" data-content="xp2-newsletter" title=""> <a href="https://www.dynamicyield.com/newsletter/"><span class="button-arrow">XP<sup>2</sup> Newsletter</span></a></li></ul></nav><div class="morph-dropdown-wrapper"><div class="dropdown-list"><div class="dropdown-tooltip"></div><ul><li class="top-link menu-item menu-item-type-post_type menu-item-object-page menu-item-home" data-content="discover-dynamic-yield" title=""> <a href="https://www.dynamicyield.com/">Discover Dynamic Yield</a></li><li id="learning-center" class="dropdown links"><a href="" class="label">Learning Center</a><div class="content"><ul><li class="top"><ul class="links-list"><li class="resources-xp hidden-mobile menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/learn/">Take your knowledge to exponential levels</a></li><li class="hidden-desktop menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/learn/">XP<sup>2</sup> Hub<span>Take your knowledge to exponential levels</span></a></li><li class="resources-paths menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/learning-paths/">Learning Paths<span>Curated courses on key skill areas</span></a></li><li class="resources-webinars menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/talks/">Talks<span>Engaging discussions taking place in CX</span></a></li><li class="resources-articles menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/articles/">Articles<span>An expansive collection of in-depth playbooks</span></a></li><li class="resources-glossary menu-item menu-item-type-custom menu-item-object-custom"><a href="/glossary-terms/">Encyclopedia<span>A glossary of experience optimization terms</span></a></li><li class="resources-use-cases menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/personalization-examples/">Inspiration Library<span>Personalization examples from real brands</span></a></li><li class="resources-maturity menu-item menu-item-type-custom menu-item-object-custom"><a href="/personalization-maturity/"><strong class="badge-new">Personalization Maturity</strong><span>How global businesses prioritize personalization</span></a></li><li class="resources-guides menu-item menu-item-type-custom menu-item-object-custom"><a href="/guides/">Guides & reports<span>Comprehensive topic-specific materials</span></a></li><li class="resources-benchmarks menu-item menu-item-type-custom menu-item-object-custom"><a href="https://marketing.dynamicyield.com/benchmarks/">Benchmarks<span>Industry performance metrics and insights</span></a></li></ul></li><li class="bottom"><a href="/rooted-personalization/" target="_self"><div class="menu-banner"><div class="menu-cta--left"> <img src="/wp-content/uploads/2023/03/rooted-icon.svg" width="60" height="60" /></div><div class="menu-cta--right"> <strong class="menu-cta-title">Rooted Personalization<span>new</span></strong> <span class="menu-cta-subtitle">Learn a scalable methodology for building an audience-based personalization program.</span> <span class="menu-cta-button">Find out more →</span></div></div></a></li></ul></div></li><li class="contact top-link walker-last mobile-only menu-item menu-item-type-post_type menu-item-object-page" data-content="xp2-newsletter" title=""> <a href="https://www.dynamicyield.com/newsletter/"><span class="button-arrow">XP<sup>2</sup> Newsletter</span></a></li></ul><div class="bg-layer" aria-hidden="true"></div></div></div></li></ul></nav></div><div class="topnav-buttons"><div class="language-switcher header-language-switcher"><ul><li>en<ul><li><a class="active" href="https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/">English</a></li><li><a href="https://www.dynamicyield.com/es/">Español</a></li><li><a href="https://www.dynamicyield.com/de/">Deutsch</a></li><li><a href="https://www.dynamicyield.com/fr/">Français</a></li><li><a href="https://www.dynamicyield.com/ja/">日本語</a></li></ul></li></li></ul></div> <a href="#" data-toggle="modal" data-target="#lc-subscription"
class="btn-desktop-only pulse-button v5 pulsed topnav-btn"> <span class="button-inner"> XP² Newsletter </span> </a> <a href="/newsletter/" class="btn-mobile-only pulse-button v5 pulsed topnav-btn"> <span class="button-inner"> XP² Newsletter </span> </a> <a href="" class="nav-trigger"> Open Nav <span class="nav-trigger-box"><span class="nav-trigger-inner"></span></span> </a></div></div></header><div id="lc-subscription" class="modal lc-subscription-popup" tabindex="-1" role="dialog" data-backdrop="false" aria-labelledby="lcSubscriptionModal"><div class="lc-container"><div class="lc-subscription-block"> <span class="lc-close-subscription-popup" data-dismiss="modal" aria-label="close"><i class="fa fa-close"></i></span><div class="lc-subscription-form-block lc-subscription-inner"><div class="lc-subscription-title-wrap"><h4 class="lc-subscription-title">Sign up for the XP² newsletter</h4><div class="lc-subscription-text">Join thousands of readers from Target, Citi, Spotify, Hulu, Google, Sephora, and other innovative brands who read our bi-weekly XP² newsletter, delivering educational content, research, and insights straight to your inbox</div><div class="lc-privacy">You may unsubscribe at any time. Visit our <a href="/privacy-policy/">privacy policy</a> to learn more about how we process your data and your rights as a data subject.</div></div><form class="mktoForm" data-formId="3876" data-formInstance="sub-popup"></form></div><div class="lc-subscription-ty-block lc-subscription-inner"><h4 class="lc-subscription-title text-center">Thanks for<br> signing up!</h4></div></div></div></div><div id="top-nav-backdrop"></div><div id="et-main-area"><div id="main-content"><div id="content-area"><header class="article-sticky-header"><div class="lc-header-container lc-flex"><div class="lc-header-logo"> <a href="https://www.dynamicyield.com/learn" title="XP Squared" class="xp-logo-container"> <svg xmlns="http://www.w3.org/2000/svg" width="112.004" height="53.42" viewBox="0 0 112.004 53.42"> <path d="M8968.651,4606.571a49.057,49.057,0,0,0-6.159,6.344q-1.263,1.485-2.782,3.339t-3.079,3.71q-1.559,1.856-2.969,3.6t-2.522,3.079q.3.445,1.261,1.781t2.227,3.19q1.26,1.856,2.745,3.97t2.856,4.044q1.373,1.931,2.486,3.488t1.633,2.375q1.484,2.153,2.708,3.858a25.747,25.747,0,0,0,2.411,2.931,12.808,12.808,0,0,0,2.449,2,7.254,7.254,0,0,0,2.819,1v.742h-24.041v-.742q1.039-.147,2.226-.333a12.155,12.155,0,0,0,2.227-.557,4.522,4.522,0,0,0,1.706-1.039,2.31,2.31,0,0,0,.668-1.707,6.3,6.3,0,0,0-.965-2.783q-.964-1.743-2.226-3.673-1.708-2.672-3.9-5.825t-5.009-7.086q-1.188,1.411-2.6,3.116t-2.819,3.451q-1.411,1.744-2.708,3.376t-2.189,2.894a32.577,32.577,0,0,0-1.854,2.968,5.867,5.867,0,0,0-.816,2.82,2.706,2.706,0,0,0,.668,1.818,5.342,5.342,0,0,0,1.707,1.3,10.711,10.711,0,0,0,2.226.816,21.722,21.722,0,0,0,2.226.445v.742H8919.6v-.742a18.531,18.531,0,0,0,6.158-3.228,45.326,45.326,0,0,0,6.53-6.2q1.112-1.261,2.745-3.228t3.413-4.081q1.783-2.115,3.414-4.118t2.745-3.339q-.445-.593-2.04-2.856t-3.6-5.083q-2-2.819-3.97-5.6t-3.153-4.415q-1.559-2.152-2.782-3.784a26.076,26.076,0,0,0-2.375-2.782,12.747,12.747,0,0,0-2.449-1.93,13.428,13.428,0,0,0-3-1.3v-.742h24.189v.742q-.965.075-2.227.3a14.111,14.111,0,0,0-2.3.593,5.268,5.268,0,0,0-1.78,1.039,2.194,2.194,0,0,0-.743,1.707,5.789,5.789,0,0,0,1.039,2.746q1.039,1.708,2.374,3.636l3.785,5.454q1.928,2.782,4.675,6.715,2.449-2.818,5.009-6.159t4.488-5.787a31.259,31.259,0,0,0,1.93-2.968,6.278,6.278,0,0,0,1.039-2.968,2.5,2.5,0,0,0-.705-1.781,6.393,6.393,0,0,0-1.707-1.262,10.075,10.075,0,0,0-2.188-.816q-1.188-.3-2.152-.445v-.742h19.367v.742A15.309,15.309,0,0,0,8968.651,4606.571Z" transform="translate(-8919.604 -4602.453)"/> <path d="M9422.068,4621.77a14.179,14.179,0,0,1-2.338,3.932,14.748,14.748,0,0,1-3.376,2.931,20.786,20.786,0,0,1-3.934,1.967,24.367,24.367,0,0,1-4.451,1.113q-2.376.373-4.787.557t-4.711.186h-4.008v6.975q0,1.263.037,2.338t.038,2.337q.072,2.376.148,4.081a5.411,5.411,0,0,0,.815,2.82,6.25,6.25,0,0,0,2.3,1.929,10.813,10.813,0,0,0,4.75,1.113v.742h-23.745v-.742a20.734,20.734,0,0,0,3.561-.89,5.8,5.8,0,0,0,2.672-1.781,3.388,3.388,0,0,0,.594-1.336,14.222,14.222,0,0,0,.334-2q.111-1.113.186-2.375t.074-2.449c0-.692.011-1.632.037-2.82s.048-2.485.074-3.9.037-2.856.037-4.341v-18.773q0-1.334-.037-2.708c-.026-.915-.074-1.781-.148-2.6a13.947,13.947,0,0,0-.371-2.226,3.157,3.157,0,0,0-.705-1.447,5.554,5.554,0,0,0-2.856-1.6q-1.745-.407-3.748-.705v-.742h1.781q1.409,0,3.3-.037t4.117-.037q2.227,0,4.415-.037t4.119-.037h3.339a69.43,69.43,0,0,1,8.682.482,19.691,19.691,0,0,1,7.123,2.337,15.1,15.1,0,0,1,5.417,5.046,13.712,13.712,0,0,1,2.152,7.717A13.3,13.3,0,0,1,9422.068,4621.77Zm-9.312-12.986a13.487,13.487,0,0,0-9.535-6.159,28.427,28.427,0,0,0-4.155-.3,20,20,0,0,0-2.412.111q-.854.112-1.372.186a5.343,5.343,0,0,1-.186.779,4.946,4.946,0,0,0-.187.853q-.073.521-.147,1.447t-.149,2.634q-.075,2.6-.111,6.307t-.037,7.272v9.5q3.042,0,5.715-.186a18.593,18.593,0,0,0,4.971-1,13.885,13.885,0,0,0,7.271-5.379,14.741,14.741,0,0,0,2.523-8.422A13.63,13.63,0,0,0,9412.756,4608.784Z" transform="translate(-9329.39 -4601.215)"/> <g transform="translate(98.35)"> <path d="M9839.271,4616.067a10.2,10.2,0,0,1,3.648-3.89,26.827,26.827,0,0,0,3.232-2.56,3.789,3.789,0,0,0,1.127-2.635,2.928,2.928,0,0,0-.646-1.939,2.637,2.637,0,0,0-4.094.456,6.113,6.113,0,0,0-.4,2.256h-3.512a8.885,8.885,0,0,1,.85-3.789q1.447-2.75,5.139-2.75a6.52,6.52,0,0,1,4.643,1.616,5.584,5.584,0,0,1,1.727,4.277,5.783,5.783,0,0,1-1.221,3.624,12.091,12.091,0,0,1-2.627,2.345l-1.447,1.026q-1.359.964-1.859,1.395a4.076,4.076,0,0,0-.844,1h8.023v3.181h-12.584A8.616,8.616,0,0,1,9839.271,4616.067Z" transform="translate(-9838.422 -4601.215)"/> </g> </svg> </a></div><div class="lc-header-title"> The role of optimization analytics in experimentation</div><div class="lc-header-right lc-flex"><ul class="lc-header-social lc-flex"><li class="et-social-icon"> <a href="#" title="Share on Facebook" class="icon" onclick="window.open('https://www.facebook.com/sharer/sharer.php?u=' + encodeURIComponent(document.URL) + '&t=' + encodeURIComponent(document.URL)); return false;"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 460.1990662 458.5138245"><path d="M460.1990662,230.0995331C460.1990662,103.0190277,357.1800232,0,230.0995331,0S0,103.0190277,0,230.0995331C0,338.011261,74.2903519,428.5578003,174.5126801,453.4230957V300.4146423H127.064209v-70.3151093h47.4484711v-30.2983704c0-78.3184433,35.4433594-114.619339,112.3326569-114.619339c14.5774536,0,39.730896,2.8583221,50.020874,5.7166443v63.7409363c-5.4308167-0.5716553-14.8633118-0.8574677-26.58255-0.8574677c-37.7300415,0-52.3075867,14.2916718-52.3075867,51.4501038v24.8674927h75.158783l-12.9122009,70.3151093h-62.246582v158.0991821C371.9082336,444.7527161,460.1990662,347.742981,460.1990662,230.0995331z"/></svg> </a></li><li class="et-social-icon"> <a href="#" title="Share on Linkedin" class="icon" onclick="window.open( 'https://www.linkedin.com/shareArticle?mini=true&url=' + encodeURIComponent(document.location.href)); return false;"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><path d="M100.28 448H7.4V148.9h92.88zM53.79 108.1C24.09 108.1 0 83.5 0 53.8a53.79 53.79 0 0 1 107.58 0c0 29.7-24.1 54.3-53.79 54.3zM447.9 448h-92.68V302.4c0-34.7-.7-79.2-48.29-79.2-48.29 0-55.69 37.7-55.69 76.7V448h-92.78V148.9h89.08v40.8h1.3c12.4-23.5 42.69-48.3 87.88-48.3 94 0 111.28 61.9 111.28 142.3V448z"/></svg> </a></li><li class="et-social-icon"> <a href="#" title="Tweet" class="icon" onclick="window.open('https://twitter.com/intent/tweet?text=' + encodeURIComponent(document.title) + ':%20' + encodeURIComponent(document.URL)); return false;"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><path d="M389.2 48h70.6L305.6 224.2 487 464H345L233.7 318.6 106.5 464H35.8L200.7 275.5 26.8 48H172.4L272.9 180.9 389.2 48zM364.4 421.8h39.1L151.1 88h-42L364.4 421.8z"/></svg> </a></li></ul><div class="topnav-buttons"> <a href="#" data-toggle="modal" data-target="#lc-subscription" class="btn-desktop-only pulse-button v5 topnav-btn"> <span class="button-inner"> XP² Newsletter </span> </a> <a href="/newsletter/" class="pulse-button v5 topnav-btn btn-mobile-only"> <span class="button-inner"> XP² Newsletter </span> </a></div></div></div><div class="lc-progress-wrap"><div class="lc-scroll-progress"></div></div></header><article id="post-97849" class="et_pb_post post-97849 lesson type-lesson status-publish has-post-thumbnail hentry article-category-data-analytics article-industry-agnostic article-tags-audience-segmentation article-tags-personalization-and-testing-frameworks article-function-bi-analytics article-type-thought-leadership course-advanced-experimentation"><div class="lesson-content-sidebar"><div class="lesson-content-sidebar-inner" style="background-color: #ffe8b1"><div class="lesson-content-sidebar-header"><h4>Advanced Experimentation Course</h4></div><style>@media only screen and (max-width: 1199px) {
.lesson-content-sidebar-header {
background-color: #ffe8b1;
}
.lesson-content-sidebar-header:after {
background-color: rgb(253,218,165);
}
}</style><div class="lesson-content-sidebar-list"><ol><li class=""> <a href="https://www.dynamicyield.com/lesson/client-side-testing-and-personalization/">Client-side testing and personalization explained</a> <span class="mark-as-read" data-id="120262"></span> <span class="article-read-mark">Already read</span></li><li class=""> <a href="https://www.dynamicyield.com/lesson/server-side-testing-and-personalization/">Server-side testing and personalization explained</a> <span class="mark-as-read" data-id="130448"></span> <span class="article-read-mark">Already read</span></li><li class=""> <a href="https://www.dynamicyield.com/lesson/frequentists-approach-to-ab-testing/">The Frequentist Approach to A/B Testing</a> <span class="mark-as-read" data-id="117864"></span> <span class="article-read-mark">Already read</span></li><li class=""> <a href="https://www.dynamicyield.com/lesson/bayesian-approach-to-ab-testing/">The Bayesian Approach to A/B Testing</a> <span class="mark-as-read" data-id="116912"></span> <span class="article-read-mark">Already read</span></li><li class="current"> <a href="https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/">The role of optimization analytics in experimentation</a> <span class="mark-as-read" data-id="97849"></span> <span class="article-read-mark">Already read</span></li><li class=""> <a href="https://www.dynamicyield.com/lesson/conversion-attribution-kpi/">Why session-based attribution is flawed in A/B tests</a> <span class="mark-as-read" data-id="20206"></span> <span class="article-read-mark">Already read</span></li><li class=""> <a href="https://www.dynamicyield.com/lesson/maximum-returns-with-3-ab-testing-rules/">Choosing the right optimization KPI for your A/B tests</a> <span class="mark-as-read" data-id="25170"></span> <span class="article-read-mark">Already read</span></li><li class=""> <a href="https://www.dynamicyield.com/lesson/5-reasons-multivariate-testing-sucks/">The complex nature of running multivariate tests</a> <span class="mark-as-read" data-id="29682"></span> <span class="article-read-mark">Already read</span></li><li class=""> <a href="https://www.dynamicyield.com/lesson/outliers-detection/">Outliers detection in A/B testing</a> <span class="mark-as-read" data-id="91549"></span> <span class="article-read-mark">Already read</span></li></ol><div class="view-all-courses-wrap"> <a href="https://www.dynamicyield.com/learning-paths/" class="view-all-courses">Discover more courses<svg xmlns="http://www.w3.org/2000/svg" width="10.928" height="9.501" viewBox="0 0 10.928 9.501" style=" "><path d="M10.929,4.753a.859.859,0,0,0-.19-.491L6.938.224A.835.835,0,0,0,5.892.16.736.736,0,0,0,5.9,1.2L8.577,4.041H.746a.733.733,0,0,0-.734.586.711.711,0,0,0,.7.84H8.577L5.9,8.3a.784.784,0,0,0-.008,1.038.8.8,0,0,0,1.037-.063l3.8-4.039a.652.652,0,0,0,.2-.483Z" transform="translate(-0.001 -0.003)"/></svg></a></div></div></div></div><div class="article-content"><div class="lesson-content-inner"><div class="article-breadcrumbs"> <a href="https://www.dynamicyield.com/learning-paths/" title="Learning Paths">Learning Paths</a> <a href="https://www.dynamicyield.com/course/advanced-experimentation/">Advanced Experimentation</a></div><h1 class="article-title">The role of optimization analytics in experimentation</h1><h2 class="article-excerpt">From ideation to execution and analysis, data and analytics can support marketers at every turn, turning any business into an insights-driven industry leader.</h2><div class="article-authors lc-flex"><div class="article-author-box lc-col lc-flex"><div class="article-author-image lc-col"><div class="entry_author_image"><img src="https://www.dynamicyield.com/wp-content/authors/JamesFlory-104.png" alt="James Flory" width="76" height="76" /></div></div><div class="article-author-title lc-col"><div class="article-author-name"> <a href="https://www.dynamicyield.com/blog/author/jamesflory/" title="Posts by James Flory" rel="author">James Flory</a></div><div class="article-author-title-text"> Director of Experimentation Strategy at Widerfunnel</div></div></div></div><div class="entry-content article-entry-content"><p></p><div id="WYNTK-Sec" class="what-you-need-to-know-sec"><p><img decoding="async" src="/wp-content/uploads/2020/09/star-icon.svg" style="width: 20px; margin-right: 5px; vertical-align: -2px;"><strong class="first">Summarize this article</strong><strong class="hidden">Here’s what you need to know:</strong></p><ul style="margin-left: 4px; font-size: 16px;"><li style="margin: 10px 0;">Experimentation analytics are crucial for turning data into actionable insights, driving business growth, and improving user experience.</li><li style="margin: 10px 0;">The 3 stages of experimentation – ideation, execution, and analysis – all rely on data and analytics to inform decision-making.</li><li style="margin: 10px 0;">Asking the right questions is key to gathering useful data. Consider user perspectives and segment data for deeper understanding.</li><li style="margin: 10px 0;">Track relevant events and engagement metrics to measure the impact of your experiments and avoid misinterpreting results.</li><li style="margin: 10px 0;">Challenge your assumptions! Use secondary metrics to uncover hidden insights and learn from both positive and negative results.</li></ul></div><p></p><p>Today, it’s not enough to be versed in analytics. You need to understand how to use analytics to solve the specific business problems you own. Those of us in the digital world may all be using the same tools—Google Analytics, or Adobe, or any other stack—but we’re using them in very specific ways for very different purposes. The nuances are critical.</p><p>SEO teams are likely hyper-focused on keywords and <a href="https://www.dynamicyield.com/glossary/bounce-rate/">bounce rates</a>, while advertising teams are looking at impressions, costs, and click-through-rates. Content teams pay attention to session metrics and pageviews, and the web development team is keeping its eye on page speed and performance.</p><p>These teams are pulling different insights out of the same collection of data, creating fractured subcultures of data analysis, and experimentation teams are not exempt. If you are leading experimentation on your organization’s site or in your product, you need to adopt a particular lens for data analysis to answer the types of questions that an experimentation role will ask of you.</p><p>In this post, I’m going to walk you through the three stages of experimentation from ideation through execution and into analysis, and highlight how analytics and data can support your <a href="https://www.dynamicyield.com/article/experience-optimization/">experience optimization</a> process at every turn.</p><p>This includes:</p><ul><li>How to use analytics data to generate and validate an <a href="https://www.dynamicyield.com/glossary/hypothesis-testing/">evidence-backed hypothesis</a>,</li><li>How to measure experiments to get insights in addition to wins and losses, and</li><li>Ultimately, how to push your experimentation program forward.</li></ul><p><em>Note: The term “Optimization Analytics” is fairly broad. Both qualitative and quantitative analysis are necessary in your optimization work, however, while <a href="https://www.widerfunnel.com/conversion-optimization-process" rel="noopener noreferrer" target="_blank">digital analytics is only one component</a> of a well-rounded optimization process, I’m going to focus here for the bulk of this post. </em></p><h2>Experiment ideation</h2><p>First things first, let’s level set on what an analytics platform really is and what it’s capable of. To a lot of people, their analytics platform is a golden goose—a treasure trove of data that holds the secret to more traffic, more revenue, and more success.</p><p>In reality, your analytics platform is like the magic 8-ball toy you had as a kid. It won’t do anything valuable or interesting until you ask it a <strong>meaningful question</strong>. If you wander aimlessly through analytics looking for an insight, you’ll likely end up exhausted and discouraged.</p><p>Instead, take a step back and consider the problems you are trying to solve. You may have a hunch, or a list of questions or ideas about your site, product, or users. Once you have those questions, use your analytics platform to either confirm or reject your hunch.</p><p>If the data backs up your initial hunch, this may indicate a meaningful insight and an impactful test idea. If it doesn’t, you may have uncovered something new to dig into, or you may have successfully closed the book on a dead-end idea.</p><p>If you find this part challenging, you’re not alone. It’s the reason companies like Google have rolled out the Insights portion of Google Analytics—to try to jumpstart the actual insights flowing out of the data. While still fairly rudimentary, these artificial intelligence-driven insights are a step in the right direction in prompting us to ask the right questions of our data.</p><p>My tips for asking the right questions are to:</p><ul><li>Put yourself in your users’ shoes,</li><li>Track events, and</li><li>Segment segment segment</li></ul><h3>Put yourself in your users’ shoes</h3><p>One of the best ways to identify a strong experiment idea is to put yourself in your users’ shoes, especially if you are new to your role or business (you’ll have a fresh perspective). Browse your website or app as if you’re a new user and look for <a href="https://www.widerfunnel.com/lift-model/" rel="noopener noreferrer" target="_blank">points of friction or opportunity</a>.</p><p>Perhaps you’re at an eCommerce company and you find that shipping costs aren’t available on your cart page; that may be a potential point of frustration for users. Now, you have a good set of questions to ask of your analytics platform:</p><ul><li>How many users <a href="https://www.dynamicyield.com/glossary/pogo-sticking/">pogo-stick</a> between your cart and the point of your checkout that contains the shipping information?</li><li>How many people exit from there?</li><li>How many convert?</li></ul><p>You’re on your way to identifying a new test idea.</p><p><strong>An Example</strong></p><p>One of our clients is an apartment rental company. Their site allows people to browse apartment buildings in their area and find units for rent. If someone finds an apartment they like, they simply click the “Phone” or “Send Message” buttons to contact the property to set up a viewing or start an application. This lead is the primary metric we are optimizing.</p><p>Our first step when partnering with this company was to browse their site and put ourselves in their users’ shoes. This surfaced a hunch: Certain tabs at the top of the apartment building pages contained valuable content that we felt was more likely to induce a conversion than others.</p><p><img fetchpriority="high" decoding="async" src="https://www.dynamicyield.com/wp-content/uploads/2019/05/DY-RiseofOptimizationAnalytics-InlineImages-01-1024x425.jpg" alt="" width="1024" height="425" /></p><p>This gave us a question to ask of analytics: Do visitors who have visited certain tabs convert at a higher rate than others? We used Google Analytics data to confirm our hunch and, sure enough, the “Reviews” tab was the most likely to result in a lead from a visitor. (Note: The photos tab is open by default on page load.)</p><p>Our hypothesis was simple: Ordering the tabs from left to right in order of likelihood to result in a conversion will result in higher performing tabs receiving more traffic, ultimately increasing conversions. The result? A <strong>+4.93% increase in leads.</strong></p><p><img decoding="async" src="https://www.dynamicyield.com/wp-content/uploads/2019/05/DY-RiseofOptimizationAnalytics-InlineImages-02-1024x424.jpg" alt="" width="1024" height="424" /></p><h3>Track relevant events in your analytics platform</h3><p>If you’ve been testing for a while, this one may seem like a no-brainer. But I am constantly surprised at how many organizations still under-leverage event-tracking. If you’re building an experimentation program from the ground up, or if you have a redesign or new rollout coming up soon, take some time to map out the important actions and events you may want to track across your site.</p><p>Build a framework to organize and <a href="https://www.dynamicyield.com/blog/google-analytics-tag-audit/">tag these events using Google Tag Manager</a> or Dynamic Tag Manager for ease of execution. Engagement with your navigation or footer, clicks to calls-to-action, scroll depths, video plays, etc. are valuable actions that may help you uncover key insights based on these user behaviors.</p><p>You may be thinking, “that’s what heatmaps are for.” <a href="https://www.widerfunnel.com/scrollmaps-and-cro/" rel="noopener noreferrer" target="_blank">Heatmaps</a> will show you what users are clicking on, but they won’t enable you to segment based on those actions and cross-reference conversion rate.</p><p>Imagine you have a product page featuring a video. A heatmap shows you that many of your users are clicking the video. In fact, it’s one of highest engaged elements on your page. The logical next step is to put more videos on your site, right?</p><p>Hold on. There’s an assumption built into that conclusion: That video views are correlated to conversions. If you are tracking video engagement in your analytics platform, you can segment users who watched the video and those who didn’t and see who <em>actually converted</em>. You may find, to your surprise, that those that don’t watch the video are more likely to convert. In this case, you have a counter-intuitive test idea; remove the video and see what happens to your conversion rate.</p><p>You may find event tracking somewhat daunting from a time and resource perspective. If that’s the case, you should consider tools like Heap or the auto-event tracking capabilities of Google Tag Manager to automate some of the work and enable event data collection automatically.</p><h3>Segment segment segment</h3><p>If they aren’t already, segments should be your best friend. Canned reports in analytics platforms will only tell you so much. Segments can unlock a whole new realm of untapped insights and custom segments are often a faster route to answering a question and providing the most granularity.</p><p>One of the most valuable ways you can use segments is to gain a better understanding of the actual users visiting pages on your site.</p><p>Analytics platforms often interchangeably report on sessions, pageviews, unique pageviews, users, etc. in pre-built reports—all of which have different definitions. Many <a href="https://www.dynamicyield.com/lesson/introduction-to-ab-testing/">A/B testing</a> tools, however, operate on a “unique visitor” level. Building segments based on users will enable you to more accurately estimate things like audience size, conversion rate, and test duration using historical data from your analytics platform.</p><div class="blog-ebook-cta" style="display: inline-block; margin-top: 10px;"><span style="font-weight: 600;"><b>Mastering Micro-Segmentation for Personalization</b></span><br /> The ultimate step-by-step guide for creating and targeting micro-segments to reach your most valuable users today</p><div class="ebook-cta-button"><a class="Read-Now-cap" style="color: #fff; text-transform: uppercase; font-weight: 400;" href="https://www.dynamicyield.com/guides/micro-segmentation-for-personalization/">READ NOW ›</a></div></div><p><strong>An Example</strong></p><p>To do this in Google Analytics, for example, build a custom segment that utilizes the “conditions” section of the “advanced options” tab and set the condition to filter users (not sessions) that have visited a specific page—say your homepage.</p><p>You can then apply that segment to any other report to analyze the behavior of users that have interacted with the homepage at some point in their journey. Adding this segment to the audience view will show you your expected audience size. Adding it to your conversion views will show you how many conversions you can expect for an experiment targeting the homepage.</p><p><img decoding="async" src="https://www.dynamicyield.com/wp-content/uploads/2019/05/Test-3-1024x525.jpg" alt="" width="1024" height="525" class="alignnone size-large wp-image-98182" /></p><p>Another very helpful component is the sequences tab, which can enable you to build segments of users that follow a specific path and compare that journey to estimate audience size or conversion rate, etc. The custom segment feature is a treasure when it comes to detailed analysis.</p><h2>Experiment execution</h2><p>Using analytics to identify an experiment idea is only half the battle. <a href="https://www.dynamicyield.com/blog/conversion-attribution-kpi/">Knowing what to track</a> and how to measure the performance of that idea-turned-hypothesis is critical. To evaluate success and to gather insights and learning along the way, you need to ensure you’re tracking the right metrics.</p><p>There’s a lot of information out there on this topic, and I’m going to break from the herd on a few best practices. But the ultimate takeaway is this: Your experiments are only as good as the metrics you are tracking, and the thought you put into these metrics.</p><h3>Define your metrics up front</h3><p>In an experiment, you are likely tracking both primary metrics and secondary metrics.</p><p>Primary metrics should be consistent across many if not all of your experiments. If yours is an eCommerce company, your primary metric might be “Revenue per Visitor (RPV)” or “Transactions.” If your company is powered by leads, it may be “Leads” or “Sales Qualified Leads;” in SaaS, it may be “Product Trials.”</p><p>Your <a href="https://www.dynamicyield.com/blog/maximum-returns-with-3-ab-testing-rules/">primary metric should be closely tied to your business</a>’ bottom line and it should be very difficult to game. That is, you shouldn’t be able to easily influence your primary metric with a simple button color test.</p><p>Secondary metrics are often more experiment-specific and will change depending on what you’re testing. You may utilize a few guardrail metrics—like “Visits” or interactions with each step of your funnel—to suss out any noise in your primary metric. You may also track engagement metrics like clicks to a CTA, or visits to a specific page likely to be impacted by the changes you’re making.</p><p>Secondary metrics exist as both a sanity check on your primary metric and a learning or storytelling opportunity when your experiment concludes.</p><h3>Track more secondary metrics than you think you should</h3><p>Your primary metric is your guiding light for go/no-go roll-out decisions, but it can’t tell the entire story on its own. You need secondary metrics to help explain <em>why</em> your primary metric went up or down.</p><p>This is especially important when an experiment has a negative impact on your primary metric. It’s never easy to present negative numbers to your boss, but it becomes a lot easier if you have supporting metrics that tell the story of <em>why </em>and<em> what next</em>.</p><p>Secondary metrics can often point to how to turn the observed negative impact into a follow-up experiment because they paint a clearer picture of why users behaved the way they did.</p><p><strong>An Experiment Example</strong></p><p>Another partner of ours is an eCommerce company. One of their landing pages was a very high-performer; we wanted to test the same experience on the homepage to see if we could replicate this success. The variation featured a hero section that matched the look and feel of that successful landing page. This change, however, resulted in a -10.6% decrease in transactions.</p><p>If we had only been tracking a single primary metric, in this case, transactions, we would have closed this experiment as a “loser” and moved on to another idea without understanding why transactions decreased. Luckily, we were tracking secondary metrics.</p><p>Among the secondary metrics being tracked were clicks to a “How it Works” video embedded in both the control and the variation. As it turned out, clicks to that link went up +153.6% in the variation! A surprising result and one that got our attention: What if video content wasn’t as helpful as we thought? What if it was actually distracting users and hurting conversion rate on the site? We set out to answer that very question with a follow-up experiment on the company’s product pages.</p><p>These pages featured a prominent video link in the hero, similar to what we had tested on the homepage. We hypothesized that by reducing its prominence, and therefore user engagement with the video, and replacing that visual real estate with a static value proposition, we could lift conversion rate on these pages. Sure enough, the experience with the less prominent video resulted in a -29.4% decrease in video views, and a 33% increase in transactions.</p><p><strong>A note.</strong></p><p>The prevailing wisdom in experimentation and statistics is that the more metrics you track, the less reliable your results are, primarily due to the <a href="https://conversionxl.com/blog/how-many-ab-test-variations/">multiple comparison problem</a>.</p><p>The reality is that, while this is true in conventional statistics, most experimentation platforms have adapted their <a href="https://www.dynamicyield.com/blog/statistical-significance/">statistical models</a> to control for this. This means that your primary metric is considered in a near vacuum and will not be impacted by additional metrics you’re tracking for insight and learning purposes.</p><p>I also want to point out that, in most cases, our experiments are not a life or death situation. The statistics developed to support <a href="https://www.dynamicyield.com/lesson/introduction-to-ab-testing/">A/B testing</a> have a deep-rooted history in clinical trials where stakes are very high and the margin of error has to be razor thin.</p><p>Unless you’re experimenting on something like a radical change to your core business strategy or pricing model, it’s unlikely that your changes require this level of mathematical scrutiny. The benefit of learning from the movement in your additional metrics likely outweighs any negative impact they may have to the reliability of your final outcome.</p><h3>Integrate your testing tool with your analytics tool</h3><p>As much as I’ve talked about making sure you define the right metrics up front, the reality is that you won’t always get that right.</p><p>Often times, our assumptions about how users will interact with an experience are wildly different from reality. Sometimes, you’ll see extreme swings in one direction or the other that are far off from your expected result and none of your secondary or “guard rail” metrics illuminate the “why.”</p><p>This is why integrating your experimentation platform with your analytics platform is so important. And if you follow my earlier advice and set up your event tracking first, you’ll be in a better position to turn a confusing result into an insight.</p><p>For example, you may find that a change in your experiment caused traffic to surge to an unexpected page that you didn’t think to track ahead of time. When your testing platform is integrated with your analytics platform, you’ll be able to utilize all the data already being collected and compare it across your variations to explain these unexpected results.</p><h2>Analytics and Experiment Analysis</h2><p>Results analysis is an essential piece of a sustainable experimentation program. Again, analytics allow you to evaluate the success of the experiment given your goals. You should be able to demonstrate the impact of your efforts and uncover new insights to <a href="https://www.dynamicyield.com/blog/never-run-out-cro-ideas/">generate a continuous stream of experiment ideas</a>.</p><h3>Determining Success</h3><p>Of course, you need to determine whether your experiments perform better (or worse) than baseline. How you do that will depend heavily on your specific environment and the metrics you identified pre-experiment.</p><p>In a perfect world, if you saw your primary metric go up, you’d roll out the variation as the new experience; and if you saw it go down, you’d stick with the control. We don’t have the luxury of a perfect world, that’s why we’re optimizing!</p><p>In the real world—especially if yours is a larger organization or a longer conversion funnel—it can be difficult to achieve material, measurable impact on your primary metric at every stage of the funnel.</p><p>It’s not uncommon to find yourself in a situation where your primary metric looks flat, but you see good movement around your secondary metrics. In some of these cases, you may want to move forward with the variation for reasons beyond solely a lift in your primary metric. Experiments that are neutral <a href="https://www.dynamicyield.com/blog/losing-ab-tests-actually-winning/">(even losers) can still be winners</a>.</p><p>For example, imagine you’re at an eCommerce company and you just ran an experiment on the homepage introducing a new featured products section. You find that transactions are flat at the end of the experiment, but engagement with the featured products is high and traffic to these specific product pages is improving as well.</p><p>You have succeeded in introducing a new promotional area on the homepage that doesn’t hurt transactions and creates an opportunity to feature products that may be high value to the business for other reasons. You also have a new promotional vehicle to experiment with. Perhaps other iterations of this area will lift conversion rates with the right featured products or messaging.</p><h3>Demonstrating Impact</h3><p>Projecting the impact of your experiments is key to getting and maintaining stakeholder buy-in, proving the value of your program, and ensuring your initiatives retain the budget they deserve.</p><p>You’re likely responsible for certain KPIs. You need to show that your experiments are having an impact on those KPIs and on your business’ bottom line. The goal of projecting the impact of your experiments is to prove the Return on Investment (ROI) of your initiatives and experiments, as well as demonstrating longer-term value and impact.</p><p>The common approach to these projections is to take the observed lift of your experiment and project that lift out over a period, commonly 12 months, to show how much additional revenue or how many additional conversions, leads, or transactions your experiment could potentially produce if it were to become the new baseline on the day the experiment was stopped.</p><p>Let’s explore a simple example to clarify that concept. Let’s say you run an experiment with one variation and a control for four weeks. Your variation results in a 5% lift in transactions at 95% confidence. Awesome! But now you need to show what that observed 5% lift means to the business in a more material way.</p><p>Over those four weeks, you received 200,000 total visitors to your experiment: 100,000 in the control and 100,000 in the variation. 9,500 people converted in the control, and 10,000 people converted in your variation (your 5% lift). You can take two approaches to project the impact of this experiment:</p><ol><li>Use the experiment data alone, or</li><li>Use experiment data combined with historical data</li></ol><p>In the first method, you are basing your projection solely on the data observed during the experiment. You aren’t considering any long-term seasonality or potential external factors outside of the experiment’s duration.</p><p>Using this method, you would project the traffic volume observed during your experiment over a full 52 weeks:</p><p><img loading="lazy" decoding="async" src="https://www.dynamicyield.com/wp-content/uploads/2019/05/DY-RiseofOptimizationAnalytics-InlineImages-04-1024x360.jpg" alt="" width="1024" height="360" /></p><p>Then, you would calculate the difference between the Control conversion rate and the Variant conversion rate on those annual visitors:</p><p><img loading="lazy" decoding="async" src="https://www.dynamicyield.com/wp-content/uploads/2019/05/DY-RiseofOptimizationAnalytics-Inline-New-1024x424.jpg" alt="" width="1024" height="424" /></p><p>That 13,000 is the projected estimated additional conversions you can expect based on the observed 5% lift during the experiment.</p><p>If you know your average order value is, say, $100, then you can take this one step further to calculate estimated additional annual revenue:</p><p><img loading="lazy" decoding="async" src="https://www.dynamicyield.com/wp-content/uploads/2019/05/DY-RiseofOptimizationAnalytics-InlineImages-06-1024x172.jpg" alt="" width="1024" height="172" /></p><p>Using historical data is another approach, which may be more accurate for your business if you have significant seasonality that the experiment didn’t observe. In this approach, you would pull the previous 12 months of traffic and conversions to the page or pages your experiment targeted, and use those numbers in the same “observed lift” calculations above.</p><p>Instead of 2,600,000 annual users and 247,000 conversions, you may find that the page actually had 3,200,000 visitors over the last 12 months, and 270,000 conversions due to some higher traffic periods that weren’t accounted for during the experiment.</p><p>None of these calculations are perfect, which is why it’s important to label them as projections and estimations, but they’ll help you show a more tangible result from your work in numbers that may appeal more to those higher up in the organization.</p><h3>Uncovering New Insights</h3><p>In the end, uncovering new insights is what it’s all about. Of course, a primary goal of marketing experimentation is to improve revenue and lift conversion rates, but insights are the key to true, long-term success.</p><p style="font-size:20px">“Insights-driven businesses bring insight, not just data, into every decision, and they know exactly how to use them for greatest advantage across the entire customer life cycle. For these firms, digital insights and what they do with them are their secret weapons to disrupt your market and steal your customers.”</em></p><p> — Brian Hopkins, Ted Schadler, and James McCormick, Authors of Forrester’s report, <em>The Insights-Driven Business</em></p><p>The tactics and suggestions outlined above will help you lay a foundation to start uncovering these insights, but it’s important to remember that your tools can’t do this for you. You will only uncover core insights if you put the effort into <a href="https://www.widerfunnel.com/resources/experiment-ideation-workbook/" rel="noopener noreferrer" target="_blank">asking the right questions</a> and collecting the right data to answer those questions.</p><p>If you do step into the realm of optimization analytics, you can join the ranks of insights-driven businesses that are at the forefront of their industries or disrupting others.</p><div class="article-separator"><svg width="54" height="26" viewBox="0 0 54 26" fill="none" xmlns="http://www.w3.org/2000/svg"> <path d="M53.0088 25.6367C52.7884 21.9564 51.8025 18.3625 50.1141 15.0849C48.4256 11.8073 46.0718 8.9181 43.2031 6.60203L43.2031 25.6367L53.0088 25.6367Z" fill="black"/> <path d="M14.4395 3.59038C10.3237 5.69174 6.82743 8.83082 4.29638 12.6973C1.76533 16.5638 0.28711 21.0239 0.0078125 25.6367L14.4395 25.6367L14.4395 3.59038Z" fill="black"/> <path d="M17.6563 2.20809L17.6562 25.6367L39.9829 25.6367L39.9829 4.36701C39.9829 4.36701 30.7303 -2.31342 17.6563 2.20809Z" fill="black"/> </svg></div><div class="article-tags-block"> <strong>Categories:</strong> <a href="https://www.dynamicyield.com/course/advanced-experimentation/">Advanced Experimentation</a></div><div class="article-tags-block"> <strong>Tags:</strong> <a href="https://www.dynamicyield.com/tags/audience-segmentation/">Audience Segmentation</a><a href="https://www.dynamicyield.com/tags/personalization-and-testing-frameworks/">Personalization and Testing Frameworks</a></div><div class="article-pagination"><h5>Continue reading</h5><div class="article-pagination-row lc-flex"> <a rel="next" style="background-color: #ffe8b1" class="article-next-link pagination-link" href="https://www.dynamicyield.com/lesson/conversion-attribution-kpi/" title="Why session-based attribution is flawed in A/B tests">Why session-based attribution is flawed in A/B tests</a><a rel="prev" style="background-color: #ffe8b1" class="article-previous-link pagination-link" href="https://www.dynamicyield.com/lesson/bayesian-approach-to-ab-testing/" title="The Bayesian Approach to A/B Testing">The Bayesian Approach to A/B Testing</a></div></div></div></div></div></article></div><section class="lc-subscription-section"><div class="lc-subscription-form-block lc-subscription-inner"><h4 class="lc-subscription-title">Sign up for the XP² newsletter</h4><div class="lc-subscription-text">Join thousands of readers from Target, Citi, Spotify, Hulu, Google, Sephora, and other innovative brands who read our bi-weekly XP² newsletter, delivering educational content, research, and insights straight to your inbox</div><form class="mktoForm" data-formId="3876" data-formInstance="sub-inline"></form><div class="lc-privacy-wrapper"><div class="lc-privacy"> You may unsubscribe at any time. Visit our <a href="/privacy-policy/">privacy policy</a> to learn more about how we process your data and your rights as a data subject.</div></div></div><div class="lc-subscription-ty-block lc-subscription-inner"><h4 class="lc-subscription-title text-center">Thanks for<br> signing up!</h4></div></section></div> <script type='text/javascript'>//DY.API('siteVar', {id: 128, value: 'James Flory'});
//DY.API('siteVar', {id: 129, value: ''});</script> <footer id="main-footer" class=""><div class="footer-top"><div class="footer-container"> <a href="https://www.dynamicyield.com/" class="dy-footer-logo"> <img src="/wp-content/uploads/2025/09/ma_symbol.svg" alt="Dynamic Yield footer logo" width="100" height="66"> </a><h4>Make hyper-personalization a reality</h4><p>Trusted by 450+ forward-thinking enterprise businesses:</p><div class="footer-clients retina_parent"> <img src="/wp-content/uploads/2023/08/customer-logos-footer.png" alt="DY Clients" width="498" height="123"></div><div class="footer-btns-wrap"> <a href="https://www.dynamicyield.com/request-demo/" class="btn-teal">Contact Sales</a> <a href="https://www.dynamicyield.com/product-demo/" class="btn-transparent">Watch a product demo<svg xmlns="http://www.w3.org/2000/svg" width="15.625" height="15.625" viewBox="0 0 15.625 15.625"> <path id="Icon_ionic-ios-play-circle" data-name="Icon ionic-ios-play-circle" d="M10.688,3.375A7.313,7.313,0,1,0,18,10.688,7.311,7.311,0,0,0,10.688,3.375Zm2.946,7.45L8.81,13.743a.156.156,0,0,1-.236-.137V7.77a.156.156,0,0,1,.236-.137l4.823,2.918A.161.161,0,0,1,13.634,10.825Z" transform="translate(-2.875 -2.875)" fill="none" stroke="rgba(0,0,0,0.8)" stroke-width="1"/> </svg> </a></div><ul class="footer-certificates"><li class="retina_parent"> <a href="https://www.dynamicyield.com/compliance/"> <img src="/wp-content/uploads/2022/05/gdpr-icon.png" alt="gdpr icon" width="139" height="70"> </a></li><li class="retina_parent"> <a href="https://www.dynamicyield.com/compliance/"> <img src="/wp-content/uploads/2023/04/SOCII-Badge.png" alt="socii icon" width="70" height="70"> </a></li><li> <a href="https://www.dynamicyield.com/compliance/"> <img src="/wp-content/uploads/2022/05/ccpa-icon.svg" alt="ccpa icon" width="117" height="40"> </a></li><li class="retina_parent"> <a href="https://www.dynamicyield.com/compliance/"> <img src="/wp-content/uploads/2022/05/iso-27001.png" alt="iso icon" width="106" height="68"> </a></li></ul></div></div><div id="et-footer-nav"><div class="footer-container"><ul id="menu-footer-menu" class="bottom-nav"><li id="menu-item-19183" class="footer_menu_title menu-item menu-item-type-custom menu-item-object-custom menu-item-19183"><a>Recommended Guides:</a></li><li id="menu-item-211968" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-211968"><a href="https://www.dynamicyield.com/article/personalization-guide/">Personalization</a></li><li id="menu-item-138198" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-138198"><a href="https://www.dynamicyield.com/lesson/introduction-to-ab-testing/">A/B Testing</a></li><li id="menu-item-147897" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-147897"><a href="https://www.dynamicyield.com/lesson/cro-plan/">Conversion Rate Optimization</a></li><li id="menu-item-19184" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-19184"><a href="/lesson/shopping-cart-abandonment-strategy/">Shopping Cart Abandonment</a></li><li id="menu-item-19185" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-19185"><a href="/lesson/product-recommendations-guide/">Product Recommendations</a></li><li id="menu-item-22805" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-22805"><a href="/lesson/promise-and-pitfalls-omnichannel-retailing/">Omnichannel Retailing</a></li><li id="menu-item-22885" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-22885"><a href="/lesson/the-economics-of-ecommerce-conversion-optimization/">eCommerce Conversion Rate Optimization</a></li><li id="menu-item-28240" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-28240"><a href="/article/50-most-important-dynamicyield-personalization-stats/">Personalization Statistics</a></li><li id="menu-item-33877" class="menu-item menu-item-type-post_type menu-item-object-post menu-item-33877"><a href="https://www.dynamicyield.com/article/tj-maxx-spired-checkout-optimization/">Checkout Optimization</a></li><li id="menu-item-123711" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-123711"><a href="/article/ecommerce-personalization-must-haves/">eCommerce Personalization</a></li><li id="menu-item-144090" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-144090"><a href="https://www.dynamicyield.com/lesson/strategizing-omnichannel-personalization/">Omnichannel Personalization</a></li><li id="menu-item-148146" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-148146"><a href="https://marketing.dynamicyield.com/benchmarks/cart-abandonment-rate/">Shopping Cart Abandonment Rate</a></li></ul><div class="footer-mach-logo retina_parent"> <img src="/wp-content/uploads/2022/05/mach-cert-22-wh.png" alt="Mach logo" width="63" height="70"></div><div class="language-switcher footer-language-switcher"><ul><li>en<ul><li><a class="active" href="https://www.dynamicyield.com/lesson/optimization-analytics-in-experimentation/">English</a></li><li><a href="https://www.dynamicyield.com/es/">Español</a></li><li><a href="https://www.dynamicyield.com/de/">Deutsch</a></li><li><a href="https://www.dynamicyield.com/fr/">Français</a></li><li><a href="https://www.dynamicyield.com/ja/">日本語</a></li></ul></li></li></ul></div></div></div><div id="footer-bottom"><div class="footer-container"><div class="footer_info_wrapper"><div id="footer-info"> <button id="ot-sdk-btn" class="ot-sdk-show-settings">Manage Cookies</button> <a href="/privacy-notice/" target="_blank" class="animlink">Privacy Notice</a> <a href="/tos/" target="_blank" class="animlink">Terms of use</a> <span>© 2026 Mastercard Dynamic Yield</span></div></div><div class="footer_social_icons"><style>.et-social-icons .et-social-icon img {
max-width: 16px;
transition: 0.2s all;
}
.et-social-icons .et-social-icon img:not(:hover) {
opacity: 0.5;
}</style><span itemscope itemtype="http://schema.org/Organization"><link itemprop="url" href="https://www.dynamicyield.com"><ul class="et-social-icons"><li class="et-social-icon"> <a itemprop="sameAs" href="https://www.linkedin.com/company/dynamic-yield" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame-5.png" alt="LinkedIn Logo" /> </a></li><li class="et-social-icon"> <a itemprop="sameAs" href="https://www.instagram.com/dynamicyield/" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame-4.png" alt="Instagram Logo" /> </a></li><li class="et-social-icon"> <a itemprop="sameAs" href="https://www.youtube.com/c/Dynamicyield" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame-3.png" alt="Youtube Logo" /> </a></li><li class="et-social-icon"> <a itemprop="sameAs" href="https://www.tiktok.com/@dynamic.yield" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame-2.png" alt="TikTok Logo" /> </a></li><li class="et-social-icon"> <a itemprop="sameAs" href="https://www.facebook.com/DynamicYield" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame-1.png" alt="Facebook Logo" /> </a></li><li class="et-social-icon"> <a itemprop="sameAs" href="https://twitter.com/DynamicYield" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame.png" alt="X Logo" /> </a></li></ul> </span></div></div></div></footer></div></div><div id="mobile_menu_overlay"></div> <script type='text/javascript' src='//cdn.dynamicyield.com/api/8765281/api_dynamic.js'></script> <script type='text/javascript' src='//cdn.dynamicyield.com/api/8765281/api_static.js'></script> <script type="text/javascript">var sbiajaxurl = "https://www.dynamicyield.com/wp-admin/admin-ajax.php";</script> <script type="text/javascript">!function(e,n){var r={"selectors":{"block":"pre.EnlighterJSRAW","inline":"code.EnlighterJSRAW"},"options":{"indent":2,"ampersandCleanup":true,"linehover":true,"rawcodeDbclick":false,"textOverflow":"break","linenumbers":true,"theme":"beyond","language":"generic","retainCssClasses":false,"collapse":false,"toolbarOuter":"","toolbarTop":"{BTN_RAW}{BTN_COPY}{BTN_WINDOW}{BTN_WEBSITE}","toolbarBottom":""},"resources":["https:\/\/www.dynamicyield.com\/wp-content\/plugins\/enlighter\/cache\/enlighterjs.min.css?tvrakvcLArVio6C","https:\/\/www.dynamicyield.com\/wp-content\/plugins\/enlighter\/resources\/enlighterjs\/enlighterjs.min.js"]},o=document.getElementsByTagName("head")[0],t=n&&(n.error||n.log)||function(){};e.EnlighterJSINIT=function(){!function(e,n){var r=0,l=null;function c(o){l=o,++r==e.length&&(!0,n(l))}e.forEach(function(e){switch(e.match(/\.([a-z]+)(?:[#?].*)?$/)[1]){case"js":var n=document.createElement("script");n.onload=function(){c(null)},n.onerror=c,n.src=e,n.async=!0,o.appendChild(n);break;case"css":var r=document.createElement("link");r.onload=function(){c(null)},r.onerror=c,r.rel="stylesheet",r.type="text/css",r.href=e,r.media="all",o.appendChild(r);break;default:t("Error: invalid file extension",e)}})}(r.resources,function(e){e?t("Error: failed to dynamically load EnlighterJS resources!",e):"undefined"!=typeof EnlighterJS?EnlighterJS.init(r.selectors.block,r.selectors.inline,r.options):t("Error: EnlighterJS resources not loaded yet!")})},(document.querySelector(r.selectors.block)||document.querySelector(r.selectors.inline))&&e.EnlighterJSINIT()}(window,console);</script><script type="text/javascript" id="rfw-script-js-extra">var rfw = {"speed":""};</script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_a5ec58aa1f222bedaf60cdced3b8a004.js?ver=2026020733" id="rfw-script-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_c1b7fbe6b1a3b777fddfe187094deb97.js?ver=2026020733" id="rfw-script-fitvid-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-includes/js/comment-reply.min.js?ver=935c7f8ce03960b02800bc8ae3231df8" id="comment-reply-js" async="async" data-wp-strategy="async"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_ea3fad8947382ebf8709507a11a805db.js" id="divi-fitvids-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/themes/Divi/js/waypoints.min.js" id="waypoints-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_845779ea879c70773af95b2f6fc394d2.js" id="magnific-popup-js"></script> <script type="text/javascript" id="divi-custom-script-js-extra">var et_custom = {"ajaxurl":"https:\/\/www.dynamicyield.com\/wp-admin\/admin-ajax.php","images_uri":"https:\/\/www.dynamicyield.com\/wp-content\/themes\/Divi\/images","et_load_nonce":"ee490c83a0","subscription_failed":"Please, check the fields below to make sure you entered the correct information.","fill":"Fill","field":"field","invalid":"Invalid email","captcha":"Captcha"};</script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_43b4a88b2828bc5c023dc913689e8da5.js?ver=20.4" id="divi-custom-script-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/themes/Divi/custom/bootstrap/js/bootstrap.min.js" id="bootstrap-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/themes/Divi/custom/slick/slick.min.js" id="slick-js"></script> <script type="text/javascript" data-ot-ignore src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_12efd5a7d441d8b7f817fc6fc685dca9.js?ver=21.85" id="customscript-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_33fa37a5effe9361e40b8525e36e3eaf.js" id="typed-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_eb8981568f49ac148e96af3241eb95c7.js?ver=21.85" id="form_validation-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_be33bce741fd746dee1a6cf31c0b5bde.js" id="retina_js-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_9a50928b26148d1c35a95957e5ef2b9d.js?ver=21.85" id="mkto-scripts-js"></script> <script type="module" src="https://www.dynamicyield.com/wp-content/themes/Divi/js/cid-validation.js?ver=21.85" id="cid-validation"></script><script type="text/javascript" src="https://www.dynamicyield.com/wp-content/plugins/wp-featherlight/js/wpFeatherlight.pkgd.min.js?ver=1.3.4" id="wp-featherlight-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_17babe2fc54117766e24bd50889c6577.js?ver=1772264034" id="article-script-js"></script> <script type="text/javascript" id="dc_conversion_api-js-extra">var ajax_obj = {"nonce":"910abf0a29","ajax_url":"https:\/\/www.dynamicyield.com\/wp-admin\/admin-ajax.php","ajax_action":"conversion_api_call"};</script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_34a6a3100a2ecbc7706af2d4c798f856.js?ver=935c7f8ce03960b02800bc8ae3231df8" id="dc_conversion_api-js"></script> <script type="text/javascript" data-noptimize>var dy_search_params = new URLSearchParams(window.location.search);
if(dy_search_params.has('utm_source')) {
localStorage.setItem('utm_source', dy_search_params.get('utm_source'));
}
if(dy_search_params.has('utm_medium')) {
localStorage.setItem('utm_medium', dy_search_params.get('utm_medium'));
}
if(dy_search_params.has('utm_content')) {
localStorage.setItem('utm_content', dy_search_params.get('utm_content'));
}
if(dy_search_params.has('utm_campaign')) {
localStorage.setItem('utm_campaign', dy_search_params.get('utm_campaign'));
}
if(dy_search_params.has('utm_term')) {
localStorage.setItem('utm_term', dy_search_params.get('utm_term'));
}
if(dy_search_params.has('mkt_tok')) {
localStorage.setItem('mkt_tok', dy_search_params.get('mkt_tok'));
}
// Also, fill in the data inside the Marketo form once its available
if(typeof window.MktoForms2 === 'object') {
MktoForms2.whenReady(function(_form) {
var form = _form.getFormElem();
console.log('MKTO FORM READY');
// Append the mkt_tok param if it exists in the URL
var ls_mkto_tok = localStorage.getItem('mkt_tok');
if(ls_mkto_tok) {
_form.addHiddenFields({
mkt_tok: ls_mkto_tok
});
_form.vals({ mkt_tok: ls_mkto_tok });
}
// Fill in the UTM data from localStorage
var dy_utms = {
'utm_source': 'txt_source__c',
'utm_medium': 'txt_medium__c',
'utm_content': 'txt_content__c',
'utm_campaign': 'txt_campaign_name__c',
'utm_term': 'txt_term__c',
'gclid': ['GCLID__c', 'GCLID_c__c']
};
for(var i = 0; i < Object.keys(dy_utms).length; i++) {
var ls_key = Object.keys(dy_utms)[i],
ls_value = localStorage.getItem(ls_key),
form_key = dy_utms[ls_key],
form_field;
if(ls_value) {
if(ls_key === 'gclid') {
ls_value = JSON.parse(ls_value).value;
}
if(typeof(form_key) === 'string') {
form_field = form[0].querySelector('input[type="hidden"][name="' + form_key + '"]');
if(form_field) {
form_field.value = ls_value;
}
} else {
for(var j = 0; j < form_key.length; j++) {
form_field = form[0].querySelector('input[type="hidden"][name="' + form_key[j] + '"]');
if(form_field) {
form_field.value = ls_value;
}
}
}
}
}
// IP Address
fetch('https://www.cloudflare.com/cdn-cgi/trace').then(function(result) {
result.text().then(function(values) {
values.split("\n").map(function(item) {
var pair = item.split('=');
if(['ip'].includes(pair[0])) {
form[0].querySelector('input[type="hidden"][name="IP__c"]').value = pair[1];
}
});
});
});
// GA ID and Referrer
var dy_utms_cookie = {};
document.cookie.split(';').forEach(function(el) {
var splitCookie = el.split('=');
var key = splitCookie[0].trim();
var value = splitCookie[1];
dy_utms_cookie[key] = value;
});
if(dy_utms_cookie["_ga"]) {
form[0].querySelector('input[type="hidden"][name="Visitor_ID__c"]').value = dy_utms_cookie["_ga"].substring(6);
}
if(dy_utms_cookie["personalize_referrer_string"]) {
form[0].querySelector('input[type="hidden"][name="Referral__c"]').value = dy_utms_cookie["personalize_referrer_string"].substring(6);
}
// ZoomInfo
if(localStorage.getItem('ZoomInfoData')) {
var zoominfo_data = JSON.parse(localStorage.getItem('ZoomInfoData'));
if(zoominfo_data) {
form[0].querySelector('input[type="hidden"][name="Alexa_Rank__c"]').value = zoominfo_data.alexaglobalrank;
form[0].querySelector('input[type="hidden"][name="Employees__c"]').value = zoominfo_data.employeesrange;
form[0].querySelector('input[type="hidden"][name="Company_Revenue_Range__c"]').value = zoominfo_data.estimatedannualrevenue;
}
}
// Also, dispatch a dataLayer event on success
_form.onSuccess(function(values, follow_up_url) {
window.dataLayer = window.dataLayer || [];
if([3739, 4802, 5094, 5110, 4946, 5076, 5116, 5171, 5172, 5173].includes(_form.getId())) {
window.dataLayer.push({
'event': 'demo_request_complete',
'eventCallback': function() {
if(follow_up_url) {
window.location.href = follow_up_url;
}
}
});
return false;
} else if([3876].includes(_form.getId())) {
window.dataLayer.push({
'event': 'newsletter_signup_complete',
'eventCallback': function() {
if(follow_up_url) {
window.location.href = follow_up_url;
}
}
});
return false;
}
});
});
}</script> <script>if(typeof(jQuery) !== 'undefined') {
$ = jQuery.noConflict();
}
function getUrlParameter(sParam) {
var sPageURL = decodeURIComponent(window.location.search.substring(1)),
sURLVariables = sPageURL.split('&'),
sParameterName,
i;
for (i = 0; i < sURLVariables.length; i++) {
sParameterName = sURLVariables[i].split('=');
if (sParameterName[0] === sParam) {
return sParameterName[1] === undefined ? true : sParameterName[1];
}
}
}
$(window).load(function(){
var videoID = getUrlParameter('video');
if(videoID !== undefined){
if($('#wistia-' + videoID + '-1').length){
var videoName = $('#wistia-' + videoID + '-1').parents('.tab-pane');
videoName = videoName[0].id
}
if($('a[href="#' + videoName +'"]').length){
$('a[href="#' + videoName +'"]').each(function(){
$(this).click();
});
}
addScript('https://fast.wistia.com/embed/medias/'+ videoID +'.jsonp');
addScript('https://fast.wistia.com/assets/external/E-v1.js');
$('body').append('<span class="wistia_embed wistia_async_'+videoID+' popover=true popoverContent=link dy-wistia-popup" style="display:inline"></span>');
window._wq = window._wq || [];
_wq.push({ id: videoID, onReady: function(video) {
var video = Wistia.api(videoID);
video.popover.show(); // open the popover
video.play(); // play the video
setTimeout(function(){video.unmute();}, 500);
}});
}
});</script> <style>#adBanner {
background-color: transparent;
height: 0;
width: 1px;
}</style><div id="wrapfabtest"><div id="adBanner"></div></div></body></html>
<!-- Dynamic page generated in 0.480 seconds. -->
<!-- Cached page generated by WP-Super-Cache on 2026-02-28 02:33:54 -->
<!-- super cache -->