Rivalry2

Turning messy data into a gold mine using Spark, Flink, and ScyllaDB
2026-02-28 19:44 Diff
https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/
At Dynamic Yield, we eat, sleep, and breathe data – it is the bedrock of our personalization technology. Our R&D department is constantly working to ensure all of the valuable information we capture through the platform fuels the development of new products and features.
Recently, we faced a real set of challenges with regards to how we stored our user’s data. Seeing as every dev team works on different features and leverages the data for their own unique purposes, naturally, portions of it would end up stored in different databases – wherever was most suitable based on the team’s technical needs. However, with data spread across numerous locations, it became more and more difficult to see the big picture. Developers began struggling to analyze the data and understand simple questions about our users. And in turn, it was harder for them to develop features based on it.
To solve this, we set out to unify it – all of it – made possible with the help of Apache Spark, Apache Flink, and ScyllaDB. In the rest of this post, I will walk through all the technical decisions we made to establish a unified database, which has not only unlocked new opportunities for our department as a whole, but also our customers.
Our attempt at unifying user data into one place

Although we didn’t tackle everything from the onset, we did want to plan on addressing these four main technical issues to effectively accomplish our goal:
Defining internal and customer needs
Choosing a database (DB) which is capable of supporting our solution
Migrating historical data into the DB
Streaming live events without any gaps between historical and streamed data
Choosing a database

Next, a few core requirements needed to be ironed out.
Should store large amounts of data – We collect a lot of data about users and have been doing so for several years now, so we needed an option that would not only be able to handle our current database but also scale as it becomes larger.
Read latency must be very low – When users come back to our customers’ sites, we quickly analyze everything we know about them to offer a personalized experience. This process of analysis and delivery has to happen almost immediately in order for the web page to load within a reasonable amount of time.
Needs to contain both historical and real-time data – We analyze user data and for us to make accurate decisions based on it, the data has to be up-to-date. Therefore, both historical user data, as well as current activity, should be taken into account.
Flexible query support – Today we may be interested in only certain pieces of data, but tomorrow we might be interested in another portion of it. Proper query support was needed to allow us this kind of flexibility.
Why we chose ScyllaDB

ScyllaDB is one of the newest NoSQL databases out there. Described as an Apache Cassandra replacement with ultra-low latency and extreme throughput, with most of Cassandra’s features implemented in C++, it’s able to present very impressive benchmarks.
Additionally, it shared other similarities with Apache Cassandra we liked:
CQL (Cassandra Query Language) support – Using CQL, we could easily write queries to fetch the data we needed.
Easy integration with Apache Spark – When writing our Spark applications, we could use the ‘spark-cassandra-connector’ in order to write and read from Scylla.
Its low read latency meant we could confidently use it for serving experiences. And because it’s not an in-memory database, it wouldn’t be as expensive. As I mentioned before, we needed to store a lot of user data, and given the number of nodes we would need, this was a big factor for us.
From a safety point of view, Scylla supports data replication according to the replication strategy of your choosing. So if one Scylla node crashes, there’s no fear of losing the data.
Architecture

After we got our DB up and running, it was time to fill it with some data.
Migrating old data to the new DB -Using Apache Spark we were able to migrate all of our data to the new Scylla DB very quickly. The spark application was pretty simple as it only needed a start and end date to get started. It then scanned the old ‘raw data’ within the time range allotted and wrote it to Scylla. This ‘raw data’ functions as our big data store – great for batch jobs, but not for random access.
*Note* we only had to run the spark application once in order to migrate all data to into Scylla DB. If we ever need to migrate data into Scylla again, we will be able to use the same application. In fact, we have used this capability more than once when developing.
Streaming online events to Scylla -As previously mentioned, one of our requirements was that Dynamic Yield’s Unified Customer Profile would always contain up-to-date data. For this purpose, we used Apache Flink – our application receives streams of user events as they occur from our Kafka topics, applies some transformations to the data to fit our schema, and then writes them to Scylla.
Data modeling

Not all data is created equal. For example, a purchase event can provide a lot more insight into a user’s preferences than a simple page view. With a large percentage of our data representing page views, we knew there wasn’t a need to store them forever.
Using a table-based format for our data modeling, we save all user events in one table. Events like purchases or add-to-carts are never deleted, while page views are stored for a three month period. Rather than saving these page views, what we do is actually extract ‘aggregated data’ from them.
We store this aggregated data, which can also include a user’s location, browsers, devices, etc., in another table. If we already know a user has visited a web page from the U.S., there is no need to save the information again when they revisit the web page from the same location and so on.
The merits of our unified customer profile

Having all of our data in one place has opened many doors for us, aiding our understanding of each user who interacts with an experience launched through the Dynamic Yield platform in a much more efficient way. And through the work we’ve done, we are now able to bestow this knowledge on our customers, who can use the rich data from what we refer to as our Unified Customer Profile (UCP) to:
Provide tailored content every time a user revisits a site or app
Leverage machine learning algorithms to match each user with the optimal experience (based on both long-term history and in-session behavior)
Develop custom individualized experiences through server-side implementation
Not to mention, meeting the requirements of GDPR became significantly easier when all of our user data was unified, another huge bonus of our efforts.
Have questions about the process we went through or want to share stories about how you unified your company’s data? Comment below!
<!DOCTYPE html> <!--[if IE 6]><html id="ie6" lang="en-US"> <![endif]--> <!--[if IE 7]><html id="ie7" lang="en-US"> <![endif]--> <!--[if IE 8]><html id="ie8" lang="en-US"> <![endif]--> <!--[if !(IE 6) | !(IE 7) | !(IE 8)  ]><!--><html lang="en-US"> <!--<![endif]--><head><meta name="msvalidate.01" content="1033997EC383621B874EB0392F261984"/><link rel="preconnect" href="//cdn.dynamicyield.com"><link rel="preconnect" href="//st.dynamicyield.com"><link rel="preconnect" href="//rcom.dynamicyield.com"><link rel="dns-prefetch" href="//cdn.dynamicyield.com"><link rel="dns-prefetch" href="//st.dynamicyield.com"><link rel="dns-prefetch" href="//rcom.dynamicyield.com"> <script data-cfasync="false">window.dataLayer = window.dataLayer || [];
        window.gtag = function() { dataLayer.push(arguments); }
        window.gtag('consent', 'default', {
            ad_storage: 'denied',
            ad_user_data: 'denied',
            ad_personalization: 'denied',
            analytics_storage: 'denied',
            functionality_storage: 'denied',
            personalization_storage: 'denied',
            security_storage: 'granted', // Only this is granted by default
        });</script> <script data-cfasync="false">(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
        new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
        j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
        '//www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
        })(window,document,'script','dataLayer','GTM-MCQHF8');</script> <script data-cfasync="false" async>// Init the adobeDataLayer array
        var adobeDataLayer = adobeDataLayer || [];

        // Only when the API is ready
        function adobeWaitForDY() {
            if (typeof(window.DY) !== 'undefined' && typeof(window.DY.AdDetection) !== 'undefined' &&
                typeof($dy) !== 'undefined' && typeof($dy(window)) !== 'undefined' && typeof($dy.renderSmartTagSlider) !== 'undefined') {
                
                var zi_data = localStorage.getItem('ZoomInfoData'),
                    ada_page_info = {
                        "pageName": "Turning messy data into a gold mine using Spark, Flink, and ScyllaDB",
                        "siteName": "Mastercard Dynamic Yield",
                        "pageTitle": "en_US :turning-messy-data-into-a-gold-mine", 
                        "pageURL": "https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/",
                        "country": "US",
                        "language": "en",
                        "dyID": DY.dyid,
                        "userStatus": "Guest"
                    };

                // Detect the hierarchy and 
                // site sections based on the URL slugs
                var current_pathname = window.location.pathname.replace(/^\/|\/$/g, '');

                if(current_pathname) {

                    var current_sitesections = current_pathname.split('/'),
                        current_hierarchy = current_sitesections.map(function(segment) {

                        return segment.split(/[-_]/).map(function(word) {
                            return word.charAt(0).toUpperCase() + word.slice(1);
                        })
                        .join(' ');
                    });

                    // Site section 1
                    if(current_sitesections[0]) {
                        ada_page_info.siteSection1 = current_sitesections[0];
                    }

                    // Site section 2
                    if(current_sitesections[1]) {
                        ada_page_info.siteSection2 = current_sitesections[1];
                    }

                    // Hierarchy
                    ada_page_info.hierarchy = current_hierarchy.join('|');
                }

                // Push the ZI params as well
                if(zi_data) {
                    zi_data = JSON.parse(zi_data);

                    if(zi_data.name) {
                        ada_page_info.ziCompany = zi_data.name;
                    }

                    if(zi_data.domain) {
                        ada_page_info.ziDomain = zi_data.domain;
                    }

                    if(zi_data.employeesrange) {
                        ada_page_info.ziEmployees = zi_data.employeesrange;
                    }

                    if(zi_data.industry) {
                        ada_page_info.ziIndustry = zi_data.industry;
                    }
                }

                // Decode HTML prior to dispatching the event
                Object.keys(ada_page_info).forEach(function(key) {
                    ada_page_info[key] = $('<textarea/>').html(ada_page_info[key]).text();
                });

                adobeDataLayer.push({
                    "event": "pageLoad", 
                    "pageInfo": ada_page_info
                });

            } else {
                setTimeout(adobeWaitForDY, 150);
            }
        }

        adobeWaitForDY();</script>  <script type="text/javascript">function OptanonWrapper() {
            console.log('OneTrust Loaded');
            DY_API_update_consent_mode();
            OneTrust.OnConsentChanged(DY_API_update_consent_mode);
        }

        function DY_API_update_consent_mode() {
            var groups = OnetrustActiveGroups.split(',');

            DY.userActiveConsent = {accepted: groups.includes('C0004')};
            DYO.ActiveConsent.updateConsentAcceptedStatus(groups.includes('C0004'));

            if(groups.includes('C0004')) {
                console.log('OneTrust_Consent_Is_Granted');
                DY.API('event',{ name: 'OneTrust_Consent_Is_Granted', properties: {}});
            }

            console.log('OneTrust - DY API is ' + (groups.includes('C0004') ? 'Enabled' : 'Disabled'));
        }
        
        window.addEventListener('OTConsentApplied', function() {
            var groups = OnetrustActiveGroups.split(',');

            if(groups.includes('C0004')) {
                console.log('OneTrust_Initial_Consent_Granted');
                DY.API('event',{ name: 'OneTrust_Initial_Consent_Granted', properties: {}});
            }
        });</script>  <script type="text/javascript">/*WebFontConfig = {
		 google: { families: [ 'Shadows+Into+Light::latin', 'Handlee::latin', 'Roboto:400,100,300,500:latin' ] }
		 };
		 (function() {
		 var wf = document.createElement('script');
		 wf.src = ('https:' == document.location.protocol ? 'https' : 'http') +
		 '://ajax.googleapis.com/ajax/libs/webfont/1/webfont.js';
		 wf.type = 'text/javascript';
		 wf.async = 'true';
		 var s = document.getElementsByTagName('script')[0];
		 s.parentNode.insertBefore(wf, s);
		 })(); */

        console.log(
            '%cLet’s personalize experiences together.\n%cDynamic Yield is hiring smart engineers like you to help solve hard problems.\n%cSee our jobs page: https://www.dynamicyield.com/careers',
            'font-size:2.2em;line-height:1.4em;margin-top:20px;color:rgb(93, 56, 185);',
            'font-size:1.1em;line-height:1.6em;color:rgb(0, 0, 0);',
            'font-size:1.1em;line-height:1.6em;margin-bottom:20px;color:rgb(0, 0, 0);'
        );

        function waitForElementToDisplay(selector, time, functionName) {
            if (document.getElementById(selector) != null) {
                functionName();
                return;
            } else {
                setTimeout(function () {
                    waitForElementToDisplay(selector, time, functionName);
                }, time);
            }
        }

        function dcWaitForJQuery(time, functionName) {
            if (window.jQuery || window.n2) {
                functionName();
                return;
            } else {
                setTimeout(function () {
                    dcWaitForJQuery(time, functionName);
                }, time);
            }
        }</script> <meta name="msvalidate.01" content="451B75B70520F4C76D67F116C9EFFE5C"/><meta property="twitter:account_id" content="4503599629852947"/><meta name="p:domain_verify" content="f8112f57c09778e892850075a87dc1cf"/><meta name="google-site-verification" content="9FbmfdC4cM7JG8sgYh6w9HhFlxvEHewltexeUVzuSfE"/><meta charset="UTF-8"/><link rel="pingback" href="https://www.dynamicyield.com/xmlrpc.php"/> <!--[if lt IE 9]> <script src="https://www.dynamicyield.com/wp-content/themes/Divi/js/html5.js" type="text/javascript"></script> <![endif]--> <script type="text/javascript">document.documentElement.className = 'js';</script> <script type="text/javascript">var getLangCode = 'en';</script> <link rel="prefetch" href="/wp-content/themes/Divi/css/custom-style.css" as="style" crossorigin="anonymous"><meta name='robots' content='index, follow, max-image-preview:large, max-snippet:-1, max-video-preview:-1' /><link rel="alternate" hreflang="en" href="https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/" /><link rel="alternate" hreflang="x-default" href="https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/" />  <script data-cfasync="false" data-pagespeed-no-defer>var gtm4wp_datalayer_name = "dataLayer";
	var dataLayer = dataLayer || [];</script> <title>Turning messy data into a gold mine using Spark, Flink, and ScyllaDB</title><link rel="preload" as="font" href="https://www.dynamicyield.com/wp-content/themes/Divi/fonts/CircularXXTT/CircularXXWeb-Bold.woff2" crossorigin/><link rel="preload" as="font" href="https://www.dynamicyield.com/wp-content/themes/Divi/fonts/CircularXXTT/CircularXXWeb-Medium.woff2" crossorigin/><link rel="preload" as="font" href="https://www.dynamicyield.com/wp-content/themes/Divi/fonts/CircularXXTT/CircularXXWeb-Book.woff2" crossorigin/><meta name="description" content="Our user&#039;s data was spread across locations. Here&#039;s how we unified it, along with all of the technical decisions we made and new opportunities it unlocked." /><link rel="canonical" href="https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/" /><meta property="og:locale" content="en_US" /><meta property="og:type" content="article" /><meta property="og:title" content="Turning messy data into a gold mine using Spark, Flink, and ScyllaDB" /><meta property="og:description" content="Our user&#039;s data was spread across locations. Here&#039;s how we unified it, along with all of the technical decisions we made and new opportunities it unlocked." /><meta property="og:url" content="https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/" /><meta property="og:site_name" content="Mastercard Dynamic Yield" /><meta property="article:publisher" content="https://www.facebook.com/DynamicYield" /><meta property="article:modified_time" content="2020-07-05T11:32:27+00:00" /><meta property="og:image" content="https://www.dynamicyield.com/wp-content/uploads/2019/11/data-goldmine-feature.png" /><meta property="og:image:width" content="795" /><meta property="og:image:height" content="320" /><meta property="og:image:type" content="image/png" /><meta name="twitter:card" content="summary_large_image" /><meta name="twitter:site" content="@DynamicYield" /><meta name="twitter:label1" content="Est. reading time" /><meta name="twitter:data1" content="6 minutes" /><meta name="twitter:label2" content="Written by" /><meta name="twitter:data2" content="Oran Hirsch" /> <script type="application/ld+json" class="yoast-schema-graph">{"@context":"https://schema.org","@graph":[{"@type":"WebPage","@id":"https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/","url":"https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/","name":"Turning messy data into a gold mine using Spark, Flink, and ScyllaDB","isPartOf":{"@id":"https://www.dynamicyield.com/#website"},"primaryImageOfPage":{"@id":"https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/#primaryimage"},"image":{"@id":"https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/#primaryimage"},"thumbnailUrl":"https://www.dynamicyield.com/wp-content/uploads/2019/11/data-goldmine-feature.png","datePublished":"2019-09-23T14:41:25+00:00","dateModified":"2020-07-05T11:32:27+00:00","description":"Our user's data was spread across locations. Here's how we unified it, along with all of the technical decisions we made and new opportunities it unlocked.","breadcrumb":{"@id":"https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/#primaryimage","url":"https://www.dynamicyield.com/wp-content/uploads/2019/11/data-goldmine-feature.png","contentUrl":"https://www.dynamicyield.com/wp-content/uploads/2019/11/data-goldmine-feature.png","width":795,"height":320},{"@type":"BreadcrumbList","@id":"https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://www.dynamicyield.com/"},{"@type":"ListItem","position":2,"name":"Articles","item":"https://www.dynamicyield.com/article/"},{"@type":"ListItem","position":3,"name":"Turning messy data into a gold mine using Spark, Flink, and ScyllaDB"}]},{"@type":"WebSite","@id":"https://www.dynamicyield.com/#website","url":"https://www.dynamicyield.com/","name":"Mastercard Dynamic Yield","description":"","publisher":{"@id":"https://www.dynamicyield.com/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://www.dynamicyield.com/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https://www.dynamicyield.com/#organization","name":"Mastercard Dynamic Yield","url":"https://www.dynamicyield.com/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https://www.dynamicyield.com/#/schema/logo/image/","url":"https://www.dynamicyield.com/wp-content/uploads/2019/10/DY-Hero_Logo-Black@x2.png","contentUrl":"https://www.dynamicyield.com/wp-content/uploads/2019/10/DY-Hero_Logo-Black@x2.png","width":458,"height":458,"caption":"Mastercard Dynamic Yield"},"image":{"@id":"https://www.dynamicyield.com/#/schema/logo/image/"},"sameAs":["https://www.facebook.com/DynamicYield","https://x.com/DynamicYield","https://www.instagram.com/dynamic.yield/","https://www.linkedin.com/company/dynamic-yield","http://www.pinterest.com/dynamicyield/","https://www.youtube.com/channel/UCPWEcR8lQRadVKYW8H4sndQ","https://en.wikipedia.org/wiki/Dynamic_Yield"]}]}</script> <link rel='dns-prefetch' href='//pages.dynamicyield.com' /><link rel='dns-prefetch' href='//cdnjs.cloudflare.com' /><link rel="alternate" type="application/rss+xml" title="Mastercard Dynamic Yield &raquo; Feed" href="https://www.dynamicyield.com/feed/" /><link rel="alternate" type="application/rss+xml" title="Mastercard Dynamic Yield &raquo; Comments Feed" href="https://www.dynamicyield.com/comments/feed/" /><link rel="alternate" type="application/rss+xml" title="Mastercard Dynamic Yield &raquo; Turning messy data into a gold mine using Spark, Flink, and ScyllaDB Comments Feed" href="https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/feed/" /><meta content="Divi v." name="generator"/><link rel='stylesheet' id='sbi_styles-css' href='https://www.dynamicyield.com/wp-content/plugins/instagram-feed/css/sbi-styles.min.css?ver=6.2.6' type='text/css' media='all' /><style id='wp-emoji-styles-inline-css' type='text/css'>img.wp-smiley, img.emoji {
		display: inline !important;
		border: none !important;
		box-shadow: none !important;
		height: 1em !important;
		width: 1em !important;
		margin: 0 0.07em !important;
		vertical-align: -0.1em !important;
		background: none !important;
		padding: 0 !important;
	}</style><link rel='stylesheet' id='wp-block-library-css' href='https://www.dynamicyield.com/wp-includes/css/dist/block-library/style.min.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><style id='classic-theme-styles-inline-css' type='text/css'>/*! This file is auto-generated */
.wp-block-button__link{color:#fff;background-color:#32373c;border-radius:9999px;box-shadow:none;text-decoration:none;padding:calc(.667em + 2px) calc(1.333em + 2px);font-size:1.125em}.wp-block-file__button{background:#32373c;color:#fff;text-decoration:none}</style><style id='global-styles-inline-css' type='text/css'>body{--wp--preset--color--black: #000000;--wp--preset--color--cyan-bluish-gray: #abb8c3;--wp--preset--color--white: #ffffff;--wp--preset--color--pale-pink: #f78da7;--wp--preset--color--vivid-red: #cf2e2e;--wp--preset--color--luminous-vivid-orange: #ff6900;--wp--preset--color--luminous-vivid-amber: #fcb900;--wp--preset--color--light-green-cyan: #7bdcb5;--wp--preset--color--vivid-green-cyan: #00d084;--wp--preset--color--pale-cyan-blue: #8ed1fc;--wp--preset--color--vivid-cyan-blue: #0693e3;--wp--preset--color--vivid-purple: #9b51e0;--wp--preset--gradient--vivid-cyan-blue-to-vivid-purple: linear-gradient(135deg,rgba(6,147,227,1) 0%,rgb(155,81,224) 100%);--wp--preset--gradient--light-green-cyan-to-vivid-green-cyan: linear-gradient(135deg,rgb(122,220,180) 0%,rgb(0,208,130) 100%);--wp--preset--gradient--luminous-vivid-amber-to-luminous-vivid-orange: linear-gradient(135deg,rgba(252,185,0,1) 0%,rgba(255,105,0,1) 100%);--wp--preset--gradient--luminous-vivid-orange-to-vivid-red: linear-gradient(135deg,rgba(255,105,0,1) 0%,rgb(207,46,46) 100%);--wp--preset--gradient--very-light-gray-to-cyan-bluish-gray: linear-gradient(135deg,rgb(238,238,238) 0%,rgb(169,184,195) 100%);--wp--preset--gradient--cool-to-warm-spectrum: linear-gradient(135deg,rgb(74,234,220) 0%,rgb(151,120,209) 20%,rgb(207,42,186) 40%,rgb(238,44,130) 60%,rgb(251,105,98) 80%,rgb(254,248,76) 100%);--wp--preset--gradient--blush-light-purple: linear-gradient(135deg,rgb(255,206,236) 0%,rgb(152,150,240) 100%);--wp--preset--gradient--blush-bordeaux: linear-gradient(135deg,rgb(254,205,165) 0%,rgb(254,45,45) 50%,rgb(107,0,62) 100%);--wp--preset--gradient--luminous-dusk: linear-gradient(135deg,rgb(255,203,112) 0%,rgb(199,81,192) 50%,rgb(65,88,208) 100%);--wp--preset--gradient--pale-ocean: linear-gradient(135deg,rgb(255,245,203) 0%,rgb(182,227,212) 50%,rgb(51,167,181) 100%);--wp--preset--gradient--electric-grass: linear-gradient(135deg,rgb(202,248,128) 0%,rgb(113,206,126) 100%);--wp--preset--gradient--midnight: linear-gradient(135deg,rgb(2,3,129) 0%,rgb(40,116,252) 100%);--wp--preset--font-size--small: 13px;--wp--preset--font-size--medium: 20px;--wp--preset--font-size--large: 36px;--wp--preset--font-size--x-large: 42px;--wp--preset--spacing--20: 0.44rem;--wp--preset--spacing--30: 0.67rem;--wp--preset--spacing--40: 1rem;--wp--preset--spacing--50: 1.5rem;--wp--preset--spacing--60: 2.25rem;--wp--preset--spacing--70: 3.38rem;--wp--preset--spacing--80: 5.06rem;--wp--preset--shadow--natural: 6px 6px 9px rgba(0, 0, 0, 0.2);--wp--preset--shadow--deep: 12px 12px 50px rgba(0, 0, 0, 0.4);--wp--preset--shadow--sharp: 6px 6px 0px rgba(0, 0, 0, 0.2);--wp--preset--shadow--outlined: 6px 6px 0px -3px rgba(255, 255, 255, 1), 6px 6px rgba(0, 0, 0, 1);--wp--preset--shadow--crisp: 6px 6px 0px rgba(0, 0, 0, 1);}:where(.is-layout-flex){gap: 0.5em;}:where(.is-layout-grid){gap: 0.5em;}body .is-layout-flow > .alignleft{float: left;margin-inline-start: 0;margin-inline-end: 2em;}body .is-layout-flow > .alignright{float: right;margin-inline-start: 2em;margin-inline-end: 0;}body .is-layout-flow > .aligncenter{margin-left: auto !important;margin-right: auto !important;}body .is-layout-constrained > .alignleft{float: left;margin-inline-start: 0;margin-inline-end: 2em;}body .is-layout-constrained > .alignright{float: right;margin-inline-start: 2em;margin-inline-end: 0;}body .is-layout-constrained > .aligncenter{margin-left: auto !important;margin-right: auto !important;}body .is-layout-constrained > :where(:not(.alignleft):not(.alignright):not(.alignfull)){max-width: var(--wp--style--global--content-size);margin-left: auto !important;margin-right: auto !important;}body .is-layout-constrained > .alignwide{max-width: var(--wp--style--global--wide-size);}body .is-layout-flex{display: flex;}body .is-layout-flex{flex-wrap: wrap;align-items: center;}body .is-layout-flex > *{margin: 0;}body .is-layout-grid{display: grid;}body .is-layout-grid > *{margin: 0;}:where(.wp-block-columns.is-layout-flex){gap: 2em;}:where(.wp-block-columns.is-layout-grid){gap: 2em;}:where(.wp-block-post-template.is-layout-flex){gap: 1.25em;}:where(.wp-block-post-template.is-layout-grid){gap: 1.25em;}.has-black-color{color: var(--wp--preset--color--black) !important;}.has-cyan-bluish-gray-color{color: var(--wp--preset--color--cyan-bluish-gray) !important;}.has-white-color{color: var(--wp--preset--color--white) !important;}.has-pale-pink-color{color: var(--wp--preset--color--pale-pink) !important;}.has-vivid-red-color{color: var(--wp--preset--color--vivid-red) !important;}.has-luminous-vivid-orange-color{color: var(--wp--preset--color--luminous-vivid-orange) !important;}.has-luminous-vivid-amber-color{color: var(--wp--preset--color--luminous-vivid-amber) !important;}.has-light-green-cyan-color{color: var(--wp--preset--color--light-green-cyan) !important;}.has-vivid-green-cyan-color{color: var(--wp--preset--color--vivid-green-cyan) !important;}.has-pale-cyan-blue-color{color: var(--wp--preset--color--pale-cyan-blue) !important;}.has-vivid-cyan-blue-color{color: var(--wp--preset--color--vivid-cyan-blue) !important;}.has-vivid-purple-color{color: var(--wp--preset--color--vivid-purple) !important;}.has-black-background-color{background-color: var(--wp--preset--color--black) !important;}.has-cyan-bluish-gray-background-color{background-color: var(--wp--preset--color--cyan-bluish-gray) !important;}.has-white-background-color{background-color: var(--wp--preset--color--white) !important;}.has-pale-pink-background-color{background-color: var(--wp--preset--color--pale-pink) !important;}.has-vivid-red-background-color{background-color: var(--wp--preset--color--vivid-red) !important;}.has-luminous-vivid-orange-background-color{background-color: var(--wp--preset--color--luminous-vivid-orange) !important;}.has-luminous-vivid-amber-background-color{background-color: var(--wp--preset--color--luminous-vivid-amber) !important;}.has-light-green-cyan-background-color{background-color: var(--wp--preset--color--light-green-cyan) !important;}.has-vivid-green-cyan-background-color{background-color: var(--wp--preset--color--vivid-green-cyan) !important;}.has-pale-cyan-blue-background-color{background-color: var(--wp--preset--color--pale-cyan-blue) !important;}.has-vivid-cyan-blue-background-color{background-color: var(--wp--preset--color--vivid-cyan-blue) !important;}.has-vivid-purple-background-color{background-color: var(--wp--preset--color--vivid-purple) !important;}.has-black-border-color{border-color: var(--wp--preset--color--black) !important;}.has-cyan-bluish-gray-border-color{border-color: var(--wp--preset--color--cyan-bluish-gray) !important;}.has-white-border-color{border-color: var(--wp--preset--color--white) !important;}.has-pale-pink-border-color{border-color: var(--wp--preset--color--pale-pink) !important;}.has-vivid-red-border-color{border-color: var(--wp--preset--color--vivid-red) !important;}.has-luminous-vivid-orange-border-color{border-color: var(--wp--preset--color--luminous-vivid-orange) !important;}.has-luminous-vivid-amber-border-color{border-color: var(--wp--preset--color--luminous-vivid-amber) !important;}.has-light-green-cyan-border-color{border-color: var(--wp--preset--color--light-green-cyan) !important;}.has-vivid-green-cyan-border-color{border-color: var(--wp--preset--color--vivid-green-cyan) !important;}.has-pale-cyan-blue-border-color{border-color: var(--wp--preset--color--pale-cyan-blue) !important;}.has-vivid-cyan-blue-border-color{border-color: var(--wp--preset--color--vivid-cyan-blue) !important;}.has-vivid-purple-border-color{border-color: var(--wp--preset--color--vivid-purple) !important;}.has-vivid-cyan-blue-to-vivid-purple-gradient-background{background: var(--wp--preset--gradient--vivid-cyan-blue-to-vivid-purple) !important;}.has-light-green-cyan-to-vivid-green-cyan-gradient-background{background: var(--wp--preset--gradient--light-green-cyan-to-vivid-green-cyan) !important;}.has-luminous-vivid-amber-to-luminous-vivid-orange-gradient-background{background: var(--wp--preset--gradient--luminous-vivid-amber-to-luminous-vivid-orange) !important;}.has-luminous-vivid-orange-to-vivid-red-gradient-background{background: var(--wp--preset--gradient--luminous-vivid-orange-to-vivid-red) !important;}.has-very-light-gray-to-cyan-bluish-gray-gradient-background{background: var(--wp--preset--gradient--very-light-gray-to-cyan-bluish-gray) !important;}.has-cool-to-warm-spectrum-gradient-background{background: var(--wp--preset--gradient--cool-to-warm-spectrum) !important;}.has-blush-light-purple-gradient-background{background: var(--wp--preset--gradient--blush-light-purple) !important;}.has-blush-bordeaux-gradient-background{background: var(--wp--preset--gradient--blush-bordeaux) !important;}.has-luminous-dusk-gradient-background{background: var(--wp--preset--gradient--luminous-dusk) !important;}.has-pale-ocean-gradient-background{background: var(--wp--preset--gradient--pale-ocean) !important;}.has-electric-grass-gradient-background{background: var(--wp--preset--gradient--electric-grass) !important;}.has-midnight-gradient-background{background: var(--wp--preset--gradient--midnight) !important;}.has-small-font-size{font-size: var(--wp--preset--font-size--small) !important;}.has-medium-font-size{font-size: var(--wp--preset--font-size--medium) !important;}.has-large-font-size{font-size: var(--wp--preset--font-size--large) !important;}.has-x-large-font-size{font-size: var(--wp--preset--font-size--x-large) !important;}
.wp-block-navigation a:where(:not(.wp-element-button)){color: inherit;}
:where(.wp-block-post-template.is-layout-flex){gap: 1.25em;}:where(.wp-block-post-template.is-layout-grid){gap: 1.25em;}
:where(.wp-block-columns.is-layout-flex){gap: 2em;}:where(.wp-block-columns.is-layout-grid){gap: 2em;}
.wp-block-pullquote{font-size: 1.5em;line-height: 1.6;}</style><link rel='stylesheet' id='rfw-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_7c036e0987ef80b703d7eb1abf922895.css?ver=2026020657' type='text/css' media='all' /><link rel='stylesheet' id='divi-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_ab3666a686d3dac03fd3032e5757637e.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='ctt-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_f1d1b2f761cf3768f427cdc5c4c2544b.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='inline-tweet-sharer-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_daa32ca85e114a0c6e5d1432c89237f3.css?ver=2.6.4' type='text/css' media='all' /><link rel='stylesheet' id='et_lb_modules-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_c0102af27938567cb82cc02e4da3a673.css?ver=2.4' type='text/css' media='all' /><link rel='stylesheet' id='et-shortcodes-css-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_40cb404848bc2e556105c8aea77f29aa.css?ver=3.0' type='text/css' media='all' /><link rel='stylesheet' id='et-shortcodes-responsive-css-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_3a30a77569bc4544b9465acbee8e5f97.css?ver=3.0' type='text/css' media='all' /><link rel='stylesheet' id='wp-featherlight-css' href='https://www.dynamicyield.com/wp-content/plugins/wp-featherlight/css/wp-featherlight.min.css?ver=1.3.4' type='text/css' media='all' /><link rel='stylesheet' id='baskerville-font-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_8c23179f030d02e0065d2054eab34279.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='markformc-font-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_0eb51906eca648ca4559d0a03b898d83.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='mackinac-font-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_9c8fec2609ebc9764a6e16471d9451ae.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='button-pulse-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_4cabc8dd29eaa9af7bf2df62f1c40af9.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='fontawesome-css' href='https://www.dynamicyield.com/wp-content/themes/Divi/custom/font-awesome/css/font-awesome.min.css' type='text/css' media='all' /><link rel='stylesheet' id='bootstrap-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_d375e60add46d8de26a78728b517660f.css' type='text/css' media='all' /><link rel='stylesheet' id='slick-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_13b1b6672b8cfb0d9ae7f899f1c42875.css' type='text/css' media='all' /><link rel='stylesheet' id='slick-theme-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_f9faba678c4d6dcfdde69e5b11b37a2e.css' type='text/css' media='all' /><link rel='stylesheet' id='custom-theme-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_57f41281576414370c141cd00d2a80a9.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='custom-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_417039b6d54561fe1e8c94ee67cd902c.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='header-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_d75ad7543415df427913a5fe91606386.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='footer-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_25400e1cfd7099cc557083769c928231.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='video_tabs_module-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_f1109a8bc8c46ec9f2205cc8603c132f.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='select2-css' href='https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.6-rc.0/css/select2.min.css' type='text/css' media='all' /><link rel='stylesheet' id='post-checklist-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_ede7f9aa4d78ac997ea5e48f5d511400.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='post-event-cards-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_d02c1d48c2c1fde18d405f087a0fe833.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='mkto-reset-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_d117c8ecde800f5dccd137c16bab71af.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='learning-center-general-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_73dd3b78cf6e21c28be39db6740774b1.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='article-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_e1d850b2d8d6fd6331ad74bd35ddde51.css?ver=1772261842' type='text/css' media='all' /><link rel='stylesheet' id='customers-shortcode-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_c6bf61e9c8beb560d64d119f58757b8b.css?ver=26.27' type='text/css' media='all' /> <script type="text/javascript" src="https://www.dynamicyield.com/wp-includes/js/jquery/jquery.min.js?ver=3.7.1" id="jquery-core-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-includes/js/jquery/jquery-migrate.min.js?ver=3.4.1" id="jquery-migrate-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_dbdab0f35217f6041eb7e2028dc94ba4.js?ver=2.6.4" id="inline-tweet-sharer-js-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/themes/Divi/js/blazy.min.js" id="blazy-js"></script> <script type="text/javascript" src="//pages.dynamicyield.com/js/forms2/js/forms2.min.js?ver=21.85" id="mkto-form-script-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_db6be3b57dba139e789fec1a78186c8e.js?ver=21.85" id="learning-center-js-js"></script> <link rel="https://api.w.org/" href="https://www.dynamicyield.com/wp-json/" /><link rel="EditURI" type="application/rsd+xml" title="RSD" href="https://www.dynamicyield.com/xmlrpc.php?rsd" /><link rel='shortlink' href='https://www.dynamicyield.com/?p=110420' /><link rel="alternate" type="application/json+oembed" href="https://www.dynamicyield.com/wp-json/oembed/1.0/embed?url=https%3A%2F%2Fwww.dynamicyield.com%2Farticle%2Fturning-messy-data-into-a-gold-mine%2F" /><link rel="alternate" type="text/xml+oembed" href="https://www.dynamicyield.com/wp-json/oembed/1.0/embed?url=https%3A%2F%2Fwww.dynamicyield.com%2Farticle%2Fturning-messy-data-into-a-gold-mine%2F&#038;format=xml" /><meta name="generator" content="WPML ver:4.6.8 stt:1,4,3,28,2;" /> <script type="text/javascript">(function(url){
	if(/(?:Chrome\/26\.0\.1410\.63 Safari\/537\.31|WordfenceTestMonBot)/.test(navigator.userAgent)){ return; }
	var addEvent = function(evt, handler) {
		if (window.addEventListener) {
			document.addEventListener(evt, handler, false);
		} else if (window.attachEvent) {
			document.attachEvent('on' + evt, handler);
		}
	};
	var removeEvent = function(evt, handler) {
		if (window.removeEventListener) {
			document.removeEventListener(evt, handler, false);
		} else if (window.detachEvent) {
			document.detachEvent('on' + evt, handler);
		}
	};
	var evts = 'contextmenu dblclick drag dragend dragenter dragleave dragover dragstart drop keydown keypress keyup mousedown mousemove mouseout mouseover mouseup mousewheel scroll'.split(' ');
	var logHuman = function() {
		if (window.wfLogHumanRan) { return; }
		window.wfLogHumanRan = true;
		var wfscr = document.createElement('script');
		wfscr.type = 'text/javascript';
		wfscr.async = true;
		wfscr.src = url + '&r=' + Math.random();
		(document.getElementsByTagName('head')[0]||document.getElementsByTagName('body')[0]).appendChild(wfscr);
		for (var i = 0; i < evts.length; i++) {
			removeEvent(evts[i], logHuman);
		}
	};
	for (var i = 0; i < evts.length; i++) {
		addEvent(evts[i], logHuman);
	}
})('//www.dynamicyield.com/?wordfence_lh=1&hid=EFB87BE306FD459FF760D4FEDF4AB307');</script> 
 <script data-cfasync="false" data-pagespeed-no-defer>var dataLayer_content = {"visitorLoginState":"logged-out","visitorType":"visitor-logged-out","pagePostType":"article","pagePostType2":"single-article","pagePostAuthor":"Oran Hirsch","pagePostDate":"September 23, 2019","pagePostDateYear":"2019","pagePostDateMonth":"09","pagePostDateDay":"23","pagePostDateDayName":"Monday","pagePostDateHour":"10","pagePostDateMinute":"41","pagePostDateIso":"2019-09-23T10:41:25-04:00","pagePostDateUnix":1569235285};
	dataLayer.push( dataLayer_content );</script> <script type="text/javascript">console.warn && console.warn("[GTM4WP] Google Tag Manager container code placement set to OFF !!!");
	console.warn && console.warn("[GTM4WP] Data layer codes are active but GTM container must be loaded using custom coding !!!");</script> <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=3.0, user-scalable=1" /><link rel="icon" type="image/png" href="/favicon-16x16.png" sizes="16x16"><link rel="icon" type="image/png" href="/favicon-32x32.png" sizes="32x32"><link rel="icon" type="image/png" href="/favicon-96x96.png" sizes="96x96"><link rel="icon" type="image/png" href="/favicon-180x180.png" sizes="180x180"><link rel="icon" type="image/png" href="/favicon-192x192.png" sizes="192x192"><link rel="icon" type="image/png" href="/favicon-194x194.png" sizes="194x194"><link rel="shortcut icon" href="/favicon.ico"> <script type="text/javascript" data-noptimize>function dy_bind_adobe_mkto_events(form, form_id, form_name) {

        // Init the adobeDataLayer object
        window.adobeDataLayer = window.adobeDataLayer || [];

        var focus_reported = false;
        var form_completed = false;
        var form_started_reported = false;
        
        // Monitor input interactions - first focus (form start)
        form.getFormElem().find('input, select').on('focus', function() {
            
            // Only if it hasn't been reported yet
            if(! focus_reported) {
                focus_reported = true;
                form_started_reported = true;
                
                adobeDataLayer.push({
                    "event": "form_start",
                    "trackAction": {
                        "formName": form_name,
                        "formID": form_id,
                        "zoomInfo": localStorage.ZoomInfoData !== undefined,
                    }
                });
            }
        });

        // Detect dropoff if the form hasn't been completed
        window.addEventListener("beforeunload", function(event) {
            if (form_completed) return;
            
            var filledLabels = [];
            var count = 0;
            
            form.getFormElem().find('input, select').each(function() {
                var value = this.value && this.value.trim();
                if (value) {
                var label = form.getFormElem().find("label[for='" + this.id + "']").text().trim();
                if (label) {
                    filledLabels.push(label);
                    count++;
                }
                }
            });
            
            if (count > 0) {
                adobeDataLayer.push({
                    "event": "form_dropoff",
                    "formInfo": {
                        "formName": form_name,
                        "formID": form_id,
                        "zoomInfo": localStorage.ZoomInfoData !== undefined,
                        "completedFields": {
                            "labels": filledLabels.join("|"),
                            "count": count.toString()
                        }
                    }
                });
            }
        });

        // Hook into successful submission
        form.onSuccess(function(values, followUpUrl) {

            if(form_started_reported) {

                // Report form completed
                form_completed = true;

                // Send an Adobe Analytics DataLayer event
                adobeDataLayer.push({
                    "event": "form_complete",
                    "formInfo": {
                        "formName": form_name,
                        "formID": form_id,
                        "zoomInfo": localStorage.ZoomInfoData !== undefined,
                        "successAction": {
                            "action": "redirect",
                            "redirectUrl": followUpUrl
                        }
                    }
                });
            }
        });
    }

    function dy_bind_adobe_cta_event(cta, cta_type, cta_section) {
        var cta_name = cta.text().trim(),
            cta_url,
            page_url = window.location.href;

        // const domainName = window.location.origin;

        if (cta.attr('href') !== undefined) {
            // cta_url = domainName + cta.attr('href');
            cta_url = cta.attr('href');
        } else {
            cta_url = page_url;
        }

        adobeDataLayer.push({
            "event": "ctaClick",
            "clickInfo": {
                "ctaName": cta_name,
                "ctaType": cta_type,
                "destinationURL": cta_url,
                "ctaLocation": page_url,
                "ctaSection": cta_section
            }
        });
    }</script> <link rel="shortcut icon" href="https://www.dynamicyield.com/wp-content/uploads/2018/11/favicon-194x194.png" /><link rel="amphtml" href="https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/amp/"></head><body class="article-template-default single single-article postid-110420 wp-featherlight-captions et_fixed_nav windows et_right_sidebar chrome et_includes_sidebar elementor-default default_header learning-center"><div id="page-container"><header id="main-header" class="et_nav_text_color_dark"
 style="background: #fff;"><div class="container-fluid clearfix"><div class="logo-combined"> <a href="https://www.dynamicyield.com/learn"
 title="XP Squared"><svg xmlns="http://www.w3.org/2000/svg" width="112.004" height="53.42" viewBox="0 0 112.004 53.42"> <path d="M8968.651,4606.571a49.057,49.057,0,0,0-6.159,6.344q-1.263,1.485-2.782,3.339t-3.079,3.71q-1.559,1.856-2.969,3.6t-2.522,3.079q.3.445,1.261,1.781t2.227,3.19q1.26,1.856,2.745,3.97t2.856,4.044q1.373,1.931,2.486,3.488t1.633,2.375q1.484,2.153,2.708,3.858a25.747,25.747,0,0,0,2.411,2.931,12.808,12.808,0,0,0,2.449,2,7.254,7.254,0,0,0,2.819,1v.742h-24.041v-.742q1.039-.147,2.226-.333a12.155,12.155,0,0,0,2.227-.557,4.522,4.522,0,0,0,1.706-1.039,2.31,2.31,0,0,0,.668-1.707,6.3,6.3,0,0,0-.965-2.783q-.964-1.743-2.226-3.673-1.708-2.672-3.9-5.825t-5.009-7.086q-1.188,1.411-2.6,3.116t-2.819,3.451q-1.411,1.744-2.708,3.376t-2.189,2.894a32.577,32.577,0,0,0-1.854,2.968,5.867,5.867,0,0,0-.816,2.82,2.706,2.706,0,0,0,.668,1.818,5.342,5.342,0,0,0,1.707,1.3,10.711,10.711,0,0,0,2.226.816,21.722,21.722,0,0,0,2.226.445v.742H8919.6v-.742a18.531,18.531,0,0,0,6.158-3.228,45.326,45.326,0,0,0,6.53-6.2q1.112-1.261,2.745-3.228t3.413-4.081q1.783-2.115,3.414-4.118t2.745-3.339q-.445-.593-2.04-2.856t-3.6-5.083q-2-2.819-3.97-5.6t-3.153-4.415q-1.559-2.152-2.782-3.784a26.076,26.076,0,0,0-2.375-2.782,12.747,12.747,0,0,0-2.449-1.93,13.428,13.428,0,0,0-3-1.3v-.742h24.189v.742q-.965.075-2.227.3a14.111,14.111,0,0,0-2.3.593,5.268,5.268,0,0,0-1.78,1.039,2.194,2.194,0,0,0-.743,1.707,5.789,5.789,0,0,0,1.039,2.746q1.039,1.708,2.374,3.636l3.785,5.454q1.928,2.782,4.675,6.715,2.449-2.818,5.009-6.159t4.488-5.787a31.259,31.259,0,0,0,1.93-2.968,6.278,6.278,0,0,0,1.039-2.968,2.5,2.5,0,0,0-.705-1.781,6.393,6.393,0,0,0-1.707-1.262,10.075,10.075,0,0,0-2.188-.816q-1.188-.3-2.152-.445v-.742h19.367v.742A15.309,15.309,0,0,0,8968.651,4606.571Z" transform="translate(-8919.604 -4602.453)"/> <path d="M9422.068,4621.77a14.179,14.179,0,0,1-2.338,3.932,14.748,14.748,0,0,1-3.376,2.931,20.786,20.786,0,0,1-3.934,1.967,24.367,24.367,0,0,1-4.451,1.113q-2.376.373-4.787.557t-4.711.186h-4.008v6.975q0,1.263.037,2.338t.038,2.337q.072,2.376.148,4.081a5.411,5.411,0,0,0,.815,2.82,6.25,6.25,0,0,0,2.3,1.929,10.813,10.813,0,0,0,4.75,1.113v.742h-23.745v-.742a20.734,20.734,0,0,0,3.561-.89,5.8,5.8,0,0,0,2.672-1.781,3.388,3.388,0,0,0,.594-1.336,14.222,14.222,0,0,0,.334-2q.111-1.113.186-2.375t.074-2.449c0-.692.011-1.632.037-2.82s.048-2.485.074-3.9.037-2.856.037-4.341v-18.773q0-1.334-.037-2.708c-.026-.915-.074-1.781-.148-2.6a13.947,13.947,0,0,0-.371-2.226,3.157,3.157,0,0,0-.705-1.447,5.554,5.554,0,0,0-2.856-1.6q-1.745-.407-3.748-.705v-.742h1.781q1.409,0,3.3-.037t4.117-.037q2.227,0,4.415-.037t4.119-.037h3.339a69.43,69.43,0,0,1,8.682.482,19.691,19.691,0,0,1,7.123,2.337,15.1,15.1,0,0,1,5.417,5.046,13.712,13.712,0,0,1,2.152,7.717A13.3,13.3,0,0,1,9422.068,4621.77Zm-9.312-12.986a13.487,13.487,0,0,0-9.535-6.159,28.427,28.427,0,0,0-4.155-.3,20,20,0,0,0-2.412.111q-.854.112-1.372.186a5.343,5.343,0,0,1-.186.779,4.946,4.946,0,0,0-.187.853q-.073.521-.147,1.447t-.149,2.634q-.075,2.6-.111,6.307t-.037,7.272v9.5q3.042,0,5.715-.186a18.593,18.593,0,0,0,4.971-1,13.885,13.885,0,0,0,7.271-5.379,14.741,14.741,0,0,0,2.523-8.422A13.63,13.63,0,0,0,9412.756,4608.784Z" transform="translate(-9329.39 -4601.215)"/> <g transform="translate(98.35)"> <path d="M9839.271,4616.067a10.2,10.2,0,0,1,3.648-3.89,26.827,26.827,0,0,0,3.232-2.56,3.789,3.789,0,0,0,1.127-2.635,2.928,2.928,0,0,0-.646-1.939,2.637,2.637,0,0,0-4.094.456,6.113,6.113,0,0,0-.4,2.256h-3.512a8.885,8.885,0,0,1,.85-3.789q1.447-2.75,5.139-2.75a6.52,6.52,0,0,1,4.643,1.616,5.584,5.584,0,0,1,1.727,4.277,5.783,5.783,0,0,1-1.221,3.624,12.091,12.091,0,0,1-2.627,2.345l-1.447,1.026q-1.359.964-1.859,1.395a4.076,4.076,0,0,0-.844,1h8.023v3.181h-12.584A8.616,8.616,0,0,1,9839.271,4616.067Z" transform="translate(-9838.422 -4601.215)"/> </g> </svg></a> <a href="https://www.dynamicyield.com/"
 title="Dynamic Yield"><svg xmlns="http://www.w3.org/2000/svg" width="112" height="14.15" viewBox="0 0 112 14.15"> <g> <path d="M8919.6,5180.139v-10.982h1.729v4.505a2.662,2.662,0,0,1,2.321-1.107c2.214,0,3.412,1.684,3.412,3.853,0,2.23-1.32,3.914-3.473,3.914a2.538,2.538,0,0,1-2.276-1.168v.986Zm3.7-6.021a2.035,2.035,0,0,0-1.987,2.305,2.071,2.071,0,0,0,1.987,2.336c1.153,0,1.986-.88,1.986-2.336S8924.473,5174.118,8923.3,5174.118Z" transform="translate(-8919.604 -5168.976)"/> <path d="M8990.422,5213.118l1.774-3.868-3.154-6.507h1.987l2.124,4.672,1.986-4.672h1.866l-4.7,10.375Z" transform="translate(-8981.609 -5198.966)"/> <path d="M9105.4,5178.79c0,.5.031,1.046.061,1.35h-1.926a5.038,5.038,0,0,1-.076-.864,2.34,2.34,0,0,1-2.094,1.046,3.926,3.926,0,0,1-.061-7.827,2.238,2.238,0,0,1,2.109.925v-4.263h1.986Zm-3.793-.272a1.9,1.9,0,0,0,1.836-2.14,1.849,1.849,0,0,0-1.836-2.078,2.127,2.127,0,0,0,0,4.217Z" transform="translate(-9078.662 -5168.976)"/> <path d="M9178.639,5212.441l1.76-3.884-3.17-6.522h2.26l1.987,4.369,1.835-4.369h2.154l-4.687,10.406Z" transform="translate(-9149.652 -5198.333)"/> <path d="M9264.135,5207.852h-2.019v-7.462h1.958v.925a2.476,2.476,0,0,1,2.153-1.123c1.851,0,2.7,1.32,2.7,2.958v4.7h-2.019V5203.5a1.312,1.312,0,0,0-1.38-1.486,1.4,1.4,0,0,0-1.395,1.547Z" transform="translate(-9225.454 -5196.688)"/> <path d="M9339.31,5203.306l1.835-.272c.425-.061.562-.274.562-.531,0-.531-.409-.971-1.259-.971a1.325,1.325,0,0,0-1.427,1.213l-1.79-.379a2.969,2.969,0,0,1,3.2-2.458c2.366,0,3.247,1.335,3.247,2.836v3.671a8.454,8.454,0,0,0,.09,1.183h-1.851a5.105,5.105,0,0,1-.076-.9,2.494,2.494,0,0,1-2.2,1.107,2.357,2.357,0,0,1-2.563-2.246A2.3,2.3,0,0,1,9339.31,5203.306Zm2.4,1.274v-.334l-1.685.258a.942.942,0,0,0-.925.94.876.876,0,0,0,.971.864A1.516,1.516,0,0,0,9341.706,5204.581Z" transform="translate(-9292.393 -5196.435)"/> <path d="M9414.173,5207.726v-7.463h1.927v.91a2.565,2.565,0,0,1,2.185-1.123,2.308,2.308,0,0,1,2.214,1.244,2.76,2.76,0,0,1,5.066,1.592v4.839h-1.956V5203.3a1.245,1.245,0,0,0-1.32-1.411,1.372,1.372,0,0,0-1.381,1.472v4.369h-2V5203.3a1.252,1.252,0,0,0-1.319-1.411,1.366,1.366,0,0,0-1.4,1.472v4.369Z" transform="translate(-9361.234 -5196.561)"/> <path d="M9535.157,5167.457a1.244,1.244,0,1,1-1.243,1.259A1.243,1.243,0,0,1,9535.157,5167.457Zm-1,11.165v-7.463h2.018v7.463Z" transform="translate(-9468.158 -5167.457)"/> <path d="M9568.379,5203.868a1.921,1.921,0,0,0,1.91,2.108,1.707,1.707,0,0,0,1.715-1.259l1.775.592a3.466,3.466,0,0,1-3.49,2.518,3.861,3.861,0,0,1-3.928-3.959,3.821,3.821,0,0,1,3.867-3.959,3.42,3.42,0,0,1,3.506,2.519l-1.807.607a1.626,1.626,0,0,0-1.652-1.26A1.9,1.9,0,0,0,9568.379,5203.868Z" transform="translate(-9497.133 -5196.435)"/> <path d="M9669.8,5212.441l1.761-3.884-3.171-6.522h2.26l1.987,4.369,1.836-4.369h2.153l-4.687,10.406Z" transform="translate(-9588.244 -5198.333)"/> <path d="M9752.258,5167.457a1.244,1.244,0,1,1-1.244,1.259A1.244,1.244,0,0,1,9752.258,5167.457Zm-1,11.165v-7.463h2.018v7.463Z" transform="translate(-9662.02 -5167.457)"/> <path d="M9790.711,5205.46a3.36,3.36,0,0,1-3.4,2.366,3.966,3.966,0,0,1-.182-7.918c2.3,0,3.687,1.472,3.687,3.868,0,.289-.031.592-.031.622h-5.34a1.818,1.818,0,0,0,1.881,1.7,1.647,1.647,0,0,0,1.7-1.137Zm-1.881-2.442a1.5,1.5,0,0,0-1.668-1.472,1.615,1.615,0,0,0-1.669,1.472Z" transform="translate(-9690.994 -5196.435)"/> <path d="M9863.529,5180.139v-10.982h2.016v10.982Z" transform="translate(-9762.492 -5168.976)"/> <path d="M9901.386,5178.79c0,.5.03,1.046.061,1.35h-1.927a5.163,5.163,0,0,1-.075-.864,2.341,2.341,0,0,1-2.094,1.046,3.926,3.926,0,0,1-.061-7.827,2.234,2.234,0,0,1,2.108.925v-4.263h1.987Zm-3.793-.272a1.9,1.9,0,0,0,1.836-2.14,1.85,1.85,0,0,0-1.836-2.078,2.127,2.127,0,0,0,0,4.217Z" transform="translate(-9789.442 -5168.976)"/> </g> </svg></a></div><div id="et-top-navigation" class="cd-morph-dropdown"><nav id="top-menu-nav" class="main-nav"><ul id="top-menu" class="nav"></li></li></li></li></li></li></li></li></li></li></li></li></li><li class="top-link menu-item menu-item-type-post_type menu-item-object-page menu-item-home" data-content="discover-dynamic-yield" title=""> <a href="https://www.dynamicyield.com/">Discover Dynamic Yield</a></li><li class="has-dropdown learning_center menu-has-cta top-link-use-cases menu-item menu-item-type-custom menu-item-object-custom menu-item-has-children" data-content="learning-center" title=""> <a href="">Learning Center</a></li><li class="contact top-link walker-last mobile-only menu-item menu-item-type-post_type menu-item-object-page" data-content="xp2-newsletter" title=""> <a href="https://www.dynamicyield.com/newsletter/"><span class="button-arrow">XP<sup>2</sup> Newsletter</span></a></li></ul></nav><div class="morph-dropdown-wrapper"><div class="dropdown-list"><div class="dropdown-tooltip"></div><ul><li class="top-link menu-item menu-item-type-post_type menu-item-object-page menu-item-home" data-content="discover-dynamic-yield" title=""> <a href="https://www.dynamicyield.com/">Discover Dynamic Yield</a></li><li id="learning-center" class="dropdown links"><a href="" class="label">Learning Center</a><div class="content"><ul><li class="top"><ul class="links-list"><li class="resources-xp hidden-mobile menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/learn/">Take your knowledge to exponential levels</a></li><li class="hidden-desktop menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/learn/">XP<sup>2</sup> Hub<span>Take your knowledge to exponential levels</span></a></li><li class="resources-paths menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/learning-paths/">Learning Paths<span>Curated courses on key skill areas</span></a></li><li class="resources-webinars menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/talks/">Talks<span>Engaging discussions taking place in CX</span></a></li><li class="resources-articles menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/articles/">Articles<span>An expansive collection of in-depth playbooks</span></a></li><li class="resources-glossary menu-item menu-item-type-custom menu-item-object-custom"><a href="/glossary-terms/">Encyclopedia<span>A glossary of experience optimization terms</span></a></li><li class="resources-use-cases menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/personalization-examples/">Inspiration Library<span>Personalization examples from real brands</span></a></li><li class="resources-maturity menu-item menu-item-type-custom menu-item-object-custom"><a href="/personalization-maturity/"><strong class="badge-new">Personalization Maturity</strong><span>How global businesses prioritize personalization</span></a></li><li class="resources-guides menu-item menu-item-type-custom menu-item-object-custom"><a href="/guides/">Guides & reports<span>Comprehensive topic-specific materials</span></a></li><li class="resources-benchmarks menu-item menu-item-type-custom menu-item-object-custom"><a href="https://marketing.dynamicyield.com/benchmarks/">Benchmarks<span>Industry performance metrics and insights</span></a></li></ul></li><li class="bottom"><a href="/rooted-personalization/" target="_self"><div class="menu-banner"><div class="menu-cta--left"> <img src="/wp-content/uploads/2023/03/rooted-icon.svg" width="60" height="60" /></div><div class="menu-cta--right"> <strong class="menu-cta-title">Rooted Personalization<span>new</span></strong> <span class="menu-cta-subtitle">Learn a scalable methodology for building an audience-based personalization program.</span> <span class="menu-cta-button">Find out more →</span></div></div></a></li></ul></div></li><li class="contact top-link walker-last mobile-only menu-item menu-item-type-post_type menu-item-object-page" data-content="xp2-newsletter" title=""> <a href="https://www.dynamicyield.com/newsletter/"><span class="button-arrow">XP<sup>2</sup> Newsletter</span></a></li></ul><div class="bg-layer" aria-hidden="true"></div></div></div></li></ul></nav></div><div class="topnav-buttons"><div class="language-switcher header-language-switcher"><ul><li>en<ul><li><a class="active" href="https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/">English</a></li><li><a href="https://www.dynamicyield.com/es/">Español</a></li><li><a href="https://www.dynamicyield.com/de/">Deutsch</a></li><li><a href="https://www.dynamicyield.com/fr/">Français</a></li><li><a href="https://www.dynamicyield.com/ja/">日本語</a></li></ul></li></li></ul></div> <a href="#" data-toggle="modal" data-target="#lc-subscription"
 class="btn-desktop-only pulse-button v5 pulsed topnav-btn"> <span class="button-inner"> XP² Newsletter </span> </a> <a href="/newsletter/" class="btn-mobile-only pulse-button v5 pulsed topnav-btn"> <span class="button-inner"> XP² Newsletter </span> </a> <a href="" class="nav-trigger"> Open Nav <span class="nav-trigger-box"><span class="nav-trigger-inner"></span></span> </a></div></div></header><div id="lc-subscription" class="modal lc-subscription-popup" tabindex="-1" role="dialog" data-backdrop="false" aria-labelledby="lcSubscriptionModal"><div class="lc-container"><div class="lc-subscription-block"> <span class="lc-close-subscription-popup" data-dismiss="modal" aria-label="close"><i class="fa fa-close"></i></span><div class="lc-subscription-form-block lc-subscription-inner"><div class="lc-subscription-title-wrap"><h4 class="lc-subscription-title">Sign up for the XP&sup2; newsletter</h4><div class="lc-subscription-text">Join thousands of readers from Target, Citi, Spotify, Hulu, Google, Sephora, and other innovative brands who read our bi-weekly XP² newsletter, delivering educational content, research, and insights straight to your inbox</div><div class="lc-privacy">You may unsubscribe at any time. Visit our <a href="/privacy-policy/">privacy policy</a> to learn more about how we process your data and your rights as a data subject.</div></div><form class="mktoForm" data-formId="3876" data-formInstance="sub-popup"></form></div><div class="lc-subscription-ty-block lc-subscription-inner"><h4 class="lc-subscription-title text-center">Thanks for<br> signing up!</h4></div></div></div></div><div id="top-nav-backdrop"></div><div id="et-main-area"><div id="main-content"><div class="lc-container"><div id="content-area"><header class="article-sticky-header"><div class="lc-header-container lc-flex"><div class="lc-header-logo"> <a href="https://www.dynamicyield.com/learn" title="XP Squared" class="xp-logo-container"> <svg xmlns="http://www.w3.org/2000/svg" width="112.004" height="53.42" viewBox="0 0 112.004 53.42"> <path d="M8968.651,4606.571a49.057,49.057,0,0,0-6.159,6.344q-1.263,1.485-2.782,3.339t-3.079,3.71q-1.559,1.856-2.969,3.6t-2.522,3.079q.3.445,1.261,1.781t2.227,3.19q1.26,1.856,2.745,3.97t2.856,4.044q1.373,1.931,2.486,3.488t1.633,2.375q1.484,2.153,2.708,3.858a25.747,25.747,0,0,0,2.411,2.931,12.808,12.808,0,0,0,2.449,2,7.254,7.254,0,0,0,2.819,1v.742h-24.041v-.742q1.039-.147,2.226-.333a12.155,12.155,0,0,0,2.227-.557,4.522,4.522,0,0,0,1.706-1.039,2.31,2.31,0,0,0,.668-1.707,6.3,6.3,0,0,0-.965-2.783q-.964-1.743-2.226-3.673-1.708-2.672-3.9-5.825t-5.009-7.086q-1.188,1.411-2.6,3.116t-2.819,3.451q-1.411,1.744-2.708,3.376t-2.189,2.894a32.577,32.577,0,0,0-1.854,2.968,5.867,5.867,0,0,0-.816,2.82,2.706,2.706,0,0,0,.668,1.818,5.342,5.342,0,0,0,1.707,1.3,10.711,10.711,0,0,0,2.226.816,21.722,21.722,0,0,0,2.226.445v.742H8919.6v-.742a18.531,18.531,0,0,0,6.158-3.228,45.326,45.326,0,0,0,6.53-6.2q1.112-1.261,2.745-3.228t3.413-4.081q1.783-2.115,3.414-4.118t2.745-3.339q-.445-.593-2.04-2.856t-3.6-5.083q-2-2.819-3.97-5.6t-3.153-4.415q-1.559-2.152-2.782-3.784a26.076,26.076,0,0,0-2.375-2.782,12.747,12.747,0,0,0-2.449-1.93,13.428,13.428,0,0,0-3-1.3v-.742h24.189v.742q-.965.075-2.227.3a14.111,14.111,0,0,0-2.3.593,5.268,5.268,0,0,0-1.78,1.039,2.194,2.194,0,0,0-.743,1.707,5.789,5.789,0,0,0,1.039,2.746q1.039,1.708,2.374,3.636l3.785,5.454q1.928,2.782,4.675,6.715,2.449-2.818,5.009-6.159t4.488-5.787a31.259,31.259,0,0,0,1.93-2.968,6.278,6.278,0,0,0,1.039-2.968,2.5,2.5,0,0,0-.705-1.781,6.393,6.393,0,0,0-1.707-1.262,10.075,10.075,0,0,0-2.188-.816q-1.188-.3-2.152-.445v-.742h19.367v.742A15.309,15.309,0,0,0,8968.651,4606.571Z" transform="translate(-8919.604 -4602.453)"/> <path d="M9422.068,4621.77a14.179,14.179,0,0,1-2.338,3.932,14.748,14.748,0,0,1-3.376,2.931,20.786,20.786,0,0,1-3.934,1.967,24.367,24.367,0,0,1-4.451,1.113q-2.376.373-4.787.557t-4.711.186h-4.008v6.975q0,1.263.037,2.338t.038,2.337q.072,2.376.148,4.081a5.411,5.411,0,0,0,.815,2.82,6.25,6.25,0,0,0,2.3,1.929,10.813,10.813,0,0,0,4.75,1.113v.742h-23.745v-.742a20.734,20.734,0,0,0,3.561-.89,5.8,5.8,0,0,0,2.672-1.781,3.388,3.388,0,0,0,.594-1.336,14.222,14.222,0,0,0,.334-2q.111-1.113.186-2.375t.074-2.449c0-.692.011-1.632.037-2.82s.048-2.485.074-3.9.037-2.856.037-4.341v-18.773q0-1.334-.037-2.708c-.026-.915-.074-1.781-.148-2.6a13.947,13.947,0,0,0-.371-2.226,3.157,3.157,0,0,0-.705-1.447,5.554,5.554,0,0,0-2.856-1.6q-1.745-.407-3.748-.705v-.742h1.781q1.409,0,3.3-.037t4.117-.037q2.227,0,4.415-.037t4.119-.037h3.339a69.43,69.43,0,0,1,8.682.482,19.691,19.691,0,0,1,7.123,2.337,15.1,15.1,0,0,1,5.417,5.046,13.712,13.712,0,0,1,2.152,7.717A13.3,13.3,0,0,1,9422.068,4621.77Zm-9.312-12.986a13.487,13.487,0,0,0-9.535-6.159,28.427,28.427,0,0,0-4.155-.3,20,20,0,0,0-2.412.111q-.854.112-1.372.186a5.343,5.343,0,0,1-.186.779,4.946,4.946,0,0,0-.187.853q-.073.521-.147,1.447t-.149,2.634q-.075,2.6-.111,6.307t-.037,7.272v9.5q3.042,0,5.715-.186a18.593,18.593,0,0,0,4.971-1,13.885,13.885,0,0,0,7.271-5.379,14.741,14.741,0,0,0,2.523-8.422A13.63,13.63,0,0,0,9412.756,4608.784Z" transform="translate(-9329.39 -4601.215)"/> <g transform="translate(98.35)"> <path d="M9839.271,4616.067a10.2,10.2,0,0,1,3.648-3.89,26.827,26.827,0,0,0,3.232-2.56,3.789,3.789,0,0,0,1.127-2.635,2.928,2.928,0,0,0-.646-1.939,2.637,2.637,0,0,0-4.094.456,6.113,6.113,0,0,0-.4,2.256h-3.512a8.885,8.885,0,0,1,.85-3.789q1.447-2.75,5.139-2.75a6.52,6.52,0,0,1,4.643,1.616,5.584,5.584,0,0,1,1.727,4.277,5.783,5.783,0,0,1-1.221,3.624,12.091,12.091,0,0,1-2.627,2.345l-1.447,1.026q-1.359.964-1.859,1.395a4.076,4.076,0,0,0-.844,1h8.023v3.181h-12.584A8.616,8.616,0,0,1,9839.271,4616.067Z" transform="translate(-9838.422 -4601.215)"/> </g> </svg> </a></div><div class="lc-header-title"> Turning messy data into a gold mine using Spark, Flink, and ScyllaDB</div><div class="lc-header-right lc-flex"><ul class="lc-header-social lc-flex"><li class="et-social-icon"> <a href="#" title="Share on Facebook" class="icon" onclick="window.open('https://www.facebook.com/sharer/sharer.php?u=' + encodeURIComponent(document.URL) + '&t=' + encodeURIComponent(document.URL)); return false;"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 460.1990662 458.5138245"><path d="M460.1990662,230.0995331C460.1990662,103.0190277,357.1800232,0,230.0995331,0S0,103.0190277,0,230.0995331C0,338.011261,74.2903519,428.5578003,174.5126801,453.4230957V300.4146423H127.064209v-70.3151093h47.4484711v-30.2983704c0-78.3184433,35.4433594-114.619339,112.3326569-114.619339c14.5774536,0,39.730896,2.8583221,50.020874,5.7166443v63.7409363c-5.4308167-0.5716553-14.8633118-0.8574677-26.58255-0.8574677c-37.7300415,0-52.3075867,14.2916718-52.3075867,51.4501038v24.8674927h75.158783l-12.9122009,70.3151093h-62.246582v158.0991821C371.9082336,444.7527161,460.1990662,347.742981,460.1990662,230.0995331z"/></svg> </a></li><li class="et-social-icon"> <a href="#" title="Share on Linkedin" class="icon" onclick="window.open( 'https://www.linkedin.com/shareArticle?mini=true&url=' + encodeURIComponent(document.location.href)); return false;"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><path d="M100.28 448H7.4V148.9h92.88zM53.79 108.1C24.09 108.1 0 83.5 0 53.8a53.79 53.79 0 0 1 107.58 0c0 29.7-24.1 54.3-53.79 54.3zM447.9 448h-92.68V302.4c0-34.7-.7-79.2-48.29-79.2-48.29 0-55.69 37.7-55.69 76.7V448h-92.78V148.9h89.08v40.8h1.3c12.4-23.5 42.69-48.3 87.88-48.3 94 0 111.28 61.9 111.28 142.3V448z"/></svg> </a></li><li class="et-social-icon"> <a href="#" title="Tweet" class="icon" onclick="window.open('https://twitter.com/intent/tweet?text=' + encodeURIComponent(document.title) + ':%20'  + encodeURIComponent(document.URL)); return false;"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><path d="M389.2 48h70.6L305.6 224.2 487 464H345L233.7 318.6 106.5 464H35.8L200.7 275.5 26.8 48H172.4L272.9 180.9 389.2 48zM364.4 421.8h39.1L151.1 88h-42L364.4 421.8z"/></svg> </a></li></ul><div class="topnav-buttons"> <a href="#" data-toggle="modal" data-target="#lc-subscription" class="btn-desktop-only pulse-button v5 topnav-btn"> <span class="button-inner"> XP² Newsletter </span> </a> <a href="/newsletter/" class="pulse-button v5 topnav-btn btn-mobile-only"> <span class="button-inner"> XP² Newsletter </span> </a></div></div></div><div class="lc-progress-wrap"><div class="lc-scroll-progress"></div></div></header><article id="post-110420" class="et_pb_post post-110420 article type-article status-publish has-post-thumbnail hentry article-category-data-analytics article-industry-agnostic article-tags-personalization-infrastructure article-function-development article-type-thought-leadership"><div class="article-content"><div class="article-head-content-inner"><div class="article-breadcrumbs"> <a href="https://www.dynamicyield.com/articles/" title="Articles">Articles</a> <a href="https://www.dynamicyield.com/articles/category/data-analytics/ ">Data &amp; Analytics</a></div><h1 class="article-title">Turning messy data into a gold mine using Spark, Flink, and ScyllaDB</h1><h2 class="article-excerpt">With our user's data spread out across numerous locations, a set of real challenges emerged. Find out how we set out to unify it, along with all the technical decisions we made and new opportunities it unlocked.</h2><div class="article-authors lc-flex"><div class="article-author-box lc-col lc-flex"><div class="article-author-image lc-col"><div class="entry_author_image"><img src="https://www.dynamicyield.com/wp-content/authors/OranHirsch-79.jpg"  alt="Oran Hirsch" width="76" height="76" /></div></div><div class="article-author-title lc-col"><div class="article-author-name"> <a href="https://www.dynamicyield.com/blog/author/oranhirsch/" title="Posts by Oran Hirsch" rel="author">Oran Hirsch</a></div><div class="article-author-title-text"> Big Data Developer in R&D</div></div></div></div></div><div class="entry-content article-entry-content"><div class="article-entry-content-inner"><p>At Dynamic Yield, we eat, sleep, and breathe data &ndash; it is the bedrock of our personalization technology. Our R&amp;D department is constantly working to ensure all of the valuable information we capture through the platform fuels the development of new products and features.&nbsp;</p><p>Recently, we faced a real set of challenges with regards to how we stored our user&rsquo;s data. Seeing as every dev team works on different features and leverages the data for their own unique purposes, naturally, portions of it would end up stored in different databases &ndash; wherever was most suitable based on the team&rsquo;s technical needs. However, with data spread across numerous locations, it became more and more difficult to see the big picture. Developers began struggling to analyze the data and understand simple questions about our users. And in turn, it was harder for them to develop features based on it.</p><p>To solve this, we set out to unify it &ndash; all of it &ndash; made possible with the help of Apache Spark, Apache Flink, and ScyllaDB. In the rest of this post, I will walk through all the technical decisions we made to establish a unified database, which has not only unlocked new opportunities for our department as a whole, but also our customers.</p><h2>Our attempt at unifying user data into one place</h2><p>Although we didn&rsquo;t tackle everything from the onset, we did want to plan on addressing these four main technical issues to effectively accomplish our goal:</p><ul><li>Defining internal and customer needs&nbsp;</li><li>Choosing a database (DB) which is capable of supporting our solution</li><li>Migrating historical data into the DB</li><li>Streaming live events without any gaps between historical and streamed data</li></ul><h3>Choosing a database&nbsp;</h3><p>Next, a few core requirements needed to be ironed out.</p><p><strong>Should store large amounts of data </strong>&#8211; We collect a lot of data about users and have been doing so for several years now, so we needed an option that would not only be able to handle our current database but also scale as it becomes larger.</p><p><strong>Read latency must be very low </strong>&#8211; When users come back to our customers&rsquo; sites, we quickly analyze everything we know about them to offer a personalized experience. This process of analysis and delivery has to happen almost immediately in order for the web page to load within a reasonable amount of time.</p><p><strong>Needs to contain both historical and real-time data </strong>&#8211; We analyze user data and for us to make accurate decisions based on it, the data has to be up-to-date. Therefore, both historical user data, as well as current activity, should be taken into account.</p><p><strong>Flexible query support </strong>&#8211; Today we may be interested in only certain pieces of data, but tomorrow we might be interested in another portion of it. Proper query support was needed to allow us this kind of flexibility.</p><h3>Why we chose ScyllaDB</h3><p>ScyllaDB is one of the newest NoSQL databases out there. Described as an Apache Cassandra replacement with ultra-low latency and extreme throughput, with most of Cassandra&rsquo;s features implemented in C++, it&rsquo;s able to present very <a href="https://www.scylladb.com/product/benchmarks/">impressive benchmarks</a>.</p><p>Additionally, it shared other similarities with Apache Cassandra we liked:&nbsp;</p><p><strong>CQL (Cassandra Query Language) support</strong> &#8211; Using CQL, we could easily write queries to fetch the data we needed.&nbsp;</p><p><strong>Easy integration with Apache Spark</strong> &#8211; When writing our Spark applications, we could use the &lsquo;<a href="https://github.com/datastax/spark-cassandra-connector">spark-cassandra-connector</a>&rsquo; in order to write and read from Scylla.</p><p>Its low read latency meant we could confidently use it for serving experiences. And because it&rsquo;s not an in-memory database, it wouldn&rsquo;t be as expensive. As I mentioned before, we needed to store a lot of user data, and given the number of nodes we would need, this was a big factor for us.</p><p>From a safety point of view, Scylla supports data replication according to the replication strategy of your choosing. So if one Scylla node crashes, there&rsquo;s no fear of losing the data.&nbsp;</p><h3>Architecture</h3><p>After we got our DB up and running, it was time to fill it with some data.</p><p><strong>Migrating old data to the new DB </strong>-Using Apache Spark we were able to migrate all of our data to the new Scylla DB very quickly. The spark application was pretty simple as it only needed a start and end date to get started. It then scanned the old &lsquo;raw data&rsquo; within the time range allotted and wrote it to Scylla. This &lsquo;raw data&rsquo; functions as our big data store &ndash; great for batch jobs, but not for random access.&nbsp;</p><p>*Note* we only had to run the spark application once in order to migrate all data to into Scylla DB. If we ever need to migrate data into Scylla again, we will be able to use the same application. In fact, we have used this capability more than once when developing.</p><p><strong>Streaming online events to Scylla </strong>-As previously mentioned, one of our requirements was that Dynamic Yield&rsquo;s Unified Customer Profile would always contain up-to-date data. For this purpose, we used Apache Flink &ndash; our application receives streams of user events as they occur from our Kafka topics, applies some transformations to the data to fit our schema, and then writes them to Scylla.</p><p><img fetchpriority="high" decoding="async" src="/wp-content/uploads/2019/09/architecture-chart.png" alt="Dynamic Yield&#039;s database architcture" width="1330" height="900" class="aligncenter size-full wp-image-103727" /></p><h3>Data modeling</h3><p>Not all data is created equal. For example, a purchase event can provide a lot more insight into a user&rsquo;s preferences than a simple page view. With a large percentage of our data representing page views, we knew there wasn&rsquo;t a need to store them forever.</p><p>Using a table-based format for our data modeling, we save all user events in one table. Events like purchases or add-to-carts are never deleted, while page views are stored for a three month period. Rather than saving these page views, what we do is actually extract &lsquo;aggregated data&rsquo; from them.</p><p>We store this aggregated data, which can also include a user&rsquo;s location, browsers, devices, etc., in another table. If we already know a user has visited a web page from the U.S., there is no need to save the information again when they revisit the web page from the same location and so on.&nbsp;</p><h3>The merits of our unified customer profile</h3><p>Having all of our data in one place has opened many doors for us, aiding our understanding of each user who interacts with an experience launched through the Dynamic Yield platform in a much more efficient way. And through the work we&rsquo;ve done, we are now able to bestow this knowledge on our customers, who can use the rich data from what we refer to as our <a href="https://www.dynamicyield.com/blog/unified-customer-profile-apis/">Unified Customer Profile</a> (UCP) to:&nbsp;</p><ul><li>Provide tailored content every time a user revisits a site or app</li><li>Leverage machine learning algorithms to match each user with the optimal experience (based on both long-term history and in-session behavior)&nbsp;</li><li>Develop custom individualized experiences through server-side implementation</li></ul><p>Not to mention, <a href="https://www.dynamicyield.com/gdpr">meeting the requirements of GDPR</a> became significantly easier when all of our user data was unified, another huge bonus of our efforts.</p><p><em>Have questions about the process we went through or want to share stories about how you unified your company&rsquo;s data? Comment below! </em></p><div class="article-separator"><svg width="54" height="26" viewBox="0 0 54 26" fill="none" xmlns="http://www.w3.org/2000/svg"> <path d="M53.0088 25.6367C52.7884 21.9564 51.8025 18.3625 50.1141 15.0849C48.4256 11.8073 46.0718 8.9181 43.2031 6.60203L43.2031 25.6367L53.0088 25.6367Z" fill="black"/> <path d="M14.4395 3.59038C10.3237 5.69174 6.82743 8.83082 4.29638 12.6973C1.76533 16.5638 0.28711 21.0239 0.0078125 25.6367L14.4395 25.6367L14.4395 3.59038Z" fill="black"/> <path d="M17.6563 2.20809L17.6562 25.6367L39.9829 25.6367L39.9829 4.36701C39.9829 4.36701 30.7303 -2.31342 17.6563 2.20809Z" fill="black"/> </svg></div><div class="article-tags-block"> <strong>Tags:</strong> <a href="https://www.dynamicyield.com/tags/personalization-infrastructure/">Personalization Infrastructure</a></div></div><div class="article-content-sidebar"><div class="article-content-sidebar-inner"><h4>Read next</h4><ul class="article-sidebar-nav"><li> <a href="https://www.dynamicyield.com/article/the-enterprise-guide-to-ip-warning/">The enterprise guide to IP warming: Boost deliverability, ensure compliance, and power seamless journeys</a><div class="article-sidebar-nav-author">Ravit Morales</div></li><li> <a href="https://www.dynamicyield.com/article/personalization-maturity-2026-top-takeaways/">Disruption redefines personalization: Why brands that accelerate maturity will win in 2026 and beyond</a><div class="article-sidebar-nav-author">Sarah Grob</div></li></ul></div></div></div></div></article></div></div><section class="lc-subscription-section"><div class="lc-subscription-form-block lc-subscription-inner"><h4 class="lc-subscription-title">Sign up for the XP&sup2; newsletter</h4><div class="lc-subscription-text">Join thousands of readers from Target, Citi, Spotify, Hulu, Google, Sephora, and other innovative brands who read our bi-weekly XP² newsletter, delivering educational content, research, and insights straight to your inbox</div><form class="mktoForm" data-formId="3876" data-formInstance="sub-inline"></form><div class="lc-privacy-wrapper"><div class="lc-privacy"> You may unsubscribe at any time. Visit our <a href="/privacy-policy/">privacy policy</a> to learn more about how we process your data and your rights as a data subject.</div></div></div><div class="lc-subscription-ty-block lc-subscription-inner"><h4 class="lc-subscription-title text-center">Thanks for<br> signing up!</h4></div></section><div class="container"><div class="articles-popular-wrapper"><h3>Most Popular in <span>Data &amp; Analytics</span></h3><div class="lc-flex articles-popular-block"><a href="https://www.dynamicyield.com/article/ga-filters/"><span class="article-popular-img-wrap"><img src="https://www.dynamicyield.com/wp-content/uploads/2014/06/analytics2.jpg" width="255" height="103" alt="" class="crp_thumb"></span><strong>Don’t Trust Your Google Analytics Data Just yet (part 2): Get Rid of Noise With These Fundamental 9 Filters</strong></a><a href="https://www.dynamicyield.com/article/apache-hbase-for-the-win/"><span class="article-popular-img-wrap"><img src="https://www.dynamicyield.com/wp-content/uploads/2014/06/Apache-HBase2.jpg" width="255" height="103" alt="" class="crp_thumb"></span><strong>Apache HBase for the Win, Part 1:  All the Ways in Which it’s Bad - and Why it’s Still Great</strong></a><a href="https://www.dynamicyield.com/article/dark-social/"><span class="article-popular-img-wrap"><img src="https://www.dynamicyield.com/wp-content/uploads/2015/06/The-Dark-Side-of-Social1.jpg" width="255" height="103" alt="" class="crp_thumb"></span><strong>Into the Great Unknown: Understanding and Leveraging Dark Social Traffic</strong></a><a href="https://www.dynamicyield.com/article/web-analytics-fail/"><span class="article-popular-img-wrap"><img src="https://www.dynamicyield.com/wp-content/uploads/2015/02/Web-Analytics-Fail.jpg" width="255" height="103" alt="" class="crp_thumb"></span><strong>Why most marketers fail at web analytics</strong></a></div></div></div></div><footer id="main-footer" class=""><div class="footer-top"><div class="footer-container"> <a href="https://www.dynamicyield.com/" class="dy-footer-logo"> <img src="/wp-content/uploads/2025/09/ma_symbol.svg" alt="Dynamic Yield footer logo" width="100" height="66"> </a><h4>Make hyper-personalization a reality</h4><p>Trusted by 450+ forward-thinking enterprise businesses:</p><div class="footer-clients retina_parent"> <img src="/wp-content/uploads/2023/08/customer-logos-footer.png" alt="DY Clients" width="498" height="123"></div><div class="footer-btns-wrap"> <a href="https://www.dynamicyield.com/request-demo/" class="btn-teal">Contact Sales</a> <a href="https://www.dynamicyield.com/product-demo/" class="btn-transparent">Watch a product demo<svg xmlns="http://www.w3.org/2000/svg" width="15.625" height="15.625" viewBox="0 0 15.625 15.625"> <path id="Icon_ionic-ios-play-circle" data-name="Icon ionic-ios-play-circle" d="M10.688,3.375A7.313,7.313,0,1,0,18,10.688,7.311,7.311,0,0,0,10.688,3.375Zm2.946,7.45L8.81,13.743a.156.156,0,0,1-.236-.137V7.77a.156.156,0,0,1,.236-.137l4.823,2.918A.161.161,0,0,1,13.634,10.825Z" transform="translate(-2.875 -2.875)" fill="none" stroke="rgba(0,0,0,0.8)" stroke-width="1"/> </svg> </a></div><ul class="footer-certificates"><li class="retina_parent"> <a href="https://www.dynamicyield.com/compliance/"> <img src="/wp-content/uploads/2022/05/gdpr-icon.png" alt="gdpr icon" width="139" height="70"> </a></li><li class="retina_parent"> <a href="https://www.dynamicyield.com/compliance/"> <img src="/wp-content/uploads/2023/04/SOCII-Badge.png" alt="socii icon" width="70" height="70"> </a></li><li> <a href="https://www.dynamicyield.com/compliance/"> <img src="/wp-content/uploads/2022/05/ccpa-icon.svg" alt="ccpa icon" width="117" height="40"> </a></li><li class="retina_parent"> <a href="https://www.dynamicyield.com/compliance/"> <img src="/wp-content/uploads/2022/05/iso-27001.png" alt="iso icon" width="106" height="68"> </a></li></ul></div></div><div id="et-footer-nav"><div class="footer-container"><ul id="menu-footer-menu" class="bottom-nav"><li id="menu-item-19183" class="footer_menu_title menu-item menu-item-type-custom menu-item-object-custom menu-item-19183"><a>Recommended Guides:</a></li><li id="menu-item-211968" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-211968"><a href="https://www.dynamicyield.com/article/personalization-guide/">Personalization</a></li><li id="menu-item-138198" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-138198"><a href="https://www.dynamicyield.com/lesson/introduction-to-ab-testing/">A/B Testing</a></li><li id="menu-item-147897" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-147897"><a href="https://www.dynamicyield.com/lesson/cro-plan/">Conversion Rate Optimization</a></li><li id="menu-item-19184" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-19184"><a href="/lesson/shopping-cart-abandonment-strategy/">Shopping Cart Abandonment</a></li><li id="menu-item-19185" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-19185"><a href="/lesson/product-recommendations-guide/">Product Recommendations</a></li><li id="menu-item-22805" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-22805"><a href="/lesson/promise-and-pitfalls-omnichannel-retailing/">Omnichannel Retailing</a></li><li id="menu-item-22885" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-22885"><a href="/lesson/the-economics-of-ecommerce-conversion-optimization/">eCommerce Conversion Rate Optimization</a></li><li id="menu-item-28240" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-28240"><a href="/article/50-most-important-dynamicyield-personalization-stats/">Personalization Statistics</a></li><li id="menu-item-33877" class="menu-item menu-item-type-post_type menu-item-object-post menu-item-33877"><a href="https://www.dynamicyield.com/article/tj-maxx-spired-checkout-optimization/">Checkout Optimization</a></li><li id="menu-item-123711" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-123711"><a href="/article/ecommerce-personalization-must-haves/">eCommerce Personalization</a></li><li id="menu-item-144090" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-144090"><a href="https://www.dynamicyield.com/lesson/strategizing-omnichannel-personalization/">Omnichannel Personalization</a></li><li id="menu-item-148146" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-148146"><a href="https://marketing.dynamicyield.com/benchmarks/cart-abandonment-rate/">Shopping Cart Abandonment Rate</a></li></ul><div class="footer-mach-logo retina_parent"> <img src="/wp-content/uploads/2022/05/mach-cert-22-wh.png" alt="Mach logo" width="63" height="70"></div><div class="language-switcher footer-language-switcher"><ul><li>en<ul><li><a class="active" href="https://www.dynamicyield.com/article/turning-messy-data-into-a-gold-mine/">English</a></li><li><a href="https://www.dynamicyield.com/es/">Español</a></li><li><a href="https://www.dynamicyield.com/de/">Deutsch</a></li><li><a href="https://www.dynamicyield.com/fr/">Français</a></li><li><a href="https://www.dynamicyield.com/ja/">日本語</a></li></ul></li></li></ul></div></div></div><div id="footer-bottom"><div class="footer-container"><div class="footer_info_wrapper"><div id="footer-info"> <button id="ot-sdk-btn" class="ot-sdk-show-settings">Manage Cookies</button> <a href="/privacy-notice/" target="_blank" class="animlink">Privacy Notice</a> <a href="/tos/" target="_blank" class="animlink">Terms of use</a> <span>&copy; 2026 Mastercard Dynamic Yield</span></div></div><div class="footer_social_icons"><style>.et-social-icons .et-social-icon img {
        max-width: 16px;
        transition: 0.2s all;
    }

    .et-social-icons .et-social-icon img:not(:hover) {
        opacity: 0.5;
    }</style><span itemscope itemtype="http://schema.org/Organization"><link itemprop="url" href="https://www.dynamicyield.com"><ul class="et-social-icons"><li class="et-social-icon"> <a itemprop="sameAs" href="https://www.linkedin.com/company/dynamic-yield" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame-5.png" alt="LinkedIn Logo" /> </a></li><li class="et-social-icon"> <a itemprop="sameAs" href="https://www.instagram.com/dynamicyield/" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame-4.png" alt="Instagram Logo" /> </a></li><li class="et-social-icon"> <a itemprop="sameAs" href="https://www.youtube.com/c/Dynamicyield" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame-3.png" alt="Youtube Logo" /> </a></li><li class="et-social-icon"> <a itemprop="sameAs" href="https://www.tiktok.com/@dynamic.yield" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame-2.png" alt="TikTok Logo" /> </a></li><li class="et-social-icon"> <a itemprop="sameAs" href="https://www.facebook.com/DynamicYield" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame-1.png" alt="Facebook Logo" /> </a></li><li class="et-social-icon"> <a itemprop="sameAs" href="https://twitter.com/DynamicYield" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame.png" alt="X Logo" /> </a></li></ul> </span></div></div></div></footer></div></div><div id="mobile_menu_overlay"></div> <script type='text/javascript' src='//cdn.dynamicyield.com/api/8765281/api_dynamic.js'></script> <script type='text/javascript' src='//cdn.dynamicyield.com/api/8765281/api_static.js'></script>  <script type="text/javascript">var sbiajaxurl = "https://www.dynamicyield.com/wp-admin/admin-ajax.php";</script> <script type="text/javascript">!function(e,n){var r={"selectors":{"block":"pre.EnlighterJSRAW","inline":"code.EnlighterJSRAW"},"options":{"indent":2,"ampersandCleanup":true,"linehover":true,"rawcodeDbclick":false,"textOverflow":"break","linenumbers":true,"theme":"beyond","language":"generic","retainCssClasses":false,"collapse":false,"toolbarOuter":"","toolbarTop":"{BTN_RAW}{BTN_COPY}{BTN_WINDOW}{BTN_WEBSITE}","toolbarBottom":""},"resources":["https:\/\/www.dynamicyield.com\/wp-content\/plugins\/enlighter\/cache\/enlighterjs.min.css?tvrakvcLArVio6C","https:\/\/www.dynamicyield.com\/wp-content\/plugins\/enlighter\/resources\/enlighterjs\/enlighterjs.min.js"]},o=document.getElementsByTagName("head")[0],t=n&&(n.error||n.log)||function(){};e.EnlighterJSINIT=function(){!function(e,n){var r=0,l=null;function c(o){l=o,++r==e.length&&(!0,n(l))}e.forEach(function(e){switch(e.match(/\.([a-z]+)(?:[#?].*)?$/)[1]){case"js":var n=document.createElement("script");n.onload=function(){c(null)},n.onerror=c,n.src=e,n.async=!0,o.appendChild(n);break;case"css":var r=document.createElement("link");r.onload=function(){c(null)},r.onerror=c,r.rel="stylesheet",r.type="text/css",r.href=e,r.media="all",o.appendChild(r);break;default:t("Error: invalid file extension",e)}})}(r.resources,function(e){e?t("Error: failed to dynamically load EnlighterJS resources!",e):"undefined"!=typeof EnlighterJS?EnlighterJS.init(r.selectors.block,r.selectors.inline,r.options):t("Error: EnlighterJS resources not loaded yet!")})},(document.querySelector(r.selectors.block)||document.querySelector(r.selectors.inline))&&e.EnlighterJSINIT()}(window,console);</script><script type="text/javascript" id="rfw-script-js-extra">var rfw = {"speed":""};</script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_a5ec58aa1f222bedaf60cdced3b8a004.js?ver=2026020657" id="rfw-script-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_c1b7fbe6b1a3b777fddfe187094deb97.js?ver=2026020657" id="rfw-script-fitvid-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-includes/js/comment-reply.min.js?ver=935c7f8ce03960b02800bc8ae3231df8" id="comment-reply-js" async="async" data-wp-strategy="async"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_ea3fad8947382ebf8709507a11a805db.js" id="divi-fitvids-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/themes/Divi/js/waypoints.min.js" id="waypoints-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_845779ea879c70773af95b2f6fc394d2.js" id="magnific-popup-js"></script> <script type="text/javascript" id="divi-custom-script-js-extra">var et_custom = {"ajaxurl":"https:\/\/www.dynamicyield.com\/wp-admin\/admin-ajax.php","images_uri":"https:\/\/www.dynamicyield.com\/wp-content\/themes\/Divi\/images","et_load_nonce":"ee490c83a0","subscription_failed":"Please, check the fields below to make sure you entered the correct information.","fill":"Fill","field":"field","invalid":"Invalid email","captcha":"Captcha"};</script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_43b4a88b2828bc5c023dc913689e8da5.js?ver=20.4" id="divi-custom-script-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/themes/Divi/custom/bootstrap/js/bootstrap.min.js" id="bootstrap-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/themes/Divi/custom/slick/slick.min.js" id="slick-js"></script> <script type="text/javascript"  data-ot-ignore src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_12efd5a7d441d8b7f817fc6fc685dca9.js?ver=21.85" id="customscript-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_33fa37a5effe9361e40b8525e36e3eaf.js" id="typed-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_eb8981568f49ac148e96af3241eb95c7.js?ver=21.85" id="form_validation-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_be33bce741fd746dee1a6cf31c0b5bde.js" id="retina_js-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_9a50928b26148d1c35a95957e5ef2b9d.js?ver=21.85" id="mkto-scripts-js"></script> <script type="module" src="https://www.dynamicyield.com/wp-content/themes/Divi/js/cid-validation.js?ver=21.85" id="cid-validation"></script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.5/js/select2.min.js" id="select2-js-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_6ee6e48031ef5dddfe05a9f11136201f.js" id="post-checklist-js-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_5113be2479418a077e699b1647e0adef.js" id="post-event-cards-js-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/plugins/wp-featherlight/js/wpFeatherlight.pkgd.min.js?ver=1.3.4" id="wp-featherlight-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_17babe2fc54117766e24bd50889c6577.js?ver=1772261842" id="article-script-js"></script> <script type="text/javascript" id="dc_conversion_api-js-extra">var ajax_obj = {"nonce":"910abf0a29","ajax_url":"https:\/\/www.dynamicyield.com\/wp-admin\/admin-ajax.php","ajax_action":"conversion_api_call"};</script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_34a6a3100a2ecbc7706af2d4c798f856.js?ver=935c7f8ce03960b02800bc8ae3231df8" id="dc_conversion_api-js"></script> <script type="text/javascript" data-noptimize>var dy_search_params = new URLSearchParams(window.location.search);

        if(dy_search_params.has('utm_source')) {
            localStorage.setItem('utm_source', dy_search_params.get('utm_source'));
        }

        if(dy_search_params.has('utm_medium')) {
            localStorage.setItem('utm_medium', dy_search_params.get('utm_medium'));
        }

        if(dy_search_params.has('utm_content')) {
            localStorage.setItem('utm_content', dy_search_params.get('utm_content'));
        }

        if(dy_search_params.has('utm_campaign')) {
            localStorage.setItem('utm_campaign', dy_search_params.get('utm_campaign'));
        }

        if(dy_search_params.has('utm_term')) {
            localStorage.setItem('utm_term', dy_search_params.get('utm_term'));
        }

        if(dy_search_params.has('mkt_tok')) {
            localStorage.setItem('mkt_tok', dy_search_params.get('mkt_tok'));
        }

        // Also, fill in the data inside the Marketo form once its available
        if(typeof window.MktoForms2 === 'object') {

            MktoForms2.whenReady(function(_form) {

                var form = _form.getFormElem();
                console.log('MKTO FORM READY');

                // Append the mkt_tok param if it exists in the URL
                var ls_mkto_tok = localStorage.getItem('mkt_tok');

                if(ls_mkto_tok) {

                    _form.addHiddenFields({
                        mkt_tok: ls_mkto_tok
                    });

                    _form.vals({ mkt_tok: ls_mkto_tok });
                }

                // Fill in the UTM data from localStorage
                var dy_utms = {
                    'utm_source': 'txt_source__c',
                    'utm_medium': 'txt_medium__c',
                    'utm_content': 'txt_content__c',
                    'utm_campaign': 'txt_campaign_name__c',
                    'utm_term': 'txt_term__c',
                    'gclid': ['GCLID__c', 'GCLID_c__c']
                };

                for(var i = 0; i < Object.keys(dy_utms).length; i++) {

                    var ls_key   = Object.keys(dy_utms)[i],
                        ls_value = localStorage.getItem(ls_key),
                        form_key = dy_utms[ls_key],
                        form_field;

                    if(ls_value) {

                        if(ls_key === 'gclid') {
                            ls_value = JSON.parse(ls_value).value;
                        }

                        if(typeof(form_key) === 'string') {
                            form_field = form[0].querySelector('input[type="hidden"][name="' + form_key + '"]');

                            if(form_field) {
                                form_field.value = ls_value;
                            }

                        } else {

                            for(var j = 0; j < form_key.length; j++) {
                                form_field = form[0].querySelector('input[type="hidden"][name="' + form_key[j] + '"]');

                                if(form_field) {
                                    form_field.value = ls_value;
                                }
                            }
                        }
                    }
                }

                // IP Address
                fetch('https://www.cloudflare.com/cdn-cgi/trace').then(function(result) {

                    result.text().then(function(values) {

                        values.split("\n").map(function(item) {
                            var pair = item.split('=');

                            if(['ip'].includes(pair[0])) {
                                form[0].querySelector('input[type="hidden"][name="IP__c"]').value = pair[1];
                            }
                        });
                    });
                });

                // GA ID and Referrer
                var dy_utms_cookie = {};

                document.cookie.split(';').forEach(function(el) {
                    var splitCookie = el.split('=');
                    var key = splitCookie[0].trim();
                    var value = splitCookie[1];
                    dy_utms_cookie[key] = value;
                });

                if(dy_utms_cookie["_ga"]) {
                    form[0].querySelector('input[type="hidden"][name="Visitor_ID__c"]').value = dy_utms_cookie["_ga"].substring(6);
                }

                if(dy_utms_cookie["personalize_referrer_string"]) {
                    form[0].querySelector('input[type="hidden"][name="Referral__c"]').value = dy_utms_cookie["personalize_referrer_string"].substring(6);
                }

                // ZoomInfo
                if(localStorage.getItem('ZoomInfoData')) {
                    var zoominfo_data = JSON.parse(localStorage.getItem('ZoomInfoData'));

                    if(zoominfo_data) {
                        form[0].querySelector('input[type="hidden"][name="Alexa_Rank__c"]').value = zoominfo_data.alexaglobalrank;
                        form[0].querySelector('input[type="hidden"][name="Employees__c"]').value = zoominfo_data.employeesrange;
                        form[0].querySelector('input[type="hidden"][name="Company_Revenue_Range__c"]').value = zoominfo_data.estimatedannualrevenue;
                    }
                }

                // Also, dispatch a dataLayer event on success
                _form.onSuccess(function(values, follow_up_url) {
                    window.dataLayer = window.dataLayer || [];

                    if([3739, 4802, 5094, 5110, 4946, 5076, 5116, 5171, 5172, 5173].includes(_form.getId())) {

                        window.dataLayer.push({
                            'event': 'demo_request_complete',
                            'eventCallback': function() {

                                if(follow_up_url) {
                                    window.location.href = follow_up_url;
                                }
                            }
                        });

                        return false;

                    } else if([3876].includes(_form.getId())) {

                        window.dataLayer.push({
                            'event': 'newsletter_signup_complete',
                            'eventCallback': function() {

                                if(follow_up_url) {
                                    window.location.href = follow_up_url;
                                }
                            }
                        });

                        return false;
                    }
                });
            });
        }</script> <script>if(typeof(jQuery) !== 'undefined') {
                $ = jQuery.noConflict();
            }
			function getUrlParameter(sParam) {
    var sPageURL = decodeURIComponent(window.location.search.substring(1)),
        sURLVariables = sPageURL.split('&'),
        sParameterName,
        i;

    for (i = 0; i < sURLVariables.length; i++) {
        sParameterName = sURLVariables[i].split('=');

        if (sParameterName[0] === sParam) {
            return sParameterName[1] === undefined ? true : sParameterName[1];
        }
    }
}
$(window).load(function(){
    var videoID = getUrlParameter('video');
    if(videoID !== undefined){
        if($('#wistia-' + videoID + '-1').length){
    	    var videoName = $('#wistia-' + videoID + '-1').parents('.tab-pane');
    	    videoName = videoName[0].id
        } 
       if($('a[href="#' + videoName +'"]').length){
            $('a[href="#' + videoName +'"]').each(function(){
            $(this).click();
            });
        }
        
        addScript('https://fast.wistia.com/embed/medias/'+ videoID +'.jsonp');
        addScript('https://fast.wistia.com/assets/external/E-v1.js');
        $('body').append('<span class="wistia_embed wistia_async_'+videoID+' popover=true popoverContent=link dy-wistia-popup" style="display:inline"></span>');
    
         window._wq = window._wq || [];
        _wq.push({ id: videoID, onReady: function(video) {
            var video = Wistia.api(videoID);
            video.popover.show(); // open the popover
            video.play(); // play the video
            setTimeout(function(){video.unmute();}, 500);
        }});
    }
});</script> <style>#adBanner {
        background-color: transparent;
        height: 0;
        width: 1px;
    }</style><div id="wrapfabtest"><div id="adBanner"></div></div></body></html>
<!-- Dynamic page generated in 0.475 seconds. -->
<!-- Cached page generated by WP-Super-Cache on 2026-02-28 01:57:22 -->

<!-- super cache -->