This post is part two of a two-part series (read part one here, if you haven’t yet) brought to you by DY Labs, an initiative by members of our Product and R&D departments who have a passion for experimentation and building developer resources for the greater digital marketing and engineering communities. This series focuses on the cohort’s latest project, Funnel Rocket, which is a serverless query engine optimized for complex user-centric queries at scale. In part two, Elad Rosenheim, our VP of Technology, walks through the inception of Funnel Rocket, how it works, and the potential applications it can be used for.
An earlier version of this post was previously published by Elad on Medium.
Funnel Rocket is a serverless query engine optimized for complex user-centric queries at scale. It allows companies to look for users matching a nuanced set of conditions over time and glean fine-grained statistics so they can better understand a given group of users and take action. It was built at Dynamic Yield to answer on-demand interactive queries quickly and cost-efficiently and is now open-sourced for use by the wider developer community. With Funnel Rocket, you can build custom ad-hoc analytics capabilities over big datasets of user activity – at a large scale and with minimal overhead in terms of operations and resource usage. Below, learn about the challenges we set out to solve with Funnel Rocket, what the building process looked like, how the engine can be a solution for engineers looking to overcome similar challenges, and if cloud native actually means anything.
The status quo of SaaS
As a SaaS vendor, success is defined by how many teams install or use your product for their websites and apps. Whether you’re selling tools for analytics, recommendations, personalization, guided onboarding, surveys, media serving, etc., one crucial feature for your customers is the ability to freely explore the complexities of end-user behavior. This requires understanding the makeup and value of distinct user groups and evolving the site accordingly, potentially tailoring different experiences on their site to different user groups (Note: this is also achievable without tracking people across the web or storing their personal info).
Marketers onboarding software for this purpose are looking for a UI that enables ad-hoc exploration, giving them the ability to look for users who’ve met a nuanced set of conditions over time, perhaps those who have completed a specific sequence of actions in a particular order. Afterward, they want to be able to access detailed analytics showing not only the size of the group but also detailed metrics on how this audience group stands out from the rest of their user base.
For a glimpse of what this type of feature looks like in the Dynamic Yield platform, jump to minute 0:27 in the video.
The lingering problem with interactive queries
Such interactive queries are a challenge to pull off using existing tools — using SQL or NoSQL. Getting the query syntax right is cumbersome, but the main issue centers around performance at scale. Consider what it takes to implement this as a GROUP BY <USER ID> query.
No query engine I know will let you define in one step the whole sequence of actions that each group (i.e. each user) should match, in order. You will need to break this into multiple sub-queries, passing intermediate data between them, requiring multiple passes through the engine. In each pass, the engine will search (per each user) for a row whose timestamp is later than the row found in the last step, for that specific user. Such complex cases are typically expressed via composition, by design. Therefore, the query engine doesn’t always offer the simplest shortcut for every imaginable need.
There’s a performance cost to that complexity, however. On a small scale, it will work fine. However, take a dataset of 20 million users and that’s 20 million groups per query. Is it doable? Yes, assuming the user activity data is bucketed in the right way. Is it any fun, fast, or cheap? Not really. (Remember, I set some lofty goals at the end of my previous post: I want maximum processing speed just when I need it and to pay for exactly the needed compute resources).
Managed query engines (e.g. Amazon’s Athena, Google’s BigQuery) do offer on-demand pricing by the amount of data actually scanned. Your queries are executed behind the scenes by a small slice of their vast army of workers. This option was simply not available when we started, and thus we’ve built our homegrown solution to this problem with Elasticsearch + custom plugins. It was cool for a period of time, but overtime became an operational and maintenance burden.
Considering the managed tools available to us now, there were some things I was looking for:
- While these managed offerings are no doubt ultra-scalable and offer an impressively rich syntax, they’re simply not meant to serve interactive end-user queries. Query latency can widely fluctuate, with some occasional hiccups — which is fine for your internal BI needs or a pipeline but becomes a real problem if you want to ensure that your users get latencies in the low single-digits.
- I wanted an engine optimized for the type of complex user-centric queries I needed to support, with the ability to find bottlenecks, optimize them and add functionality natively.
- I wanted a tool that is suited for fast on-demand interactive requests but also for big batches of scheduled queries, optimizing each use case for cost while continuing to maintain control of the SLA.
The user-centric query feature is a core element of the Dynamic Yield platform, so it was important to us to have control of our capabilities, meaning that a “build” option was on the table. Of course, we didn’t set out to match the extensive feature-set of general-purpose query engines – we set out to solve one problem well. The result? Funnel Rocket.
The building blocks of Funnel Rocket
Here’s a glance at some of the core components of the solution we set out to build:
-
Pandas for data crunching at the single-task level, coupled with a few helpers — notably Apache Arrow, with perhaps Numba for critical performance pain points. The general idea was to have Pandas-based workers processing one Parquet file at a time out of a much bigger dataset.
- Serverless as a first-class option to support scaling and paying for only the actual compute time (currently supporting AWS Lambda). The solution also needed to be able to happily run on old-school machines or containers.
-
Redis for assisting with orchestration and making the code as environment-agnostic as possible, no less than for metadata storage.
- Cloud Storage (currently S3 or compatible stores) to hold datasets. Funnel Rocket does not have an “ingestion” phase — it reads datasets that were prepared and written to S3 by other components at their own pace.
To merge all of these together, we wrote an API server and task worker components in Python 3 that are pretty lightweight. The API server has endpoints for managing datasets and running queries, expressed in JSON format. It takes care to validate a given query, invoke the needed workers, track their progress, retry tasks as necessary, and aggregate the results back to the client. Additionally, the client has the option to ask for progress updates via HTTP streaming.
Here’s how it works, from the ground up:
Icon credit: Noun Project
Pandas-based workers
Pandas and its concept of DataFrames are widely used in the data engineering/science communities. Pandas itself is a library embedded within a single process, and we planned for each worker process to handle one file at a time out of the whole dataset, rather than worrying about any larger context. Additionally, Pandas is fairly performant and feature-rich out of the box. If it doesn’t offer some needed functionality, you can have it run Python code (though that’s pretty slow). On the other end, you can use Numba to generate optimized LLVM-compiled code from carefully written Python functions. In my experience, that does deliver the (performance) goods.
We use Apache Arrow to augment Pandas with excellent support for Parquet files and will probably use it much more going forward. Unlike Pandas, it supports nested data structures natively and has a growing feature set for computation and querying of its own. When a worker process is allocated a task, it transforms the JSON query syntax into a series of in-memory operations over a DataFrame and returns the aggregated results for the single file it was assigned to process. But for each worker to process independently, files in the dataset must already be partitioned (or “bucketed”) by user ID into a set of “n files,” so that each file holds data about a distinct set of users, as well as guarantee each user’s data is gathering in a single file. That shuffling stage is arguably the biggest performance pain in big data. However, this is only executed when a dataset is being created / updated, rather than for each query.
Running serverless — or server-full
Serverless is quite polarizing, I know; the whole paradigm, the proprietary implementations, the latency, the constraints… I did not come into this project sold on the idea, to be honest. For this use-case, however, AWS Lambda did prove itself to be reliable, fast, and (gasp) cost-efficient. Additionally, 99% of the code is agnostic to it and can be easily extended to support other implementations.
There are several reasons why it works well for Funnel Rocket:
Latency
Queries in Funnel Rocket are measured in seconds, not milliseconds. When Lambda function instances are cold, they will normally take ±3 – 5 seconds to get to the point where the handler function starts running (that time does include running all imports in the handler’s source file).
Luckily, Funnel Rocket serves a rather “power user” feature, so while there tends to be few customers concurrently using the feature, each user typically runs multiple iterations of a query over their own dataset, progressively tweaking conditions to zoom in on the user population. This means that for most queries, not only are Lambda functions warm, but performance can further benefit from local caching of data files — if you can make that work for serverless. More on that in a bit…
Cost
The per/second cost is indeed higher than all less-managed options. It’s a spectrum, really: from spot instances, through reserved VMs to on-demand ones, to Fargate, etc. — the more you need to worry about, the less you pay in compute (and more in operations). However, you only pay for actual processing time, from the exact millisecond your entry point starts execution until the millisecond it ends. That excludes the bootstrap time and the function instance staying warm for a while to be reused. We found this model to fit our query patterns well: maximum scale when there’s a query, zero resources, and cost at all other times. As a side effect, the cost is directly correlated to milliseconds of compute used — for each query request Funnel Rocket returns the exact cost back.
Ease of invocation
Funnel Rocket uses the asynchronous Lambda invocation API, making it easier to launch hundreds of jobs (or more) quickly, without needing to block waiting on all API calls to complete. Internally, AWS manages such calls with a queue, but we’ve found that it adds no meaningful latency in normal operation. This mode has an important extra benefit: in moments of momentary pressure, you mostly avoid rate limiting on concurrent function executions.
Serverless, however, is not the only way to fly. I wanted the ability to have a small army of workers that can run locally, on physical machines, VMs, or containers — while still making it easy to scale up or down, distribute work, handle failures, etc. That’s where Redis comes in.
Managing it all with Redis
Two traits of Redis, taken together, make it a natural choice for a wide range of functionality:
- It has a wealth of data structures over its basic key-value abstraction: lists, sets and ordered sets, atomic increments, HyperLogLogs, streams, and much more.
- Operations are atomic by nature, giving you a pretty strong guarantee that only one consumer will pop a specific element from a list or set (I realize there are always some nuanced caveats and edge cases, particularly around fail-overs).
Redis is used by Funnel Rocket to:
- Hold metadata on registered datasets and their schema.
- Report the current status and results of all tasks in an in-flight query. Instead of needing to communicate directly, workers write all updates to Redis as tasks progress, and these updates are polled by the API server. The API server knows whether a task succeeded, failed, or was “lost in space” through its status in Redis. It does not need to rely on AWS Lambda or any other runtime environment to get this information.
- Number consecutive attempts at running a failed/lost task using atomic counters. Each attempt of a task has its own unique attempt number. If multiple attempts for the same task end up succeeding, the API server will take care to only use the results of one.
- Add work to the queue in the non-serverless mode. In the “classic process” mode, each worker is simply blocked on a Redis list to access work to do. In this mode, when a query starts the API server enqueues task requests in a logical queue, from which tasks are populated by workers. Statuses and retries are handled as above through Redis as well but regardless of the task invocation mode. No load balancer is required; workers don’t know each other, nor does the API server know them directly.
- Apply best-effort optimization for data locality.
In the case of repeated queries over the same dataset, how do we make warm functions work on the same file they’ve downloaded before? There is no direct way to call a specific warm instance. Instead, the API server lets the workers choose for themselves which file to work on. When a query starts, it publishes the list of files as a Redis set. Then it invokes workers, asking them to each take one file from the set. Those with matching local files will attempt to grab that same file again. Others will grab a part at random. This mechanism is not guaranteed to always have maximum efficiency, but it does reduce orchestration needs to a minimum. The API server meanwhile, knows nothing.
Multiple API server instances can run concurrently, only sharing the list of registered datasets between them. A portion may run ad-hoc queries via serverless workers, while others run batch work through cheap spot-instances at scheduled times. Both deployment options push much of the complexity into battle-tested tools and have Redis as their single stateful component.
Let’s talk about benchmarking
As a basic benchmark, I’ve created a dataset that mimics user behavioral data (page views, events, clicks on campaigns, etc.) to reflect real data we collect in Dynamic Yield as closely as possible.
I’ve made two versions of the dataset: a smaller one with 100 million rows, the other with 500 million. Each row holds a user ID, a timestamp, the type of activity and related data fields (URLs, browser type, country, event type, product SKU, etc.).
- The datasets are split into 100 and 500 Parquet files stored in S3, respectively.
- Each file weighs a bit less than 40mb, and the small dataset weighs 3.8 GB in total, with the larger one clocking in at 19.6GB.
- Note: this can be very misleading. Parquet files are typically highly-compressed. The exact same files saved as CSV are about 10x the size, which would make the datasets weigh 38 & 196 GB.
- The Lambda function was configured to use 1768 MB RAM (per instance) so that each instance gets one full vCPU. It doesn’t need all that RAM necessarily, but will simply run more slowly with less as it’ll get a smaller CPU share (see here).
I’ve run a basic funnel query, the results of which you can see below: find the users who have performed a pageview, then added a product to cart, then made a purchase — and return the matching user count at each step.
For both datasets, I measured the time-to-run and compute cost in two scenarios: when all Lambda function instances are cold, and when all are warm. Each scenario ran 10 times.
Results
For 100 million rows:
Scenario
Min Time
Max Time
Avg Time
Min Cost
Max Cost
Avg Cost
All Cold2.3 secs6.5 secs5.43 secs0.15 cents0.4 cents0.35 centsAll Warm1.78 secs4.56 secs2.2 secs0.13 cents0.146 cents0.14 cents
For 500 million rows:
Scenario
Min Time
Max Time
Avg Time
Min Cost
Max Cost
Avg Cost
All Cold8.56 secs10.39 secs9.17 secs2.77 cents3.93 cents3.08 centsAll Warm3.42 secs5.51 secs4.34 secs0.7 cents1.7 cents0.98 cents
Things to note:
- Time is the total time-to-run of the query from the viewpoint of the API server. However, cost is the total billed amount for the Lambda runs, based directly on total milliseconds of actual handling time * units of allocated RAM * cost per unit/ms. I did not include per-request Lambda and S3 fees, which are, for this use case, much lower.
- Admittedly, these are preliminary results. More scenarios could be added here, with more iterations so that histograms of total duration, cost, and task completion over time can be added.
Final takeaways and musings
Building Funnel Rocket has left me with a lingering thought: Is “cloud native” a real thing? It was personally exciting for me to see all of this working in practice. Of course, there are missing features to wish for, surely a few bugs, and a bunch of optimizations and improvements yet to done (see the high-level roadmap). But all-in-all, I believe the software that resulted encapsulates what “cloud native” is about:
- Having a large functional task broken down into smaller pieces of work that can be run in multiple runtime environments so that availability and cost are optimized depending on what you need (interactive vs. batch, more compute cost vs more operations, etc.)
- Relying on existing, managed constructs to scale and orchestrate. “The Cloud” may be someone else’s computer, but it also offers many other services to build with: cloud storage, managed databases, serverless/containerized environments, and more.
- Having modest requirements for every component (compared to a Spark executor, in this case). It should start quickly, take (relatively) little RAM, and just do its work while also reporting metrics for observability.
- Lastly: it should also work fully on your computer (We use docker-compose, MinIO, and docker-lambda to support that).
This exercise has led me to finally believe in cloud native, and if you’re interested in learning more about Funnel Rocket and playing around with it yourself, access it on Github. And if you’re interested in joining our growing Product and R&D teams, here at Dynamic Yield, check out our open roles.
<!DOCTYPE html> <!--[if IE 6]><html id="ie6" lang="en-US"> <![endif]--> <!--[if IE 7]><html id="ie7" lang="en-US"> <![endif]--> <!--[if IE 8]><html id="ie8" lang="en-US"> <![endif]--> <!--[if !(IE 6) | !(IE 7) | !(IE 8) ]><!--><html lang="en-US"> <!--<![endif]--><head><meta name="msvalidate.01" content="1033997EC383621B874EB0392F261984"/><link rel="preconnect" href="//cdn.dynamicyield.com"><link rel="preconnect" href="//st.dynamicyield.com"><link rel="preconnect" href="//rcom.dynamicyield.com"><link rel="dns-prefetch" href="//cdn.dynamicyield.com"><link rel="dns-prefetch" href="//st.dynamicyield.com"><link rel="dns-prefetch" href="//rcom.dynamicyield.com"> <script data-cfasync="false">window.dataLayer = window.dataLayer || [];
window.gtag = function() { dataLayer.push(arguments); }
window.gtag('consent', 'default', {
ad_storage: 'denied',
ad_user_data: 'denied',
ad_personalization: 'denied',
analytics_storage: 'denied',
functionality_storage: 'denied',
personalization_storage: 'denied',
security_storage: 'granted', // Only this is granted by default
});</script> <script data-cfasync="false">(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
'//www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','GTM-MCQHF8');</script> <script data-cfasync="false" async>// Init the adobeDataLayer array
var adobeDataLayer = adobeDataLayer || [];
// Only when the API is ready
function adobeWaitForDY() {
if (typeof(window.DY) !== 'undefined' && typeof(window.DY.AdDetection) !== 'undefined' &&
typeof($dy) !== 'undefined' && typeof($dy(window)) !== 'undefined' && typeof($dy.renderSmartTagSlider) !== 'undefined') {
var zi_data = localStorage.getItem('ZoomInfoData'),
ada_page_info = {
"pageName": "From DY Labs: Introducing Funnel Rocket, an open-source serverless query engine",
"siteName": "Mastercard Dynamic Yield",
"pageTitle": "en_US :funnel-rocket-serverless-query-engine",
"pageURL": "https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/",
"country": "US",
"language": "en",
"dyID": DY.dyid,
"userStatus": "Guest"
};
// Detect the hierarchy and
// site sections based on the URL slugs
var current_pathname = window.location.pathname.replace(/^\/|\/$/g, '');
if(current_pathname) {
var current_sitesections = current_pathname.split('/'),
current_hierarchy = current_sitesections.map(function(segment) {
return segment.split(/[-_]/).map(function(word) {
return word.charAt(0).toUpperCase() + word.slice(1);
})
.join(' ');
});
// Site section 1
if(current_sitesections[0]) {
ada_page_info.siteSection1 = current_sitesections[0];
}
// Site section 2
if(current_sitesections[1]) {
ada_page_info.siteSection2 = current_sitesections[1];
}
// Hierarchy
ada_page_info.hierarchy = current_hierarchy.join('|');
}
// Push the ZI params as well
if(zi_data) {
zi_data = JSON.parse(zi_data);
if(zi_data.name) {
ada_page_info.ziCompany = zi_data.name;
}
if(zi_data.domain) {
ada_page_info.ziDomain = zi_data.domain;
}
if(zi_data.employeesrange) {
ada_page_info.ziEmployees = zi_data.employeesrange;
}
if(zi_data.industry) {
ada_page_info.ziIndustry = zi_data.industry;
}
}
// Decode HTML prior to dispatching the event
Object.keys(ada_page_info).forEach(function(key) {
ada_page_info[key] = $('<textarea/>').html(ada_page_info[key]).text();
});
adobeDataLayer.push({
"event": "pageLoad",
"pageInfo": ada_page_info
});
} else {
setTimeout(adobeWaitForDY, 150);
}
}
adobeWaitForDY();</script> <script type="text/javascript">function OptanonWrapper() {
console.log('OneTrust Loaded');
DY_API_update_consent_mode();
OneTrust.OnConsentChanged(DY_API_update_consent_mode);
}
function DY_API_update_consent_mode() {
var groups = OnetrustActiveGroups.split(',');
DY.userActiveConsent = {accepted: groups.includes('C0004')};
DYO.ActiveConsent.updateConsentAcceptedStatus(groups.includes('C0004'));
if(groups.includes('C0004')) {
console.log('OneTrust_Consent_Is_Granted');
DY.API('event',{ name: 'OneTrust_Consent_Is_Granted', properties: {}});
}
console.log('OneTrust - DY API is ' + (groups.includes('C0004') ? 'Enabled' : 'Disabled'));
}
window.addEventListener('OTConsentApplied', function() {
var groups = OnetrustActiveGroups.split(',');
if(groups.includes('C0004')) {
console.log('OneTrust_Initial_Consent_Granted');
DY.API('event',{ name: 'OneTrust_Initial_Consent_Granted', properties: {}});
}
});</script> <script type="text/javascript">/*WebFontConfig = {
google: { families: [ 'Shadows+Into+Light::latin', 'Handlee::latin', 'Roboto:400,100,300,500:latin' ] }
};
(function() {
var wf = document.createElement('script');
wf.src = ('https:' == document.location.protocol ? 'https' : 'http') +
'://ajax.googleapis.com/ajax/libs/webfont/1/webfont.js';
wf.type = 'text/javascript';
wf.async = 'true';
var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(wf, s);
})(); */
console.log(
'%cLet’s personalize experiences together.\n%cDynamic Yield is hiring smart engineers like you to help solve hard problems.\n%cSee our jobs page: https://www.dynamicyield.com/careers',
'font-size:2.2em;line-height:1.4em;margin-top:20px;color:rgb(93, 56, 185);',
'font-size:1.1em;line-height:1.6em;color:rgb(0, 0, 0);',
'font-size:1.1em;line-height:1.6em;margin-bottom:20px;color:rgb(0, 0, 0);'
);
function waitForElementToDisplay(selector, time, functionName) {
if (document.getElementById(selector) != null) {
functionName();
return;
} else {
setTimeout(function () {
waitForElementToDisplay(selector, time, functionName);
}, time);
}
}
function dcWaitForJQuery(time, functionName) {
if (window.jQuery || window.n2) {
functionName();
return;
} else {
setTimeout(function () {
dcWaitForJQuery(time, functionName);
}, time);
}
}</script> <meta name="msvalidate.01" content="451B75B70520F4C76D67F116C9EFFE5C"/><meta property="twitter:account_id" content="4503599629852947"/><meta name="p:domain_verify" content="f8112f57c09778e892850075a87dc1cf"/><meta name="google-site-verification" content="9FbmfdC4cM7JG8sgYh6w9HhFlxvEHewltexeUVzuSfE"/><meta charset="UTF-8"/><link rel="pingback" href="https://www.dynamicyield.com/xmlrpc.php"/> <!--[if lt IE 9]> <script src="https://www.dynamicyield.com/wp-content/themes/Divi/js/html5.js" type="text/javascript"></script> <![endif]--> <script type="text/javascript">document.documentElement.className = 'js';</script> <script type="text/javascript">var getLangCode = 'en';</script> <link rel="prefetch" href="/wp-content/themes/Divi/css/custom-style.css" as="style" crossorigin="anonymous"><meta name='robots' content='index, follow, max-image-preview:large, max-snippet:-1, max-video-preview:-1' /><link rel="alternate" hreflang="en" href="https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/" /><link rel="alternate" hreflang="x-default" href="https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/" /> <script data-cfasync="false" data-pagespeed-no-defer>var gtm4wp_datalayer_name = "dataLayer";
var dataLayer = dataLayer || [];</script> <title>Introducing Funnel Rocket, an open-source serverless query engine</title><link rel="preload" as="font" href="https://www.dynamicyield.com/wp-content/themes/Divi/fonts/CircularXXTT/CircularXXWeb-Bold.woff2" crossorigin/><link rel="preload" as="font" href="https://www.dynamicyield.com/wp-content/themes/Divi/fonts/CircularXXTT/CircularXXWeb-Medium.woff2" crossorigin/><link rel="preload" as="font" href="https://www.dynamicyield.com/wp-content/themes/Divi/fonts/CircularXXTT/CircularXXWeb-Book.woff2" crossorigin/><meta name="description" content="Learn about Funnel Rocket, a serverless query engine optimized for complex user-centric queries at scale, built by DY Labs." /><link rel="canonical" href="https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/" /><meta property="og:locale" content="en_US" /><meta property="og:type" content="article" /><meta property="og:title" content="Introducing Funnel Rocket, an open-source serverless query engine" /><meta property="og:description" content="Learn about Funnel Rocket, a serverless query engine optimized for complex user-centric queries at scale, built by DY Labs." /><meta property="og:url" content="https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/" /><meta property="og:site_name" content="Mastercard Dynamic Yield" /><meta property="article:publisher" content="https://www.facebook.com/DynamicYield" /><meta property="article:published_time" content="2021-07-02T05:50:39+00:00" /><meta property="og:image" content="https://www.dynamicyield.com/wp-content/uploads/2021/07/funnel-rocket-social-2.png" /><meta property="og:image:width" content="1200" /><meta property="og:image:height" content="628" /><meta property="og:image:type" content="image/png" /><meta name="author" content="Elad Rosenheim" /><meta name="twitter:card" content="summary_large_image" /><meta name="twitter:creator" content="@DynamicYield" /><meta name="twitter:site" content="@DynamicYield" /><meta name="twitter:label1" content="Written by" /><meta name="twitter:data1" content="Elad Rosenheim" /><meta name="twitter:label2" content="Est. reading time" /><meta name="twitter:data2" content="16 minutes" /> <script type="application/ld+json" class="yoast-schema-graph">{"@context":"https://schema.org","@graph":[{"@type":"Article","@id":"https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/#article","isPartOf":{"@id":"https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/"},"author":[{"@id":"https://www.dynamicyield.com/#/schema/person/7498e40640dac0dd78aca5c263b97ecf"}],"headline":"From DY Labs: Introducing Funnel Rocket, an open-source serverless query engine","datePublished":"2021-07-02T05:50:39+00:00","dateModified":"2021-07-02T05:50:39+00:00","mainEntityOfPage":{"@id":"https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/"},"wordCount":3166,"commentCount":0,"publisher":{"@id":"https://www.dynamicyield.com/#organization"},"image":{"@id":"https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/#primaryimage"},"thumbnailUrl":"https://www.dynamicyield.com/wp-content/uploads/2021/07/funnel-rocket-blog-2.png","keywords":["Engineering","Funnel Rocket","R&D"],"articleSection":["Product"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/#respond"]}]},{"@type":"WebPage","@id":"https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/","url":"https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/","name":"Introducing Funnel Rocket, an open-source serverless query engine","isPartOf":{"@id":"https://www.dynamicyield.com/#website"},"primaryImageOfPage":{"@id":"https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/#primaryimage"},"image":{"@id":"https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/#primaryimage"},"thumbnailUrl":"https://www.dynamicyield.com/wp-content/uploads/2021/07/funnel-rocket-blog-2.png","datePublished":"2021-07-02T05:50:39+00:00","dateModified":"2021-07-02T05:50:39+00:00","description":"Learn about Funnel Rocket, a serverless query engine optimized for complex user-centric queries at scale, built by DY Labs.","breadcrumb":{"@id":"https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/#primaryimage","url":"https://www.dynamicyield.com/wp-content/uploads/2021/07/funnel-rocket-blog-2.png","contentUrl":"https://www.dynamicyield.com/wp-content/uploads/2021/07/funnel-rocket-blog-2.png","width":795,"height":320},{"@type":"BreadcrumbList","@id":"https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://www.dynamicyield.com/"},{"@type":"ListItem","position":2,"name":"From DY Labs: Introducing Funnel Rocket, an open-source serverless query engine"}]},{"@type":"WebSite","@id":"https://www.dynamicyield.com/#website","url":"https://www.dynamicyield.com/","name":"Mastercard Dynamic Yield","description":"","publisher":{"@id":"https://www.dynamicyield.com/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://www.dynamicyield.com/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https://www.dynamicyield.com/#organization","name":"Mastercard Dynamic Yield","url":"https://www.dynamicyield.com/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https://www.dynamicyield.com/#/schema/logo/image/","url":"https://www.dynamicyield.com/wp-content/uploads/2019/10/DY-Hero_Logo-Black@x2.png","contentUrl":"https://www.dynamicyield.com/wp-content/uploads/2019/10/DY-Hero_Logo-Black@x2.png","width":458,"height":458,"caption":"Mastercard Dynamic Yield"},"image":{"@id":"https://www.dynamicyield.com/#/schema/logo/image/"},"sameAs":["https://www.facebook.com/DynamicYield","https://x.com/DynamicYield","https://www.instagram.com/dynamic.yield/","https://www.linkedin.com/company/dynamic-yield","http://www.pinterest.com/dynamicyield/","https://www.youtube.com/channel/UCPWEcR8lQRadVKYW8H4sndQ","https://en.wikipedia.org/wiki/Dynamic_Yield"]},{"@type":"Person","@id":"https://www.dynamicyield.com/#/schema/person/7498e40640dac0dd78aca5c263b97ecf","name":"Elad Rosenheim","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https://www.dynamicyield.com/#/schema/person/image/01b84b68e35295cd1c8b122801839de2","url":"https://secure.gravatar.com/avatar/219d268be8777d79c07971bba27a457a?s=96&d=mm&r=g","contentUrl":"https://secure.gravatar.com/avatar/219d268be8777d79c07971bba27a457a?s=96&d=mm&r=g","caption":"Elad Rosenheim"},"description":"Elad is the VP of Technology at Dynamic Yield, in charge of overseeing current and future architecture, mentoring developers, and designing for scale.","url":"https://www.dynamicyield.com/blog/author/elad/"}]}</script> <link rel='dns-prefetch' href='//pages.dynamicyield.com' /><link rel='dns-prefetch' href='//cdnjs.cloudflare.com' /><link rel="alternate" type="application/rss+xml" title="Mastercard Dynamic Yield » Feed" href="https://www.dynamicyield.com/feed/" /><link rel="alternate" type="application/rss+xml" title="Mastercard Dynamic Yield » Comments Feed" href="https://www.dynamicyield.com/comments/feed/" /><link rel="alternate" type="application/rss+xml" title="Mastercard Dynamic Yield » From DY Labs: Introducing Funnel Rocket, an open-source serverless query engine Comments Feed" href="https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/feed/" /><meta content="Divi v." name="generator"/><link rel='stylesheet' id='sbi_styles-css' href='https://www.dynamicyield.com/wp-content/plugins/instagram-feed/css/sbi-styles.min.css?ver=6.2.6' type='text/css' media='all' /><style id='wp-emoji-styles-inline-css' type='text/css'>img.wp-smiley, img.emoji {
display: inline !important;
border: none !important;
box-shadow: none !important;
height: 1em !important;
width: 1em !important;
margin: 0 0.07em !important;
vertical-align: -0.1em !important;
background: none !important;
padding: 0 !important;
}</style><link rel='stylesheet' id='wp-block-library-css' href='https://www.dynamicyield.com/wp-includes/css/dist/block-library/style.min.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><style id='classic-theme-styles-inline-css' type='text/css'>/*! This file is auto-generated */
.wp-block-button__link{color:#fff;background-color:#32373c;border-radius:9999px;box-shadow:none;text-decoration:none;padding:calc(.667em + 2px) calc(1.333em + 2px);font-size:1.125em}.wp-block-file__button{background:#32373c;color:#fff;text-decoration:none}</style><style id='global-styles-inline-css' type='text/css'>body{--wp--preset--color--black: #000000;--wp--preset--color--cyan-bluish-gray: #abb8c3;--wp--preset--color--white: #ffffff;--wp--preset--color--pale-pink: #f78da7;--wp--preset--color--vivid-red: #cf2e2e;--wp--preset--color--luminous-vivid-orange: #ff6900;--wp--preset--color--luminous-vivid-amber: #fcb900;--wp--preset--color--light-green-cyan: #7bdcb5;--wp--preset--color--vivid-green-cyan: #00d084;--wp--preset--color--pale-cyan-blue: #8ed1fc;--wp--preset--color--vivid-cyan-blue: #0693e3;--wp--preset--color--vivid-purple: #9b51e0;--wp--preset--gradient--vivid-cyan-blue-to-vivid-purple: linear-gradient(135deg,rgba(6,147,227,1) 0%,rgb(155,81,224) 100%);--wp--preset--gradient--light-green-cyan-to-vivid-green-cyan: linear-gradient(135deg,rgb(122,220,180) 0%,rgb(0,208,130) 100%);--wp--preset--gradient--luminous-vivid-amber-to-luminous-vivid-orange: linear-gradient(135deg,rgba(252,185,0,1) 0%,rgba(255,105,0,1) 100%);--wp--preset--gradient--luminous-vivid-orange-to-vivid-red: linear-gradient(135deg,rgba(255,105,0,1) 0%,rgb(207,46,46) 100%);--wp--preset--gradient--very-light-gray-to-cyan-bluish-gray: linear-gradient(135deg,rgb(238,238,238) 0%,rgb(169,184,195) 100%);--wp--preset--gradient--cool-to-warm-spectrum: linear-gradient(135deg,rgb(74,234,220) 0%,rgb(151,120,209) 20%,rgb(207,42,186) 40%,rgb(238,44,130) 60%,rgb(251,105,98) 80%,rgb(254,248,76) 100%);--wp--preset--gradient--blush-light-purple: linear-gradient(135deg,rgb(255,206,236) 0%,rgb(152,150,240) 100%);--wp--preset--gradient--blush-bordeaux: linear-gradient(135deg,rgb(254,205,165) 0%,rgb(254,45,45) 50%,rgb(107,0,62) 100%);--wp--preset--gradient--luminous-dusk: linear-gradient(135deg,rgb(255,203,112) 0%,rgb(199,81,192) 50%,rgb(65,88,208) 100%);--wp--preset--gradient--pale-ocean: linear-gradient(135deg,rgb(255,245,203) 0%,rgb(182,227,212) 50%,rgb(51,167,181) 100%);--wp--preset--gradient--electric-grass: linear-gradient(135deg,rgb(202,248,128) 0%,rgb(113,206,126) 100%);--wp--preset--gradient--midnight: linear-gradient(135deg,rgb(2,3,129) 0%,rgb(40,116,252) 100%);--wp--preset--font-size--small: 13px;--wp--preset--font-size--medium: 20px;--wp--preset--font-size--large: 36px;--wp--preset--font-size--x-large: 42px;--wp--preset--spacing--20: 0.44rem;--wp--preset--spacing--30: 0.67rem;--wp--preset--spacing--40: 1rem;--wp--preset--spacing--50: 1.5rem;--wp--preset--spacing--60: 2.25rem;--wp--preset--spacing--70: 3.38rem;--wp--preset--spacing--80: 5.06rem;--wp--preset--shadow--natural: 6px 6px 9px rgba(0, 0, 0, 0.2);--wp--preset--shadow--deep: 12px 12px 50px rgba(0, 0, 0, 0.4);--wp--preset--shadow--sharp: 6px 6px 0px rgba(0, 0, 0, 0.2);--wp--preset--shadow--outlined: 6px 6px 0px -3px rgba(255, 255, 255, 1), 6px 6px rgba(0, 0, 0, 1);--wp--preset--shadow--crisp: 6px 6px 0px rgba(0, 0, 0, 1);}:where(.is-layout-flex){gap: 0.5em;}:where(.is-layout-grid){gap: 0.5em;}body .is-layout-flow > .alignleft{float: left;margin-inline-start: 0;margin-inline-end: 2em;}body .is-layout-flow > .alignright{float: right;margin-inline-start: 2em;margin-inline-end: 0;}body .is-layout-flow > .aligncenter{margin-left: auto !important;margin-right: auto !important;}body .is-layout-constrained > .alignleft{float: left;margin-inline-start: 0;margin-inline-end: 2em;}body .is-layout-constrained > .alignright{float: right;margin-inline-start: 2em;margin-inline-end: 0;}body .is-layout-constrained > .aligncenter{margin-left: auto !important;margin-right: auto !important;}body .is-layout-constrained > :where(:not(.alignleft):not(.alignright):not(.alignfull)){max-width: var(--wp--style--global--content-size);margin-left: auto !important;margin-right: auto !important;}body .is-layout-constrained > .alignwide{max-width: var(--wp--style--global--wide-size);}body .is-layout-flex{display: flex;}body .is-layout-flex{flex-wrap: wrap;align-items: center;}body .is-layout-flex > *{margin: 0;}body .is-layout-grid{display: grid;}body .is-layout-grid > *{margin: 0;}:where(.wp-block-columns.is-layout-flex){gap: 2em;}:where(.wp-block-columns.is-layout-grid){gap: 2em;}:where(.wp-block-post-template.is-layout-flex){gap: 1.25em;}:where(.wp-block-post-template.is-layout-grid){gap: 1.25em;}.has-black-color{color: var(--wp--preset--color--black) !important;}.has-cyan-bluish-gray-color{color: var(--wp--preset--color--cyan-bluish-gray) !important;}.has-white-color{color: var(--wp--preset--color--white) !important;}.has-pale-pink-color{color: var(--wp--preset--color--pale-pink) !important;}.has-vivid-red-color{color: var(--wp--preset--color--vivid-red) !important;}.has-luminous-vivid-orange-color{color: var(--wp--preset--color--luminous-vivid-orange) !important;}.has-luminous-vivid-amber-color{color: var(--wp--preset--color--luminous-vivid-amber) !important;}.has-light-green-cyan-color{color: var(--wp--preset--color--light-green-cyan) !important;}.has-vivid-green-cyan-color{color: var(--wp--preset--color--vivid-green-cyan) !important;}.has-pale-cyan-blue-color{color: var(--wp--preset--color--pale-cyan-blue) !important;}.has-vivid-cyan-blue-color{color: var(--wp--preset--color--vivid-cyan-blue) !important;}.has-vivid-purple-color{color: var(--wp--preset--color--vivid-purple) !important;}.has-black-background-color{background-color: var(--wp--preset--color--black) !important;}.has-cyan-bluish-gray-background-color{background-color: var(--wp--preset--color--cyan-bluish-gray) !important;}.has-white-background-color{background-color: var(--wp--preset--color--white) !important;}.has-pale-pink-background-color{background-color: var(--wp--preset--color--pale-pink) !important;}.has-vivid-red-background-color{background-color: var(--wp--preset--color--vivid-red) !important;}.has-luminous-vivid-orange-background-color{background-color: var(--wp--preset--color--luminous-vivid-orange) !important;}.has-luminous-vivid-amber-background-color{background-color: var(--wp--preset--color--luminous-vivid-amber) !important;}.has-light-green-cyan-background-color{background-color: var(--wp--preset--color--light-green-cyan) !important;}.has-vivid-green-cyan-background-color{background-color: var(--wp--preset--color--vivid-green-cyan) !important;}.has-pale-cyan-blue-background-color{background-color: var(--wp--preset--color--pale-cyan-blue) !important;}.has-vivid-cyan-blue-background-color{background-color: var(--wp--preset--color--vivid-cyan-blue) !important;}.has-vivid-purple-background-color{background-color: var(--wp--preset--color--vivid-purple) !important;}.has-black-border-color{border-color: var(--wp--preset--color--black) !important;}.has-cyan-bluish-gray-border-color{border-color: var(--wp--preset--color--cyan-bluish-gray) !important;}.has-white-border-color{border-color: var(--wp--preset--color--white) !important;}.has-pale-pink-border-color{border-color: var(--wp--preset--color--pale-pink) !important;}.has-vivid-red-border-color{border-color: var(--wp--preset--color--vivid-red) !important;}.has-luminous-vivid-orange-border-color{border-color: var(--wp--preset--color--luminous-vivid-orange) !important;}.has-luminous-vivid-amber-border-color{border-color: var(--wp--preset--color--luminous-vivid-amber) !important;}.has-light-green-cyan-border-color{border-color: var(--wp--preset--color--light-green-cyan) !important;}.has-vivid-green-cyan-border-color{border-color: var(--wp--preset--color--vivid-green-cyan) !important;}.has-pale-cyan-blue-border-color{border-color: var(--wp--preset--color--pale-cyan-blue) !important;}.has-vivid-cyan-blue-border-color{border-color: var(--wp--preset--color--vivid-cyan-blue) !important;}.has-vivid-purple-border-color{border-color: var(--wp--preset--color--vivid-purple) !important;}.has-vivid-cyan-blue-to-vivid-purple-gradient-background{background: var(--wp--preset--gradient--vivid-cyan-blue-to-vivid-purple) !important;}.has-light-green-cyan-to-vivid-green-cyan-gradient-background{background: var(--wp--preset--gradient--light-green-cyan-to-vivid-green-cyan) !important;}.has-luminous-vivid-amber-to-luminous-vivid-orange-gradient-background{background: var(--wp--preset--gradient--luminous-vivid-amber-to-luminous-vivid-orange) !important;}.has-luminous-vivid-orange-to-vivid-red-gradient-background{background: var(--wp--preset--gradient--luminous-vivid-orange-to-vivid-red) !important;}.has-very-light-gray-to-cyan-bluish-gray-gradient-background{background: var(--wp--preset--gradient--very-light-gray-to-cyan-bluish-gray) !important;}.has-cool-to-warm-spectrum-gradient-background{background: var(--wp--preset--gradient--cool-to-warm-spectrum) !important;}.has-blush-light-purple-gradient-background{background: var(--wp--preset--gradient--blush-light-purple) !important;}.has-blush-bordeaux-gradient-background{background: var(--wp--preset--gradient--blush-bordeaux) !important;}.has-luminous-dusk-gradient-background{background: var(--wp--preset--gradient--luminous-dusk) !important;}.has-pale-ocean-gradient-background{background: var(--wp--preset--gradient--pale-ocean) !important;}.has-electric-grass-gradient-background{background: var(--wp--preset--gradient--electric-grass) !important;}.has-midnight-gradient-background{background: var(--wp--preset--gradient--midnight) !important;}.has-small-font-size{font-size: var(--wp--preset--font-size--small) !important;}.has-medium-font-size{font-size: var(--wp--preset--font-size--medium) !important;}.has-large-font-size{font-size: var(--wp--preset--font-size--large) !important;}.has-x-large-font-size{font-size: var(--wp--preset--font-size--x-large) !important;}
.wp-block-navigation a:where(:not(.wp-element-button)){color: inherit;}
:where(.wp-block-post-template.is-layout-flex){gap: 1.25em;}:where(.wp-block-post-template.is-layout-grid){gap: 1.25em;}
:where(.wp-block-columns.is-layout-flex){gap: 2em;}:where(.wp-block-columns.is-layout-grid){gap: 2em;}
.wp-block-pullquote{font-size: 1.5em;line-height: 1.6;}</style><link rel='stylesheet' id='rfw-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_7c036e0987ef80b703d7eb1abf922895.css?ver=2026020705' type='text/css' media='all' /><link rel='stylesheet' id='divi-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_ab3666a686d3dac03fd3032e5757637e.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='ctt-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_f1d1b2f761cf3768f427cdc5c4c2544b.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='inline-tweet-sharer-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_daa32ca85e114a0c6e5d1432c89237f3.css?ver=2.6.4' type='text/css' media='all' /><link rel='stylesheet' id='et_lb_modules-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_c0102af27938567cb82cc02e4da3a673.css?ver=2.4' type='text/css' media='all' /><link rel='stylesheet' id='et-shortcodes-css-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_40cb404848bc2e556105c8aea77f29aa.css?ver=3.0' type='text/css' media='all' /><link rel='stylesheet' id='et-shortcodes-responsive-css-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_3a30a77569bc4544b9465acbee8e5f97.css?ver=3.0' type='text/css' media='all' /><link rel='stylesheet' id='wp-featherlight-css' href='https://www.dynamicyield.com/wp-content/plugins/wp-featherlight/css/wp-featherlight.min.css?ver=1.3.4' type='text/css' media='all' /><link rel='stylesheet' id='baskerville-font-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_8c23179f030d02e0065d2054eab34279.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='markformc-font-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_0eb51906eca648ca4559d0a03b898d83.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='mackinac-font-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_9c8fec2609ebc9764a6e16471d9451ae.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='button-pulse-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_4cabc8dd29eaa9af7bf2df62f1c40af9.css?ver=935c7f8ce03960b02800bc8ae3231df8' type='text/css' media='all' /><link rel='stylesheet' id='fontawesome-css' href='https://www.dynamicyield.com/wp-content/themes/Divi/custom/font-awesome/css/font-awesome.min.css' type='text/css' media='all' /><link rel='stylesheet' id='bootstrap-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_d375e60add46d8de26a78728b517660f.css' type='text/css' media='all' /><link rel='stylesheet' id='slick-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_13b1b6672b8cfb0d9ae7f899f1c42875.css' type='text/css' media='all' /><link rel='stylesheet' id='slick-theme-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_f9faba678c4d6dcfdde69e5b11b37a2e.css' type='text/css' media='all' /><link rel='stylesheet' id='custom-theme-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_57f41281576414370c141cd00d2a80a9.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='custom-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_417039b6d54561fe1e8c94ee67cd902c.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='header-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_d75ad7543415df427913a5fe91606386.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='footer-style-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_25400e1cfd7099cc557083769c928231.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='video_tabs_module-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_f1109a8bc8c46ec9f2205cc8603c132f.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='select2-css' href='https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.6-rc.0/css/select2.min.css' type='text/css' media='all' /><link rel='stylesheet' id='post-checklist-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_ede7f9aa4d78ac997ea5e48f5d511400.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='post-event-cards-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_d02c1d48c2c1fde18d405f087a0fe833.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='mkto-reset-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_d117c8ecde800f5dccd137c16bab71af.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='learning-center-general-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_73dd3b78cf6e21c28be39db6740774b1.css?ver=26.27' type='text/css' media='all' /><link rel='stylesheet' id='article-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_e1d850b2d8d6fd6331ad74bd35ddde51.css?ver=1772305539' type='text/css' media='all' /><link rel='stylesheet' id='customers-shortcode-css' href='https://www.dynamicyield.com/wp-content/cache/autoptimize/css/autoptimize_single_c6bf61e9c8beb560d64d119f58757b8b.css?ver=26.27' type='text/css' media='all' /> <script type="text/javascript" src="https://www.dynamicyield.com/wp-includes/js/jquery/jquery.min.js?ver=3.7.1" id="jquery-core-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-includes/js/jquery/jquery-migrate.min.js?ver=3.4.1" id="jquery-migrate-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_dbdab0f35217f6041eb7e2028dc94ba4.js?ver=2.6.4" id="inline-tweet-sharer-js-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/themes/Divi/js/blazy.min.js" id="blazy-js"></script> <script type="text/javascript" src="//pages.dynamicyield.com/js/forms2/js/forms2.min.js?ver=21.85" id="mkto-form-script-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_db6be3b57dba139e789fec1a78186c8e.js?ver=21.85" id="learning-center-js-js"></script> <link rel="https://api.w.org/" href="https://www.dynamicyield.com/wp-json/" /><link rel="alternate" type="application/json" href="https://www.dynamicyield.com/wp-json/wp/v2/posts/160220" /><link rel="EditURI" type="application/rsd+xml" title="RSD" href="https://www.dynamicyield.com/xmlrpc.php?rsd" /><link rel='shortlink' href='https://www.dynamicyield.com/?p=160220' /><link rel="alternate" type="application/json+oembed" href="https://www.dynamicyield.com/wp-json/oembed/1.0/embed?url=https%3A%2F%2Fwww.dynamicyield.com%2Fblog%2Ffunnel-rocket-serverless-query-engine%2F" /><link rel="alternate" type="text/xml+oembed" href="https://www.dynamicyield.com/wp-json/oembed/1.0/embed?url=https%3A%2F%2Fwww.dynamicyield.com%2Fblog%2Ffunnel-rocket-serverless-query-engine%2F&format=xml" /><meta name="generator" content="WPML ver:4.6.8 stt:1,4,3,28,2;" /> <script type="text/javascript">(function(url){
if(/(?:Chrome\/26\.0\.1410\.63 Safari\/537\.31|WordfenceTestMonBot)/.test(navigator.userAgent)){ return; }
var addEvent = function(evt, handler) {
if (window.addEventListener) {
document.addEventListener(evt, handler, false);
} else if (window.attachEvent) {
document.attachEvent('on' + evt, handler);
}
};
var removeEvent = function(evt, handler) {
if (window.removeEventListener) {
document.removeEventListener(evt, handler, false);
} else if (window.detachEvent) {
document.detachEvent('on' + evt, handler);
}
};
var evts = 'contextmenu dblclick drag dragend dragenter dragleave dragover dragstart drop keydown keypress keyup mousedown mousemove mouseout mouseover mouseup mousewheel scroll'.split(' ');
var logHuman = function() {
if (window.wfLogHumanRan) { return; }
window.wfLogHumanRan = true;
var wfscr = document.createElement('script');
wfscr.type = 'text/javascript';
wfscr.async = true;
wfscr.src = url + '&r=' + Math.random();
(document.getElementsByTagName('head')[0]||document.getElementsByTagName('body')[0]).appendChild(wfscr);
for (var i = 0; i < evts.length; i++) {
removeEvent(evts[i], logHuman);
}
};
for (var i = 0; i < evts.length; i++) {
addEvent(evts[i], logHuman);
}
})('//www.dynamicyield.com/?wordfence_lh=1&hid=65887E5D4B76DD927FFD2D0B34B2D125');</script>
<script data-cfasync="false" data-pagespeed-no-defer>var dataLayer_content = {"visitorLoginState":"logged-out","visitorType":"visitor-logged-out","pagePostType":"post","pagePostType2":"single-post","pageCategory":["product"],"pagePostAuthor":"Elad Rosenheim","pagePostDate":"July 2, 2021","pagePostDateYear":"2021","pagePostDateMonth":"07","pagePostDateDay":"02","pagePostDateDayName":"Friday","pagePostDateHour":"01","pagePostDateMinute":"50","pagePostDateIso":"2021-07-02T01:50:39-04:00","pagePostDateUnix":1625190639};
dataLayer.push( dataLayer_content );</script> <script type="text/javascript">console.warn && console.warn("[GTM4WP] Google Tag Manager container code placement set to OFF !!!");
console.warn && console.warn("[GTM4WP] Data layer codes are active but GTM container must be loaded using custom coding !!!");</script> <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=3.0, user-scalable=1" /><link rel="icon" type="image/png" href="/favicon-16x16.png" sizes="16x16"><link rel="icon" type="image/png" href="/favicon-32x32.png" sizes="32x32"><link rel="icon" type="image/png" href="/favicon-96x96.png" sizes="96x96"><link rel="icon" type="image/png" href="/favicon-180x180.png" sizes="180x180"><link rel="icon" type="image/png" href="/favicon-192x192.png" sizes="192x192"><link rel="icon" type="image/png" href="/favicon-194x194.png" sizes="194x194"><link rel="shortcut icon" href="/favicon.ico"> <script type="text/javascript" data-noptimize>function dy_bind_adobe_mkto_events(form, form_id, form_name) {
// Init the adobeDataLayer object
window.adobeDataLayer = window.adobeDataLayer || [];
var focus_reported = false;
var form_completed = false;
var form_started_reported = false;
// Monitor input interactions - first focus (form start)
form.getFormElem().find('input, select').on('focus', function() {
// Only if it hasn't been reported yet
if(! focus_reported) {
focus_reported = true;
form_started_reported = true;
adobeDataLayer.push({
"event": "form_start",
"trackAction": {
"formName": form_name,
"formID": form_id,
"zoomInfo": localStorage.ZoomInfoData !== undefined,
}
});
}
});
// Detect dropoff if the form hasn't been completed
window.addEventListener("beforeunload", function(event) {
if (form_completed) return;
var filledLabels = [];
var count = 0;
form.getFormElem().find('input, select').each(function() {
var value = this.value && this.value.trim();
if (value) {
var label = form.getFormElem().find("label[for='" + this.id + "']").text().trim();
if (label) {
filledLabels.push(label);
count++;
}
}
});
if (count > 0) {
adobeDataLayer.push({
"event": "form_dropoff",
"formInfo": {
"formName": form_name,
"formID": form_id,
"zoomInfo": localStorage.ZoomInfoData !== undefined,
"completedFields": {
"labels": filledLabels.join("|"),
"count": count.toString()
}
}
});
}
});
// Hook into successful submission
form.onSuccess(function(values, followUpUrl) {
if(form_started_reported) {
// Report form completed
form_completed = true;
// Send an Adobe Analytics DataLayer event
adobeDataLayer.push({
"event": "form_complete",
"formInfo": {
"formName": form_name,
"formID": form_id,
"zoomInfo": localStorage.ZoomInfoData !== undefined,
"successAction": {
"action": "redirect",
"redirectUrl": followUpUrl
}
}
});
}
});
}
function dy_bind_adobe_cta_event(cta, cta_type, cta_section) {
var cta_name = cta.text().trim(),
cta_url,
page_url = window.location.href;
// const domainName = window.location.origin;
if (cta.attr('href') !== undefined) {
// cta_url = domainName + cta.attr('href');
cta_url = cta.attr('href');
} else {
cta_url = page_url;
}
adobeDataLayer.push({
"event": "ctaClick",
"clickInfo": {
"ctaName": cta_name,
"ctaType": cta_type,
"destinationURL": cta_url,
"ctaLocation": page_url,
"ctaSection": cta_section
}
});
}</script> <link rel="shortcut icon" href="https://www.dynamicyield.com/wp-content/uploads/2018/11/favicon-194x194.png" /><link rel="amphtml" href="https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/amp/"></head><body class="post-template-default single single-post postid-160220 single-format-standard wp-featherlight-captions et_fixed_nav et_right_sidebar gecko et_includes_sidebar elementor-default default_header"><div id="page-container"><header id="main-header" class="et_nav_text_color_dark"
style="background: #fff;"><div class="container-fluid clearfix"><div class="dy-header-logo"> <a href="https://www.dynamicyield.com/"
class="main-logo-container"><div id="logo_symbol"><svg xmlns="http://www.w3.org/2000/svg" xml:space="preserve" id="Layer_1" x="0" y="0" version="1.1" viewBox="0 0 266.8 164.9"><g id="XMLID_23_"><path fill="#ff5f00" d="M97.3 17.5h72.2v129.7H97.3z"/><path id="XMLID_35_" fill="#eb001b" d="M101.9 82.4c0-26.3 12.3-49.7 31.5-64.8-14-11-31.7-17.6-51-17.6C36.9-.1 0 36.8 0 82.4c0 45.5 36.9 82.5 82.5 82.5 19.2 0 36.9-6.6 51-17.6-19.3-15.2-31.6-38.6-31.6-64.9z"/><path d="M266.8 82.4c0 45.5-36.9 82.5-82.5 82.5-19.2 0-36.9-6.6-51-17.6 19.2-15.1 31.5-38.5 31.5-64.8s-12.3-49.7-31.5-64.8c14-11 31.7-17.6 51-17.6 45.6-.2 82.5 36.7 82.5 82.3zM259 133.5v-2.7h1.1v-.5h-2.7v.5h1.1v2.7h.5zm5.3 0v-3.2h-.8l-1 2.2-1-2.2h-.8v3.2h.6v-2.4l.9 2.1h.6l.9-2.1v2.4h.6z" fill="#f79e1b"/></g></svg></div> </a></div><div id="et-top-navigation" class="cd-morph-dropdown"><nav id="top-menu-nav" class="main-nav"><ul id="top-menu" class="nav"></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li><li class="has-dropdown product menu-item menu-item-type-custom menu-item-object-custom menu-item-has-children" data-content="product" title=""> <a href="">Product</a></li><li class="has-dropdown solutions menu-item menu-item-type-custom menu-item-object-custom menu-item-has-children" data-content="solutions" title=""> <a href="">Solutions</a></li><li class="has-dropdown why_dy menu-item menu-item-type-custom menu-item-object-custom menu-item-has-children" data-content="why-dynamic-yield" title="Why DY"> <a href="">Why Dynamic Yield</a></li><li class="has-dropdown company menu-has-cta menu-item menu-item-type-custom menu-item-object-custom menu-item-has-children" data-content="company" title=""> <a href="">Company</a></li><li class="has-dropdown learning_center menu-has-cta top-link-use-cases menu-item menu-item-type-custom menu-item-object-custom menu-item-has-children" data-content="learning-center" title=""> <a href="">Learning Center</a></li><li class="contact top-link walker-last mobile-only menu-item menu-item-type-post_type menu-item-object-page" data-content="connect-with-us" title=""> <a href="https://www.dynamicyield.com/contact/"><span class="button-arrow">Connect with us</span></a></li></ul></nav><div class="morph-dropdown-wrapper"><div class="dropdown-list"><div class="dropdown-tooltip"></div><ul><li id="product" class="dropdown links"><a href="" class="label">Product</a><div class="content"><ul><li class="intro"><div class="exp_os-wrapper"><h2>Experience OS</h2><ul class="links-list"><li class=" menu-item menu-item-type-custom menu-item-object-custom"><a href="https://www.dynamicyield.com/experience-os/"><span class="menu-banner-img-wrap"><img class="menu-banner-img-desktop" src="/wp-content/uploads/2024/10/expos-nav-updated.png" width="218" height="131" /><img class="menu-banner-img-mobile" src="/wp-content/uploads/2024/10/expos-nav-updated-mobile.png" width="114" height="98" /></span><span class="menu-cta-text-wrap"><span class="menu-cta-subtitle">Get an overview of Dynamic Yield’s product and see why we’re known as the Experience OS.</span> <span class="menu-cta-button">Learn more →</span></span></a></li></ul></div></li><li class="left"><div class="key_caps-wrapper"><h2>Core Capabilities</h2><ul class="links-list"><li class="key-caps-segment menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/segmentation/"><svg version="1.1" id="audience" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 30 30" style="enable-background:new 0 0 30 30;" xml:space="preserve"><path id="audience-green" fill="#4FB294" d="M9.291,30H30V9.291C18.562737,9.291,9.291,18.562737,9.291,30z"/><path id="audience-black" d="M12.527,15.947C9.240353,12.385597,4.806085,10.093244,0,9.471V30h7.639 C7.632218,24.897682,9.35532,19.943764,12.527,15.947z"/><circle id="audience-circle" fill="#4FB294" cx="15" cy="4.967" r="4.967"/></svg>Segmentation<span>Identify, build, and analyze audiences</span></a></li><li class="key-caps-omnichannel menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/targeting/"><svg version="1.1" id="personalization" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 20.987 30" style="enable-background:new 0 0 20.987 30;" xml:space="preserve"><path id="personalization-black-2" d="M20.987,19.507c0,5.795401-4.698099,10.4935-10.4935,10.4935S0,25.302401,0,19.507H20.987z"/><path id="personalization-pink" fill="#EE3968" d="M18.143,10.245c0,4.224978-3.425022,7.649999-7.65,7.649999S2.842999,14.469978,2.842999,10.245 H18.143z"/><circle id="personalization-black-1" cx="10.494" cy="4.307" r="4.307"/></svg>Targeting<span>Serve each customer with unique offers</span></a></li><li class="key-caps-recs menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/recommendations/"><svg version="1.1" id="recs" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 30 30" style="enable-background:new 0 0 30 30;" xml:space="preserve"><path id="recs_4" d="M15.953,30H30V15.953H15.953V30z"/> <path id="recs_3" d="M0,15.953V30h14.047V15.953H0z"/><path id="recs_2" fill="#FFA354" d="M30,14.047V0H15.953v14.047H30z"/><path id="recs_1" d="M14.047,0H0v14.047h14.047V0z"/></svg>Recommendations<span>Algorithmically predict customers’ interests</span></a></li><li class="key-caps-email menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/journey-orchestration/"><svg version="1.1" id="trigger" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 26.017 30" style="enable-background:new 0 0 26.017 30;" xml:space="preserve"><path id="trigger-black" d="M14.017,12h12l-12,18V12z"/><path id="trigger-purple" fill="#5D39B9" d="M12,18H0L12,0V18z"/></svg>Journey Orchestration<span>Reach customers at critical moments</span></a></li><li class="key-caps-abtesting menu-item menu-item-type-custom menu-item-object-custom"><a href="/optimization/"><svg version="1.1" id="testing" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" viewBox="0 0 28.713001 30" style="enable-background:new 0 0 28.713001 30;" xml:space="preserve"><path id="testing-blue" fill="#6BDADA" d="M15.831,0c7.114532,0,12.882001,5.767468,12.882001,12.882S22.945532,25.764,15.831,25.764l0,0V0z"/><path id="testing-black" d="M12.882,30C5.767468,30,0,24.232533,0,17.118S5.767468,4.236,12.882,4.236V30z"/></svg>Optimization<span>Experiment on any digital property</span></a></li><li class="key-caps-search menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/search/"><svg width="16" height="16" viewBox="0 0 16 16" fill="none" xmlns="http://www.w3.org/2000/svg"> <g clip-path="url(#clip0_4082_5186)" id="cap-search-svg-bg"> <path d="M11.6757 9.81054C8.73958 9.81054 6.35089 7.42177 6.35089 4.48562C6.35089 2.62899 7.30664 0.992133 8.75134 0.0388074C8.50309 0.0155936 8.25193 0.00238037 7.99762 0.00238037C3.58066 0.00238037 0 3.58304 0 8C0 12.417 3.58066 15.9976 7.99762 15.9976C12.4146 15.9976 15.9952 12.417 15.9952 8C15.9952 7.86848 15.9916 7.73782 15.9853 7.60785C15.0166 8.94122 13.4462 9.81054 11.6757 9.81054Z" fill="black"/> <path id="cap-search-svg-circle" d="M11.6757 0.161346C11.2226 0.161346 10.7857 0.231773 10.3749 0.361649C8.62429 0.915114 7.35156 2.55436 7.35156 4.48561C7.35156 6.87003 9.29141 8.80988 11.6757 8.80988C13.5373 8.80988 15.1277 7.62753 15.7358 5.97437C15.9066 5.51 16 5.00851 16 4.48561C16 2.10119 14.0602 0.161346 11.6757 0.161346Z" fill="#FFC61E"/> </g> <defs> <clipPath id="clip0_4082_5186"> <rect width="16" height="16" fill="white"/> </clipPath> </defs> </svg>Search<span>Help customers find what they want</span></a></li></ul></div></li><li class="right"><div class="scale-wrapper"><h2>Made for Scale</h2><ul class="links-list"><li class="key_caps-element menu-item menu-item-type-custom menu-item-object-custom"><a href="/element/">Element<span>A groundbreaking new dimension of hyper-personalization</span></a></li><li class="key_caps-adaptml menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/ai/">AI & Automation<span>Increase automation, lower operational costs, and drive more revenue with a holistic AI personalization system</span></a></li><li class="key_caps-apis menu-item menu-item-type-custom menu-item-object-custom"><a href="https://www.dynamicyield.com/experience-apis/">Experience APIs<span>Give developers the tools they need to build the best customer experiences</span></a></li><li class="key_caps-enterprise menu-item menu-item-type-custom menu-item-object-custom"><a href="https://www.dynamicyield.com/enterprise-grade-personalization/">Enterprise solutions<span>Scale with security, privacy, and compliance, made for the enterprise</span></a></li></ul></div></li></ul><a href="/shopping-muse/" target="" class="banner"><div class="menu-banner"><span class="menu-cta-text"><strong>Meet Shopping Muse: </strong> Conversational commerce meets cutting-edge personalization.</span> <span class="menu-cta-button">Learn more →</span></div></a></div></li><li id="solutions" class="dropdown links"><a href="" class="label">Solutions</a><div class="content"><ul><li class="left"><div class="industries-wrapper"><h2>Industries</h2><ul class="links-list"><li class="industry-ecomm menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/ecommerce/">eCommerce</a></li><li class="industry-finserv menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/financial-services/">Financial Services</a></li><li class="industry-qsr menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/restaurants/">Restaurants</a></li><li class="industry-grocery menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/grocery/">Grocery</a></li><li class="industry-b2b menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/b2b-ecommerce/">B2B eCommerce</a></li><li class="industry-travel menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/travel/">Travel</a></li><li class="industry-gaming menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/gaming/">iGaming</a></li><li class="industry-media menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/media/">Media</a></li></ul></div></li><li class="right"><div class="channels-wrapper"><h2>Channels</h2><ul class="links-list"><li class="channels-web menu-item menu-item-type-custom menu-item-object-custom"><a href="/web/">Web<span>Create customer-centric web experiences</span></a></li><li class="channels-email menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/channels/email/">Email<span>Personalize emails with a no-code builder</span></a></li><li class="channels-mobile-apps menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/mobile/">Apps<span>Optimize mobile app experiences</span></a></li><li class="channels-ads menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/advertising/">Ads<span>Improve customer acquisition results</span></a></li></ul></div></li></ul></div></li><li id="why-dynamic-yield" class="dropdown links"><a href="" class="label">Why Dynamic Yield</a><div class="content"><ul><li class="impact-wrapper"><h2>Proven impact and success services</h2><ul class="links-list"><li class=" menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/clients/">Select customers</a></li><li class=" menu-item menu-item-type-custom menu-item-object-custom"><a href="/case-studies/">Case studies</a></li><li class=" menu-item menu-item-type-custom menu-item-object-custom"><a href="/customer-success/">Customer success services</a></li></ul></li><li class="scale-wrapper"><h2>Built for scale</h2><ul class="links-list"><li class=" menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/enterprise-grade-personalization/">Enterprise-grade solutions</a></li><li class=" menu-item menu-item-type-custom menu-item-object-custom"><a href="/adaptml/">AI Technology</a></li><li class=" menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/gdpr-and-privacy/">GDPR and data privacy</a></li><li class=" menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/security/">Security</a></li><li class=" menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/compliance/">Compliance</a></li></ul></li><li class="network-wrapper"><h2>Robust partner network</h2><ul class="links-list"><li class=" menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/partners/">Partner library</a></li><li class=" menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/partner-program/">Partnership program</a></li><li class=" menu-item menu-item-type-custom menu-item-object-custom"><a href="/partners/#technology-partners">Tech Partners and Integrations</a></li></ul></li><li class="resources-wrapper"><h2>Extensive product resources</h2><ul class="links-list"><li class=" menu-item menu-item-type-custom menu-item-object-custom"><a href="https://support.dynamicyield.com/hc/en-us">Knowledge Base</a></li><li class=" menu-item menu-item-type-custom menu-item-object-custom"><a href="http://academy.dynamicyield.com/">Academy & Certification</a></li><li class=" menu-item menu-item-type-custom menu-item-object-custom"><a href="https://support.dynamicyield.com/hc/en-us/community/topics">Community</a></li><li class=" menu-item menu-item-type-custom menu-item-object-custom"><a href="https://dy.dev/">API documentation</a></li></ul></li></ul><a href="https://www.dynamicyield.com/personalization-awards/" target="_self" class="banner"><div class="menu-banner"><div class="menu-cta--left"> <img src="/wp-content/uploads/2023/11/2023-award-badge.svg" width="60" height="60" /></div><div class="menu-cta--right"> <strong class="menu-cta-title">The Personalization Awards</strong> <span class="menu-cta-subtitle">Celebrating customers and partners setting the gold-standard for personalization</span> <span class="menu-cta-button">Learn more →</span></div></div></a></div></li><li id="company" class="dropdown button"><a href="" class="label">Company</a><div class="content"><ul class="links-list"><li class=" menu-item menu-item-type-custom menu-item-object-custom"><a href="https://www.mastercard.com/us/en/for-the-world/about-us.html">About us</a></li><li class=" menu-item menu-item-type-custom menu-item-object-custom"><a href="/blog/">Blog</a></li><li class=" menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/labs/">DY Labs</a></li><li class=" menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/careers/">Careers</a></li><li class=" menu-item menu-item-type-custom menu-item-object-custom"><a href="https://events.dynamicyield.com/">Events</a></li><li class=" menu-item menu-item-type-custom menu-item-object-custom"><a href="https://www.mastercard.com/us/en/news-and-trends/stories.html">Press</a></li><li class=" menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/awards-and-recognitions/">Awards</a></li><li class=" menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/contact/">Contact us</a></li><li class=" menu-item menu-item-type-custom menu-item-object-custom"><a href="/request-demo/">Request a demo</a></li></ul></div></li><li id="learning-center" class="dropdown links"><a href="" class="label">Learning Center</a><div class="content"><ul><li class="top"><ul class="links-list"><li class="resources-xp hidden-mobile menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/learn/">Take your knowledge to exponential levels</a></li><li class="hidden-desktop menu-item menu-item-type-custom menu-item-object-custom"><a href="/learn/">XP<sup>2</sup> Hub<span>Take your knowledge to exponential levels</span></a></li><li class="resources-paths menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/learning-paths/">Learning Paths<span>Curated courses on key skill areas</span></a></li><li class="resources-webinars menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/talks/">Talks<span>Engaging discussions taking place in CX</span></a></li><li class="resources-articles menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/articles/">Articles<span>An expansive collection of in-depth playbooks</span></a></li><li class="resources-glossary menu-item menu-item-type-custom menu-item-object-custom"><a href="/glossary-terms/">Encyclopedia<span>A glossary of experience optimization terms</span></a></li><li class="resources-use-cases menu-item menu-item-type-post_type menu-item-object-page"><a href="https://www.dynamicyield.com/personalization-examples/">Inspiration Library<span>Personalization examples from real brands</span></a></li><li class="resources-maturity menu-item menu-item-type-custom menu-item-object-custom"><a href="https://www.dynamicyield.com/personalization-maturity/"><strong class="badge-new">Personalization Maturity</strong><span>How global businesses prioritize personalization</span></a></li><li class="resources-guides menu-item menu-item-type-custom menu-item-object-custom"><a href="/guides/">Guides & reports<span>Comprehensive topic-specific materials</span></a></li><li class="resources-benchmarks menu-item menu-item-type-custom menu-item-object-custom"><a href="https://marketing.dynamicyield.com/benchmarks/">Benchmarks<span>Industry performance metrics and insights</span></a></li></ul></li><li class="bottom"><a href="/rooted-personalization/" target="_self"><div class="menu-banner"><div class="menu-cta--left"> <img src="/wp-content/uploads/2023/03/rooted-icon.svg" width="60" height="60" /></div><div class="menu-cta--right"> <strong class="menu-cta-title">Rooted Personalization<span>new</span></strong> <span class="menu-cta-subtitle">Learn a scalable methodology for building an audience-based personalization program.</span> <span class="menu-cta-button">Find out more →</span></div></div></a></li></ul></div></li><li class="contact top-link walker-last mobile-only menu-item menu-item-type-post_type menu-item-object-page" data-content="connect-with-us" title=""> <a href="https://www.dynamicyield.com/contact/"><span class="button-arrow">Connect with us</span></a></li></ul><div class="bg-layer" aria-hidden="true"></div></div></div></li></ul></nav></div><div class="topnav-buttons"><div class="language-switcher header-language-switcher"><ul><li>en<ul><li><a class="active" href="https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/">English</a></li><li><a href="https://www.dynamicyield.com/es/">Español</a></li><li><a href="https://www.dynamicyield.com/de/">Deutsch</a></li><li><a href="https://www.dynamicyield.com/fr/">Français</a></li><li><a href="https://www.dynamicyield.com/ja/">日本語</a></li></ul></li></li></ul></div> <a class="pulse-button v5 pulsed nav_btn" href="/request-demo/"> <span class="button-inner"> Talk to sales </span> </a> <a href="" class="nav-trigger"> Open Nav <span class="nav-trigger-box"><span class="nav-trigger-inner"></span></span> </a></div></div></header><div id="top-nav-backdrop"></div><div id="et-main-area"><div id="main-content"><div class="lc-container"><div id="content-area" class="clearfix"><header class="article-sticky-header"><div class="lc-header-container lc-flex"><div class="lc-header-logo"></div><div class="lc-header-title"> From DY Labs: Introducing Funnel Rocket, an open-source serverless query engine</div><div class="lc-header-right lc-flex"><ul class="lc-header-social lc-flex"><li class="et-social-icon"> <a href="#" title="Share on Facebook" class="icon" onclick="window.open('https://www.facebook.com/sharer/sharer.php?u=' + encodeURIComponent(document.URL) + '&t=' + encodeURIComponent(document.URL)); return false;"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 460.1990662 458.5138245"><path d="M460.1990662,230.0995331C460.1990662,103.0190277,357.1800232,0,230.0995331,0S0,103.0190277,0,230.0995331C0,338.011261,74.2903519,428.5578003,174.5126801,453.4230957V300.4146423H127.064209v-70.3151093h47.4484711v-30.2983704c0-78.3184433,35.4433594-114.619339,112.3326569-114.619339c14.5774536,0,39.730896,2.8583221,50.020874,5.7166443v63.7409363c-5.4308167-0.5716553-14.8633118-0.8574677-26.58255-0.8574677c-37.7300415,0-52.3075867,14.2916718-52.3075867,51.4501038v24.8674927h75.158783l-12.9122009,70.3151093h-62.246582v158.0991821C371.9082336,444.7527161,460.1990662,347.742981,460.1990662,230.0995331z"/></svg> </a></li><li class="et-social-icon"> <a href="#" title="Share on Linkedin" class="icon" onclick="window.open( 'https://www.linkedin.com/shareArticle?mini=true&url=' + encodeURIComponent(document.location.href)); return false;"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><path d="M100.28 448H7.4V148.9h92.88zM53.79 108.1C24.09 108.1 0 83.5 0 53.8a53.79 53.79 0 0 1 107.58 0c0 29.7-24.1 54.3-53.79 54.3zM447.9 448h-92.68V302.4c0-34.7-.7-79.2-48.29-79.2-48.29 0-55.69 37.7-55.69 76.7V448h-92.78V148.9h89.08v40.8h1.3c12.4-23.5 42.69-48.3 87.88-48.3 94 0 111.28 61.9 111.28 142.3V448z"/></svg> </a></li><li class="et-social-icon"> <a href="#" title="Tweet" class="icon" onclick="window.open('https://twitter.com/intent/tweet?text=' + encodeURIComponent(document.title) + ':%20' + encodeURIComponent(document.URL)); return false;"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><path d="M389.2 48h70.6L305.6 224.2 487 464H345L233.7 318.6 106.5 464H35.8L200.7 275.5 26.8 48H172.4L272.9 180.9 389.2 48zM364.4 421.8h39.1L151.1 88h-42L364.4 421.8z"/></svg> </a></li></ul><div class="topnav-buttons"> <a href="#" data-toggle="modal" data-target="#lc-subscription" class="btn-desktop-only pulse-button v5 topnav-btn"> <span class="button-inner"> XP² Newsletter </span> </a> <a href="/newsletter/" class="pulse-button v5 topnav-btn btn-mobile-only"> <span class="button-inner"> XP² Newsletter </span> </a></div></div></div><div class="lc-progress-wrap"><div class="lc-scroll-progress"></div></div></header><article id="post-160220" class="et_pb_post post-160220 post type-post status-publish format-standard has-post-thumbnail hentry category-product tag-engineering tag-funnel-rocket tag-rd"><div class="article-content"><div class="article-breadcrumbs"> <a href="https://www.dynamicyield.com/blog/" title="Blog">Blog</a><ul class="post-categories"><li><a href="https://www.dynamicyield.com/blog/category/product/" rel="category tag">Product</a></li></ul></div><h1 class="article-title">From DY Labs: Introducing Funnel Rocket, an open-source serverless query engine</h1><h2 class="article-excerpt">Learn about Funnel Rocket, a serverless query engine optimized for complex user-centric queries at scale, built at Dynamic Yield to answer on-demand interactive queries quickly and cost-efficiently, and now open-sourced for use by the wider developer community.</h2><div class="article-authors lc-flex"><div class="article-author-box lc-col lc-flex"><div class="article-author-image lc-col"><div class="entry_author_image"><img src="https://www.dynamicyield.com/wp-content/authors/elad-47.png" alt="Elad Rosenheim" width="76" height="76" /></div></div><div class="article-author-title lc-col"><div class="article-author-name"> <a href="https://www.dynamicyield.com/blog/author/elad/" title="Posts by Elad Rosenheim" rel="author">Elad Rosenheim</a></div><div class="article-author-title-text"> VP of Technology, Dynamic Yield</div></div></div></div><div class="entry-content article-entry-content"><p><i><span style="font-weight: 400;">This post is part two of a two-part series (<a href="https://www.dynamicyield.com/blog/dy-labs-building-blocks-open-source">read part one here</a>, if you haven’t yet) brought to you by DY Labs, an initiative by members of our Product and R&D departments who have a passion for experimentation and building developer resources for the greater digital marketing and engineering communities. This series focuses on the cohort’s latest project, </span></i><a href="https://github.com/DynamicYieldProjects/funnel-rocket"><i><span style="font-weight: 400;">Funnel Rocket</span></i></a><i><span style="font-weight: 400;">, which is a serverless query engine optimized for complex user-centric queries at scale. In part two, Elad Rosenheim, our VP of Technology, walks through the inception of Funnel Rocket, how it works, and the potential applications it can be used for.</span></i></p><p><i><span style="font-weight: 400;">An earlier version of this post was previously published by Elad </span></i><a href="https://elad-rosenheim.medium.com/funnel-rocket-a-cloud-native-query-engine-30bc142163b1" target="_blank" rel="noopener noreferrer"><i><span style="font-weight: 400;">on Medium</span></i></a><i><span style="font-weight: 400;">.</span></i></p><p><span style="font-weight: 400;">Funnel Rocket is a serverless query engine optimized for complex user-centric queries at scale. It allows companies to look for users matching a nuanced set of conditions over time and glean fine-grained statistics so they can better understand a given group of users and take action. It was built at Dynamic Yield to answer on-demand interactive queries quickly and cost-efficiently and is now open-sourced for use by the wider developer community. With Funnel Rocket, you can build custom ad-hoc analytics capabilities over big datasets of user activity – at a large scale and with minimal overhead in terms of operations and resource usage. Below, learn about the challenges we set out to solve with Funnel Rocket, what the building process looked like, how the engine can be a solution for engineers looking to overcome similar challenges, and if cloud native actually means anything.</span></p><h2><span style="font-weight: 400;">The status quo of SaaS</span></h2><p><span style="font-weight: 400;">As a SaaS vendor, success is defined by how many teams install or use your product for their websites and apps. Whether you’re selling tools for analytics, recommendations, personalization, guided onboarding, surveys, media serving, etc., one crucial feature for your customers is the ability to freely explore the complexities of end-user behavior. This requires understanding the makeup and value of distinct user groups and evolving the site accordingly, potentially tailoring different experiences on their site to different user groups (Note: this is also achievable without tracking people across the web or storing their personal info).</span></p><p><span style="font-weight: 400;">Marketers onboarding software for this purpose are looking for a UI that enables ad-hoc exploration, giving them the ability to look for users who’ve met a nuanced set of conditions over time, perhaps those who have completed a specific sequence of actions in a particular order. Afterward, they want to be able to access detailed analytics showing not only the size of the group but also detailed metrics on how this audience group stands out from the rest of their user base.</span></p><div class="wistia_responsive_padding" style="margin-top: 10px !important; padding: 56.25% 0 28px 0; position: relative;"><div class="wistia_responsive_wrapper" style="height: 100%; left: 0; position: absolute; top: 0; width: 100%;"><iframe class="wistia_embed" src="//fast.wistia.net/embed/iframe/mttbyvec0m?videoFoam=true" name="wistia_embed" width="100%" height="100%" frameborder="0" scrolling="no" allowfullscreen="allowfullscreen"></iframe></div></div><p style="text-align: center;"><i><span style="font-weight: 400;">For a glimpse of what this type of feature looks like in the Dynamic Yield platform, jump to minute 0:27 in the video.</span></i></p><h3><strong>The lingering problem with interactive queries</strong></h3><p><span style="font-weight: 400;">Such interactive queries are a challenge to pull off using existing tools — using SQL or NoSQL. Getting the query syntax right is cumbersome, but the main issue centers around performance at scale. Consider what it takes to implement this as a GROUP BY <USER ID> query.</span></p><p><span style="font-weight: 400;">No query engine I know will let you define in one step the whole sequence of actions that each group (i.e. each user) should match, in order. You will need to break this into multiple sub-queries, passing intermediate data between them, requiring multiple passes through the engine. In each pass, the engine will search (per each user) for a row whose timestamp is later than the row found in the last step, for that specific user. Such complex cases are typically expressed via composition, by design. Therefore, the query engine doesn’t always offer the simplest shortcut for every imaginable need.</span></p><p><span style="font-weight: 400;">There’s a performance cost to that complexity, however. On a small scale, it will work fine. However, take a dataset of 20 million users and that’s 20 million groups per query. Is it doable? Yes, assuming the user activity data is bucketed in the right way. Is it any fun, fast, or cheap? Not really. (Remember, I set some lofty goals at the end of </span><a href="https://www.dynamicyield.com/blog/dy-labs-building-blocks-open-source"><span style="font-weight: 400;">my previous post</span></a><span style="font-weight: 400;">: I want maximum processing speed just when I need it and to pay for exactly the needed compute resources).</span></p><p><span style="font-weight: 400;">Managed query engines (e.g. Amazon’s Athena, Google’s BigQuery) do offer on-demand pricing by the amount of data actually scanned. Your queries are executed behind the scenes by a small slice of their vast army of workers. This option was simply not available when we started, and thus we’ve built our homegrown solution to this problem with Elasticsearch + custom plugins. It was cool for a period of time, but overtime became an operational and maintenance burden.</span></p><p><span style="font-weight: 400;">Considering the managed tools available to us now, there were some things I was looking for:</span><span style="font-weight: 400;"><br /> </span></p><ul><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">While these managed offerings are no doubt ultra-scalable and offer an impressively rich syntax, they’re simply not meant to serve interactive end-user queries. Query latency can widely fluctuate, with some occasional hiccups — which is fine for your internal BI needs or a pipeline but becomes a real problem if you want to ensure that your users get latencies in the low single-digits.</span></li><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">I wanted an engine optimized for the type of complex user-centric queries I needed to support, with the ability to find bottlenecks, optimize them and add functionality natively.</span></li><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">I wanted a tool that is suited for fast on-demand interactive requests but also for big batches of scheduled queries, optimizing each use case for cost while continuing to maintain control of the SLA.</span></li></ul><p><span style="font-weight: 400;">The user-centric query feature is a core element of the Dynamic Yield platform, so it was important to us to have control of our capabilities, meaning that a “build” option was on the table. Of course, we didn’t set out to match the extensive feature-set of general-purpose query engines – we set out to solve one problem well. The result? </span><a href="https://github.com/DynamicYieldProjects/funnel-rocket"><span style="font-weight: 400;">Funnel Rocket</span></a><span style="font-weight: 400;">.</span></p><h2><span style="font-weight: 400;">The building blocks of Funnel Rocket</span></h2><p><span style="font-weight: 400;">Here’s a glance at some of the core components of the solution we set out to build:</span><span style="font-weight: 400;"><br /> </span></p><ol><li style="font-weight: 400;" aria-level="1"><a href="https://pandas.pydata.org/" target="_blank" rel="noopener noreferrer"><span style="font-weight: 400;">Pandas</span></a><span style="font-weight: 400;"> for data crunching at the single-task level, coupled with a few helpers — notably </span><a href="https://arrow.apache.org/" target="_blank" rel="noopener noreferrer"><span style="font-weight: 400;">Apache Arrow</span></a><span style="font-weight: 400;">, with perhaps </span><a href="http://numba.pydata.org/" target="_blank" rel="noopener noreferrer"><span style="font-weight: 400;">Numba</span></a><span style="font-weight: 400;"> for critical performance pain points. The general idea was to have Pandas-based workers processing one Parquet file at a time out of a much bigger dataset.</span></li><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">Serverless as a first-class option to support scaling and paying for only the actual compute time (currently supporting AWS Lambda). The solution also needed to be able to happily run on old-school machines or containers.</span></li><li style="font-weight: 400;" aria-level="1"><a href="https://redis.io/" target="_blank" rel="noopener noreferrer"><span style="font-weight: 400;">Redis</span></a><span style="font-weight: 400;"> for assisting with orchestration and making the code as environment-agnostic as possible, no less than for metadata storage.</span></li><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">Cloud Storage (currently S3 or compatible stores) to hold datasets. Funnel Rocket does not have an “ingestion” phase — it reads datasets that were prepared and written to S3 by other components at their own pace.</span></li></ol><p><span style="font-weight: 400;">To merge all of these together, we wrote an API server and task worker components in Python 3 that are pretty lightweight. The API server has endpoints for managing datasets and running queries, expressed in JSON format. It takes care to validate a given query, invoke the needed workers, track their progress, retry tasks as necessary, and aggregate the results back to the client. Additionally, the client has the option to ask for progress updates via HTTP streaming.</span></p><p><span style="font-weight: 400;">Here’s how it works, from the ground up:</span></p><p><img fetchpriority="high" decoding="async" class=" wp-image-160221 alignnone" src="https://www.dynamicyield.com/wp-content/uploads/2021/07/funnel-rocket-inline.png" alt="funnel rocket flowchart" width="1084" height="542" /></p><p style="text-align: center;"><i><span style="font-weight: 400;">Icon credit: </span></i><a href="https://thenounproject.com/" target="_blank" rel="noopener noreferrer"><i><span style="font-weight: 400;">Noun Project</span></i></a></p><h3><strong>Pandas-based workers</strong></h3><p><span style="font-weight: 400;">Pandas and its concept of DataFrames are widely used in the data engineering/science communities. Pandas itself is a library embedded within a single process, and we planned for each worker process to handle one file at a time out of the whole dataset, rather than worrying about any larger context. Additionally, Pandas is fairly performant and feature-rich out of the box. If it doesn’t offer some needed functionality, you can have it run Python code (though that’s pretty slow). On the other end, you can use Numba to generate optimized LLVM-compiled code from carefully written Python functions. In my experience, that does deliver the (performance) goods. </span></p><p><span style="font-weight: 400;">We use Apache Arrow to augment Pandas with excellent support for Parquet files and will probably use it much more going forward. Unlike Pandas, it supports nested data structures natively and has a growing feature set for computation and querying of its own. When a worker process is allocated a task, it transforms the JSON query syntax into a series of in-memory operations over a DataFrame and returns the aggregated results for the single file it was assigned to process. But for each worker to process independently, files in the dataset must already be partitioned (or “</span><a href="https://blog.clairvoyantsoft.com/bucketing-in-spark-878d2e02140f" target="_blank" rel="noopener noreferrer"><span style="font-weight: 400;">bucketed</span></a><span style="font-weight: 400;">”) by user ID into a set of “n files,” so that each file holds data about a distinct set of users, as well as guarantee each user’s data is gathering in a single file. That shuffling stage is arguably the biggest performance pain in big data. However, this is only executed when a dataset is being created / updated, rather than for each query. </span></p><h3><strong>Running serverless — or server-full</strong></h3><p><span style="font-weight: 400;">Serverless is quite polarizing, I know; the whole paradigm, the proprietary implementations, the latency, the constraints… I did not come into this project sold on the idea, to be honest. For this use-case, however, AWS Lambda did prove itself to be reliable, fast, and (gasp) cost-efficient. Additionally, 99% of the code is agnostic to it and can be easily extended to support other implementations.</span></p><p><span style="font-weight: 400;">There are several reasons why it works well for Funnel Rocket:</span></p><p><b>Latency</b></p><p><span style="font-weight: 400;">Queries in Funnel Rocket are measured in seconds, not milliseconds. When Lambda function instances are cold, they will normally take ±3 – 5 seconds to get to the point where the handler function starts running (that time does include running all imports in the handler’s source file).</span></p><p><span style="font-weight: 400;">Luckily, Funnel Rocket serves a rather “power user” feature, so while there tends to be few customers concurrently using the feature, each user typically runs multiple iterations of a query over their own dataset, progressively tweaking conditions to zoom in on the user population. This means that for most queries, not only are Lambda functions warm, but performance can further benefit from local caching of data files — if you can make that work for serverless. More on that in a bit…</span></p><p><b>Cost</b></p><p><span style="font-weight: 400;">The per/second cost is indeed higher than all less-managed options. It’s a spectrum, really: from spot instances, through reserved VMs to on-demand ones, to Fargate, etc. — the more you need to worry about, the less you pay in compute (and more in operations). However, you only pay for actual processing time, from the exact millisecond your entry point starts execution until the millisecond it ends. That excludes the bootstrap time and the function instance staying warm for a while to be reused. We found this model to fit our query patterns well: maximum scale when there’s a query, zero resources, and cost at all other times. As a side effect, the cost is directly correlated to milliseconds of compute used — for each query request Funnel Rocket returns the exact cost back.</span></p><p><b>Ease of invocation</b></p><p><span style="font-weight: 400;">Funnel Rocket uses the asynchronous Lambda invocation API, making it easier to launch hundreds of jobs (or more) quickly, without needing to block waiting on all API calls to complete. Internally, AWS manages such calls with a queue, but we’ve found that it adds no meaningful latency in normal operation. This mode has an important extra benefit: in moments of momentary pressure, you mostly avoid rate limiting on concurrent function executions.</span></p><p><span style="font-weight: 400;">Serverless, however, is not the only way to fly. I wanted the ability to have a small army of workers that can run locally, on physical machines, VMs, or containers — while still making it easy to scale up or down, distribute work, handle failures, etc. That’s where Redis comes in.</span></p><h3><strong>Managing it all with Redis</strong></h3><p><span style="font-weight: 400;">Two traits of Redis, taken together, make it a natural choice for a wide range of functionality:</span></p><ol><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">It has a wealth of data structures over its basic key-value abstraction: lists, sets and ordered sets, atomic increments, HyperLogLogs, streams, and much more.</span></li><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">Operations are atomic by nature, giving you a pretty strong guarantee that only one consumer will pop a specific element from a list or set (I realize there are always some nuanced caveats and edge cases, particularly around fail-overs).</span></li></ol><p><span style="font-weight: 400;">Redis is used by Funnel Rocket to:</span><span style="font-weight: 400;"><br /> </span></p><ul><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">Hold metadata on registered datasets and their schema.</span></li><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">Report the current status and results of all tasks in an in-flight query. Instead of needing to communicate directly, workers write all updates to Redis as tasks progress, and these updates are polled by the API server. The API server knows whether a task succeeded, failed, or was “lost in space” through its status in Redis. It does not need to rely on AWS Lambda or any other runtime environment to get this information.</span></li><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">Number consecutive attempts at running a failed/lost task using atomic counters. Each attempt of a task has its own unique attempt number. If multiple attempts for the same task end up succeeding, the API server will take care to only use the results of one.</span></li><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">Add work to the queue in the non-serverless mode. In the “classic process” mode, each worker is simply blocked on a Redis list to access work to do. In this mode, when a query starts the API server enqueues task requests in a logical queue, from which tasks are populated by workers. Statuses and retries are handled as above through Redis as well but regardless of the task invocation mode. No load balancer is required; workers don’t know each other, nor does the API server know them directly.</span></li><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">Apply best-effort optimization for data locality. </span></li></ul><p><span style="font-weight: 400;">In the case of repeated queries over the same dataset, how do we make warm functions work on the same file they’ve downloaded before? There is no direct way to call a specific warm instance. Instead, the API server lets the workers choose for themselves which file to work on. When a query starts, it publishes the list of files as a Redis set. Then it invokes workers, asking them to each take one file from the set. Those with matching local files will attempt to grab that same file again. Others will grab a part at random. This mechanism is not guaranteed to always have maximum efficiency, but it does reduce orchestration needs to a minimum. The API server meanwhile, knows nothing.</span></p><p><span style="font-weight: 400;">Multiple API server instances can run concurrently, only sharing the list of registered datasets between them. A portion may run ad-hoc queries via serverless workers, while others run batch work through cheap spot-instances at scheduled times. Both deployment options push much of the complexity into battle-tested tools and have Redis as their single stateful component.</span></p><h2><span style="font-weight: 400;">Let’s talk about benchmarking</span></h2><p><span style="font-weight: 400;">As a basic benchmark, I’ve created a dataset that mimics user behavioral data (page views, events, clicks on campaigns, etc.) to reflect real data we collect in Dynamic Yield as closely as possible.</span></p><p><span style="font-weight: 400;">I’ve made two versions of the dataset: a smaller one with 100 million rows, the other with 500 million. Each row holds a user ID, a timestamp, the type of activity and related data fields (URLs, browser type, country, event type, product SKU, etc.).</span><span style="font-weight: 400;"><br /> </span></p><ul><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">The datasets are split into 100 and 500 Parquet files stored in S3, respectively.</span></li><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">Each file weighs a bit less than 40mb, and the small dataset weighs 3.8 GB in total, with the larger one clocking in at 19.6GB.</span><ul><li style="font-weight: 400;" aria-level="2"><span style="font-weight: 400;">Note: this can be very misleading. Parquet files are typically highly-compressed. The exact same files saved as CSV are about 10x the size, which would make the datasets weigh 38 & 196 GB.</span></li></ul></li><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">The Lambda function was configured to use 1768 MB RAM (per instance) so that each instance gets one full vCPU. It doesn’t need all that RAM necessarily, but will simply run more slowly with less as it’ll get a smaller CPU share (</span><a href="https://engineering.opsgenie.com/how-does-proportional-cpu-allocation-work-with-aws-lambda-41cd44da3cac"><span style="font-weight: 400;">see here</span></a><span style="font-weight: 400;">).</span><span style="font-weight: 400;"><br /> </span></li></ul><p><span style="font-weight: 400;">I’ve run a basic funnel query, the results of which you can see below: find the users who have performed a pageview, then added a product to cart, then made a purchase — and return the matching user count at each step.</span></p><p><span style="font-weight: 400;">For both datasets, I measured the time-to-run and compute cost in two scenarios: when all Lambda function instances are cold, and when all are warm. Each scenario ran 10 times.</span></p><h2><span style="font-weight: 400;">Results</span></h2><p><img decoding="async" class=" wp-image-160227 alignnone" src="https://www.dynamicyield.com/wp-content/uploads/2021/07/funnel-rocket-inline-1.png" alt="funnel rocket query time" width="952" height="476" /></p><p><img loading="lazy" decoding="async" class=" wp-image-160233 alignnone" src="https://www.dynamicyield.com/wp-content/uploads/2021/07/funnel-rocket-inline-2-copy.png" alt="funnel rocket cost in cents" width="950" height="475" /></p><p><span style="font-weight: 400;">For 100 million rows:</span></p><table style="height: 264px;" width="803"><thead style="color: #fff; font-weight: 700!important; font-size: 17px; background: linear-gradient(to right, #3a82b5 1%,#51a997 100%); filter: progid:DXImageTransform.Microsoft.gradient( startColorstr='#3a82b5', endColorstr='#51a997',GradientType=1 );"><tr><td style="border: 1px solid #D1DBE1; color: #fff; line-height: 0.8em; font-size: 16px;"><p style="color: #ffffff; line-height: 1.3em; font-size: 18px; text-align: center;"><strong>Scenario</strong></p></td><td style="border: 1px solid #D1DBE1; color: #fff; line-height: 1em; font-size: 16px;"><p style="color: #ffffff; line-height: 1.3em; font-size: 18px; text-align: center;"><strong>Min Time</strong></p></td><td style="border: 1px solid #d1dbe1; color: #ffffff; line-height: 1em; font-size: 16px; text-align: center;"><p style="color: #fff; line-height: 1.3em; font-size: 18px;"><strong>Max Time</strong></p></td><td style="border: 1px solid #d1dbe1; color: #ffffff; line-height: 1em; font-size: 16px; text-align: center;"><p style="color: #fff; line-height: 1.3em; font-size: 18px;"><strong>Avg Time</strong></p></td><td style="border: 1px solid #D1DBE1; color: #fff; line-height: 1em; font-size: 16px;"><p style="color: #ffffff; line-height: 1.3em; font-size: 18px; text-align: center;"><strong>Min Cost</strong></p></td><td style="border: 1px solid #d1dbe1; color: #ffffff; line-height: 1em; font-size: 16px; text-align: center;"><p style="color: #fff; line-height: 1.3em; font-size: 18px;"><strong>Max Cost</strong></p></td><td style="border: 1px solid #D1DBE1; color: #fff; line-height: 1em; font-size: 16px;"><p style="color: #ffffff; line-height: 1.3em; font-size: 18px; text-align: center;"><strong>Avg Cost</strong></p></td></tr></thead><tbody><tr><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">All Cold</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">2.3 secs</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">6.5 secs</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">5.43 secs</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">0.15 cents</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">0.4 cents</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">0.35 cents</td></tr><tr><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">All Warm</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">1.78 secs</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">4.56 secs</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">2.2 secs</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">0.13 cents</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">0.146 cents</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">0.14 cents</td></tr></tbody></table><p>For 500 million rows:</p><table style="height: 264px;" width="803"><thead style="color: #fff; font-weight: 700!important; font-size: 17px; background: linear-gradient(to right, #3a82b5 1%,#51a997 100%); filter: progid:DXImageTransform.Microsoft.gradient( startColorstr='#3a82b5', endColorstr='#51a997',GradientType=1 );"><tr><td style="border: 1px solid #D1DBE1; color: #fff; line-height: 0.8em; font-size: 16px;"><p style="color: #ffffff; line-height: 1.3em; font-size: 18px; text-align: center;"><strong>Scenario</strong></p></td><td style="border: 1px solid #D1DBE1; color: #fff; line-height: 1em; font-size: 16px;"><p style="color: #ffffff; line-height: 1.3em; font-size: 18px; text-align: center;"><strong>Min Time</strong></p></td><td style="border: 1px solid #d1dbe1; color: #ffffff; line-height: 1em; font-size: 16px; text-align: center;"><p style="color: #fff; line-height: 1.3em; font-size: 18px;"><strong>Max Time</strong></p></td><td style="border: 1px solid #d1dbe1; color: #ffffff; line-height: 1em; font-size: 16px; text-align: center;"><p style="color: #fff; line-height: 1.3em; font-size: 18px;"><strong>Avg Time</strong></p></td><td style="border: 1px solid #D1DBE1; color: #fff; line-height: 1em; font-size: 16px;"><p style="color: #ffffff; line-height: 1.3em; font-size: 18px; text-align: center;"><strong>Min Cost</strong></p></td><td style="border: 1px solid #d1dbe1; color: #ffffff; line-height: 1em; font-size: 16px; text-align: center;"><p style="color: #fff; line-height: 1.3em; font-size: 18px;"><strong>Max Cost</strong></p></td><td style="border: 1px solid #D1DBE1; color: #fff; line-height: 1em; font-size: 16px;"><p style="color: #ffffff; line-height: 1.3em; font-size: 18px; text-align: center;"><strong>Avg Cost</strong></p></td></tr></thead><tbody><tr><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">All Cold</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">8.56 secs</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">10.39 secs</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">9.17 secs</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">2.77 cents</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">3.93 cents</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">3.08 cents</td></tr><tr><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">All Warm</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">3.42 secs</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">5.51 secs</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">4.34 secs</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">0.7 cents</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">1.7 cents</td><td style="height: auto; border: 1px solid #d1dbe1; text-align: center;">0.98 cents</td></tr></tbody></table><p><b>Things to note:</b><b><br /> </b></p><ol><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">Time is the total time-to-run of the query from the viewpoint of the API server. However, cost is the total billed amount for the Lambda runs, based directly on total milliseconds of actual handling time * units of allocated RAM * cost per unit/ms. I did not include per-request Lambda and S3 fees, which are, for this use case, much lower.</span></li><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">Admittedly, these are preliminary results. More scenarios could be added here, with more iterations so that histograms of total duration, cost, and task completion over time can be added.</span></li></ol><h2><span style="font-weight: 400;">Final takeaways and musings</span></h2><p><span style="font-weight: 400;">Building Funnel Rocket has left me with a lingering thought: Is “cloud native” a real thing? It was personally exciting for me to see all of this working in practice. Of course, there are missing features to wish for, surely a few bugs, and a bunch of optimizations and improvements yet to done (</span><a href="https://github.com/DynamicYieldProjects/funnel-rocket#high-level-roadmap"><span style="font-weight: 400;">see the high-level roadmap</span></a><span style="font-weight: 400;">). But all-in-all, I believe the software that resulted encapsulates what “cloud native” is about:</span><span style="font-weight: 400;"><br /> </span></p><ul><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">Having a large functional task broken down into smaller pieces of work that can be run in multiple runtime environments so that availability and cost are optimized depending on what you need (interactive vs. batch, more compute cost vs more operations, etc.)</span></li><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">Relying on existing, managed constructs to scale and orchestrate. “The Cloud” may be someone else’s computer, but it also offers many other services to build with: cloud storage, managed databases, serverless/containerized environments, and more.</span></li><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">Having modest requirements for every component (compared to a Spark executor, in this case). It should start quickly, take (relatively) little RAM, and just do its work while also reporting metrics for observability.</span></li><li style="font-weight: 400;" aria-level="1"><span style="font-weight: 400;">Lastly: it should also work fully on your computer (We use docker-compose, </span><a href="https://min.io/" target="_blank" rel="noopener noreferrer"><span style="font-weight: 400;">MinIO,</span></a><span style="font-weight: 400;"> and </span><a href="https://github.com/lambci/docker-lambda" target="_blank" rel="noopener noreferrer"><span style="font-weight: 400;">docker-lambda</span></a><span style="font-weight: 400;"> to support that). </span></li></ul><p><span style="font-weight: 400;">This exercise has led me to finally believe in cloud native, and if you’re interested in learning more about Funnel Rocket and playing around with it yourself, <a href="https://github.com/DynamicYieldProjects/funnel-rocket">access it on Github</a>. And if you’re interested in joining our growing Product and R&D teams, here at Dynamic Yield, <a href="https://www.dynamicyield.com/careers/">check out our open roles</a>.</span></p></div></article><div style="clear: both;"></div></div></div><div class="article-content"><div class="article-separator"><svg width="54" height="26" viewBox="0 0 54 26" fill="none" xmlns="http://www.w3.org/2000/svg"> <path d="M53.0088 25.6367C52.7884 21.9564 51.8025 18.3625 50.1141 15.0849C48.4256 11.8073 46.0718 8.9181 43.2031 6.60203L43.2031 25.6367L53.0088 25.6367Z" fill="black"/> <path d="M14.4395 3.59038C10.3237 5.69174 6.82743 8.83082 4.29638 12.6973C1.76533 16.5638 0.28711 21.0239 0.0078125 25.6367L14.4395 25.6367L14.4395 3.59038Z" fill="black"/> <path d="M17.6563 2.20809L17.6562 25.6367L39.9829 25.6367L39.9829 4.36701C39.9829 4.36701 30.7303 -2.31342 17.6563 2.20809Z" fill="black"/> </svg></div><div class="article-tags-block"> <strong>Categories:</strong><ul class="post-categories"><li><a href="https://www.dynamicyield.com/blog/category/product/" rel="category tag">Product</a></li></ul></div><div class="article-tags-block"><strong>Tags:</strong><a href="https://www.dynamicyield.com/blog/tag/engineering/" rel="tag">Engineering</a><a href="https://www.dynamicyield.com/blog/tag/funnel-rocket/" rel="tag">Funnel Rocket</a><a href="https://www.dynamicyield.com/blog/tag/rd/" rel="tag">R&D</a></div></div></div><section class="lc-subscription-section"><div class="lc-subscription-form-block lc-subscription-inner"><h4 class="lc-subscription-title">Sign up for the XP² newsletter</h4><div class="lc-subscription-text">Join thousands of readers from Target, Citi, Spotify, Hulu, Google, Sephora, and other innovative brands who read our bi-weekly XP² newsletter, delivering educational content, research, and insights straight to your inbox</div><form class="mktoForm" data-formId="3876" data-formInstance="sub-inline"></form><div class="lc-privacy-wrapper"><div class="lc-privacy"> You may unsubscribe at any time. Visit our <a href="/privacy-policy/">privacy policy</a> to learn more about how we process your data and your rights as a data subject.</div></div></div><div class="lc-subscription-ty-block lc-subscription-inner"><h4 class="lc-subscription-title text-center">Thanks for<br> signing up!</h4></div></section><div class="container"><div class="related-posts-wrapper"><div id="dc_crp_related" class="crp_related"><h3>YOU MIGHT ALSO LIKE</h3><ul><li><a href="https://www.dynamicyield.com/blog/dy-labs-building-blocks-open-source/"><img src="https://www.dynamicyield.com/wp-content/uploads/2021/07/funnel-rocket-blog-1.png" alt="" class="crp_thumb"><div class="crp_title">Going for gold: The building blocks of effective open-source software </div><div class="crp_button button">READ MORE</div></a></li><li><a href="https://www.dynamicyield.com/blog/personalization-reporting/"><img src="https://www.dynamicyield.com/wp-content/uploads/2018/09/DY_NewDashboard_Blog_795x320_V2.jpg" alt="" class="crp_thumb"><div class="crp_title">Personalization reporting – powerful insights to help drive your experimentation efforts</div><div class="crp_button button">READ MORE</div></a></li><li><a href="https://www.dynamicyield.com/blog/personalized-holiday-emails/"><img src="https://www.dynamicyield.com/wp-content/uploads/2020/10/DY-XmasInboxes-Blog-795x320-1.jpg" alt="" class="crp_thumb"><div class="crp_title">Personalized holiday email campaigns to inspire gift givers</div><div class="crp_button button">READ MORE</div></a></li></ul></div></div></div><div class="et_pb_section post_slider_row test_posts_carousel select-clients-carousel et_section_regular"
style="background-color:#f9f9f9;"><div class="et_pb_row"><div class="et_pb_column et_pb_column_4_4"><div class="et_pb_text et_pb_bg_layout_light et_pb_text_align_left"><div class="case-studies-carousel-title">Driving Big Results with Personalization</div><div class="posts_carousel"><ul id="posts_carousel"><li class="243971"><a href="https://www.dynamicyield.com/case-studies/valamar/"><img class="b-lazy" src="data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==" data-src="https://www.dynamicyield.com/wp-content/uploads/2026/02/mc-cm-valmar-excerpt-544x672-1.jpg" width="281" height="347" /></a><div class="grid_title">How Valamar converts travel intent into bookings with real-time context</div></li><li class="243414"><a href="https://www.dynamicyield.com/case-studies/saks-fifth-avenue/"><img class="b-lazy" src="data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==" data-src="https://www.dynamicyield.com/wp-content/uploads/2025/11/mc-cm-saks-5thave-excerpt-544x672_v2.jpg" width="281" height="347" /></a><div class="grid_title">What makes Saks’ personalization engine a game-changer for driving 10% more conversions?</div></li><li class="242646"><a href="https://www.dynamicyield.com/case-studies/bergzeit/"><img class="b-lazy" src="data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==" data-src="https://www.dynamicyield.com/wp-content/uploads/2025/09/mc-cm-bergzeit-excerpt-544x672-1.jpg" width="281" height="347" /></a><div class="grid_title">Bringing the in-store expert online: How Bergzeit reinvented gear discovery with conversational commerce</div></li><li class="237681"><a href="https://www.dynamicyield.com/case-studies/leroymerlin/"><img class="b-lazy" src="data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==" data-src="https://www.dynamicyield.com/wp-content/uploads/2025/01/library-thumbnail.png" width="281" height="347" /></a><div class="grid_title">32% of Total Purchases from AdaptML Recommendations</div></li><li class="230927"><a href="https://www.dynamicyield.com/case-studies/signet/"><img class="b-lazy" src="data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==" data-src="https://www.dynamicyield.com/wp-content/uploads/2024/10/signet-new-library-thumbnail.jpg" width="281" height="347" /></a><div class="grid_title">+88% Conversion Uplift with Mastercard Predictive Spend Insights</div></li><li class="224233"><a href="https://www.dynamicyield.com/case-studies/personalization-breeze/"><img class="b-lazy" src="data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==" data-src="https://www.dynamicyield.com/wp-content/uploads/2024/07/library-thumbnail.png" width="281" height="347" /></a><div class="grid_title">Making Personalization a Breeze for Leading Financial Institutions</div></li><li class="221699"><a href="https://www.dynamicyield.com/case-studies/tui/"><img class="b-lazy" src="data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==" data-src="https://www.dynamicyield.com/wp-content/uploads/2024/06/tui-library-thumbnail.png" width="281" height="347" /></a><div class="grid_title">+10.3% uplift in add-to-cart rate</div></li><li class="215474"><a href="https://www.dynamicyield.com/case-studies/sky/"><img class="b-lazy" src="data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==" data-src="https://www.dynamicyield.com/wp-content/uploads/2024/02/library-thumbnail-2.png" width="281" height="347" /></a><div class="grid_title">39% decrease in same-month cancellations</div></li><li class="215658"><a href="https://www.dynamicyield.com/case-studies/electrolux/"><img class="b-lazy" src="data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==" data-src="https://www.dynamicyield.com/wp-content/uploads/2024/02/library-thumbnail-1.png" width="281" height="347" /></a><div class="grid_title">16% contribution to D2C revenue from Dynamic Yield</div></li><li class="215340"><a href="https://www.dynamicyield.com/case-studies/on/"><img class="b-lazy" src="data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==" data-src="https://www.dynamicyield.com/wp-content/uploads/2024/02/library-thumbnail.png" width="281" height="347" /></a><div class="grid_title">16% of revenue from recommendations</div></li></ul></div><style>ul#posts_carousel:not(.slick-initialized) {
display: flex;
flex-direction: row;
overflow: visible;
width: 10000px;
}
.posts_carousel {
overflow-x: hidden;
}
#posts_carousel .slick-list {
max-width: 873px;
margin: auto;
left: 5px;
}
#posts_carousel li {
margin-right: 10px;
margin-bottom: 10px;
}
#posts_carousel li img{
box-shadow: 3px -1px 10px #C3C3C3;
}
#posts_carousel .slick-list:after {
content: '';
position: absolute;
right: 0;
top: 0;
bottom: 0;
width: 10px;
background: #f9f9f9;
}
#posts_carousel .slick-prev, #posts_carousel .slick-next {
width: 45px;
height: 45px;
}
#posts_carousel.slick-slider .slick-prev:before {
background: url('/wp-content/themes/Divi/images/arrow_left.svg')50% no-repeat;
opacity: 0.5;
width: 40px;
height: 40px;
}
#posts_carousel.slick-slider .slick-next:before {
background: url('/wp-content/themes/Divi/images/arrow_right.svg')50% no-repeat;
opacity: 0.5;
width: 40px;
height: 40px;
}
.post_slider_row {
overflow-x: hidden;
}
@media(max-width: 980px){
#posts_carousel {
max-width: 450px;
margin: auto;
}
#posts_carousel .slick-list {
width: 278px;
max-width: 95%;
margin: auto;
left: auto;
}
#posts_carousel li {
margin-right: 0;
}
#posts_carousel .slick-list:after {
display: none;
}
}
@media(max-width: 480px){
.posts_carousel{
display: inline;
}
#posts_carousel .slick-prev {
left: -25px;
z-index: 9;
}
#posts_carousel .slick-next {
right: -25px;
z-index: 9;
}
}
@media(max-width: 360px){
#posts_carousel.slick-slider .slick-next:before {
position: relative;
right: -10px;
}
#posts_carousel.slick-slider .slick-prev:before {
position:relative;
left: -5.5px;
}
}</style><script type="text/javascript">$=jQuery.noConflict();
$(".posts_carousel").css({visibility: "visible"});
$(window).load(function(){
$("#posts_carousel").slick({
centerMode: false,
waitForAnimate: false,
slidesToShow: 3,
arrows: true,
dots: false,
autoplaySpeed: 5,
draggable: true,
swipeToSlide: true,
responsive: [
{
breakpoint: 981,
settings: {
slidesToShow: 1
}
},
]
});
$(".slider-widget ul").css({opacity: "1"});
});
$("#posts_carousel").on("beforeChange", function(event, slick, currentSlide, nextSlide){
var imagesToLoad = $(".posts_carousel .b-lazy");
var slickCarouselBlazy = new Blazy();
slickCarouselBlazy.load(imagesToLoad);
});
let pageUrl = window.location.href;
let csCard = $(".posts_carousel a");
if(csCard) {
csCard.on("click", function(e) {
let ctaUrl = $(this).attr("href"),
ctaName = $(this).parent().find(".grid_title").text().trim();
adobeDataLayer.push({
"event": "tileClick",
"clickInfo": {
tileName: ctaName,
destinationURL: ctaUrl,
tileSection : "Posts Carousel",
tileLocation: pageUrl,
}
});
});
}</script> </div></div></div></div> <script type='text/javascript'>//DY.API('siteVar', {id: 128, value: 'Elad Rosenheim'});
//DY.API('siteVar', {id: 129, value: 'Product'});</script> <footer id="main-footer" class=""><div class="footer-top"><div class="footer-container"><h4>Make hyper-personalization a reality</h4><p>Transform simple customer transactions into enduring relationships and unlock a new horizon of growth</p><div class="footer-btns-wrap"> <a href="https://www.dynamicyield.com/request-demo/" class="btn-teal">Contact Sales</a> <a href="https://www.dynamicyield.com/product-demo/" class="btn-transparent">Watch a product demo<svg xmlns="http://www.w3.org/2000/svg" width="15.625" height="15.625" viewBox="0 0 15.625 15.625"> <path id="Icon_ionic-ios-play-circle" data-name="Icon ionic-ios-play-circle" d="M10.688,3.375A7.313,7.313,0,1,0,18,10.688,7.311,7.311,0,0,0,10.688,3.375Zm2.946,7.45L8.81,13.743a.156.156,0,0,1-.236-.137V7.77a.156.156,0,0,1,.236-.137l4.823,2.918A.161.161,0,0,1,13.634,10.825Z" transform="translate(-2.875 -2.875)" fill="none" stroke="rgba(0,0,0,0.8)" stroke-width="1"/> </svg> </a></div><ul class="footer-certificates"><li class="retina_parent"> <a href="https://www.dynamicyield.com/compliance/"> <img src="/wp-content/uploads/2022/05/gdpr-icon.png" alt="gdpr icon" width="139" height="70"> </a></li><li class="retina_parent"> <a href="https://www.dynamicyield.com/compliance/"> <img src="/wp-content/uploads/2023/04/SOCII-Badge.png" alt="socii icon" width="70" height="70"> </a></li><li> <a href="https://www.dynamicyield.com/compliance/"> <img src="/wp-content/uploads/2022/05/ccpa-icon.svg" alt="ccpa icon" width="117" height="40"> </a></li><li class="retina_parent"> <a href="https://www.dynamicyield.com/compliance/"> <img src="/wp-content/uploads/2022/05/iso-27001.png" alt="iso icon" width="106" height="68"> </a></li></ul></div></div><div class="footer-container"><div id="footer-widgets"><div class="footer-widget"><div id="nav_menu-2" class="fwidget et_pb_widget widget_nav_menu"><h4 class="title">Key Capabilities</h4><div class="menu-core-capabilities-container"><ul id="menu-core-capabilities" class="menu"><li id="menu-item-230872" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-230872"><a href="https://www.dynamicyield.com/segmentation/">Segmentation</a></li><li id="menu-item-5913" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-5913"><a href="https://www.dynamicyield.com/targeting/">Targeting</a></li><li id="menu-item-198647" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-198647"><a href="https://www.dynamicyield.com/recommendations/">Recommendations</a></li><li id="menu-item-198648" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-198648"><a href="https://www.dynamicyield.com/journey-orchestration/">Journey Orchestration</a></li><li id="menu-item-230873" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-230873"><a href="https://www.dynamicyield.com/optimization/">Optimization</a></li></ul></div></div></div><div class="footer-widget"><div id="nav_menu-13" class="fwidget et_pb_widget widget_nav_menu"><h4 class="title">Industries</h4><div class="menu-industries-container"><ul id="menu-industries" class="menu"><li id="menu-item-51518" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-51518"><a href="/ecommerce/">eCommerce</a></li><li id="menu-item-51520" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-51520"><a href="/financial-services/">Financial Services</a></li><li id="menu-item-224041" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-224041"><a href="/restaurants/">Restaurants</a></li><li id="menu-item-224042" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-224042"><a href="/grocery/">Grocery</a></li><li id="menu-item-51519" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-51519"><a href="/b2b-ecommerce/">B2B eCommerce</a></li><li id="menu-item-13777" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-13777"><a href="https://www.dynamicyield.com/travel/">Travel</a></li><li id="menu-item-19895" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-19895"><a href="https://www.dynamicyield.com/gaming/">iGaming</a></li><li id="menu-item-13775" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-13775"><a href="https://www.dynamicyield.com/media/">Media</a></li></ul></div></div></div><div class="footer-widget"><div id="nav_menu-11" class="fwidget et_pb_widget widget_nav_menu"><h4 class="title">Why Dynamic Yield</h4><div class="menu-why-dynamic-yield-container"><ul id="menu-why-dynamic-yield" class="menu"><li id="menu-item-51524" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-51524"><a href="/customer-success/">Industry-Leading Services</a></li><li id="menu-item-31658" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-31658"><a href="https://www.dynamicyield.com/clients/">Global Scale</a></li><li id="menu-item-13718" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-13718"><a href="https://www.dynamicyield.com/case-studies/">Real Impact</a></li><li id="menu-item-224043" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-224043"><a href="/ai/">Personalization AI</a></li><li id="menu-item-224044" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-224044"><a href="/partners/">Rich Partner Network</a></li><li id="menu-item-224045" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-224045"><a href="/enterprise-grade-personalization/">Enterprise-Grade Security</a></li></ul></div></div></div><div class="footer-widget"><div id="nav_menu-5" class="fwidget et_pb_widget widget_nav_menu"><h4 class="title">Company</h4><div class="menu-company-container"><ul id="menu-company" class="menu"><li id="menu-item-242875" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-242875"><a href="https://www.mastercard.com/us/en/for-the-world/about-us.html">About Us</a></li><li id="menu-item-19360" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-19360"><a href="https://events.dynamicyield.com/">Events</a></li><li id="menu-item-242874" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-242874"><a href="https://www.mastercard.com/us/en/news-and-trends/stories.html">Press</a></li><li id="menu-item-130411" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-130411"><a href="https://www.dynamicyield.com/blog/">Blog</a></li><li id="menu-item-226" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-226"><a href="https://www.dynamicyield.com/contact/">Contact</a></li><li id="menu-item-198646" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-198646"><a href="https://www.dynamicyield.com/partner-program/">Become a Partner</a></li><li id="menu-item-11264" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-11264"><a href="/request-demo/">Request Demo</a></li></ul></div></div><div id="text-6" class="fwidget et_pb_widget widget_text"><div class="textwidget"><ul style="min-width: 150px; margin-top: -15px;"><li><a style="color: #fff; font-weight: 500;" href="/careers/">We’re hiring!</a></li></ul></div></div></div><div class="lc-footer-nav-block"><div class="lc-footer-nav-header"> <a href="https://www.dynamicyield.com/learn/" class="lc-footer-logo"></a> <span>Take your knowledge to exponential levels</span></div><div class="lc-footer-menu"><ul id="menu-lc-menu-footer" class="menu"><li id="menu-item-212992" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-212992"><a href="https://www.dynamicyield.com/articles/">Articles</a></li><li id="menu-item-212993" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-212993"><a href="https://www.dynamicyield.com/learning-paths/">Learning Paths</a></li><li id="menu-item-212994" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-212994"><a href="https://www.dynamicyield.com/rooted-personalization/">Rooted Personalization</a></li><li id="menu-item-212995" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-212995"><a href="https://www.dynamicyield.com/personalization-examples/">Personalization Examples</a></li><li id="menu-item-212996" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-212996"><a href="https://www.dynamicyield.com/guides/">Guides</a></li><li id="menu-item-212997" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-212997"><a href="https://www.dynamicyield.com/talks/">Talks</a></li><li id="menu-item-212999" class="menu-item menu-item-type-post_type menu-item-object-page menu-item-212999"><a href="https://www.dynamicyield.com/glossary-terms/">Encyclopedia</a></li><li id="menu-item-212998" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-212998"><a href="https://marketing.dynamicyield.com/benchmarks/">Benchmarks</a></li></ul></div></div></div></div><div id="et-footer-nav"><div class="footer-container"><ul id="menu-footer-menu" class="bottom-nav"><li id="menu-item-19183" class="footer_menu_title menu-item menu-item-type-custom menu-item-object-custom menu-item-19183"><a>Recommended Guides:</a></li><li id="menu-item-211968" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-211968"><a href="https://www.dynamicyield.com/article/personalization-guide/">Personalization</a></li><li id="menu-item-138198" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-138198"><a href="https://www.dynamicyield.com/lesson/introduction-to-ab-testing/">A/B Testing</a></li><li id="menu-item-147897" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-147897"><a href="https://www.dynamicyield.com/lesson/cro-plan/">Conversion Rate Optimization</a></li><li id="menu-item-19184" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-19184"><a href="/lesson/shopping-cart-abandonment-strategy/">Shopping Cart Abandonment</a></li><li id="menu-item-19185" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-19185"><a href="/lesson/product-recommendations-guide/">Product Recommendations</a></li><li id="menu-item-22805" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-22805"><a href="/lesson/promise-and-pitfalls-omnichannel-retailing/">Omnichannel Retailing</a></li><li id="menu-item-22885" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-22885"><a href="/lesson/the-economics-of-ecommerce-conversion-optimization/">eCommerce Conversion Rate Optimization</a></li><li id="menu-item-28240" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-28240"><a href="/article/50-most-important-dynamicyield-personalization-stats/">Personalization Statistics</a></li><li id="menu-item-33877" class="menu-item menu-item-type-post_type menu-item-object-post menu-item-33877"><a href="https://www.dynamicyield.com/article/tj-maxx-spired-checkout-optimization/">Checkout Optimization</a></li><li id="menu-item-123711" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-123711"><a href="/article/ecommerce-personalization-must-haves/">eCommerce Personalization</a></li><li id="menu-item-144090" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-144090"><a href="https://www.dynamicyield.com/lesson/strategizing-omnichannel-personalization/">Omnichannel Personalization</a></li><li id="menu-item-148146" class="menu-item menu-item-type-custom menu-item-object-custom menu-item-148146"><a href="https://marketing.dynamicyield.com/benchmarks/cart-abandonment-rate/">Shopping Cart Abandonment Rate</a></li></ul><div class="footer-mach-logo retina_parent"> <img src="/wp-content/uploads/2022/05/mach-cert-22-wh.png" alt="Mach logo" width="63" height="70"></div><div class="language-switcher footer-language-switcher"><ul><li>en<ul><li><a class="active" href="https://www.dynamicyield.com/blog/funnel-rocket-serverless-query-engine/">English</a></li><li><a href="https://www.dynamicyield.com/es/">Español</a></li><li><a href="https://www.dynamicyield.com/de/">Deutsch</a></li><li><a href="https://www.dynamicyield.com/fr/">Français</a></li><li><a href="https://www.dynamicyield.com/ja/">日本語</a></li></ul></li></li></ul></div></div></div><div id="footer-bottom"><div class="footer-container"><div class="footer_info_wrapper"><div id="footer-info"> <button id="ot-sdk-btn" class="ot-sdk-show-settings">Manage Cookies</button> <a href="/privacy-notice/" target="_blank" class="animlink">Privacy Notice</a> <a href="/tos/" target="_blank" class="animlink">Terms of use</a> <span>© 2026 Mastercard Dynamic Yield</span></div></div><div class="footer_social_icons"><style>.et-social-icons .et-social-icon img {
max-width: 16px;
transition: 0.2s all;
}
.et-social-icons .et-social-icon img:not(:hover) {
opacity: 0.5;
}</style><span itemscope itemtype="http://schema.org/Organization"><link itemprop="url" href="https://www.dynamicyield.com"><ul class="et-social-icons"><li class="et-social-icon"> <a itemprop="sameAs" href="https://www.linkedin.com/company/dynamic-yield" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame-5.png" alt="LinkedIn Logo" /> </a></li><li class="et-social-icon"> <a itemprop="sameAs" href="https://www.instagram.com/dynamicyield/" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame-4.png" alt="Instagram Logo" /> </a></li><li class="et-social-icon"> <a itemprop="sameAs" href="https://www.youtube.com/c/Dynamicyield" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame-3.png" alt="Youtube Logo" /> </a></li><li class="et-social-icon"> <a itemprop="sameAs" href="https://www.tiktok.com/@dynamic.yield" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame-2.png" alt="TikTok Logo" /> </a></li><li class="et-social-icon"> <a itemprop="sameAs" href="https://www.facebook.com/DynamicYield" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame-1.png" alt="Facebook Logo" /> </a></li><li class="et-social-icon"> <a itemprop="sameAs" href="https://twitter.com/DynamicYield" class="icon" target="_blank"> <img src="https://www.dynamicyield.com/wp-content/uploads/2025/09/Frame.png" alt="X Logo" /> </a></li></ul> </span></div></div></div></footer></div></div><div id="mobile_menu_overlay"></div> <script type="text/javascript">window.DY = window.DY || {};
DY.recommendationContext = { type: "POST", data: ['160220'], lng: "en_US" };</script> <script type='text/javascript' src='//cdn.dynamicyield.com/api/8765281/api_dynamic.js'></script> <script type='text/javascript' src='//cdn.dynamicyield.com/api/8765281/api_static.js'></script> <script type="text/javascript">var sbiajaxurl = "https://www.dynamicyield.com/wp-admin/admin-ajax.php";</script> <script type="text/javascript">!function(e,n){var r={"selectors":{"block":"pre.EnlighterJSRAW","inline":"code.EnlighterJSRAW"},"options":{"indent":2,"ampersandCleanup":true,"linehover":true,"rawcodeDbclick":false,"textOverflow":"break","linenumbers":true,"theme":"beyond","language":"generic","retainCssClasses":false,"collapse":false,"toolbarOuter":"","toolbarTop":"{BTN_RAW}{BTN_COPY}{BTN_WINDOW}{BTN_WEBSITE}","toolbarBottom":""},"resources":["https:\/\/www.dynamicyield.com\/wp-content\/plugins\/enlighter\/cache\/enlighterjs.min.css?tvrakvcLArVio6C","https:\/\/www.dynamicyield.com\/wp-content\/plugins\/enlighter\/resources\/enlighterjs\/enlighterjs.min.js"]},o=document.getElementsByTagName("head")[0],t=n&&(n.error||n.log)||function(){};e.EnlighterJSINIT=function(){!function(e,n){var r=0,l=null;function c(o){l=o,++r==e.length&&(!0,n(l))}e.forEach(function(e){switch(e.match(/\.([a-z]+)(?:[#?].*)?$/)[1]){case"js":var n=document.createElement("script");n.onload=function(){c(null)},n.onerror=c,n.src=e,n.async=!0,o.appendChild(n);break;case"css":var r=document.createElement("link");r.onload=function(){c(null)},r.onerror=c,r.rel="stylesheet",r.type="text/css",r.href=e,r.media="all",o.appendChild(r);break;default:t("Error: invalid file extension",e)}})}(r.resources,function(e){e?t("Error: failed to dynamically load EnlighterJS resources!",e):"undefined"!=typeof EnlighterJS?EnlighterJS.init(r.selectors.block,r.selectors.inline,r.options):t("Error: EnlighterJS resources not loaded yet!")})},(document.querySelector(r.selectors.block)||document.querySelector(r.selectors.inline))&&e.EnlighterJSINIT()}(window,console);</script><script type="text/javascript" id="rfw-script-js-extra">var rfw = {"speed":""};</script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_a5ec58aa1f222bedaf60cdced3b8a004.js?ver=2026020705" id="rfw-script-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_c1b7fbe6b1a3b777fddfe187094deb97.js?ver=2026020705" id="rfw-script-fitvid-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-includes/js/comment-reply.min.js?ver=935c7f8ce03960b02800bc8ae3231df8" id="comment-reply-js" async="async" data-wp-strategy="async"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_ea3fad8947382ebf8709507a11a805db.js" id="divi-fitvids-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/themes/Divi/js/waypoints.min.js" id="waypoints-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_845779ea879c70773af95b2f6fc394d2.js" id="magnific-popup-js"></script> <script type="text/javascript" id="divi-custom-script-js-extra">var et_custom = {"ajaxurl":"https:\/\/www.dynamicyield.com\/wp-admin\/admin-ajax.php","images_uri":"https:\/\/www.dynamicyield.com\/wp-content\/themes\/Divi\/images","et_load_nonce":"b06e7d0c21","subscription_failed":"Please, check the fields below to make sure you entered the correct information.","fill":"Fill","field":"field","invalid":"Invalid email","captcha":"Captcha"};</script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_43b4a88b2828bc5c023dc913689e8da5.js?ver=20.4" id="divi-custom-script-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/themes/Divi/custom/bootstrap/js/bootstrap.min.js" id="bootstrap-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/themes/Divi/custom/slick/slick.min.js" id="slick-js"></script> <script type="text/javascript" data-ot-ignore src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_12efd5a7d441d8b7f817fc6fc685dca9.js?ver=21.85" id="customscript-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_33fa37a5effe9361e40b8525e36e3eaf.js" id="typed-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_eb8981568f49ac148e96af3241eb95c7.js?ver=21.85" id="form_validation-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_be33bce741fd746dee1a6cf31c0b5bde.js" id="retina_js-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_9a50928b26148d1c35a95957e5ef2b9d.js?ver=21.85" id="mkto-scripts-js"></script> <script type="module" src="https://www.dynamicyield.com/wp-content/themes/Divi/js/cid-validation.js?ver=21.85" id="cid-validation"></script><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.5/js/select2.min.js" id="select2-js-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_6ee6e48031ef5dddfe05a9f11136201f.js" id="post-checklist-js-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_5113be2479418a077e699b1647e0adef.js" id="post-event-cards-js-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/plugins/wp-featherlight/js/wpFeatherlight.pkgd.min.js?ver=1.3.4" id="wp-featherlight-js"></script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_17babe2fc54117766e24bd50889c6577.js?ver=1772305539" id="article-script-js"></script> <script type="text/javascript" id="dc_conversion_api-js-extra">var ajax_obj = {"nonce":"4a0670c385","ajax_url":"https:\/\/www.dynamicyield.com\/wp-admin\/admin-ajax.php","ajax_action":"conversion_api_call"};</script> <script type="text/javascript" src="https://www.dynamicyield.com/wp-content/cache/autoptimize/js/autoptimize_single_34a6a3100a2ecbc7706af2d4c798f856.js?ver=935c7f8ce03960b02800bc8ae3231df8" id="dc_conversion_api-js"></script> <script type="text/javascript" data-noptimize>var dy_search_params = new URLSearchParams(window.location.search);
if(dy_search_params.has('utm_source')) {
localStorage.setItem('utm_source', dy_search_params.get('utm_source'));
}
if(dy_search_params.has('utm_medium')) {
localStorage.setItem('utm_medium', dy_search_params.get('utm_medium'));
}
if(dy_search_params.has('utm_content')) {
localStorage.setItem('utm_content', dy_search_params.get('utm_content'));
}
if(dy_search_params.has('utm_campaign')) {
localStorage.setItem('utm_campaign', dy_search_params.get('utm_campaign'));
}
if(dy_search_params.has('utm_term')) {
localStorage.setItem('utm_term', dy_search_params.get('utm_term'));
}
if(dy_search_params.has('mkt_tok')) {
localStorage.setItem('mkt_tok', dy_search_params.get('mkt_tok'));
}
// Also, fill in the data inside the Marketo form once its available
if(typeof window.MktoForms2 === 'object') {
MktoForms2.whenReady(function(_form) {
var form = _form.getFormElem();
console.log('MKTO FORM READY');
// Append the mkt_tok param if it exists in the URL
var ls_mkto_tok = localStorage.getItem('mkt_tok');
if(ls_mkto_tok) {
_form.addHiddenFields({
mkt_tok: ls_mkto_tok
});
_form.vals({ mkt_tok: ls_mkto_tok });
}
// Fill in the UTM data from localStorage
var dy_utms = {
'utm_source': 'txt_source__c',
'utm_medium': 'txt_medium__c',
'utm_content': 'txt_content__c',
'utm_campaign': 'txt_campaign_name__c',
'utm_term': 'txt_term__c',
'gclid': ['GCLID__c', 'GCLID_c__c']
};
for(var i = 0; i < Object.keys(dy_utms).length; i++) {
var ls_key = Object.keys(dy_utms)[i],
ls_value = localStorage.getItem(ls_key),
form_key = dy_utms[ls_key],
form_field;
if(ls_value) {
if(ls_key === 'gclid') {
ls_value = JSON.parse(ls_value).value;
}
if(typeof(form_key) === 'string') {
form_field = form[0].querySelector('input[type="hidden"][name="' + form_key + '"]');
if(form_field) {
form_field.value = ls_value;
}
} else {
for(var j = 0; j < form_key.length; j++) {
form_field = form[0].querySelector('input[type="hidden"][name="' + form_key[j] + '"]');
if(form_field) {
form_field.value = ls_value;
}
}
}
}
}
// IP Address
fetch('https://www.cloudflare.com/cdn-cgi/trace').then(function(result) {
result.text().then(function(values) {
values.split("\n").map(function(item) {
var pair = item.split('=');
if(['ip'].includes(pair[0])) {
form[0].querySelector('input[type="hidden"][name="IP__c"]').value = pair[1];
}
});
});
});
// GA ID and Referrer
var dy_utms_cookie = {};
document.cookie.split(';').forEach(function(el) {
var splitCookie = el.split('=');
var key = splitCookie[0].trim();
var value = splitCookie[1];
dy_utms_cookie[key] = value;
});
if(dy_utms_cookie["_ga"]) {
form[0].querySelector('input[type="hidden"][name="Visitor_ID__c"]').value = dy_utms_cookie["_ga"].substring(6);
}
if(dy_utms_cookie["personalize_referrer_string"]) {
form[0].querySelector('input[type="hidden"][name="Referral__c"]').value = dy_utms_cookie["personalize_referrer_string"].substring(6);
}
// ZoomInfo
if(localStorage.getItem('ZoomInfoData')) {
var zoominfo_data = JSON.parse(localStorage.getItem('ZoomInfoData'));
if(zoominfo_data) {
form[0].querySelector('input[type="hidden"][name="Alexa_Rank__c"]').value = zoominfo_data.alexaglobalrank;
form[0].querySelector('input[type="hidden"][name="Employees__c"]').value = zoominfo_data.employeesrange;
form[0].querySelector('input[type="hidden"][name="Company_Revenue_Range__c"]').value = zoominfo_data.estimatedannualrevenue;
}
}
// Also, dispatch a dataLayer event on success
_form.onSuccess(function(values, follow_up_url) {
window.dataLayer = window.dataLayer || [];
if([3739, 4802, 5094, 5110, 4946, 5076, 5116, 5171, 5172, 5173].includes(_form.getId())) {
window.dataLayer.push({
'event': 'demo_request_complete',
'eventCallback': function() {
if(follow_up_url) {
window.location.href = follow_up_url;
}
}
});
return false;
} else if([3876].includes(_form.getId())) {
window.dataLayer.push({
'event': 'newsletter_signup_complete',
'eventCallback': function() {
if(follow_up_url) {
window.location.href = follow_up_url;
}
}
});
return false;
}
});
});
}</script> <script>if(typeof(jQuery) !== 'undefined') {
$ = jQuery.noConflict();
}
function getUrlParameter(sParam) {
var sPageURL = decodeURIComponent(window.location.search.substring(1)),
sURLVariables = sPageURL.split('&'),
sParameterName,
i;
for (i = 0; i < sURLVariables.length; i++) {
sParameterName = sURLVariables[i].split('=');
if (sParameterName[0] === sParam) {
return sParameterName[1] === undefined ? true : sParameterName[1];
}
}
}
$(window).load(function(){
var videoID = getUrlParameter('video');
if(videoID !== undefined){
if($('#wistia-' + videoID + '-1').length){
var videoName = $('#wistia-' + videoID + '-1').parents('.tab-pane');
videoName = videoName[0].id
}
if($('a[href="#' + videoName +'"]').length){
$('a[href="#' + videoName +'"]').each(function(){
$(this).click();
});
}
addScript('https://fast.wistia.com/embed/medias/'+ videoID +'.jsonp');
addScript('https://fast.wistia.com/assets/external/E-v1.js');
$('body').append('<span class="wistia_embed wistia_async_'+videoID+' popover=true popoverContent=link dy-wistia-popup" style="display:inline"></span>');
window._wq = window._wq || [];
_wq.push({ id: videoID, onReady: function(video) {
var video = Wistia.api(videoID);
video.popover.show(); // open the popover
video.play(); // play the video
setTimeout(function(){video.unmute();}, 500);
}});
}
});</script> <style>#adBanner {
background-color: transparent;
height: 0;
width: 1px;
}</style><div id="wrapfabtest"><div id="adBanner"></div></div></body></html>