• Latest
Data Engineering for Product Experimentation

Data Engineering for Product Experimentation

December 24, 2022
Samsung builds a massive 17,400 sq ft centerfield display for the New York Mets stadium

Samsung builds a massive 17,400 sq ft centerfield display for the New York Mets stadium

March 29, 2023
Is It An MMO? #3 — Blue Protocol May Be Just An Action RPG, But It Sure Seems Like An MMO

Is It An MMO? #3 — Blue Protocol May Be Just An Action RPG, But It Sure Seems Like An MMO

March 29, 2023
Configuring Database Connection Parameters – DZone

Configuring Database Connection Parameters – DZone

March 29, 2023
Beats teams up with ‘Girls Don’t Cry’ for special edition Beats Flex earbuds

Beats teams up with ‘Girls Don’t Cry’ for special edition Beats Flex earbuds

March 29, 2023
Amplitude Studios blends genres to create the action-packed Endless Dungeon – PlayStation.Blog

Amplitude Studios blends genres to create the action-packed Endless Dungeon – PlayStation.Blog

March 29, 2023
Hellsinger update aims to banish achievement bugs

Hellsinger update aims to banish achievement bugs

March 29, 2023
Rapid Debugging With Proper Exception Handling

Rapid Debugging With Proper Exception Handling

March 29, 2023
M2 Pro Mac mini now $49 off in Wednesday’s best deals, Apple Watch Leather Link bands $52, more

M2 Pro Mac mini now $49 off in Wednesday’s best deals, Apple Watch Leather Link bands $52, more

March 29, 2023
Final Fantasy Pixel Remaster Pre-Orders Being Charged, Launch Date Reveal “Not Quite Ready”

Final Fantasy Pixel Remaster Pre-Orders Being Charged, Launch Date Reveal “Not Quite Ready”

March 29, 2023
A Big Adventure, Tails of Iron – PlayStation.Blog

A Big Adventure, Tails of Iron – PlayStation.Blog

March 29, 2023
Synchronized Method: BoyFriend Threads and GirlFriend Object [Video]

Synchronized Method: BoyFriend Threads and GirlFriend Object [Video]

March 29, 2023
Monster-taming RPG Cassette Beasts gets Game Pass launch date

Monster-taming RPG Cassette Beasts gets Game Pass launch date

March 29, 2023
Advertise with us
Wednesday, March 29, 2023
Bookmarks
  • Login
  • Register
GetUpdated
  • Game Updates
  • Mobile Gaming
  • Playstation News
  • Xbox News
  • Switch News
  • MMORPG
  • Game News
  • IGN
  • Retro Gaming
  • Tech News
  • Apple Updates
  • Jailbreak News
  • Mobile News
  • Software Development
  • Photography
  • Contact
No Result
View All Result
GetUpdated
No Result
View All Result
GetUpdated
No Result
View All Result
ADVERTISEMENT

Data Engineering for Product Experimentation

December 24, 2022
in Software Development
Reading Time:4 mins read
0 0
0
Share on FacebookShare on WhatsAppShare on Twitter


Data engineering is a broad field and is often used as a catch-all term to signify a variety of different works. Anything that involves ingestion, storage, processing, or serving of data can constitute data engineering, and the nature of work also varies meaningfully based on the domain of the data. In this article, we focus specifically on data engineering for supporting product experimentation which is rapidly developing to be a necessary core competency for all organizations that aim to be data-driven.

Simply put, experimentation data engineering is the process of designing, building, and maintaining systems and infrastructure for collecting, storing, and analyzing data from experiments.

Broad Components of an Experimentation Platform.

Broad Components of an Experimentation Platform.

The image above details the high-level components that are part of any mature Experimentation Platform. Each of these components generates data that needs to be ingested and managed effectively by the experimentation data engineering function. 

What’s Unique About Experimentation Data Engineering?

Experimentation Spans Multiple Domains 

Data engineering teams can do their best work when they understand the domain of their stakeholders and anticipate their needs effectively. 

In companies with a strong experimentation culture, experimentation is leveraged for all aspects of the business:

  1. Non-member / not-yet-customer conversion or acquisition experiments.
  2. Customer engagement and retention experiments.
  3. Algorithm experiments
  4. Outbound marketing experiments
  5. New partner or payment integration experiments.
  6. New business model experiments.

Each of these types of experiments has its own unique challenges since they are focused on very different domains with very different stakeholder sets. Further, the complexity and velocity of experimentation could vary significantly, requiring different operational support models. The excellent publication “Online Controlled Experiments and A/B Tests” gives an excellent overview of online experimentation for readers that are interested in diving deeper.

Experimentation Data Has a Variety of Functional Stakeholders

Further, experimentation data needs to support many different types of analyses aimed at different functions in the organization:

  • Reporting/Business Intelligence type Analyses: The ultimate goal of experiments is to understand the impact of some product or infrastructure change on some business KPI. This analysis is eventually consumed by business stakeholders like Product Managers and other executives.
  • Operational/Diagnostic Analyses:  Experiments, by definition, are new features driven by new code changes against a production “stable” experience. This means that experimental data can often be associated with bugs or other issues, which require an increased need for operational and diagnostic analyses to ensure the fidelity of the experiment. Further, the lifecycle of each experiment also needs to be maintained with appropriate metadata. These analyses are intended to be done by data scientists and engineers.
  • Scientific Analyses: Experiments are a method to perform causal inference on the effect of a change on a metric of interest. Causal inference is a scientific field of study that is increasingly becoming a high priority for organizations, much like Machine Learning is. For most basic experiments, while simple statistical hypothesis testing may be sufficient, increasingly, we are seeing the advent of complex techniques like CUPED and other model-driven causal effect estimation methods that need to be applied to experimental data. This requires a significantly higher level of data quality guarantees and further novel data system architectures to enable the computation of these novel statistics. Further, since this is an area of active research, experimentation data needs to be flexible enough to allow for a lot of ad-hoc analyses. The key stakeholders for this are actual scientists and statisticians.

Experimentation Data Requires a Platform-Thinking Mindset 

Given the variety of different stakeholders and use cases that experimentation data needs to support, to truly scale and enable organizations to become data-driven, experimentation data engineering teams need to think of themselves as creating a platform product, i.e., focus on the building blocks and capabilities that are core to any experimentation setting and enable the customers of the platform to mix and match and extend as necessary. 

Recommendations for Creating a Strong Experimentation Data Engineering Team

  • Focus on Self-Service and Enablement: Without this approach, experimentation data engineering teams will likely start drowning in support requests
  • Invest in foundational data quality tooling and processes: Errors or inconsistencies in the data can have significant impacts on the validity and reliability of the experiment results, and problems compound if not fixed early.
  • Build strong relationships on all sides: Software engineering teams that produce the data, data science teams that consume the data, and ultimately product and business teams that make decisions on the recommendations based on the data. Treat every one of these partners as equal stakeholders and build proactive relationships. Data engineering teams often treat only the Data Scientists as their stakeholders, which may not always be sufficient. 
  • Always think in terms of building blocks, reusability, and APIs.

Conclusion

The field of data engineering and the practice of experimentation as a technical capability are both rapidly evolving. It is clear that experimentation is a crucial aspect of data management for all organizations, along with business intelligence and reporting and machine learning. To this extent, we are also seeing a rapid increase in the number of companies being developed around providing easier experimentation capabilities as a service, and the concept of an experimentation platform is emerging as a core infrastructural component for technology companies.



Source link

ShareSendTweet
Previous Post

YSL OBAMA TALES A GUILTY PLEA IN THE YSL RICO CASE. WACK 100 SPEAKS ON IT. WACK 100 CLUBHOUSE

Next Post

The EASIEST way to self pop your lower back #shorts #back #pop #stretch #health

Related Posts

Configuring Database Connection Parameters – DZone

March 29, 2023
0
0
Configuring Database Connection Parameters – DZone
Software Development

This article shows how we can configure database connection parameters dynamically in WSO2 EI 7.1.0. This is useful when we...

Read more

Rapid Debugging With Proper Exception Handling

March 29, 2023
0
0
Rapid Debugging With Proper Exception Handling
Software Development

As a developer, often the most irritating part of development is debugging. Most programmers I have worked with would echo...

Read more
Next Post
The EASIEST way to self pop your lower back #shorts #back #pop #stretch #health

The EASIEST way to self pop your lower back #shorts #back #pop #stretch #health

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

© 2021 GetUpdated – MW.

  • About
  • Advertise
  • Privacy & Policy
  • Terms & Conditions
  • Contact

No Result
View All Result
  • Game Updates
  • Mobile Gaming
  • Playstation News
  • Xbox News
  • Switch News
  • MMORPG
  • Game News
  • IGN
  • Retro Gaming
  • Tech News
  • Apple Updates
  • Jailbreak News
  • Mobile News
  • Software Development
  • Photography
  • Contact

Welcome Back!

Login to your account below

Forgotten Password? Sign Up

Create New Account!

Fill the forms bellow to register

All fields are required. Log In

Retrieve your password

Please enter your username or email address to reset your password.

Log In
Are you sure want to unlock this post?
Unlock left : 0
Are you sure want to cancel subscription?