Discover the "Future Cost Per Conversion" for Brand Advertising! The Impact of Data Clean Rooms × Machine Learning

Video ads are essential for branding. Yet quantifying their business contribution has been difficult.

Shiraishi Shintaro, Automobile Fire Insurance

──Could you start by outlining this initiative?

Sugawara: This case involves introducing the "Nurturing Score," developed by Dentsu Inc. using data clean rooms and machine learning, into Saison Automobile & Fire Insurance's media planning.

This score enables real-time estimation of "mid-to-long-term business outcomes"—such as those confirmed after six months—for branding video ads, which were previously difficult to evaluate as mid-to-long-term initiatives.

Press Release:
Develops New Metric "Nurturing Score" to Evaluate Mid-to-Long-Term Effects of Digital Advertising
 

 

Brand advertising via TV and video, where conversion measurement was difficult. The Nurturing Score enables measuring the effectiveness of these ads using the same metrics as retargeting ads.

──What challenges prompted its introduction?

Shiraishi: Broadly speaking, the auto insurance market is divided into "agency-based" sales, where sales representatives sell face-to-face, and "direct-based" sales, where consumers complete all procedures online themselves. The "Adult Auto Insurance" we sell is direct-based.

The market share is approximately 90% for agency-based and 10% for direct-to-consumer. For us, targeting switches from the larger agency-based segment has the greatest business impact.

However, auto insurance isn't something people can purchase anytime. They typically sign up either during their annual renewal or when buying a car. Therefore, if someone with "more than six months left on their policy" sees our ad, they can't immediately sign a new contract, so it doesn't lead to direct acquisition.

For those "actively considering applying right now," we pursue efficiency through digital measures like retargeting ads. However, research shows that even during this comparison phase, "the brand image the user already holds" significantly influences acquisition.

Therefore, our digital advertising budget has been allocated primarily across the following two categories:

● Upper Funnel (Top Funnel / Middle Funnel)
→ [Awareness-focused] Video ads for branding aimed at people who are still far from renewal
 

● Bottom Funnel
→ [Acquisition-focused] Static image ads like search ads and retargeting ads targeting those actively considering a decision

Sugawara: So, engaging in brand communication with potential customers before they actively start reviewing options—to boost awareness and recall—ultimately leads to consideration and acquisition, right?

Auto insurance has a relatively long contract cycle, making prior branding through TV and video ads crucial. However, the challenge was measuring "how much retargeting ads and search ads during the comparison/application consideration phase actually contributed to acquiring new contracts."

Shiraishi: Here's the problem: unlike "search ads" or "retargeting ads," which measure the path from ad exposure to new contract acquisition in real time, revealing their direct business contribution, the contribution of brand ads to awareness and recall to business results often isn't clear until much later—like six months down the line.

Sugawara: For measuring digital ad effectiveness, we have the "CPA (Cost Per Acquisition)" metric visible in daily management dashboards. From a CPA perspective, video ads targeting customers whose contract renewals are still far off are overwhelmingly disadvantageous compared to retargeting ads targeting those currently considering a purchase.

Nevertheless, we've measured the business contribution of upper-funnel initiatives using alternative methods, such as conducting attribution analysis or developing metrics like the True Lift Model(※) using data clean rooms. However, these are still analyses based on short-term performance metrics and cannot evaluate "future results" that remain uncertain until time passes.

※True Lift Model
Conversions among ad exposed users include "users who would have converted even without ad exposure." The True Lift Model separates these users from "users who converted purely due to ad exposure," enabling measurement of the true lift effect.

 

Sugawara: While no company underestimates branding, when allocating advertising budgets, they demand concrete numerical explanations and evidence like "We spent this much budget and achieved these results." Consequently, we consistently felt the need to measure brand advertising, such as video ads, using the same metrics as search ads or retargeting ads, and to do so in real-time daily.

This challenge applies not only to auto insurance but to many products when implementing strategies to increase brand mindshare. That's why we developed the Nurturing Score.

Making the impossible possible!? What unexpected facts emerged from the Nurturing Score, which "measures future results"?

Dentsu Inc., Kota Sugawara

──So the Nurturing Score is the metric used to measure the business contribution of video ads for branding, placing them on the same playing field as retargeting ads.

Sugawara: Traditionally, management dashboards only showed "short-term ad effectiveness"—like 7-day results—which could be reflected in media operations and PDCA cycles. When measuring effectiveness over those 7 days, if a video ad's CPA was ¥10,000, retargeting might acquire a conversion for just ¥500—a massive difference.

However, the awareness and recall gained through video ads could potentially lead to new contracts six months or a year later. The CPA visible in the management console is just the tip of the iceberg. We believe the true value changes when viewed over the medium to long term. That was the key development focus.

The cost required to acquire a single conversion is the "CPA (Cost Per Acquisition)". While we were measuring actual user behavior in the short term, this metric showed retargeting and search ads as overwhelmingly advantageous.

──But measuring future CVR (conversion rate) or CPA six months or a year ahead seems difficult by common sense.

Sugawara: What made this possible were the "data clean room" and "machine learning." The data clean room allows us to analyze various attribute data from major platform users and connect it to ad delivery, all within a privacy-protected environment. Additionally, Saison has accumulated several years' worth of data on past ad delivery results. We used this vast historical data for machine learning to build a "predictive model" that derives the relationship between nurturing scores and CVR.

Building this predictive model in a tailor-made way, closely aligned with each client's specific KPIs, was impossible using the standard AdsManager (ad submission and management interface). It was only achievable because of the Data Clean Room.

To build the model, we set conversion (i.e., acquiring new contracts) as the "dependent variable" and performed regression analysis based on "independent variables" such as platform users' ad exposure history, search history, browsing history, interest attributes, demographics, and location. This created a detailed predictive model identifying "the specific attributes of people who will actually convert over the long term."

Applying this predictive model to "users currently exposed to ads" enables real-time, highly accurate calculation of each individual's "future conversion rate." This is the approach of the Nurturing Score. This new metric allows simultaneous evaluation of video ads and retargeting ads across the funnel, at the same time and in the same moment.

Within the data clean room, we built a predictive model that uses various user data as "explanatory variables" to measure scores against the "target variable" of conversion.

A predictive model built using machine learning based on the platform's vast data. In the proof-of-concept experiment, it demonstrated extremely high accuracy, achieving over 70% in cross-validation.

──So, the predictive model can measure the "medium-to-long-term CVR and CPA" for every single user exposed to the video ad, simultaneously with the ad delivery.

Sugawara: The predictive model's accuracy has undergone rigorous statistical validation. From the perspective of predicting the "medium-to-long-term purchase probability for the entire target audience," it maintains high precision suitable for evaluating ad effectiveness. This allows us to clearly explain "how effective" video ads were, on the same timeframe as retargeting ads. As a result, we've actually seen cases where video ads demonstrate superior CPA when viewed in terms of medium-to-long-term outcomes.

When comparing the CPA of video ads and retargeting ads over the medium to long term using the nurturing score, there were cases where video ads showed greater acquisition efficiency.

Shiraishi: I was surprised by the finding that "mid-to-long-term video ads can actually offer better acquisition cost efficiency." There was always a lingering uncertainty about why video ads seemed less efficient than retargeting. Now that we can clearly demonstrate video ads' business contribution, we can allocate budgets with greater confidence.

For video ads, we utilize menus like "TrueView," "Bumper," and "VAC (Video Action Campaign)" on YouTube. Among these, the VAC strategy had the most significant impact this time.

<Google's Video Ad Menu>

・TrueView = Video ads (6+ seconds) that play within YouTube video content. Skippable.

・Bumper = Video ads (under 6 seconds) that play within YouTube video content. Non-skippable.

・VAC = Shorter video ads focused on targeting and conversion acquisition. Often include functionality like directing to landing pages.
&nbsp;

Historically, when measuring "short-term effectiveness," static image retargeting ads were overwhelmingly superior in CPA, while video ads were generally considered inefficient. However, we can now demonstrate with data that video ads deliver higher CPA in the medium to long term.

Shiraishi: Since VAC is a menu with high conversion potential, we managed it with a slightly different purpose among video ads, focusing on "short-term conversion tracking." However, for that purpose, targeting and retargeting ads were far more efficient at acquisition, making it challenging to determine how to allocate budgets to VAC going forward.

The introduction of the nurturing score revealed that "VAC holds potential for mid-to-long-term effectiveness." This allowed us to make the decision to increase the budget for VAC by switching the objective to the upper funnel: .

The nurturing score can solve various marketing challenges

──Is this predictive model strictly tailored to "Otona no Jidosha Hoken"?

Sugawara: Yes. In this case, Saison Automobile & Fire Insurance was planning media around Google products digitally, in addition to TV. So we built the predictive model using the data clean room "Google Ads Data Hub."

Dentsu Group analysts accessed Google Ads Data Hub to analyze attribute data serving as explanatory variables in a non-personally identifiable manner.

Dentsu Group has been actively utilizing data clean rooms since 2016, with over 2,300 projects already utilizing this service. Analyses combining data clean rooms with "machine learning," like this one, are rare even internationally, and the development of the nurturing score leverages this capability.

──This time it was Google, but does the specific data used as explanatory variables change depending on the data clean room being utilized?

Sugawara: Yes. The available data differs by platform. For example, Yahoo! offers unique access to "search data" and "Yahoo! Shopping purchase data." Shiraishi, do you have any future aspirations for how you'd like to utilize the nurturing score?

Shiraishi: Since we create media plans using Google products, being able to view "awareness ads" and "acquisition ads" together in one place is incredibly useful. In the future, it would be fantastic if we could integrate all media, including TV, compare nurturing scores, and plan accordingly.

Sugawara: Dentsu Inc. also offers "STADIA," a digital ad delivery and effectiveness verification service based on actual TV viewing data. We aim to calculate the medium-to-long-term effects for users who have been exposed to TV commercials.

Additionally, we have established "TOBIRAS," a business system that supports the centralized management of analytics operations across multiple data clean rooms with differing specifications and data. We want to promote more cross-functional utilization of the nurturing score.

Shiraishi: Another point is that when explaining budget allocations internally, we're often asked, "How much would conversions drop if we stopped this advertising initiative?" – essentially, calculating the impact of discontinuation. Since nurturing scores measure medium-to-long-term "future" figures, we want to leverage them as supporting material for this explanation.

Sugawara: Understanding that provides strong logic for continuing investment. We anticipated this kind of application when we developed the model.

Shiraishi: From a nurturing perspective, we also want to know the number and characteristics of "users with high nurturing scores who haven't been exposed to ads."

Sugawara: On that point, major platforms can analyze tens of millions or even hundreds of millions of users at the individual ID level. By applying predictive models not only to users who actually engaged with ads but also to those before ad exposure, we can visualize the entire market landscape – essentially showing "how many high-nurturing-score individuals exist on this platform."

By treating everyone before they see ads as "potential customers," we can approach the entire market as one large prospect list, enhancing the ROI of nurturing and branding efforts.

Furthermore, we can segment users by their "conversion likelihood" and understand "how much reach we currently have with this segment and how much room for growth remains." We can also conduct fixed-point observations to track how much users within that segment move before and after new brand communications or campaigns.

Shiraishi: In the auto insurance market, 90% are agency-based and 10% direct. This 10% direct segment is highly accustomed to online insurance and includes people who switch insurers annually. However, if we only target these users because their CPA is favorable, we won't reach agency-based users.

Therefore, it would be very helpful if we could adjust scores based on attributes, such as "agency-based and also high-scoring," rather than just targeting high-score users. Since they likely exhibit distinctly different search behaviors, we should see a difference.

Sugawara: Since web browsing behavior differs between agency-type and direct-type users, we could likely account for that difference as a variable. If we can integrate first-party data with user consent, it should be possible.

Shiraishi: If we could assign "colors" to the likelihood of conversion this way, it would also allow us to incorporate sub-KPIs like "Are we effectively delivering to this segment via this media?" when setting KPIs. This would likely lead to a more precise design for the perception flow too(※).

※Perception Flow
= Unlike the customer journey centered on consumer actions, this concept focuses on the transformation of consumers' perceptions. It is used as an overall blueprint for marketing.

&nbsp;

──MMM is also gaining attention as a method for predicting future advertising effectiveness based on historical data. How would you differentiate the roles of MMM and nurturing scores?

Sugawara: MMM is a purely statistical approach, not machine learning. Since its strengths and objectives differ from nurturing scores, I don't see them as competing; rather, they should be used appropriately depending on the situation.

The Nurturing Score can be measured at the "user level," so it's intended for weekly or monthly reviews to adjust operations, allocate budgets, and update media plans with nurturing as the goal. MMM, on the other hand, focuses on how to update and run PDCA cycles for cross-media budget optimization every half-year or annually.

We see the Nurturing Score as a bridge metric, enabling the use of MMM for long-term PDCA cycles and short-term PDCA based on daily CPA.

Shiraishi: Beyond utilizing the nurturing score, we also use MMM on cycles like six months or a year to conduct a "health check" on whether the planning is statistically optimal. By driving both wheels, we aim to update our media planning with stronger evidence.

Sugawara: Data clean rooms combined with machine learning offer significant potential for various applications. We hope they will be utilized as a foundation for solving clients' actual marketing challenges, as demonstrated in this case study. Thank you for joining us today!