Recognising and Building the Optimal Customer Experience
Back to Blog

Recognising and Building the Optimal Customer Experience

  • Ben Healy
  • 22 min read

Over the past several years, more and more teams in Enterprise Data Management have been tasked with a succinct need from the C-Suite:

We need to clean up our data and utilise it as much as possible to ensure better business outcomes.

On the surface this seems like a no-brainer, but leads to two questions:

Why is this so important all of the sudden?

How do we go about this?

What follows is an attempt to answer the industry trends driving this demand, and provide a high level overview of the concepts and best practices involved.

Why now?

Regardless of industry, the dream of any enterprise collecting data around their engagements with customers and prospects has been one thing:

To set our business apart, I want to understand every way in which we engage with customers/prospects, and use that information to give them the best experience possible.

As a consumer this seems rather intuitive. I’m most loyal to brands that give me the most value with their products or services, and provide the least friction in the process. Airlines make the travel process easiest (and the most enjoyable) for their most loyal customers. Retailers offer economic benefits or exclusive products/services if they feel I’ll be a repeat buyer, or in order to get me in the door for the first time.

But from a business perspective, delivering the best experience to customers has only gotten more and more difficult.

Sure, if I grab coffee at the same local cafe most mornings, the barista may get my order started before I reach the counter, throw in a pastry for free, or even have a more cordial conversation with me than less frequent visitors. It’s relatively easy for them to make my mornings pleasant for one key reason: they recognise me and likely only interact with me in the one medium where they are familiar with me – in-store. Personally I think this is one of several reasons why most people prefer spending their money at local businesses.

Comparing this to a global enterprise, the variables in customer experience expand drastically.  A global retailer interacts with exponentially more customers, via far more employees, and across a multitude of mediums, with all of this producing a lot more data against these interactions. This makes personalisation all the more difficult.

Today’s technology landscape presents a massive opportunity to engage with customers like never before. Studies by BCG and Google in 2023 show that companies provide anywhere from 20 to 500 customer interactions in the buying journey, a drastic increase from an average of nine in 2014*.

With that opportunity comes an even bigger challenge: using data to make sure it is done right.  The data behind these interactions comes from a countless number of sources, very few of which tell you the story in a consistent manner. Through innovations in cloud computing and software applications, and with the evolving practice of enterprise data management, many companies have taken the first step towards optimising customer experience, which is getting all available data centralised in one place. The bigger step is what lies ahead: making those sources of data effectively talk to each other, and using what you hear to make that next interaction as relevant and meaningful as possible.

Before we tackle how this can be done, let’s take time to understand the two primary channel categories by which companies recognise and interact with consumers and how the nuances of how that data is captured in each channel.

Marketing and customer success: Owned and operated channel engagements

The majority of effective customer interactions most often lie within a company’s own properties, both physical and digital. Marketing and customer service within your owned channels are arguably the best places to cater the customer experience. If I host a party, there’s a lot more control and ability to cater the experience in my own home than if I chose an external location.

Owned channels have evolved drastically over the past decade. Through evolving technologies, this has moved from a handful of mediums such as:

  • Brick and Mortar locations (i.e. in-person)
  • Website
  • Call Centre/Customer Service
  • Email
  • Direct Mail
  • SMS
  • In-App

To a much larger swath of channels, including:

  • Mobile website/applications
  • Chatbots
  • In-store digital experiences
  • IoT Applications
  • Increased and more personalised Newsletter Programs
  • QR Codes
  • Microsites and ancillary digital properties

All of these channels provide some sort of data back to enterprises on how the interaction went, but as mentioned above those data sets all come in their own unique flavours (ex. event logs, website/app analytics, customer satisfaction surveys, etc.).

Gaining a comprehensive view of all interactions with a single customer through these sources, often referred to as “Customer 360,” is a key objective of most enterprises today. In 2022, 82% of respondents in a Gartner survey listed this as a primary goal for their business, however only 14% of respondents felt they’d actually achieved this full view.  A proliferation of technology solutions have emerged to solve this very problem (more on this later), but challenges still exist in effectively recognising the right customers across these channels.

Advertising: Engaging with customers and prospects outside of your four walls

The second category of interaction channels is where things can get complex. Advertising continues to evolve at a rapid pace, primarily due to the boom of digital channels that have emerged and quickly dominated consumer share of attention over recent years. A media or advertising plan in the early 2000’s would likely contain a mix of channels such as:

  • Out of Home
  • Print (Newspaper and Magazine)
  • TV
  • Digital

But with the rise of smartphones and internet connected devices, the media mix has come to include far more categories such as:

  • Premium Digital (ad buys directly with household name web publishers)
  • Programmatic (wide-spread digital buys across any and all relevant web inventory)
  • Mobile web and Mobile Apps
  • Connected Television

The rise of digital inventory came with a variety of solutions to ensure messages to customers were relevant and encouraged another interaction/purchase with your company. Tools like retargeting pixels and Data Management Platforms (DMP) enabled businesses to track and categorise their digital interactions with customers, and re-engage them with the most relevant offerings moving forward. Digital also opened a door for other parties to provide businesses information on consumer interests and behaviours outside of what they may have gained in their interactions, primarily through third-party data offerings, which prompted more consumer privacy regulations.

Recognition and digital privacy

Privacy legislation is not a new concept. The primary focus of current legislation is to inform consumers of their privacy rights in the digital space, namely the General Data Protection Regulation (GDPR) which went into effect in the EU in 2018, as well as various state-specific privacy laws in the US, starting with the California Consumer Privacy Act (CCPA) in 2020.  Even the US Constitution alludes to the right to privacy within the First, Third, Fourth, and Fifth Amendments. More recent history has brought legislation has also been focused on individual rights as it relates to healthcare (HIPPA in 1996), minors (COPPA in 1998), and financial data (Gramm Leach Bliley in 1999).

Third-party cookies have historically been fuel for customer insights and recognition in the digital space; but today half of the open web is cookieless and companies are driving better business results without them. For example, hospitality leader Omni Hotels & Resorts saw four times more advertising effectiveness by leveraging signal-less solutions from Google’s Display & Video 360 and LiveRamp to build deeper relationships with their customers.

Recognition in a new era

As channels for interactions have increased, and data collection practices from these channels have shifted, new categories of solutions have risen to the challenge. What follows is an overview of these solutions and thoughts on best practices.

Identity and entity resolution

The process of “cleaning up data” and building a “Customer 360” view can be best summed up with the concepts of identity and entity resolution. These two concepts are often confused, and I find it best to think of them in a square-rectangle fashion. More specifically:

Entity resolution (rectangle) is the overarching concept in the practice of data management, whereby a business attempts to unify all of their disparate records (ie customer interactions), within a single channel or across multiple, and tie them to a single entity. An “entity” could mean many things, such as a business/account, household, or other depending on the business’ objectives.  Entity resolution can help a business answer questions such as:

  • Where do we see multiple customers in the same household?  Should we adjust our messaging based on this, or limit our engagements to the key buyer in the house to avoid waste/over inundation?
  • For B2B businesses: What accounts are most or least engaged, and how can we cater our messages to fit the specific needs of those companies?

Identity resolution (square) is the specific type of entity resolution by which a business attempts to tie disparate records to a single individual or household.

The latter is the most common (and difficult) goal in enterprise data management today.  In an ideal state, this is how I could recognise one customer across all channels, and thus have the most relevant interactions possible.

If I work in a call centre and my company is practicing identity resolution, they could look up the phone number in an inbound call against our database and provide me with a profile of everything we know about that person before I even pick up the phone.  That way I may know the specific order they’re checking in on, or the product they’ve purchased and are having an issue with, without the customer having to tell me.

If I work in marketing and my company practices identity resolution, I could customise the email I send customers to exclude products they already have, or offer promotions/discounts to ones they’ve specifically shown interest in in the past.

If I work in business intelligence, identity resolution could help me better analyse and showcase the optimal customer journey, meaning what number of customer interactions, across which channels, and in what order produce the most sales or conversions for my company.

Both identity and entity resolution open up a wealth of opportunities for businesses, agnostic of business function or department.  The underlying processes can be incredibly complex and should very much be customised to a businesses’ needs, but the desired outcomes are relatively consistent: a single profile of an individual or entity, which encapsulates all engagements they’ve had with your business, tied to a single ID (Customer/Account ID) in your database.

Resolution tools

To accomplish a full view of the customer through identity resolution practices, three primary categories of tools have emerged in the market.

Customer data platforms (CDPs)

CDPs come in a variety of shapes and sizes, but their providers promote a few primary features:

  • Data unification and profile creation: This is where entity and identity resolution play a key role.  CDPs offer features that give users the ability to bring in all relevant datasets, build relationships between them, and use matching rules to create unified profiles consisting of all interactions with a single customer or overarching entity.
  • Profile segmentation, analysis, and journey orchestration: Following the above work, CDPs then provide users with tools which allow them to analyse those profiles to determine where the best interactions occur, and how a business can drive more of them to put as many customers in the “optimal journey” with their business as possible.  They also offer the ability to bucket or segment users based on similar behaviours/interactions.
  • Actioning on insights: Lastly, CDPs provide users with the ability to take these profiles or “Golden Records” and the associated insights against them, and actually take action on your learnings. This most often manifests as built in messaging tools or integrations to external partners across the marketing (ex email/SMS platforms) and advertising (ex social, programmatic, CTV) ecosystems.

Standalone identity resolution/graph builder tools 

Outside of CDPs, there also exist many providers who offer solutions needed to build unified profiles as a standalone.  These solutions can be thought of as a “rules engine”, whereby users can map in all of the relevant data sets for their business into a single environment, then build rules to unify and/or dedupe profiles based on what common keys sit within each dataset.  These “solutions” may come as a simple UI or API that can sit atop a business’ own data warehouse or data infrastructure, or as a more managed service offering where a provider brings along a team of experts to help build to unified profiles (either in the provider’s environment or your own).

Enrichment beyond first-party data

While collecting and unifying all of your interactions with customers into a single view is the ideal state for most businesses today, there exists within this process two common limitations often faced.

Issue #1: Inability to unify data sets

Entity and identity resolution tools rely on one critical piece in the process of unifying data sets:  A common key that can be used to link the data. This is done via strict (exact) matches across the keys, as well as fuzzy matching which uses a more probabilistic method (ex. A record with “Benjamin Healy” could use a fuzzy match to link to another record with “Ben Healy”).

The big question here is: what happens if there’s no common key between data sets, or I’m not confident in a more probabilistic approach? Let’s use an example to illustrate:

Company A is an e-commerce retailer. They’re using an identity resolution tool to clean up their data across sources, namely their shipping/transactions and registered users.

One record from their shipping/transactions includes the below information from a 2020 purchase:

  FirstName      Last Name      Address     Email   Product            Date
  Benjamin        Healy   123 Main St.    [email protected]          Shoes   March 1, 2020    

Another record from their registration database includes the below information from a recent signup:

First Name       Last Name              Email          Date
    Ben        Healy     [email protected]      February 3rd, 2024

Company A wants to tie any purchases a customer made prior to signing up to their profile, but in the above example there’s no real common key across both records.  Sure, they could use fuzzy matching to tie the name fields together, but is that the best approach?  After all, there are a lot of Ben Healy’s out there (I frequently trade emails with one who has a similar Gmail address).

Issue #2: Lack of first-party data to build a better experience

I often recognise my neighbours around New York, but without a meaningful interaction between us those moments likely won’t go past a simple hello or awkward half-smile. I could certainly stop them on the street and ask them 21 questions, but if you know anything about New York culture they’d probably only remember me as the overly-friendly guy or the one who made them late to work. Oftentimes it takes a shared contact to break the ice in these situations or learn more about them. My wife may have told me about a brief conversation with them in our communal laundry room, or my building super mentioned they’re renovating their kitchen.  Small insights like this can enable a more meaningful interaction in the future, without someone feeling like their privacy has been breached or as if I’m conducting a survey to find common ground.

This same concept applies in business and customer interactions. At the end of the day we’re all interacting with businesses for a specific purpose, and anything outside of this can feel like oversharing or simply a waste of time. We’ve all probably been on a customer service call where the agent may be a little too chatty, and it feels like the conversation is getting in the way of the answer we came for.

Enrichment with contact and attribute data

Let’s look at the concept of profile enrichment.

Contact enrichment providers offer more PII touchpoints which can be applied to your profiles.  Let’s say my company does not often require emails to be provided in customer interactions, but we want to start doing more email marketing to promote personalised offers. Aside from changing how we interact with customers to require they give us their email, we could work with an enrichment provider to connect emails in their data sources to our existing profiles.

Attribute enrichment providers offer more details on consumer demographics, interests, or behaviours to be applied to your profiles. A business may feel they have a comprehensive view of their interactions with a single customer, however this is far from a full view of that person.  Let’s say my company sells moderately-priced clothing, but wants to branch into more luxury/high-end apparel as well. By working with an enrichment provider, my company could learn more about our existing customers’ outside of how we’ve interacted with them to date. A provider may be able to tell me more about their financial status (ex: credit score or household income), or perhaps their interest/purchase behaviours with other brands in the luxury space (ex: Often visits luxury boutiques or frequent buyers in the luxury apparel category). Once appended to my existing profiles, I can use these additional attributes to determine which customers might be best to introduce to this new offering as we test the waters.

When evaluating enrichment providers it is important to understand how they source the data they offer, and more specifically how they have permissible rights to offer this data to their partners and ensure end consumers have the ability to opt out of these databases as needed.

Second-party data and data clean rooms

In the past few years, marketers have seen the overwhelming benefits of data collaboration. I feel this has surfaced as a byproduct of a series of tailwinds and headwinds:

Headwinds

  • Unified customer views, with limited profile content: Businesses are using EDM practices and technology solutions to build a better view of customer interactions.  Along this road they’re wondering “how do I get more information on consumers outside of what we gain from our interactions?” Rather than working with an enrichment partner, they’re looking for ways to work with new or existing business partners to learn more about their customers. For example, an airline may want to expand their partnership with a credit card provider for better co-marketing efforts.  A high-end automotive manufacturer may want to work with a luxury retailer to learn more about mutual customers, using real data to go beyond their anecdotal assumption that their brands have strong synergies. The possibilities are endless.
  • New revenue sources through data as a service (DaaS): Many businesses are realising in their EDM journeys that they have a wealth of data, and are exploring ways they can offer the data itself as a product/service to their partners. A grocer may want to offer a CPG brand deeper insights into what kinds of purchases are made in their stores.  A movie theater may want to offer production studios deeper insights around ticket purchases.  The CPG/retail industries were the first to realise the potential value data sharing could unlock, and thus a proliferation of retail media networks have surfaced. However many other industries are realising they can utilise a similar model of data sharing to unlock mutual value within their networks.

Tailwinds

  • Signal loss in advertising: The reality is an information gap exists when it comes to prospecting or gaining new customers. This is where second-party relationships will play a key role in data collaboration. Rather than simply selecting an off-the-shelf audience of third-party data, businesses are taking a more concentrated approach by exploring with other businesses how they can securely share data on potential prospects. If two companies feel they have synergies across their brands, second-party data collaboration and co-marketing efforts can help both parties to interact with new prospects from the other business’ existing customer base in a much more relevant manner.
  • Increased Data Protections: With increased investment in first-party data has come a more concerted focus around how to ensure this data stays protected. While stakeholders want to utilise or monetise their shiny new data assets, they want to do so in a way which does not betray consumer trust or get them in the wrong headlines.  This is especially applicable among media inventory providers, particularly the walled gardens.  These businesses hold a wealth of data about users of their platforms, and must find a way to securely provide insights on users to their advertiser partners to justify future investments.  For all parties involved, utilising data for better business outcomes, while also limiting their attack surface in the process, has become critical.

Altogether, the above themes have resulted in a growing interest in data clean rooms.  At its base level, data clean rooms provide a neutral space whereby two or more parties can share data in a configurable, “least privilege” manner.  Many clean rooms also focus on the concept of “zero data movement”, meaning neither party necessarily has to explicitly send their data outside of their own walls to collaborate.

Clean room offerings have risen in a variety of places, namely:

  • Cloud native environments: Cloud providers such as Snowflake, AWS, GCP, Azure, and Databricks have built clean room solutions within their environments, allowing parties who have a shared presence within the provider’s cloud ecosystem to securely share data.
  • Independent clean rooms: LiveRamp and InfoSum are examples of a few providers who offer clean rooms independent of any cloud. These solutions often take a “fully neutral” stance, meaning they do not require a business to utilise a particular cloud, or for the business’ potential collaboration partners to be on the same cloud. These providers also offer add-on solutions to set themselves apart, including identity layers (share data on a neutral ID space rather than PII), analytics/visualisation tools, and connection to media execution platforms to name a few.
  • CDPs: CDPs have also begun to offer clean room extensions of their base offerings, with the idea that two or more businesses using their CDPs to build their “Customer 360” can also utilise the CDP to share the product of this work securely with other parties.
  • Walled garden clean rooms: Lastly, many of the walled garden media inventory providers have built their own clean rooms. This is in an effort to provide advertisers with meaningful insights on campaign performance to justify future campaigns, while also keeping their intellectual property (rich user data) secure.

While data clean rooms are a relatively new category, their application continues to grow as market leaders showcase ROI against new business avenues. Though they may not directly handle identity/entity resolution needs, they sit as a clear answer for the next step for businesses who want to further enrich or monetise their optimised customer interactions.

Considerations

With a wealth of options available in the market, deciding how to best move forward with a data collaboration strategy for your business can be a bit overwhelming (to say the least).  What follows are some points of consideration when determining what tools or solutions are the best investment for your business.

Recognition, resolution, and privacy limitations

CDPs in particular promise the dream of all relevant data sitting in one place, and tied to a single identifier.  While this is feasible across most channels, the reality is that in practice some channels require logical separation from others.  While this may initially be interpreted as a limitation when working with one vendor or provider, in truth this is more a byproduct of today’s global privacy landscape.  This is most relevant as it pertains to customer interactions across advertising channels.

When working with customers I often find it best to relay this in the context of a consumer.  At the end of the day we’re all consumers of one business or another, and framing from this point of view helps to better exemplify why certain practices are simply not feasible.

Tying known interactions to pseudonymous ones at an individual person level is not a new practice. It is at the core of measuring media and used to attribute real-world sales or conversions to media executions in order to justify future investments in advertising.  Players in the advertising industry offer solutions to measure this in a variety of ways, such as a platform like Meta or Google offering a conversion API to upload sales data to be tied to exposures/actions in their platform, or independent measurement providers using a common identity space to attribute row level media exposures to individual transactions.

The key piece here is that translating data from a known space to a pseudonymous one is a one-way street.  This ensures that trust is maintained and that businesses do not “reverse engineer” data from outside sources to reveal more about me to the business than I have agreed to.  The advertising industry ensures these guardrails through a couple key concepts:

  • Pseudonymisation: Use of an identity graph and linkages between channels to build a pseudonymous representation of individuals or households. This results in a separate ID space from any PII, which can be used as the bridge between known and unknown data sources.  For example, LiveRamp uses RampID as their pseudonymous identifier, whereby they can resolve both PII and Media/Device data (cookies, Mobile Ad IDs, etc.) to the same ID space for deeper analysis.  This again is one way in that PII can be resolved to RampIDs, but RampIDs cannot be used to back track to PII.
  • Privacy preserving techniques: The various ways by which a user’s identity can be obfuscated from row level PII.  This includes complex concepts such as k-anonymity (aggregating analysis to require a certain level of users per row of output data) and noise injection (adding a percentage of random signal to an analysis to prevent leakage of sensitive data), with the goal being to provide accurate measurement without sacrificing user privacy.

Required features and integrations

The technology industry has seen a massive wave of M&A activity coming out of the pandemic.  While there’s certainly numerous benefits to end users when two companies combine their offerings, it also raises concerns among leadership and procurement teams who are trying to understand “what exactly are we buying?”  A recent survey by the Wall Street Journal*** showed CIOs are increasingly concerned about M&A causing “over-bundling” in the software space, leading to premium price tags for solutions which many buyers find include lots of features they simply do not need.

MarTech, AdTech, and specifically entity/identity resolution tools are unfortunately not immune to this trend.  When evaluating solutions in the market, it’s become increasingly important to understand what key business issues are we trying to solve, and what offerings can help us do so (and nothing more). This requires striking the right balance between “need to have” and “nice to have” from a features perspective, while comparing this against costs around software acquisition, implementation, and ongoing maintenance.

A response to this concern has been the increased number of “composable” offerings brought to market in the software space.  At a high level, this model looks to unbundle features so buyers can pick and choose only what they need for their business. This is most prevalent in the CDP space, where composable CDPs such as Lytics, Hightouch, and others offer a “lighter-weight” solution meant to sit on top of a businesses’ existing data warehouse or infrastructure.

From a features perspective, a second consideration is understanding how the provider’s solution will integrate with the rest of my technology stack. This is important from both the inbound (can I easily incorporate all the data I need into your system?) and outbound (once I’ve cleaned up my customer data, can I easily action on it from your platform and/or deliver to other execution platforms?) facets. Digging into the provider’s marketplace of “Connections” or partner integrations is crucial in the evaluation process to ensure a new investment does not create yet another silo in your architecture.

Graph customisation

Specific to entity/identity resolution, it is incredibly important to understand the methodology by which these providers give you the ability to better recognise your customers across channels.  Their tools should offer customisation or specific configuration capabilities based on your specific business needs. How the data is stitched together should not be a black box to you and your users, as there is no one-size fits all model for entity/identity resolution.

For example, there is a common need among global enterprises to unify their data, but not over consolidate it.  This could be due to their data collection practices (i.e. one business unit/geographic region does not have permissible rights to know about another’s customer interactions), or more simply based on their ideal interactions with customers (i.e. a business works in both B2B and B2C capacities, and they interact with even the same individual customer in different ways across each). This is where a more complex resolution framework is required, often resulting in multiple “customer graphs” for each business unit’s specific needs.

Reaping benefits across the full enterprise

Lastly, when considering how to approach an entity/identity resolution project, one of the most important questions to ask is “How can we do so in a way which allows all parts of our business to reap the benefits?”  Or, if every customer interaction is being pulled into a single view, all members of your organisation who manage those interactions should be able to easily access and action on your unified profiles.

While this may seem intuitive, this initially was a harsh reality amongst enterprises when adopting a CDP.  If all profile unification efforts happen in the CDP, but only a handful of users in your business have logins, will it really drive optimal outcomes within every channel?  This challenge, among others, is where the “warehouse first” approach has become more and more critical to enterprise-wide success.  Data warehouses sit as the lowest common denominator across all parts of the business, and thus your best recognition of customers should live there for it to be fully leveraged.  This mindset has produced a variety of solutions, including the cloud-native independent identity tools we discussed, CDP features allowing work done in their platform to be easily “synced” to your warehouse, and most recently a host of applications offered directly within cloud provider marketplaces to enable business to build things their own way.

Understanding your full enterprise technology stack, and how customer recognition can flow throughout, is critical in evaluating your approach. Accidentally “gatekeeping” a successful identity project will only hinder its future adoption.

Conclusion

Customer recognition and the practices and technologies behind it are quite complex.  My hope is this blog sheds some light on the drivers leading us to the “why” around Customer 360, and more importantly the some thoughts on which path (“how”) might make most sense for your business. I truly believe optimising how you recognise and engage with customers will lead to the best business outcomes possible. If you ever find yourself doubting the need, I always find it helpful to remind myself that I’m somebody’s customer, and outside of their products/services, my willingness to take the next step is almost always based on how they’ve treated me in the past – that’s what knowing our customers is all about.

*Touchpoints and the Revolution in Omnichannel; Boston Consulting Group; February 2023

**Gartner 402 Marketing Survey; Gartner; January 2022

***Tech M&A Raises Fears Over Software Pricing, Bundling for CIOs; The Wall Street Journal; February 2024