An explainer of how data enrichment works, and what to look for if you want to leverage it with your organization.
You’re probably using data enrichment daily without knowing it.
Google’s autocomplete feature, for instance, works with it: it takes raw data (the letters you type on your keyboard) and enriches it to match it with an enormous database of (almost) all possible words.
The result: a smarter tool that improves user experience.
But did you know data enrichment (a.k.a. data augmentation or appending) is at the core of numerous online businesses these days? And did you know it’s easier than ever to get started with it?
Don’t worry if you’re confused. In this post I’ll go over the key concepts of data enrichment in a general context. I’ll then show why it’s such a must-have feature for online fraud prevention.
So first, let’s break down the basics:
What is Data Enrichment?
Data enrichment is a process that takes raw data points and merges them with similar data points in a larger database. The database can be internal or maintained by a third party service. Most companies use data enrichment to know more about a customer based on one piece of information, like an email address or phone number.
How Does it Help Businesses?
Imagine you’re the owner of an online store who wants to send a special offer to a customer, but all you have is an email address. You can’t personalize the email. You can’t suggest a product to buy. You’re left guessing.
Now what you can do, however, is search a directory which links that email address with a name, gender, age and purchase history. Congratulations – you’ve just performed manual data enrichment!
The key point to understand is that, in our day and age, the more data you have, the smarter your business decisions can be.
This is especially true for companies who lack crucial data, for instance when:
- Moving to a new market
- Trying to keep up with trends
- Starting a new business (like moving from brick and mortar to online)
- Trying to reduce customer friction by only collecting the essential info
- Looking to improve targeting
- Reducing fraud rates
Understanding Automated Data Enrichment Processes
Now of course, for organizations, that kind of manual work mentioned above simply isn’t feasible. Which is why they can integrate an API that automatically acquires the data, and searches for it into the right database.
Some Businesses Built On Data Enrichment
Automating data enrichment is at the core of the modern digital world. In fact, it is the process that allows businesses in a number of verticals to exist:
Lending: Credit scoring is built on data enrichment. Banks or loaning providers access third-party / alternative databases which help them create a complete profile of the customers they’re dealing with (and hopefully reject potential defaulting customers).
In fact, the entire process of underwriting risk would be impossible without data enrichment – especially when working digitally. As you are originally in the dark about your potential customers, trust alone won’t do. You need to prepare yourself against bad agents by gathering as much info as possible.
Fraud prevention: Similarly, online businesses can reduce fraud rates by creating better user profiles. A single data point like an email address, device used or IP address, can be enriched to create a full picture of the user.
A good example would be our own email lookup tool, or email analysis module. Your user enters their email address at the onboarding stage, and we automate a search that aggregates incredibly precise information, such as whether it is connected to social media sites, if the domain is valid, how old the address is, etc.. As you can imagine, that simple process can go a long way in reducing fraud rates in the long run.
Insurance: Insurance providers tend to categorize their customers based on various data-points and enrich the specific dataset. Once they have all the info they need, they can provide relevant insurance deals or products based on the risk related to the customer.
So data enrichment in that sense is used both as a segmentation tool, and a targeting one. You can use it to refine your business processes in order to be more efficient at your job.
Marketing: Another example where customer segmentation becomes more precise with data enrichment. Marketing companies target individuals with more relevant offers and adverts by getting to know their audiences.
Retail: The best illustration here is Amazon’s feature that suggests similar products. The data Amazon has about you can be simple (the page you are browsing), but by linking it to their immense database of customer purchases, they can intelligently recommend products for upselling.
This ability to aggregate data and create meaningful insights for their goals (upsell) is one of the reasons the retailing giant is so far ahead of competitors in the online sphere. Although it should be said that most online stores now use big data as an intrinsic part of their business model.
Does Data Enrichment Affect Privacy Policies?
That’s a good question, especially since data privacy has been a major topic in recent years. As the number of data breaches suffered by big organizations shows no signs of slowing down, governments had to step in and set up drastic precautions to protect user data, such as the GDRP or ISO27001.
Now to ensure you comply with both these regulations, your data enrichment service should source its data from open and social source.
Not following this guideline could put you at risk of breaking the rules in your region, which could incur fines and needless legal battles.
Ok, I Want to Enrich My Own Business Data – Now What?
The good news is that there are more and more companies providing data enrichment these days. The challenge is in finding one that really meets your needs. So here are a few things to consider:
- Manual or automated? Some data enrichment options work great for specific queries. For instance, if you only need to know more about the odd loan applicant. For large scale operations, you’ll need to work with a third party data provider / aggregator. Which brings us to the topic of…
- Integration: Do you want to work via an API? Or purchase the database and automate the search yourself? For custom integrations, a single point makes it easier for developers, but it’s not always available.
- Data quality and legality: how fresh is the data you are acquiring? And does the company delivering it meet legal requirements for data protection like the GDPR?
- Pricing: there shouldn’t be a ton of variation here, as most third party data enrichment companies charge a micro fee for each check.
And last but not least, Middleware options, which I’ll explain in more details below.
How Machine Learning Completes Data Enrichment
Getting the enriched data is one thing. Interpreting it is another. In fact, one rule to remember is that, unless you are a trained data scientist, you are more likely to make poor decisions when looking at large volumes of data.
This is where it’s worth understanding the role that machine learning can play. The technology works wonders as middleware between the deluge of data you are about to receive, and the intelligent humans who will make sense of it.
So if your data enrichment service provides a scoring system, for instance, it’s important to understand how it works, and how the models are built, because they will need to be tweaked and supervised eventually.
This is the core difference between an opaque, or blackbox system, versus a whitebox system, which lets you peer into the rules via human-readable words. If you only get the score, you might feel at the mercy of the algorithms without really getting a sense of how things work.
Here is an example of how a whitebox system and human intelligence can be combined in the context of fraud prevention, where a data enrichment system gives a score of how risky a transaction is.
And it works wonders: companies using our end-to-end fraud prevention solution, which includes data enrichment and a machine learning engine, reduce their fraud rates on average by 70-80%
Conclusion – Data Enrichment + Machine Learning = Smarter Business Decisions
Data enrichment isn’t really anything new. But how it’s performed these days is what makes all the difference. Businesses who want to remain competitive and grow need it more than ever – especially when it’s combined with the power of machine learning analysis.
Thankfully, companies don’t have to design a full data enrichment system from scratch, as they can simply hire the services of a third-party company. At SEON, we make it easier than ever to enrich data from an email or phone number, for instance with our Intelligence Tool, which works as a simple Chrome extension.
So whether you choose SEON for fraud prevention, or any other tools that can help meet your goals of better user experience or improved targeted marketing, I hope this primer on the topic will convince you of all the great possibilities data enrichment can offer.