Machine Learning or Manual Reviews in Fraud Detection?

Machine Learning or Manual Reviews in Fraud Detection?

Author avatar

by Florian

Machine Learning fraud detection is a popular buzzword: should it always replace manual fraud review?

There has seldom been a Risk Ops conference in the past few years that didn’t address the topic of machine learning in fraud detection. Some go as far as saying it will completely replace manual reviews.

After all, manual reviews are indeed slow, time-consuming, and intensive. Worse, they are often unnecessary, as research from Juniper showed, small merchants can process up to 42% of all orders – only to reject 2.3% of them.

Clearly, something must be done to improve the efficiency of manual reviews. But should we get rid of them altogether?

What is Fraud Detection with Machine Learning?

In fraud detection, machine learning algorithms are trained on your historical data to suggest risk rules. You can then implement the rules to block or allow certain user actions.

Manual Reviews: Effective But Not Foolproof

Historically, manually reviewing user info and transaction details was the only way to go. As we entered the digital age, companies began leveraging technology to process payments. This meant a huge surge in volume and speed. Understandably, fraud officers were overwhelmed.

Luckily, technology was also there to help. Through early software programming, the first rule based solutions were born. The rules were hardcoded, which filtered suspicious details for manual review.

There are three key challenges with manual reviews:

  • False positives: having static rules can result in a high number of false positives, or cases when a legitimate action is flagged as fraudulent.
  • Fixed outcomes: fraud is adaptive, and so is a modern digital business. A rule that worked great one day might become completely obsolete the next based on consumer trends.
  • Hard to scale: the biggest issue is that each rule must be created, tested and reviewed by a staff member. This is time-consuming and resource-heavy.

While these solutions and techniques reduced fraud managers’ workload, fraudsters became adept at quickly understanding the rules and finding loopholes in them. This led to the new era of risk assessment relying on big data to glean a complete picture of user behaviour.

How Can AI help in Fraud Detection of Credit Cards?

Your AI-driven system should be able to look at all your historical payment data and suggest better risk rules for your business. If you have correctly marked which credit card payments went through, and which ones resulted in a chargeback, the system may notice patterns and suggest rules to make sure only good future payments go through.

Blackbox Machine Learning to the Rescue

Instead of staffing a huge team that will set up static rules for every use case, fraud managers began to understand they could use artificial intelligence, or in this case, ML (machine learning), as part of their anti-fraud system integration.

With Machine Learning, your first immediate benefit is the ability to process enormous amounts of data. No human could trawl through as much payment info, login details and IP addresses as a computer. And if the amount scales over the years, machines can still process them without batting a (digital) eyelid.

This means a talent for recognising patterns that humans might not initially spot. In technical terms, we are talking about confusion matrix analysis, and it can work with any data, from currency to age groups.

 For instance, a system that performs fraud detection using machine learning can take into account browser resolutions to flag suspicious values (as affiliate link fraudsters load content from sites with invisible frames). It would have been a stroke of genius for a fraud manager, but a cakewalk for a fraud detection API.

Unsupervised Protection

Algorithms don’t need breaks, holidays or sleep. Fraud attacks can happen 24/7, but even the best fraud managers might come to work on Monday morning with a backlog of manual reviews. Machines can ease up the process by sorting through the obviously fraudulent or acceptable cases.

Your ML fraud management system will look at all the data points available historically and in real-time, and create a risk model you can use in your business. The idea of course is to use these models unsupervised so that you can sit back, relax, and let the machines decide what is legitimate or fraudulent.

If you have a large number of low-risk payments, for instance as a bank or payment processor, it may be best to use a black box machine learning model.

BlackBox Machine Learning and High-Risk Transactions

Even if you want to scale the number of payments you process, there is a use case when manual reviews can’t be beaten: dealing with high-risk payments:

  • You are a store that sells high-value goods or services (high-end electronics, jewellery, rare product, etc…)
  • You have to comply with AML checks 

In that case, it’s not worth letting the system run by itself. A machine learning model is great at spotting unusual patterns, but it lacks a human’s understanding of the real-world context.

Which is to say, there will initially be a higher volume of false positives and negatives, which human analysts will need to correct themselves.

In this scenario, you will likely need more manual reviews (and the staff to perform them). Luckily, there is a solution when white box fraud detection using machine learning becomes a definite advantage.

I love where the machines have taken us because AI and Machine Learning can take out those things that are very very bad, but a lot of the time there’s false positives. And if you’re teaching a machine in a certain way you have to deal with the fallout from that. So there’s that grey space in the middle and that’s where I like my teams to operate.”

Jacqueline Hart, Head of Trust & Safety at Patreon, as heard on the SEON podcast.

Machine Learning as Extra Intelligence

Sometimes, your focus isn’t just to block or accept transactions, but rather to give all the right information to your risk analysts as fast as possible. You always get these cases that fall into a grey area where the best algorithms can’t help by themselves.

This is precisely where you can use a whitebox machine learning system to suggest rules. With a whitebox system, you get a full understanding of the rules, and a human will have the final say on it. 

Transparency helps our analysts get a fuller picture of the problem and its potential solution. They can even test the rules on your own historical data and tweak them to get better results in a sandbox environment.

how SEON's whitebox machine learning fraud suggestions look
How SEON’s whitebox machine learning suggestions look

Tips on Combining Machine Learning and Manual Fraud Reviews

Whitebox machine learning rules are essentially suggestions. You can review them with analysts, but also other departments such as the sales and marketing teams to see if they really make sense.

After implementing an automated fraud management system, it can be helpful to review the declined payments where the risk score was very close to the manual review threshold. This will help you tweak the rules to have fewer grey areas over time, as long as you repeat the process regularly. Similarly, you can understand where the system declined a low-risk action, to better calibrate your set up in the long run. 

Finally, it’s also possible to allow the customer to trigger manual reviews themselves. If the risk score is between the DECLINE and REVIEW threshold, you could get their input to reduce your customer insult rate.

So What Are the Benefits of Machine Learning Fraud Management?

Because machines have a much easier job processing a large dataset than humans, what you get is an ability to slice and dice huge amounts of information.

This amounts to:
  • Faster and more efficient detection: the system gets to quickly identify suspicious patterns and behaviours that might have taken human agents months to establish.
  • Reduce manual review time: similarly, the amount of time spent on manually reviewing information can be drastically reduced when you let machines analyse all the data points for you.
  • Predict better with large datasets: the more data you feed a machine learning engine, the more trained it becomes. That is to say, while large datasets can sometimes make it challenging for humans to find patterns, it’s actually the opposite with an AI-driven system.
  • Cost-effective solution: unlike hiring more RiskOps agents, you only need one machine-learning system to go through all the data you throw at it, regardless of the volume. This is ideal for businesses with seasonal ebbs and flows in traffic, checkouts, or signups. A machine learning system is a great ally to scale up your company without increasing risk management costs drastically at the same time.

Key Takeaways

Machine Learning cannot exist in a vacuum. While a black box system makes sense for businesses with a lot of low-risk transactions, it’s not always ideal. 

But thanks to a fast decision time, better and more accurate overview of patterns and anomalies, an AI-based fraud system can certainly complement manual reviews. 

In short, machines are fantastic at processing and memorising knowledge; humans are still better at applying it. This is why we believe combining both insights and machine learning fraud detection is the ultimate way to fight frbad agentsaud online.

Share article

Learn more about our products


Author avatar
Communication Specialist

Florian helps tech startups and global leaders organise their thoughts, find their voices, and connect with customers worldwide.

Sign up to our newsletter