Key Risk Indicators are powerful metrics, but they are harder to understand than KPIs. Here’s all you need to know about both.
Should you use KPIs or KRIs to measure your risk team’s success?
The short answer to the question is that you need both. But because there still seems to be confusion between the two types of metrics and their usage, we wanted to break them down in an in-depth post.
So without further ado, let’s start by looking at what Key Risk Indicators are, and how they can help your RiskOps, or risk operations.
What Are Key Risk Indicators (KRIs)?
A Key Risk Indicator, or KRI, is a measure that indicates how damaging an activity might be. It’s a key feature of RiskOps analysis, whose goal is to predict how likely an action is to hurt the company, either financially or because of a bad reputation.
This is especially useful for upcoming projects, whether it’s to take on more transactions, attend a public event, or launch a new product.
KRIs Vs KPIs – What Are the Differences?
The key difference between a risk management KPI and a KRI is that key performance indicators are designed to measure how well (or badly) things are going using historical data. Key Risk Indicators, on the other hand, point to future adverse impact.
In other words, KRIs can be used to measure risk that hasn’t happened yet, which is useful for unveiling new growth opportunities, or assessing which processes need to be optimised.
A good way to think about it is by answering the following questions:
- How well is your risk team doing? You’re looking at fraud KPIs.
- How likely is it that you are not anticipating every risk factor? This is where KRIs shine.
Why KRIs Are Important
As mentioned above, KRIs help us see where risk could potentially exist. Using them can help with the following scenarios:
- Anticipate new risk factors,
- Justify additional headcount,
- Identify risk that doesn’t yet damage the company,
- Set up defences before new risk vectors arise,
- Organise team roles in anticipation of new risks,
This is in contrast with fraud KPIs, which are used for example to:
- Measure existing team performance,
- Individual performance review,
- Calculate ROI per agent,
- Overall team performance,
- Set reasonable team goals.
Some Examples of Useful Fraud KPIs
For this example, we’ll look at KPIs that are relevant for online stores and e-commerce, but many of them can be adapted to any kind of transaction, be it a SaaS or a B2B business.
Order Approval Rates
Before looking at how to reduce fraud, you need to check what percentage of your transactions are approved. There are various schools of thought on how to best calculate that number, but it’s important to take into account things such as whether auto declines come from the issuer bank, payment gateway or your own fraud prevention system.
Chargeback disputes are the bane of any fraud manager’s existence. Should the rates trend above the dreaded 1% rate, an online store might even be labelled high risk, and lose a partnership with the card network.
The challenge is that chargeback rates tend to be calculated differently depending on the credit card processor. It’s important to take these differences into account when calculating your own rates, and you can even break them down further by looking at individual payment methods (for instance PayPal vs WePay).
Average Manual Review Time Per Agent
A self-explanatory metric, but one that can be extremely useful to justify a promotion, assign various workloads, or to initiate a performance review.
Checkout Abandonment Rates
A useful KPI to share with marketing, as they may use the numbers to test out automatic email campaigns. For fraud and payment, it’s useful to look at how much friction your payment gateways and prevention tools add to the customer journey.
For instance, if the checkout abandonment rates boom after implementing extra prevention checks, you could look at dynamic friction solutions (when extra KYC checks are only initiated after risk scores go above a set threshold).
Cost Per Analysis
One of the best ways to calculate ROI on your fraud prevention solution, and to understand the full cost of fraud at your company. You should include all the expenses related to manual and automatic review, lost customer lifetime values from declined orders, and how much is saved when a fraud attempt is caught by the system.
This is especially useful when you work with a pay-per-API call fraud prevention engine. As we’ve previously covered, a chargeback-guarantee model may offer better value on paper, but also a strong incentive to be conservative when taking risks, resulting in a higher rate of false positives (which would impact your overall revenue in the long run).
Fraud to Sales Ratio
A simple metric that is useful both as a KPI and risk indicator. If the number rises too steeply, you know you’ll need to consider other fraud prevention solutions or strategies.
A Holistic Approach to Risk Management
In the context of fraud prevention, KRIs help risk managers with their balancing act. On the one hand, they want to block as many fraudulent transactions as possible. On the other, they want to accept as many transactions as possible. If you were to block all transactions, the fraud rates would drop down to 0%.
So a standard KPI for measuring fraud rates would look like:
Fraud = chargebacks + refunds / total accepted transactions in a given time period.
But these results also need to weigh in your acceptance rate (ratio of approved vs declined transactions).
Moreover, factoring false positives into the equation can be tricky, because you may lose more than the value of a transaction. A false positive could turn a loyal customer towards competitors, which means your customer lifetime value (CLV) and customer acquisition costs (CAC) are also wasted.
So we’re now already looking at a much more complex equation:
Cost of fraud = transaction value + chargeback fees vs. false decline = transaction value + CLV + CAC
As you can see, the cost of fraud can be a very strong KRI because it gives us a better view of how fraud affects various business areas.
More importantly, you can use that number to spot when something goes wrong. If, say, a payment service goes down, you should immediately see a change in the numbers.
Using KRIs to Calculate KPIs (and Vice Versa)
An interesting point is that you can actually use these large scale metrics to focus on more granular KPIs. Following our example above, you could use the cost of fraud value to:
- Monitor the risk team’s performance per shift,
- Look at individual agent’s performance,
- Measure the cost of whitelisting,
- Measure the time lost to catching fraud,
Interesting (and very useful questions you could answer include):
- How much cost did you save by flagging fraud?
- How expensive was it to miss transactions that turned into chargebacks?
- How much do you save when an agent finds a false positive?
And crucially, you could estimate the value of hidden or invisible risk. For instance, how much do you save when closing sleeper fraudulent accounts (by calculating the average value of all accounts belonging to the same multi-accounting criminal rings).
Concrete KRI Example: Working With a Risky Payment Gateway
To better understand KRIs and see how they require a holistic approach to RiskOps, let’s look at a concrete example. Let’s say you just started working with one of the dozens of new payment gateways that arrive on the scene every year, but that their performance is raising some eyebrows.
You start noticing that payments often fail due to server downtime from their end. This prompts you to gather some metrics, namely:
- Mean Time Between Server Failures (MTBSF): the average amount of time (in days) elapsed between two failures of the payment gateway’s systems, measured from the initial failure until the next one.
This simple metric can then help us look at:
- Monthly difference between MTBSF: to see if there is a pattern throughout the year.
- MTBSF versus amount of payments processed: to see if you are overloading them with requests.
The first actionable fix would be to mitigate risk by using a second provider as fallback. You could even re-route some transactions towards other gateways to avoid overload.
If things don’t look better after a while, you could use the KRI to justify abandoning this partnership altogether. This perfectly illustrates how a good KRI can help make business decisions that aren’t necessarily related to team performance, but rather with how likely a process is to hurt your company in the long run.
How to Share Your KRIs
Once you have found a satisfactory way to calculate the metrics, it’s up to you to decide how transparent you are with them. They can be useful to bring up to upper management, especially if you foresee a drastic increase in risk.
If they touch upon more personal performance (for instance those looking at cost saved per agent), you might have to mask the agent’s identity by using IDs. Too much open data may make people uneasy and actually disincentivise the worst performing team members.
However, sharing enough data with the team may help them self-regulate and help each other without requiring additional push from management.
You can also use KRIs to justify promotions, bonuses, or internal fraud management training if needed. It’s also useful to know if you should invest in specific software or improved infrastructure, as you can measure ROI against concrete numbers (hours saved, false positive rates, fraud rates, fraud prevention pricing, etc..).
Key Takeaways – Looking at Risk Beyond Fraud Rates
The key question many businesses fail to answer is: how do you measure something that isn’t there?
More specifically, how can you fight fraud if you’re not even certain that you are a target? Let’s not forget that the ultimate goal of fraudsters is to not get caught. If they are successful, you’ll have an excellent fraud rate – but that won’t mean you’re not under attack. Conversely, if your fraud prevention system is too rigid and treats every customer as a fraudster, you’ll lose out on business.
The solution is to take a holistic approach to risk management and RiskOps, and to measure potential threats as well as existing ones. This is precisely where KRIs or Key Risk Indicators can help.
Learn more about our products
Florian helps tech startups and global leaders organise their thoughts, find their voices, and connect with customers worldwide.