Stripe's ML team used a transformer approach on payment transactions to double fraud detection overnight
I came across this seriously fascinating post from Gautam over at Stripe. He is the Applied Machine Learning leader at the company.
Too often, organisations hire ML/AI people such as Gautam and they essentially get nothing done. They're often stuck in proof-of-concept hell, with little support from the executive team to change that.
Not so, it seems, at Stripe.
Gautam and team have done something that I'd suggest is bordering on, if not actually, revolutionary.
They took the transformer model popularised by the likes of ChatGPT (and proposed by Google researchers back in 2017) and they applied it to card transactions.
Big deal, you say.
Yeah. Wait.
You're imagining rows of Excel spreadsheets, right? That's what pops up in my head when you talk about financial institutions managing thousands of transactions per minute.
There are no relationships there. Right?
Surely, it's not like language? The fundamental relationship or attention-focused approach with transformers (i.e. "The quick brown fox jumped over the..." - what is the next word?) cannot work with payment transaction details.
Or can it?
"We wondered whether an LLM-style approach could work here," explains Gautam.
He continues: "It wasn’t obvious that it would—payments is like language in some ways (structural patterns similar to syntax and semantics, temporally sequential) and extremely unlike language in others (fewer distinct ‘tokens’, contextual sparsity, fewer organizing principles akin to grammatical rules)."
Turns out, yeah... it does actually work.
"You can think of the result as a vast distribution of payments in a high-dimensional vector space," explains Gautam.
(I'm immediately thinking of the Tesseract near the end of Interstellar)
He continues: "Payments that share similarities naturally cluster together: transactions from the same card issuer are positioned closer together, those from the same bank even closer, and those sharing the same email address are nearly identical."
The net result is that using this methodology, Stripe can immediately and easily identify anything that just doesn't look right.
By organising payments data in this way, the team have been able to reduce fraudulent attacks from an already impressive 59% to 97% "overnight."
Fascinating. Absolutely fascinating.
This is really compelling work from the Stripe team.
Do read his full post on LinkedIn.
Gautum finishes his post by suggesting that "payments have semantic meaning" a little "words in a sentence."
This isn't something that would ever have occurred to me. But if you think about it, it makes perfect, perfect sense. Witness, for example:
- Ewan buys £10 burger at Five Guys in London near Leicester Square at 1pm.
- Ewan buys £7.99 McDonalds meal at Waterloo Station at 7pm.
There is a lot of semantic meaning to these things. The location. The time. The purchase amount and the payee – not to mention, the actual product data (i.e. item level receipt data – which is probably not yet available or exposed in most respects – that's where companies like Receiptable come in).
