There’s something magical about paying with cards. It is extremely unsafe to do it, and yet it is the most common way to buy things. Everywhere, all the time.
It’s not like the insecurity of card payments is a secret. On the contrary: cards are not safe by design. We’ve covered a few times in The Playbook how naive and insecure it is to share with an unknown website the 16 numbers on your card. These numbers open the gates to your bank account, and often to highly expensive, unsecured loans.
But the world keeps rolling: you pay with card, and so does many other people worldwide.
Credit cards are the technical debt of the payments industry. They work so well, no other payment method has been able to topple them down as the de facto standard way to purchase things, decades after they were invented.
And sure, we’ve got Apple and Google trying very hard at replacing the plastic card with smartphones.
But smartphones and plastic cards alike are mere conduits of what a payment card really is: those not-so-secret 16 numbers, the expiration date and the CVV. It is the data, not the vessel, that’s valuable.
Our physical experience at the counter has experience a whirlwind of changes. We used to swipe and sign on the check; we now tap to pay, or scan a QR code. It’s not unlikely that some other way to authorize a payment will arise soon.
That’s the vessel part. The data part? It’s all the same since 1958: there’s a piece of data that an authority recognizes as your credentials.
This, in essence, is what tokens really are.
I’m Alvaro Duran, and this is The Payments Engineer Playbook. Over the last two weeks, I’ve been exploring how payment systems are made reliable, not with infrastructure, but with software. In order to provide the desired “always-on” capabilities that payment systems require, engineers often rely on software tricks rather than esoteric infra configurations.
This is the third installment of a 4-part series that expands on a talk I gave a few weeks ago, where I discussed these topics:
Tokenization (this article)
Fallbacks
Today, I’m talking about tokens: those uuid references that fly around every time some payment fails. They’re a crucial component of how payment systems orchestrate retries, because they’re a reference to those insecure credentials you’re not supposed to have, or share.
Tokens belong to the part of the payments infrastructure that’s changed a lot without anyone outside the industry really noticing. But, for payments engineers, tokens are not what they used to be, not even five years ago.
And, with Mastercard pledging to ban manual card entry by 2030, tokens are going to become foundational for payments, if they aren’t already.
This article focuses on
How Tokens solve the credit cards’ insecure protocol
The differences between Card-on-file, DPAN, MPAN, and network tokens
What do people mean by “interoperability”, and its implications for privacy
Why would you use tokens for one time payments
Yeah, some AI (I wish I didn’t have to, but it’s relevant).
On top of that, I’ve added some useful references at the end if you want to learn even more.
Enough intro, let’s dive in.
If people start talking about tokens and I haven’t removed my engineer hat, I’d probably think they’re talking about pointers and memory addresses.
Because they’re pretty much the same thing.
You’ve got credit card data on your hands. Believe me, you don’t want to have it. It’s data that screams to be stolen, like when Ashley Madison got their users’ email addresses leaked.
You don’t want to have credit data on your hands. What do you do? You ask somebody else to have it for you. Some specialized company, with lots of systems, protocols, auditors, you name it. Security as a Service.
But you need to use that data! After all, it’s how you get paid: by passing it to Visa or Mastercard, so that the cardholder bank gets notified of the payment request.
Tokens solve this very problem, and they do it in a way analogous to pointers. You have the data stored somewhere safe, and continually reference it with a token, a piece of data that becomes a bunch of nonsense when it gets stolen.
Which means that tokens can be categorized by the only two properties that they really have:
Who stores the valuable data
How long is the token useful
Let’s see how this plays out in detail.
Tokens used to be the responsibility of merchants.
They stored the card details in an isolated environment, and used tokens to avoid having the PCI auditors put obstacles on the whole payment system. Card data lived within the scope of PCI, tucked away in a corner. Tokens where used everywhere else.
These are called card-on-file tokens.
This is of course something that only big companies can do. But big companies are often those that integrate with multiple payment providers. And that’s why, soon after implementing card-on-file tokens, most big companies leverage that to retry payments without the customer really knowing what’s going on.
The process goes like this: the customer keys in their card details. In a secure manner, those credentials enter the business’s sytem, which stores those credentials and generates a token that points to them. And the business uses that token to authorize a payment with Provider A.
Here’s the magic. Last week, we talked about the pain that Do Not Honor errors are. They’re the result of banks’ terrible software and data mangling. And, importantly, do not reflect badly on the cardholder.
So, what does the business do when they get a DNH error from Provider A? They have a token that’s “single-use”, but those terms and conditions that we always agree on often allow for a timeframe, not a strict number of uses.
So they reuse that single use token on Provider B.
This is often how orchestration works at big companies, where resources are abundant and what matters is squeezing the most juice out the interaction.
The alternative is having to return that error back to the user, impacting their experience, and potentially leaving money on the table.
But card-on-file tokens are so 2010s. It is when tokens are scoped beyond a single merchant when they really shine.
There’s one company to thank for the explosion of payment tokens: Apple.
In 2014, in order to be able to pay with your smartphone, Apple partner with Visa and Mastercard and spur the development of what’s called network tokens.
Network being Visa, Mastercard. The payment networks.
Network tokens aren’t constrained by the merchant that uses them to request the payment. On one end, it is Apple’s own device the one that generates the single-use cryptogram (this one is really single use by design). On the other end, the payment network is able to decrypt that cryptogram, much like HTTPS allows for secure bidirectional communication online. The smartphone and the payment network share a secret, and the merchant doesn’t even know what the credit card credentials really are.
Apple invented the passthrough wallet: the smartphone is a secure store for credit card credentials, that’s also outside the scope of PCI, because what’s stored by your smartphone is another token called DPAN, or Device Primary Account Number.
And they’re also interoperable. DPANs work regardless of the merchant you buy from. But here’s the catch: they are unique per device.
One of the key selling points of digital wallets is that the DPAN hides the credit card details from the merchant.
That, however, doesn’t mean they can’t track you.
The DPAN is specific to the device you’re using. Losing your phone means getting a new DPAN, and using the same card on your iPhone and your Mac creates two of them.
Plus, data brokers might buy this data from merchants, because they can use it.
— Payment Cards Are An Insecure Protocol
A token is as interoperable as it is not private. A token that can be used everywhere can be used to track you everywhere.
There are also MPANs, or Merchant PANs, which are unique to the business you sell to, and are useful when the business sells you a subscription.
Which means that the business can now reuse your credit card details securely. But it also means that it can track you within their own ecosystem.
Bots used to be always fraudulent. Now, not so much.
Agentic Commerce is the idea that some bots may actually represent real humans who really want to buy, and not fraudulent attempts to use stole credentials.
So, now, businesses need to know, at checkout, whether a clearly automated payment request comes from a bot (which is bad) or an agent (which is good).
It’s an amusing distinction. Why not keep on banning everything that’s not human?
Well, because Generative AI has become really good at dealing with online resources, thanks to what’s called Model Context Protocol, or MCP.
The issue of privacy becomes now more vital than it was before. I can direct an agent to buy things from me after generating a single use token that it can use to certify that I’m OK with the purchase.
Tokens are the stepping stone for Agentic Commerce. None of this can happen if we naively share our credit card credentials as plain text.
But the most mindboggling thing to me is that Agentic Commerce falls neatly into the most innovative of the categories that I described in Payment Patterns: the Async/With Action:
Let me warn you: pretty much nobody is accepting payments from this category.
Right now, that is.
The best known payment method that’s asynchronous and requires user action is Bitcoin […]. Bitcoin sits on a different league of complexity because transactions are initiated by the sender. In order to authorize the transaction, the customer has to digitally sign it and broadcast it to the network.
[…] Bear in mind that this category is where most innovation is happening, and I wouldn’t be surprised if Stripe’s acquisition of Bridge triggers some action in this space.
— Payments Patterns: Elements of Reusable Money Software
Much of the business strategy regarding tokenization, in the payments industry and in fintech more generally (like crypto trading or stablecoins) is founded on a single goal.
To become the source of truth.
Tokens are useful only when there is a trusted environment that certifies that they represent actual data. Becoming a universal source of truth for payment tokens is the Holy Grail of payment companies.
Which is why Visa and Mastercard have silently pushed for interchange fee benefits for those merchants that make payment requests with tokenized data versus the plain credit card details.
And this, when payments online could be intermediated by agents, could be worth its weight in gold.
If you want to learn more about tokenization, Glenbrook Partners is the place to go. They have an introductory podcast episode on tokenization, yes, but it is such a recurrent topic for them, they have even acknowledged that they just can’t stop talking about tokens. I suggest you give these two a go.
That’s it for this week on The Payments Engineer Playbook. See you next week for the final installment of the Code-first Reliability series!
P. S.: Share this email with someone who might be interested in tokens. They’ll appreciate it!