How to Scale a Ledger, Part I

At Modern Treasury, we are lucky to work with some of the largest fintechs and marketplaces in the US. We started with an API that allows companies to integrate bank rails in their products. Over time, however, we quickly realized our customers needed a way to track the high volumes of transactions and balances in their platforms.

Companies that move money need a ledger: a scalable data store for financial transactions and balances. It seems simple. But as developers who work with money know, and as the news will attest to, this is actually a hard problem in computer science. This 6-part series is a summary of everything we learned helping our customers scale ledgers from scratch.

Why use a ledger database?

As financial services increasingly become embedded into mobile and web apps, more companies are facing the challenge of storing, transferring, and converting money. Typically, companies start by tracking money within their domain objects: the price of a ride share is a property of the ride; the monthly payout to a gym is a sum across bookings; the amount left to pay on a loan is stored on the loan object.

This approach tends to break down at scale:

• Companies need more sophisticated financial reporting as they grow. Every movement of money must be immutably recorded, keeping track of the source and destination of each dollar.
• Stitching together a clean history of financial events becomes impossible as data models become more complex, companies migrate to service-oriented architectures, or multiple fintech SaaS products that aren’t natively integrated are introduced.
• Performance degrades as the number of users increases. A payout cron job starts missing a bank-imposed end-of-day deadline. Credit card authorization decisions take longer than the card network allows.
• Failure cases (we overcharged, a bank account debit was rejected due to insufficient funds, a transaction was fraudulent) are more frequent and difficult to reverse.

All of these problems can be solved with a ledger database.

At its core, a ledger database is a simple data model—Accounts, Entries, and Transactions. A common refrain from engineers is: “a ledger is simple to build, but the business logic on top of a ledger is complex.” In fact, a well-designed and scalable core ledger database can simplify business logic and reduce time to market.

A scalable ledger database provides these guarantees:

1. Immutability: every change to the ledger is recorded such that past states of the ledger can always be retrieved.
2. Double-entry: the API enforces that money cannot be moved without specifying the source and destination of funds, following double-entry accounting principles.
3. Concurrency controls: money can’t be double-spent, even when transactions are written out of order or in parallel.
4. Efficient aggregations: it’s fast to compute aggregations of financial events, such as the sum of all events in a particular time frame.

We aim to show why any company that moves money at scale should invest in a ledger database.

The fintech stack

Product engineers speak the language of their domain models—orders, rides, and bookings. As their products scale, these engineers start hearing another language from the finance team—credits, debits, assets, and liabilities. Invariably, engineers get assigned the dreaded task of translating our domain models into double-entry accounting.

This is no easy feat. Financial events:

1. Come from disparate sources. The app may rely on multiple fintech SaaS solutions (like issuer processors, card processors, and ACH processors). App data models may be spread across multiple databases and be owned by different teams of engineers. Corporate bank accounts contain fungible cash—attributing each dollar to its source is not possible.
2. Speak different languages. Every fintech SaaS product has its own set of APIs that require separate integrations. App data models may be a mix of new and legacy systems, and documentation is likely scarce. As we know well at Modern Treasury, banks send transaction information in a wide variety of arcane formats that are difficult to parse.
3. Are generated from mutable underlying records. None of the sources of financial events have a rock-solid, consistent way to reconstruct why a pool of money was a certain value at a certain time.

Many companies that move money as part of their products take the translation approach, however. Finance teams learn to accept that reconciliation can only happen accurately at the aggregate level. Attributing money movement back to the source application event is always a manual task, and in some cases, isn’t possible.

A product leader at a large consumer payments company recently told us that every month, the finance team would notice that a few million dollars had gone missing in their ledger, kicking off a mad scramble to figure out where the money had gone. Every engineer that has built money moving software has a similar story. In these stories, it’s not the funds that go missing—they’re still sitting in a bank account. It’s the attribution that’s lost—the database record of who they actually belong to.

Even worse, these ledger errors ultimately become customer-facing. A small business on a marketplace platform will ask questions when a sale they expected to appear in a payout isn’t there. A digital wallet customer will churn if they send money to another customer, but it never arrives. Regulators will ask hard questions if a loan customer initiates an on-time payment, but your system marks them as past due because the payment was recorded late.

The solution is to be double-entry and immutable at the source of financial events.

Just like a unique index in a relational database gives engineers confidence, a ledger database that enforces double-entry rules makes it not just difficult for money to go missing, but architecturally impossible.

Why doesn’t every company that moves money use a double-entry ledger to power their products? Because building a ledger that both follows double-entry rules and is performant enough to run live apps is a difficult engineering problem. The most successful fintech companies have taken years and dozens of senior engineers to migrate their single-entry systems to double-entry. Only recently has it been possible for companies to get the benefits of double-entry accounting without a massive up-front build and costly maintenance by using purpose-built ledger databases like Modern Treasury Ledgers.

Next Steps

This is just the first chapter of a broader technical paper with a more comprehensive overview of the importance of solid ledgering and the amount of investment it takes to get right. If you want to learn more, download the paper, or get in touch.

Read the rest of the series:

Part II | Part III | Part IV | Part V | Part VI