Recon Diaries, Entry II: Banks are the Enigma

In the first entry of the Recon Diaries, we explained what reconciliation is and why it’s important. We illustrated the need for two separate sources of truth—one internal, one external—that map against one another to find incongruities. These incongruities represent one of two possibilities:

1. Expected money movement that did not happen.
2. Unexpected money movement that did happen.

This process is important because reconciliation underpins a strong product offering and a sound business operation. It confirms that a business is accurately managing, tracking, and attributing payments both within their own business (i.e., first-party) and on behalf of others as part of their products (i.e., third-party).

In this post, we’ll dive into the process of reconciliation, illustrate the reconciliation “enigma,” and explain how the Modern Treasury “Recon Squad” cracks reconciliation to achieve 99.9% accuracy.

An Example of the Difficulties

Let’s illustrate with an example. We can revisit the University of Modern Treasury (UMT), the esteemed institution for higher learning. As you might recall, UMT built an internal ledger and operational process to capture and memorialize all expected payments for their development office. As a result of these efforts, the school now has two separate, standardized data sets that they can reconcile:

1. An internal database (i.e., company ledger) that collects expected money movement.
2. An external database (i.e., the bank ledger) that captures actual money movement through the banking system.

However, as UMT compares these two data sets, they begin to notice an issue. The two datasets often don’t appear “apples-to-apples.” In fact, sometimes, the datasets don’t resemble each other much at all; the UMT internal database is rich with transactional and contextual information, whereas the bank statement is sparse and cryptic.

Due to these differences in data, the process of reconciling these transactions, although properly architected, remains operationally intensive and inefficient.

Why is Reconciliation Difficult?

Reconciliation becomes a difficult matching game because the banking system introduces data disruptions while processing the payments. This means that the payment message that enters the banking system doesn’t resemble the payment message that exits it.

There are many causes of these disruptions; in fact, too many to list in this article—payments, as we know, revolve around the edge cases. For simplicity, we’ll focus on a few of the structural features of the banking system that make data disturbances inevitable, even if all goes right (i.e., the happy path). These include: Batching, Timing, and Data.

It’s worth noting that these features are a product of the history of finance, and a consequence of the inherent limitations of mainframe banking infrastructure, which is built almost entirely on COBOL.  Matt Levine captures the strangeness nicely: 

“Everything in finance is accreted on top of a lot of other things in finance. Everything is weird and counterintuitive, and you often have to have a sense of financial history and market practice to understand why anyone is doing any of the things they’re doing.”

The US payment system and reconciliation is, unfortunately, no different.

1. Batching

The US banking system is a batch processor, which means that the banks process payments in aggregate batches at specific times throughout the day, as opposed to processing individual transactions as they happen. This means that if UMT sends ten ACH debit transactions to their bank, the bank might return only three transactions that collate those ten debits into three line items.

Batch processing grew out of the development of clearing houses. In the 18th and 19th century, check volume grew substantially, which made bilateral (i.e., bank-to-bank) check clearing operationally untenable. As a result, the first check clearing house was established in New York City in 1853. The clearing house facilitated the exchange of checks, and calculated the daily net settlement for each bank. With net settlement, banks fund or draw upon their settlement accounts, which helps to reduce the amount of deposits a bank must hold in reserve and mitigates interbank counterparty risk.

Net settlement underpins many of the most common bank payment methods, including checks and the Automated Clearing House (ACH).

2. Timing

As a corollary to the batching process above, bank payment networks also don’t settle transactions in real-time. In fact, depending upon the payment rail, it can take anywhere from one to seven days for a transaction to clear and settle. 

This creates significant reconciliation difficulties. One must often match transactions against payments that were sent days earlier. In addition, for companies doing batch reconciliation, they must make sure to reconcile corresponding data sets across varying settlement times no less.

3. Data

Bank transaction reports do not contain sufficient data to properly reconcile all transactions. For example, wire payments rarely include enough data to correctly attribute to the appropriate sender. ACH payment reports have tightly structured data feeds that limit the ability to consistently match against your internal systems.

Bank payment files are a product of their environment. They are exceptionally spartan because they had to be. The US electronic payment system was established in the 1970s during the early days of computing. Compute and storage were limited; processing was far less capable than it is today.

To process these electronic payment messages, banks relied upon (and actually still rely upon) mainframe computers running on COBOL codebases. To give a sense, COBOL engineers wrote dates into their programs with two digits instead of four, “71” vs. “1971”, because, as a COBOL engineer recalled, “all programs had to be very memory conscious–every byte used to be expensive.”

In sum, payment systems were not designed with reconciliation in mind. They were designed to be processed at high speed and low fuss by underpowered computers with stringent data considerations.

How This Impacts Companies

Due to this legacy, the process of reconciliation takes inordinate time and effort. It’s a task designed to frustrate.

The UMT Development office, and companies across the world, dedicate massive effort and time to manually decipher the recon puzzle by mapping payments on their internal dataset against the output on the bank statement. Try this simplified matching exercise below. How long did it take you?

Of course, doing this work manually doesn’t scale. As UMT expands, adds more fundraising levers, and grows payment volume, they must also augment their back-end operations teams to meet their reconciliation needs. In fact, beyond UMT, across the economy, companies, both small and at the Fortune 500 level, hire armies of employees to manage their reconciliation processes.

Fortunately, it’s a problem that computers and software can solve.

Reconciliation as Cryptography

At a conceptual level, reconciliation presents an interesting code-breaking challenge. Instead of an encipherer, encrypting a message to make it unintelligible, reconciliation has a legacy banking system that convolutes payment messages via batching, asynchronous clearing, and inherent data limitations. Nevertheless, the processes and goals are strikingly similar:

1. Cryptography involves overlaying two messages—one that is encrypted, and one that is not—and finding the key to prove that they are the same.
2. Reconciliation involves overlaying two payment messages—a transmitted payment file, and a received bank transaction file—and proving that they match, or reconcile.

In both cases, the objective is to show that the two messages that may appear apples-to-oranges, are, in fact, equal. Consider the two workflows below:

To summarize, the realities of the US banking system (i.e., the batching, the timing, the limitations of mainframe bank computers written in COBOL) function as encryptions by introducing noise or data distortion into a payment message, which is then encoded within a separate data file, called a BAI2 (Step 4). Reconciling a payment involves mapping that BAI2 back to the original payment message and––to extend the analogy––unraveling the puzzle, breaking the code to identify the match.

As demonstrated above, deciphering this puzzle can be achieved on a small scale manually via investigation and brute force. However, at a certain level of size and complexity, it becomes difficult to do without a repeatable program, a predictable de-cipher.

The “Recon Squad” are Code Breakers

Ciphers code messages predictably. Code breakers aim to “break the cipher” by finding the key, which then reliably unravels the coded message. To do this, the code-breakers search for predictability and redundancy in the encoded messages. They look for patterns.

In the clip from the film The Imitation Game above (although certainly apocryphal), “Cilly” at the start of a certain message was enough of a pattern to break the German Enigma code.

Similarly, the banking system—and even specific banks, we’ve found—introduce noise within payment messages in predictable ways. Therefore, by using similar techniques to code-breakers, like pattern identification, the “Recon Squad” at Modern Treasury is able to solve reconciliation (with 99.9% accuracy). The team employs strategies and techniques, which include formatting and data enrichment, to break the reconciliation enigma and arrive at near-perfect automatic reconciliation.

In the posts ahead, we’ll talk about some of these patterns and tactics. In the meantime, if you want to learn more, reach out to us.