Vibing Banking Software

I don’t often talk about current affairs, but James Cuff has goaded me into this one. So here I am giving you yet another opinion on IBM, COBOL and Claude Code. At least this time though its coming from someone who has spent most of his life working inside banks.

I’ll admit upfront, I’ve never written, or even read, a single line of COBOL in my professional career. So, feel free to ignore this if you like. Even without that though, I can say with some confidence that I don’t think Anthropic’s claimed ability to vibe your way out of a legacy COBOL codebase will be true anytime this decade for two major reasons.

And neither of them depends on Claude’s ability to actually do the job,

Firstly, the techbros and analysts that are now pushing this vision have clearly never worked in a bank. These are institutions that have written their own programming languages. I’ve come across at least 2 in my short career. And I don’t mean little DSLs either, I’ve seen more of those than I can remember. I mean full on compiled programming languages. This is an industry that decided it would create its own LLM rather than adopt one from the frontier labs. People working in technology in banks probably saw that announcement and groaned. Not because they don’t believe Claude can do the code migration but because they know it will be 20 years before they can get their hands on it. 

Yes, I’m aware of pilot programmes to provide access to tools such as Claude and Codex. A handful of experimental users with no access to any critical codebases is not going to change things anytime soon.

The second is to do with how banks test their software. Yes, there are unit tests and so forth, but the largest and most complex systems still depend, to a huge degree, on manual testing by QA teams. I am aware of exceptions to this, both teams within a bank and some places that have been more successful than others with the whole “idea to prod in a day” concept than others. Those tend to be much newer and actively maintained systems. Forget COBOL you’d be surprised at how much most banks depend on 20+ year old C and C++ code that is probably riddled with security holes. And the testing processes that go with them. If your test cycle (including regulatory sign offs where required) can span months, compressing software development doesn’t help as much as you might think. Especially if that test cycle often results in a few bugs. (Hint: it always does). Without fast feedback the velocity gains brought by AI are largely lost.

So, everything is right with the world then, yes? I’m not so sure.

I have this fun little vantage point where I still often work embedded with our clients and experience what it is like working in technology inside a bank and then on the same day might work on an internal project at HMx Labs that isn’t bound by the same constraints. Several years ago, I did a little experiment where I tried to perform the same task inside a client and at HMx. There was a 100x difference in timescale and that was before ChatGPT took over the world. I suspect that gap is going to be even greater now. I can’t see this as being sustainable for the banking industry. Something is going to have to give.