AI Adoption (in FSI)
I don’t often talk about AI but I thought I’d share a few insights and join a few dots on what I’m seeing on AI adoption within financial services.
There’s a lot of talk lately about where the AI industry is in the Gartner hype cycle. I won’t try and answer that directly, but I thought I’d share some insights from my own conversations.
Before I get to that though, I think we need to reverse a few weeks (yea I know that’s centuries in AI timelines). I’ve previously stated that one of the biggest problems with AI tooling is the indeterminate nature of its output. Exactly a month ago, Thinking Machines published a paper on fixing this.
The second major barrier to adoption is of course, the rather kindly named, hallucinations. Or just being plain wrong in normal people speak. Also last month, OpenAI published a paper on why AI models hallucinate. There’s also been previous work on detecting hallucinations though I’ve not seen much practical output resulting from this.
If, and I say that with a large degree of caution and scepticism, if the AI industry can solve both of these and provide models that are deterministic in their output and do not hallucinate, we can actually think about how to integrate them into real business processes. (Though I concede these may not be models you’ll want to use in the manner in which you’ve become accustomed to today for things such a brain storming or idea generation).
Which leaves only one major unsolved problem. Economics.
Which is where some of my recent conversations come into play. I still won’t chance my arm and say where we are on the Gartner hype cycle, but I am seeing an increased interest in adopting AI for automating business processes within financial services.
There’s a key difference to what I read about how the rest of the world is doing it.
Much like cloud adoption, the process needs to start with what the automation will cost. Both to develop and adopt and for ongoing operation (CapEx and OpEx). To this end I’m not seeing a mad panic to go out and buy thousands of GB300s. I am seeing a large desire to share the existing compute infrastructure (including sometimes large GPU estates used for financial risk) with AI workloads.
I am seeing a desire to run this expensive compute at utilisation levels of over 80% in a similar way to the HPC estate. Utilisation rates of millions of dollars of GPUs in the 20% region is not going to be acceptable.
To achieve this there’s a good degree of interest in running smaller, quantized and/or distilled models, for specific use cases rather than frontier models.
If you’ve made it this far, you’re clearly interested in this. I’m meeting with a few friends to talk about this on Tuesday evening. Nothing major, just a few key players having an interesting conversation. DM me if you want to join us.
You know what… this is probably the most optimistic I’ve been LLMs. Hopefully the market implodes and I can pick some up cheap to get this all worked out 😁.
