Flux Framework, ISC & HPC Club
Super quick intro to Flux Framework and their workshop at ISC 2025. Oh and yea its HPC Club day!
It’s HPC Club day! And since it’s all about schedulers this time, I thought I’d talk about one of the two workshops I did at ISC last week. Flux: Next Generation Workload Management.
But before I get into that… Earlier this week, as I was sat in a meeting talking about HPC data planes, I was asked when on earth I get the time to write. Just as Monday’s scheduled post dropped into someone’s inbox with a quiet but quite audible ping 😁. Whilst you’re reading this on Wednesday (probably) I’m writing this on a sunny Tuesday evening sat under a tree at Northwood Recreation Ground as I wait for my daughter to complete her Cubs session, she’s playing with paracord right now. So I the answer is, I write whenever and wherever I find the time.
Anyway, back to the topic at hand. It’s rare that I see something billed as next generation and actually agree. Mostly it’s the same old stuff. Tweaked at the edges. A fresh layer of the most fashionable tech pasted over the top. Whilst Flux Framework isn’t quite next generation (at least to me) just yet, I am quietly confident that it very much will be.
Huge thankyou to the team from Lawrence Livermore National Laboratory for the workshop and a particular thankyou to Vanessa Sochat for putting up with my incessant questions and ramblings in the couple of weeks leading up to it!
I won’t even attempt to teach you what little I learnt about it during the workshop, but I did want to talk a little about one specific feature. In a world, and ISC, where everyone is talking about how the future of HPC is going to be heterogenous (it already is quite frankly) I see very little being done to actually deal with that. Flux is one of those rare exceptions.
You see it models resources, be that CPUs, RAM or accelerators as a graph. Workloads are then matched against this graph. But that’s not all. It is possible to add new and arbitrary things as resources within this graph. Want to add CPU architecture? Even CPU flags? No problem. Data? You got it. Ice cream shops? Vanessa even has your back for that! (I’m not even kidding).
I’m genuinely gutted that we aren’t able to have one of the Flux team at HPC Club later today but hopefully some of the above can help influence our conversation.
See you all this evening!
More about Flux Framework
Sub 3min intro to Flux Framework
ISC Workshop slides (another big thanks vsoch for pushing these last night!):
flux-framework
