The SC #134
Issue #134 of The SC. Weekly supercomputing news. Updates from the OCP Global Summit with rack designs, accelerators and networking with a side of opinions about where that’s taking us.
A good chunk of the HPC/ supercomputing news from last week comes to us courtesy of the OCP Global Summit which took place in San Jose. As always, all the details are down below but there’s a couple of interesting patterns that aren’t which I wanted to bring up.
I appreciate it may not be a stated goal of the OCP to attempt any form of standardisation of hardware specifications, non the less, we are in a rather interesting position with Nvidia’s latest releases to OCP of its Vera Rubin based updates for rack scale computing vis a vis the Helios architecture which built upon the double width racks provided by AMD. i.e. we have two very different, competing, hardware layouts for large scale accelerated compute. Which only gets more interesting with AMD similarly named, but completely different, single width Helion racks which were also announced last week.
Either way, we are definitely staring down the barrel of a unit of compute creeping from a 19 inch rack mount sled to the entire rack. Not only are we seeing increasing rack scale designs and deployments we are also seeing sleds that are tightly coupled to those designs. In a similar way (and for partially similar reasons including reduced sizes) as we see in consumer hardware ,we are seeing designs move from standardised parts to bespoke (even if open sourced) designs. Although the initial signs aren’t exactly promising, I’m hoping this doesn’t lead to the same kinds of levels of repairability and interoperability and lock in that we see in consumer hardware (think Macbook Pro).
And then you have all of the high speed networking standards and complexity layered on top of that. Topic for another day, I think.
Surely though, all of this has got to make your run of the mill Top 500 supercomputer or enterprise deployment cheaper and easier? So much of what was once completely bespoke to a literal single machine is now becoming defined in OCP standards. Plural. Choice is good thing, but sometimes the limitation can foster more progress. A 19” standardised rack size, MPI, HTTP none of them are perfect, but there being just one (commonly used) made a lot of things easier.
Intel has gone from being on its death bed to having not only strong CPU sales but back in the GPU market with its Crescent Island air cooled PCIE chips. These seems to be firmly targeted to the AI inference market with large VRAM sizes to boot. Couple this with Microsoft’s announcement of increased investment into Maia (their own AI inference accelerator) with the 300 coming in 2027 in even larger quantities than the 200 and it seem everyone wants to take a chunk out of Nvidia’s business.
Curiously, that was a topic that came up quite a lot at HPC Club last week. It seems the immediate question people had if given the opportunity to acquire cloud capacity across multiple providers easily was to then ask about how to use multiple architectures (including accelerators) easily.
Lastly, it’s always fun to watch the AI bros slowly discover things the rest of the grownups in the room have known for around three decades at this point. Measuring utilisation at the CPU/GPU level is important for understanding how well your hardware is used or how well your software can make use of it. It’s worse than useless at understanding if your training run will complete or if you’ll deliver your risk reports on time. Not a single risk system I’ve ever come across has an observability stack that monitors utilisation as its primary goal. Sure, you have to be able to tie back to that, but progress on your actual task, how many portfolios and risk types have completed, how long they took compared to previous runs far closer to being front of mind. I have no doubt similar methodologies prevail across weather/ climate modelling, EDA etc.
Have a good week folks.
All the News in Depth
Cloud and vendor releases for Supercomputing & HPC (and AI too if you change the filters)
Nvidia open sources more of its data centre rack scale tech used for Vera Rubin
Microsoft goes deeper on its own accelerators with Maia 300
Intel looks increasingly less dead and even back in the GPU game
HMx Labs Updates
A few thoughts about the intersection of creativity, art, code and AI.
HPC Club last week went rather well! If you missed it you can find details about ORB here (Flamur requests you give the repo a star please!) . We’re looking into making the presentation slides and recording available.
Off Topic
I had a lunch meeting with an AI startup earlier this week. I was more than a little surprised at their lack of vision or understanding of what causes AI pilots in the enterprise space to fail. I then came across this video in my feed that covered many of the same points I tried to make to the founders of that startup.
Sort of related to the piece above about AI but more directly visible, AI created interfaces
Know someone else who might like to read this newsletter? Forward this on to them or even better, ask them to sign up here: https://cloudhpc.news
