The SC #135
Issue #135 of The SC. Weekly supercomputing news. Updates from Nvidia GTC and I do my best and giving you a take that doesn’t just echo the hype.
It’s rather serendipitous that the week in which Nvidia chose to announce DSX Pod, designed to accelerate deployment of AI factories, is the same week in which Microsoft pulled out of its deal with Nscale to use its Loughton, UK data centre due to a lack of power.
The idea behind DSX Pod isn’t really new. It follows a familiar pattern trod by things such Huf Houses, pre-fabricated concrete, block based ship building or even modern aircraft construction. I’m not sure it’s particularly novel even within the context of supercomputing with container based supercomputers existing, even if they are somewhat niche.
I’m less interested in what this means for building new AI factories than I am in what it means for more classical users of supercomputing. Where you’d once have engaged, possibly a consultancy plus the likes of Bull or Atos to design, procure and install a supercomputer, can you now just place a call to an ODM and have a number of DSX Pods delivered and hooked up? Critically, will this actually be any cheaper?
Coming back to the power constraints, of course, Nvidia and partners have things within the DSX ecosystem to address this too. It looks very interesting though we are definitely talking about tech in the lab rather than in the field. Links below for more details.
The other announcement from GTC last week that caught my eye was cuDSS, a new CUDA based library designed to accelerate the sparse system equations often used in engineering simulations. There’s certainly a good story here and even practical real world use cases showing around 10x acceleration. That sounds amazing. I think it’s nowhere near enough.
While time to solution is an important metric in supercomputing problems, on its own it means very little. What constrained the time to solution? How concurrent is the problem? If we’re generous and assume that the 10x improvement is a genuine 10x in the limit case, i.e. this is the only way to buy further improvements in speed and the problem can’t just be made more concurrent, then in those use cases where time to solution is the key metric, this could be important. That’s a pretty small niche within a niche though.
More critically, for most commercial use cases the 10x speedup is not going to be enough to offset the additional cost of running on GPU vs CPU. I’ve seen this movie before. Moving financial risk analytics moving to GPU and then sometimes back to CPU compute. CPU based compute can cost as little as $0.01 per vCPU hour. At current prices we’re looking at around 100x that for a H100 for an hour. Even with a 10x speedup the commercials of moving that calculation to GPU based compute are going to be questionable, especially if it can be made more concurrent and use cloud compute on par for what you eat basis.
Before you throw your engineering team all in to migrate everything to cuDSS and order a new DSX Pod, maybe just run the math. Check if your workload can break down to run more concurrently first too.
Lastly a couple of updates from us at HMx Towers. We’re hiring! Further details in the link below. Secondly, I’m experimenting with being even less coherent in a post from last week as a precursor to perhaps sharing a little more about how we’re building out HMx Labs. Let’s see how that plays out.
Have a good week all.
All the News in Depth
Cloud and vendor releases for Supercomputing & HPC (and AI too if you change the filters)
Lack of power continues to frustrate data centre rollouts
Nvidia DSX Pod and the rest of the ecosystem as well as third party take from HPC Wire
Karman over-provision GPUs in the power envelope
And Emerald AI goes the other way
Nvidia cuDSS and HPC Wire's take
HMx Labs Updates
We’re hiring! If you know someone who might be interested in coming joining us please point them my way, I’d really appreciate it!
Not that many of my random thought are particularly coherent anyway, but I thought I’d try a little experiment with one that’s even less cogent. Depending on how well this works I might start sharing a little more in this vein, the gritty bits of building out HMx Labs. Frugality vs Profligacy.
My baby sister got married last weekend, that’s pretty off topic right? 😁
Slightly back on track, we saw a huge number of cybersecurity vulnerabilities disclosed last week. Not particularly supercomputing adjacent, but how long till they are?
Fireship summary because, well, at least he makes it funny and otherwise you’d just cry
Know someone else who might like to read this newsletter? Forward this on to them or even better, ask them to sign up here: https://cloudhpc.news
