AMD's Freshly-Baked MI350: An Interview with the Chief Architect

That top comment doesn't seem to have engaged completely with the context here. AMD fumbled trillions of dollars of value creation by mis-identifying what their hardware was for. Or perhaps it is more correct to say by being too dogmatic about what their hardware is for. They weren't in a position to be picky. They had a choice - they could continue making trillion-dollar mistakes until their board got sacked and the exec team replaced. Or they could maybe listen to some of the people who were technically correct regardless of their size in the market.

George is just some dude and I doubt AMD paid him much attention anywhere through this saga, but AMD had screwed up to the point where he could give some precise commentary about how they'd managed to duck and weave to avoid the overwhelming torrent of money trying to rush in and buy graphics hardware. They should make some time in their busy schedules to talk with people like that.

People get on board with George Hotz because they share the frustration of using ROCm on consumer GPUs, where the experience has been insultingly dreadful to the point where I decided to postpone buying new AMD GPUs for at least a decade.

I'm not quite sure why he decided to pivot to datacenter GPUs where AMD has shown at least some commitment to ROCm. The intersection between users of tinygrad and people who use MI350s should essentially be George himself and no one else.

Most of those willing to work with AMD are very small players (with some notable exceptions). They are likely hopeful that the small players will grow.

So nVidia has a huge software lead because of open source developers like you? Or because people employed by nVidia write closed source high performance drivers and kernels? Or because the people employed by Meta and Google that wrote Torch and Tensorflow built it on nVidia?

I am really sympathetic to the complaints. It would just be incredibly useful to have competition and options further down the food chain. But the argument that this is a core strategic mistake makes no sense to me.

> if researchers don't get access to hardware X, they write their paper using hardware Y (Nvidia).

There are plenty of research institutions that can easily spend >$250k on computational resources. Many routinely spend multiples of that volume.

They'll be fine.

Serious researchers use HPC clusters not desktop workstations. Currently the biggest HPC cluster in North America has AMD GPUs. I think it'll be fine.

A big chunk of NVidia's current price is a reflection of lacking meaningful competition. So straight comparison isn't quite fair: if AMD started to do better, the gap would shrink from both ends.

Now imagine you don't have the resources to satisfy N00000 customers. What do you do?

The hobby market should be considered as a pipeline to future customers. It doesn't mean AMD should drop everything and cater specifically to them, but they'd be foolish to ignore them altogether.

The hobby market is how you get 'market default' N years later

I mean broke compared to Meta or OpenAI.

AI research used to be fringe and not well funded.

Back in those days, 99.9% of hardware was Xeon.

The higher power requirements of datacenter GPUs are mostly from running hardware well past a reasonable point on the efficiency curve to eke slightly higher generational increases in performance. Nvidia for example has three versions of the RTX PRO 6000. One runs at 600W while the other two run at 300W. The main differences are the power target and the cooling solutions. If I recall correctly, the performance difference between them is less than 20%, despite a 50% reduction in power consumption, for what is effectively the same hardware. This can be confirmed by changing the power target of the 600W version to 300W and benchmarking the before and after. Plenty of people have made similar observations of AMD’s hardware.

That said, I am confident that Nvidia will continue serve those of us who want our own hardware.