Hard drives on backorder for two years as AI data centers trigger HDD shortage

DO NOT assume SMART is reliable. You can wipe SMART stats or write any values you want.

You have to actually examine the real bits on the drive. Resellers don't want to take the time to actually zero a drive, they usually just nuke the partition table.

You also need to physically examine the drive. Corroded fingerprints on the PCB, wear on the port contacts, scratches from mounting rails, etc.

That's how it found out that the last "new" drive I bought on Amazon was actually a used Backblaze drive. It contained terabytes of customer data, and a shit ton of cleartext files. SMART, of course, reported it was a brand new drive with zero hours. Cleartext logs on the drive showed many thousands of hours of runtime.

Physical examination is the only reliable method.

Question for all of you more knowledgeble than I: can SMART data be tampered with? When I get, say, a refurbished Mac from Apple, I'm trusting Apple won't stoop to that. But a SSD vendor I've never heard of?

I have given up on Amazon and similar "market makers" (bol.com is a regional one here) completely. Too much fraud. Instead I use specific vertical or store as sourced outlets.

Nearly any product you can buy from Amazon, even when it says shipped from Amazon, is suspect.

I wouldn't shop there at all. It's a literal scam market. Allegedly.

> avoid Amazon/ebay channels,

But why ? They are the good ones. TEMU and Shein bad. /s

It’s a cartel just like the OPEC oil cartel.

Worse than nukes race? I really much prefer being slightly inconvenienced when buying hard drives to risking being obliterated along with billions of other people and having all of civilisation wiped out with hundreds or thousands of years until recovery.

Maybe the real paperclip maximizer was the irrational exuberance we made along the way :')

> Worse than nukes race, which at least did not affect common man At least not visibly, we were deprived of a clean source of surgical steel and many in the areas were uncompensated for sicknesses they experienced as a result of testing

I just find it so odd that the oh so environmentally conscious tech sector is now building out some of the most resource intensive facilities on the planet. It's like environmentalism was just a con.

> No one really gains anything.

That's not true I've seen loads of amusing videos on Instagram.

>Worse than nukes race

That certainly is a take.

Maybe when the bubble bursts there will be lots of good used hardware at massive discounts?

Don’t worry, OpenAI is about to implode.

All that manufacturing demand led to enough free machinery to get those free Ubunutu CD's! ;)

That's somehow unsurprising given how many AOL CDs I saw at checkout of every grocery store

Intelligence. We're lacking intelligence.

Water? Clean Air? ROI?

* True innovation

Every tech startup circa 2023 doesn't do anything without smearing AI all over their front page.

Makes me wonder about the things that go into datacenters and power plants:

- Air conditioners

- generators

- Racks/Cages

- fire supression

- Concrete, Steel, power lines

etc

A coherent regulatory framework? An honest conversation about whether we need all this?

Profit too. We lack profit.

I have a hypothesis that it was AI, not COVID/sanctions/etc, that was mainly responsible for the 2020-to-ongoing "chip shortage." Ignoring companies with their own fabs (Intel) and companies with pre-existing reserved-capacity contracts with fabs (Apple), everyone else is stuck waiting in line behind batch after batch of fab orders from Nvidia.

Downstream of that, AI is effectively also responsible for the current generation of game consoles never declining in price.

Because game consoles are fixed platforms that continue to be manufactured over 5+ years, normally the most expensive parts in the system (the CPU and GPU) would gradually get cheaper to manufacture [and in turn, cheaper to buy] over the course of the console's lifetime—which was often passed onto the consumer in the form of the console's MSRP gradually decreasing. Either the process node for the console's silicon design would stay fixed, and demand for this process node would gradually decrease as larger fab customers move on to newer nodes, decreasing the (effectively auction-based) pricing for fab time on the older node; or the console manufacturer['s silicon vendor] would put their silicon design through a process-shrink, and so, while still paying top dollar for use of the fab's newest node, would be getting more chips per wafer out of that, and again could charge less.

But instead, what we've seen since the start of the AI boom is that there's no longer any price-reduced timeslots to be sold to manufacture the low-BOM parts for these price-sensitive console-maker customers. Instead, both Nvidia and AMD are now getting such high value out of even the older nodes, that 1. the fabs know they can squeeze them, charging full price for those slots as well; and 2. both Nvidia and AMD, in their roles as silicon vendors to the console makers, haven't been able to justify using (very much of) the time they're paying so much for to fab low-BOM-cost parts to fulfill their pre-existing outstanding purchase orders, when they could instead be fulfilling much-higher-margin new POs from the hyperscalers.

Thus every console of the ninth generation (PS5, Xbox Series, Switch) still selling for their launch price (with no help from process shrinks); thus none of these consoles having been able to be produced in excess of demand to the point that supply-and-demand ever drove retail prices below MSRP; and thus the only tenth-gen console so far, the Switch 2, taking ~4 years longer than anticipated to release†.

---

† Nintendo were very likely waiting for Nvidia to run off enough of the Tegra T239 with a sufficiently low passed-on fab cost, for Nintendo to both 1. be able to build a backstock and non-run-dry pipeline of Switch 2s, and 2. be able to be positive-margin on charging the same price as the Switch 1 for them. They waited four years, and neither thing ever happened; so they eventually just gave up and priced the Switch 2 higher, and also built out an entirely novel D2C + online-marketplace-partner distribution pipeline so they could ration the tiny initial supply of units they had been able to build with the chips Nvidia had supplied them so far.

Though, that being said, Nintendo actually got a double whammy here. They were also waiting for fast NAND to come down in price, so that they could have physical game cards manufactured for a trivial BOM price while still enabling the "direct GPU disk-streamed assets" pipeline that games of the last generation had begun relying on. Obviously, as today's article points out, that hasn't been happening either! Thus game key cards; thus SD Express cards only beginning to trickle out, with no sizes above 128GiB available at the Switch 2's launch time; and thus those SD Express cards being ridiculously priced for their capacity compared to equivalent transfer-speed + die-size NAND (as seen in e.g. low-profile/flush-mount flash drives.)

* Time to wade through all the AI slop

* Trust that what we are watching or reading is human-sourced and real

* Time for real creators' unique voices to remain unique

* Hope for society's future

Well, at least the paperclip maximizer is not running out of water and carbon.

Definitely lacking profit. OpenAI is on track to lose $60B this year.

Don't forget rising energy prices.

jobs...

Yup. Walked into a microcenter a few weeks ago and was surprised and saddened by how expensive everything has gotten.

Chia was/is WORM (write once , read many) so not sure how these drives were heavily abused? As a storage nerd, I bought many 18tb drives for chia and they are still spinning just fine in my storage arrays. It’s been five years.

Yes, but QLC has much higher density.

I think it's the higher density that makes it better for cold storage, which generally has infrequent access, and more reads than writes.

Hence the QLC's endurance being "sufficient for cold storage".

Cold storage normally doesn't have frequent writes or frequent reads.

Ssd for cold storage seems like an odd choice in itself. If that’s genuinely done due to availability then we really are in for a wild ride

I think it is solidigm that has started to argue that with a 128 TB QLC drive constant writes at the maximum write rate will hit the drive’s endurance limit at about 4.6 years. The perf/TB of these drives is better than HDDs. The cost per TB when you factor in server count, switches, power, etc., is argued to favor huge QLC drives too.

In short: Aggressive overprovisioning.

Enterprise SSDs are not expensive only because they have better flash chips, but they have much more of them.

A top of the line write oriented SSD comes with 4-7x more capacity than what it says on the tin, but that extra capacity is used for cell replacement rather than capacity itself.

Mixed use comes with 2-4x overprovisioning, and read oriented is around 2x IIRC.

$5 a TB is insanely cheap and someone just getting rid of stuff and not really caring about the price. Nothing has ever been that cheap at retail or manufacturer recertified. Lowest was like $9-10 last year and now we are sitting at more like $15.

Cloud storage and on-prem storage vendors have long provided deterministic block-level dedupe via hashing, without expensive LLM content analysis.

GPUs yes, but there'll be no HDDs making it alive, they'll get destroyed to protect whatever rubbish they had on.

My understanding is that refurbished GPUs from e.g. crypto mining are something people are warned away from, because they've often been run into the ground. Are AI usage practices different?

> And when the AI bubble bursts, "refurbished" HDDs and GPUs will flood the market.

GPUs? No way. The datacenter cards don't even have video output ports, and I think the chips destined for AI / ML training also have everything video/render related removed from the silicon, makes for more yield.

And the other way around, using (cheap) consumer GPUs in servers, I think at least NVDA tries to prevent that with driver-based DRM, so there won't be any flooding coming from there either.

If this bubble pops you might need that money for food when bananas go from $1.50 to $150.00

> This is a long term good thing. ... By 2028 or so we'll have a 50% drop in price-per-storage for these components.

Per the op:

> and the ongoing DRAM shortage is proof of this, with memory kits costing more than double what they did just a few months ago.

> While enterprise-grade QLC SSDs would entirely power this pivot, Sandisk has already raised NAND prices by 50%, according to another DigiTimes report, after initially warning of a 10% increase two months ago.

So you're basically saying prices may return to normal in two years, and that's somehow a good thing compared to them not being inflated in the first place?

> By 2028 or so we'll have a 50% drop in price-per-storage for these components.

Do you mean relative to six months ago, or now? Because a lot of the prices have already more than doubled.

(I’m upset because the computer I’ve been planning to build, which three months ago would have come to around ₹90,000, is now up to ₹1,20,000 and climbing week by week, half due to price increases on the same part, half due to forced substitutions on RAM since the cheaper 32GB 6400MT/s DDR5 sticks are completely unavailable. And looking into laptops, for the first time ever I’m seeing manufacturet SODIMM or SSD upgrades being cheaper than aftermarket.)

Like the GPUs? How they were up since the bitcoin craze and are still up to now? Yeah I dont think these prices are ever going back down. Specially not since people are buying at these prices and will continue to pay these prices. Maybe HDDs will get phased out though, since you might as well buy SSDs if you are overpaying that much.

Will the AI bubble last until 2028? I’m still unclear how AI will return even 10% of this investment in profit.

Same. Wish I bought 2x 12TB drives now rather than 2x8TB.

Same here, though I did it a bit earlier. Data center SSD's I bought a while back for 50 euros each from Ebay now sell for over 200 euros. Similar increase for HDD's too.

Maybe in 3-5 years there will be a fantastic time to upgrade again, after AI bubble has burst and slightly used stuff gets dumped from data centers to online marketplaces.

I think it's still going to be a while. Go talk to anyone who isn't really tech savvy and you still hear all the same hype, now more than ever. None of them can tell you how they've actually used it to create much, if any value, but they still think it's the second coming of Christ.

The funniest thing about this is that, with high GPU prices, rising RAM costs, and now increasing SSD prices, Apple will end up producing the most affordable PCs.