Shai-Hulud Themed Malware Found in the PyTorch Lightning AI Training Library

>This might just be the frequency illusion at play, but there seem to have been a number of high-profile supply chain attacks of late in major packages.

It's real. As of the beginning of April we'd had 7 in the past 12 months vs 9 in the two decades before that: https://www.jefftk.com/p/more-and-more-extensive-supply-chai...

> idiot friendly tooling is somewhat lacking for projects which start as hobby/amateur code but end up being a dependency in many other projects.

Historically, extra-security-scanned artefact handling has been a paid enterprise option. Whereas the less secure option is the much-less-hassle default.

IDK how good a business model this is, I suspect not very.

People are ramming tons of code into places without ever looking at it, it would follow that supply chain attacks would increase thusly.

FWIW left-pad was not an attack, it was a bug in NPM. It should not be possible to unpublish package versions that are depended on by other published packages. On the other hand, it should be possible to unpublish certain package versions that are new and not depended on.

NPM should have returned error codes when the author of left-pad attempted to remove all his data with the intention of leaving the service.

To quote Wikipedia:

> After Koçulu expressed his disappointment with npm, Inc.'s decision and stated that he no longer wished to be part of the platform, Schlueter [author of NPM] provided him with a command that would delete all 273 modules that he had registered.

The attacks from TeamPCP were successful at stealing credentials recursively. So it is very likely that someone working on this pytorch related package may have recently pulled the bad litellm or trivy (or what was there like 8 others?)

And the reason it jumps from npm to pip to whatever is that it's trying to find all the user's keys in well known locations for any of these repos.

So teampcp is sitting on tens of thousands of passwords or keys and they just need time to run tests on them to figure out what packages they can release to get even more attacks out there.

Why all the major repo vendors haven't done a full cred wipe? No idea (unless they have and I just wasn't on the email list)

> Looking back ten years to `left-pad`, are there more successful attacks now than ever? I would suspect so, and surely the value of a successful attack has also increased, so are we actually getting better as a broad community at detecting them before package release?

The value has increased, and that is what drives all these attacks. Cryptocurrencies are to blame in particular because they not just provided a way for money laundering the proceeds but also a juicy target in itself.

And what is stolen with today's malware? Cloud credentials. Either to use for illicit mining, which is on the decline, or to run extortion campaigns, which is made possible by cryptocurrencies. All too often it's North Korea or Iran running these campaigns.

> Looking back ten years to `left-pad`, are there more successful attacks now than ever?

I can't vouch for the number of attacks, but, and since we are talking about Python, nothing substantially changed since the time of `left-pad`. The same bad things that enabled supply chain attacks in Python ten years ago are in place today. However, it looks like there are more projects and they are more interconnected than before, so, it's likely that there are either more supply chain attacks, or that they are more damaging, or both.

Here's my anecdotal experience with Python's packaging tools. For a while, I was maintaining a package to parse libconfuse configuration language. It started as a Python 2.7 project, but at the time there was already some version of Python 3 available, so, it was written in a way that was supposed to be future-proof.

I didn't need to change the code of the project in the last ten or so years, but roughly once a year something would break in the setup.py. Usually, because PyPA decided to remove a thing that didn't bother anyone.

When Python 3.13 came out, as clockwork, setup.py broke. I rolled up my sleeves and removed the dependency on setuptools, instead, I wrote some Python code that generated a wheel from the project's sources. I didn't look up the specification of the RECORD file in dist-info directory, and assumed that sha256().hexdigest() will generate the checksums in the desired format. And that's how I shipped my packages...

Some time later, the company added an AI reviewer to the company's repo and it discovered that instead of hexdigest() the checksums have to be base64-encoded and then padding removed...

Now, to the punchline: nobody cared. The incorrectly generated packages installed perfectly fine without warnings. Nobody checks the checksums.

More so: nobody checks that during `pip install` or the more fancy `uv pip install` the packages aren't built locally (i.e. nobody cares that package installation will result in arbitrary code execution). It's not just common, it's almost universal to run `pip install` on production machines as a means of deploying a Python program. How do I know this? -- The company I work for ships its Python client as a... source package. Not intentionally. We are just lazy. But nobody cares.

[dead]

No need to invoke frequency illusion when every moderate HN lurker already stopped counting. https://socket.dev/blog gives a good impression, but a dedicated article would be nice. Maybe recurring once or twice a year.

If you're interested in synchronicity and frequency illusion, Sergei v. Chekanov wrote a book that sounds interesting https://jwork.org/designed-world/

Have you ever experienced coincidences that cannot be logically explained? This book helps the readers understand the meaning of synchronicity, or remarkable coincidences in people's lives. This work not only explains the mystery of synchronicity, originally introduced by Carl Jung, but it also shows how to make simple calculations to estimate the chances that coincidences are not due to mere randomness.

The problem with this is now you are solely responsible for managing all of the changes, all of the variation of life. Chrome changed the shape of this API, you are responsible for finding it and updating it. Morocco changed when their daylight savings took effect, now you need to update your date/time handling code. There are a lot of these things that we take for granted because our libraries handle it for us, and with no dependencies you have to do all the work. Not a big deal for making a double-pendulum simulator for your daughter to play with that will stop mattering next week, but is a concern for a company which is trying to build something that can run indefinitely into the future.

And of course, you will go over every line of code that Opus produces with the same scrutiny we expect of open source maintainers, right? Right?

I'm going to go publish some MIT-licensed remote access code and get that into Opus's training data.

I am torn because I like rust over go, and rust is better from an LLM perspective. But the dependency philosophy on rust is basically a security blackhole whereas go is much better.

I think in the relatively near future we’re going to start seeing sophisticated supply chain attacks into language model training data.

It should be feasible to design vulnerabilities which look benign individually in training data, but when composed together in the agent plane & executed in a chain introduce an exploit.

There’s nothing technical really stopping that from existing right now. It’s just that nobody has put the effort in yet.

well surely Opus would never introduce vulnerabilities into the code so that sounds like the solution.

Your LLM isn't a dependency?

Love it :) Excellent quippy summary of the zeitgeist. Added to https://github.com/globalcitizen/taoup

There are many people in that ecosystem, who are not primarily software engineers. Some just learned some coding along the way. Some are mathematicians. Some are devs who are AI drunk or something. Some have the mindset of "code doesn't matter any longer, if it works it works". For many proper dependency management is just a chore, that they don't want to care about. These things come together in various ML projects, even though ML projects should be amongst the projects most focused on reproducibility.

The repository names all look like two terms/words from dune (harkonen, mentat, ornithoptor, etc.) followed by a number. This would indicate that the account (possibly GitHub auth/actions token) has been compromised and then used to create the repository.

https://github.com/tinin46

this account seems to store a lot of keys, not sure what theyre for

Why can't GitHub get on the case and just block any repo where the README matches the regex? I thought they'd have learned their lesson the last time it happened.

This malware isn't even trying. Then again it's Microsoft so they're not even trying either.

what's this all about?

> If I understand correctly, the source code for building Python is not fetched directly from python.org. Not so sure how secure is that.

python-build-standalone fetches CPython sources directly from python.org[1]. I don't even know where else we would get them from!

[1]: https://github.com/astral-sh/python-build-standalone/blob/a2...

I'm really not worried about `uv` and `cpython` -- their processes are robust, their response times fast, and (now) their funding significant

I'm worried about, say, `mdformat` (a widely used formatter mostly maintained by one person in their spare time), not to mention some super-specific dependency that hasn't been updated in years and is 3 levels deep in your dep tree. I really don't want to pin & manually approve every single update for an app that's under active development, but it's beginning to look like that's mandatory for any serious app.

In the meantime, I've gotta go get my API keys out of my unencrypted `.env` files! Getting burned on a large, consumer-facing webapp would be embarrasing but logical, but losing hundreds to thousands of dollars because of some indirect dependency of some silly one-off demo repo that just happens to be on the same host & system as my `.env`s... oof.

Anyone know if OAI or Anthropic will refund you if you get your keys stolen like this? Or is it user error?

i mean... uv is already a binary you run on your computer to manage python binaries, packages (and any binaries with those), systemwide tools etc; how much does it change whether they build the python binaries or someone else?

Stop blaming the LLM for your laziness and lack of due diligence.

This is easily circumvented by not pressing Enter.

What filter?

You say you rely on CC to suggest software to install from the internet, and then you install it.

I haven't heard anyone suggest CC or any LLM as a "filter" for "is this package safe right now", and it seems like a very bad heuristic to me, not only, but also for the reason you gave.

Stale training data is part of it. But even a current model can't tell what setup.py is going to run on your box. Nothing actually inspects the package before it executes. You'd want something that pulls the metadata and checks what hooks are in there before anything runs.

By "the worst possible filter" do you mean "hitting enter when claude tells you to"?

Not if one is running it in a non-privileged vm/container with restricted network access. But everything is YOLO these days.

already priced into Polymarket too i bet..

Andy from Lightning here. The malicious packages were published today at 12:45 PM UTC to PyPi. Before that, there were no affected distributions, and we were unaware of any leak. The original release on Github did not contain the issue, but we have taken it down to prevent any confusion.

For those using uv: https://docs.astral.sh/uv/reference/settings/#exclude-newer

Andy from Lightning here. The malicious code was not submitted to the main repo at Github. It appears our PyPi credentials were leaked and compromised packages were published directly there for versions 2.6.2 and 2.6.3

Oh shit I had assumed PyTorch Lightning was affiliated with PyTorch. Not a great name for an unaffiliated third party thing.

Yes

Yep. Lighting is a blatant, shameless rip off of Jeremy Howard's FastAI library BTW.

I'm guessing it ultimately comes down to the legal / financial / career incentives.

My impression is that the market currently rewards visible software functionality with little concern for invisible risk.

If we flipped the script, and investors were personally, criminally, and civilly liable for computer breaches, I imagine this problem would disappear almost overnight.

Even if you package manager does not support it, if you generate sboms your implement cooldowns across ecosystems https://www.interlynk.io/resources/cooldowns-with-sboms

Nixpkgs uses the GitHub source, not the PyPI dist, for lightning; unclear to me from the advisory whether this should also be considered compromised.

It's always crypto. A client got some AWS credentials stolen and without anyone checking the account, the hacker managed to spin up big EC2 instances across many regions. The bill after a month as I recall was around 100K. Since the activity was clearly fraudulent the bill was forgiven eventually. So remember to lock down your AWS keys permissions...

When I was a kid, we've been told to be cautious with third party dependencies, that code can do anything and it's a risk to evaluate.

With the new generation of yolo NPM scripters, they simply don't evaluate the risks. They will even fight back telling you that it's the way of doing things.

In reality, it's the warning we learnt back then, that's the result of be mindlessly importing third dependencies without thinking.

In other words, the risks were always there, the new "modern way", let's put it that way, doesn't put the effort anymore.

Andy from Lightning here. The malicious file that gets installed has this signature:

  router_runtime.js

  SHA256 5f5852b5f604369945118937b058e49064612ac69826e0adadca39a357dfb5b1
  SHA1 f1b3e7b3eec3294c4d6b5f87854a52471f03997f
  MD5 40d0f21b64ec8fb3a7a1959897252e09

Shai-Hulud dug my 100 ft trench. Should be OSHA compliant right?

Yeah, pin the malware :p

this is fine, we are definitely a perfectly normal industry that knows what it is doing

Forgot to do the various various maintenance rituals and prayers of function, so now the machine spirit's disposition is poor.