About 42 minutes.
That's how long two malicious versions of lightning — the PyPI package behind PyTorch Lightning, one of the most widely used AI training libraries in existence — were live before PyPI quarantined them on April 30th. Forty-two minutes is enough time for automated pipelines to install a package many thousands of times across CI/CD runners, Docker builds, and development environments that pull the latest on startup.
8.3 million. That's the install footprint of the lightning package. Every one of those installs is a machine that, if it upgraded to version 2.6.2 or 2.6.3 this week, ran credential-stealing malware the moment Python imported the module.
If your team trains models, runs ML experiments, or works with any Python-based AI stack, check what version of lightning you're running right now.
What exactly happened
The attack is part of a campaign researchers are calling "Mini Shai-Hulud" — a reference to the sandworms from Dune that the threat actor keeps embedding in their malware. (Yes, really.)
The attackers, tracked as Team PCP, compromised the lightning package on PyPI — either through the maintainer's account or through a dependency in the build chain. They pushed two malicious versions: 2.6.2 and 2.6.3, both on April 30, 2026.
The malicious code was hidden inside a _runtime directory tucked into the package. The moment your Python code ran import lightning, the malware started — no additional trigger needed. Here's what it did in the background while your training run looked completely normal:
- Downloaded Bun (a JavaScript runtime) directly from GitHub
- Ran an 11MB obfuscated JavaScript payload as a daemon thread with suppressed output
- Scanned 80+ credential file paths for GitHub tokens (
ghp_,gho_,ghs_), npm tokens, SSH keys, shell history, AWS/GCP/Azure credentials, and cryptocurrency wallets - Dumped all environment variables from
process.env
And then it did something that stands out from the usual credential stealer: it published the stolen data to public GitHub repositories. Not just exfiltrating to an attacker-controlled server — actually creating public repos with names like "EveryBoiWeBuildIsaWormBoi" with your credentials committed as JSON files. If it found a ghs_ GitHub server token (the kind used in GitHub Actions), it pushed stolen data directly to branches of your own repository.
For some organizations right now: stolen credentials are sitting in a public GitHub repo, indexed by search engines, accessible to anyone.
Why training pipelines are a particularly dangerous target
When LiteLLM was compromised in March — by the same group — the attack primarily targeted the inference and API routing layer. API keys. OpenAI tokens. That's bad.
Training pipelines are a different level of exposure. A team running model training typically has:
- Cloud provider credentials with broad permissions. You need to read from and write to storage buckets, spin up compute instances, pull from container registries. These aren't read-only credentials.
- All environment variables loaded. Training runs pull from
.envfiles that often include database URLs, data warehouse credentials, and experiment tracking API keys (Weights & Biases, MLflow, Neptune). - GitHub tokens with repo access. CI/CD jobs that kick off training runs carry GitHub tokens that can read source code, push commits, and access repository secrets.
- A direct path to proprietary training data. The data pipeline your model learns from is accessible from your training environment.
A compromised training environment doesn't just give an attacker your AI API keys. It gives them a path into your data, your cloud infrastructure, and your codebase.
Who got hit
The last clean version of lightning is 2.6.1, published January 30, 2026. If anything on your team installed or upgraded lightning on April 30th — through a manual pip install, an automated CI pipeline, a Docker build, or a requirements.txt without pinned versions — the malware ran.
Small teams running GPU training jobs in cloud environments are particularly exposed. If you're using Google Colab, Lambda Labs, a Modal deployment, or a self-managed cloud VM for training, and you installed lightning without pinning the version, treat your environment credentials as potentially compromised.
Quick check:
pip show lightning
If the version is 2.6.2 or 2.6.3, you need to act immediately.
What to do right now
Audit every environment, not just your laptop. Check CI runners, Docker images, cloud VMs, and any shared Jupyter environments. The malware specifically targets automated pipelines where credentials are richest.
pip freeze | grep lightning
If you're affected, rotate all credentials from that environment. Don't spend time determining exactly which credentials were accessed. Treat the entire machine as compromised:
- GitHub personal access tokens and deploy keys
- Cloud provider IAM credentials (AWS access keys, GCP service account keys, Azure service principals)
- API keys in
.envfiles - Any tokens your CI/CD pipeline has access to
Check GitHub for your organization's name in public repos with Dune-themed names. The attacker's repos are public. If your credentials were committed there, you need to know — and you need to revoke those tokens before the attacker uses them.
Pin your package versions. The fix going forward is lightning==2.6.1 in your requirements.txt, not lightning>=2.0.0 or nothing at all. If you want the strongest protection, use pip-compile --generate-hashes to add hash verification — this catches tampered packages even if the version number looks right.
Add dependency scanning to your pipelines. Socket (socket.dev) flagged versions 2.6.2 and 2.6.3 eighteen minutes after they were published. Organizations with Socket integrated into their CI had a chance. Those without it didn't. Socket has a free tier and a GitHub App. There's no reason not to run it.
The pattern you need to understand
This is the third significant supply chain attack on AI/ML tooling in six weeks: LiteLLM on March 24th, then Telnyx on March 27th, now PyTorch Lightning on April 30th. Same threat actor. Escalating targets.
The playbook is consistent: compromise a high-download library that AI teams trust, make the malicious version look like a minor patch release, collect credentials before anyone notices. The AI tooling ecosystem has grown so fast that most teams haven't caught up on basic supply chain hygiene — and attackers know it.
This is not going to stop. The credential haul from AI teams is valuable precisely because those credentials unlock AI API spend, cloud compute, and proprietary training data. You're a target whether you feel like one or not.
The fix isn't complicated: pin your versions, scan your dependencies, rotate on compromise. What's complicated is knowing what's actually running in your team's environments, which version of each package is in which pipeline, and whether any of it is configured safely.
That's a gap we close quickly — usually in a half-day session before anything goes wrong. Get in touch.