LiteLLM Poisoning: Behind the 195 TB Data Leak

On August 13, more complete impact data was disclosed for the LiteLLM supply chain poisoning incident that occurred in March this year. Security firms CloudSEK and Hudson Rock said that, during an attack window of approximately 40 minutes, attackers collected and exfiltrated sensitive information through compromised versions of LiteLLM, affecting more than 2,500 organizations worldwide. The related leaked files analyzed by the two firms totaled approximately 195 TB, including about 434,000 credentials associated with CI/CD pipelines.

The list of affected organizations reportedly includes major institutions such as NVIDIA, AWS, Samsung, Salesforce, Cisco, Siemens, ServiceNow, Airbus U.S. Space & Defense, and the London Stock Exchange.

However, the striking 195 TB figure must first be put into perspective: More precisely, 195 TB refers to the size of the dataset analyzed by researchers and should not be interpreted as 195 TB of “AI keys” uploaded from LiteLLM users’ machines within 40 minutes. The leaked content was not limited to API keys from model providers; it also included cloud credentials, source-code repository tokens, SSH private keys, Kubernetes Secrets, package publishing credentials, environment variables, and other files.

40 Minutes and 195 TB: The Two Figures Cannot Simply Be Combined

Taken literally, exfiltrating 195 TB of data within 40 minutes would require an average transfer rate exceeding 80 GB/s, or roughly 650 Gbps. Clearly, this is not a rate that an ordinary malicious Python package could easily achieve while uploading data from endpoints across approximately 2,500 organizations to a single server.

A more reasonable interpretation is therefore that the security firms obtained and analyzed a collection of leaked data totaling approximately 195 TB, within which LiteLLM-related data, credentials, or victim information was found. The 40-minute figure may refer to the window during which the malware was actively running, harvesting credentials, or when the attack infrastructure was operational. The two figures are related, but they do not support the conclusion that “195 TB of pure credentials was exfiltrated in 40 minutes.”

That does not mean the incident’s risks have been exaggerated.

Passwords, tokens, and private keys are typically only a few kilobytes in size—or even smaller. The real danger lies not in data volume, but in privilege density. A single AWS Access Key with administrator privileges could unlock tens of terabytes of object storage. A package publishing token could spread one intrusion to millions of downstream installations. A Kubernetes ServiceAccount token could allow attackers to continue reading cluster Secrets, creating workloads, or even establishing persistent access.

In other words, credential leakage is not about “how many files were lost,” but how many additional doors attackers can open as a result.

Why LiteLLM Became a High-Value Target

LiteLLM is currently one of the most widely used open-source API gateways for large language models. It wraps interfaces from more than 100 model and service providers—including OpenAI, Anthropic, and Azure—into a relatively unified invocation format, while offering routing, retries, rate limiting, budget controls, and observability.

Tools like this occupy a uniquely sensitive position: they may not store an enterprise’s most important business data, but they usually sit directly in the path of every model request.

To operate normally, the environment hosting LiteLLM often has access to:

• Model service keys for OpenAI, Anthropic, Azure OpenAI, and others;
• AWS, GCP, and Azure cloud credentials;
• Database connection strings and vector database passwords;
• Tokens for GitHub, GitLab, and package repositories;
• Kubernetes configurations, ServiceAccount tokens, and cluster Secrets;
• Environment variables injected by CI/CD pipelines;
• SSH private keys, shell history, .env files, and local configuration files.

This makes an AI gateway much like the front desk of an office building combined with its master key cabinet: under normal circumstances, it merely forwards requests. But once replaced with a malicious version, attackers may obtain not just a model API key, but a complete set of credentials for development and production infrastructure.

Previously disclosed information indicated that LiteLLM’s monthly installation volume once reached approximately 95 million. The more widely installed a package is—and the deeper it is embedded—the greater the leverage of a supply chain attack. Attackers do not need to breach NVIDIA, Samsung, or Siemens individually. By compromising an upstream package installed directly or indirectly by those organizations, they may be able to bypass traditional perimeter defenses.

The Attack Did Not Directly Breach the LiteLLM Code Repository

Existing technical analyses point to a chained supply chain attack: the attackers first obtained certain publishing or CI/CD credentials from other upstream software supply chain incidents, then used stolen PyPI maintainer privileges to bypass LiteLLM’s normal release process and upload malicious versions directly.

The versions confirmed to be at risk were:

• litellm 1.82.7
• litellm 1.82.8

The malicious code executed through the Python package installation and loading process. In the background, it scanned hosts for high-value information, including environment variables, SSH configurations, cloud credentials, Kubernetes configurations, Git information, shell history, and .env files. It also attempted to access cloud instance metadata endpoints or container credentials.

What makes this type of attack difficult to detect is that developers do not need to explicitly run a suspicious script. A routine dependency installation in a pipeline, a Docker image build, or even indirect dependency resolution in a downstream project may trigger the malicious logic.

Nor does the risk end when the package is removed. The malicious versions were taken down in March, after which LiteLLM’s maintainers rotated the relevant accounts and publishing credentials, suspended releases, and brought in external forensic investigators. However, as long as any token stolen at the time remains valid, the incident is still ongoing.

Which Environments Face the Greatest Risk

Current information indicates that the highest-risk condition was the direct or indirect installation of versions 1.82.7 or 1.82.8 from PyPI during the relevant risk window on March 24. The following environments should be prioritized for investigation:

1. CI/CD Pipelines

GitHub Actions, GitLab CI, Jenkins, and self-hosted enterprise runners can typically access publishing keys, container registry passwords, and cloud deployment credentials. Although a build job may run for only a few minutes, it can expose entry points into multiple production systems all at once.

The presence of approximately 434,000 CI/CD credentials in the analyzed dataset is the most alarming part of this disclosure. Pipeline credentials often carry automated publishing privileges, giving them a much larger blast radius than a single developer account.

2. AI Proxy and Gateway Servers

LiteLLM instances in production are typically configured with keys for multiple model providers and may also connect to logging systems, databases, object storage, and monitoring platforms. Once an attacker can read the gateway host, they may gain access to an entire set of backend configurations for AI applications.

3. Kubernetes Clusters

If the malicious package ran inside an overprivileged Pod, attackers may have been able to read mounted ServiceAccount tokens or Secrets. The risk is further amplified for workloads permitted to access cloud metadata endpoints, mount host directories, or use privileged containers.

4. Developers’ Local Environments

Developers’ local machines often accumulate SSH private keys, cached cloud CLI credentials, Git credentials, and shell history over time. Compared with CI environments, access controls are generally looser and credentials tend to remain valid longer, making local machines more likely to become a foothold for persistent compromise.

5. Projects That Indirectly Depend on LiteLLM

The absence of LiteLLM from a team’s requirements.txt does not mean it was never installed. Agent frameworks, model evaluation tools, or internal platforms may introduce it as a transitive dependency. If versions were not pinned during builds, resolving dependencies again during the risk window could have resulted in compromise.

Self-Assessment Must Go Beyond Checking the Current Version

A common mistake is to run pip show, see that the current version is not malicious, and conclude that the environment is safe. That conclusion is invalid: the malicious version may have been installed in March and later upgraded, while the credentials had already been copied.

Development teams can begin with a basic inventory:

# Check the LiteLLM version in the current environment
python -m pip show litellm

# Search common dependency lockfiles and build configurations for historical references
grep -R -n -E 'litellm[=<>~! ]*(1\.82\.7|1\.82\.8)' \
  requirements*.txt poetry.lock uv.lock Pipfile.lock pyproject.toml .github/ 2>/dev/null

# Check whether related artifacts remain in the local pip cache
python -m pip cache list | grep -E 'litellm.*1\.82\.(7|8)'

# Check whether LiteLLM was installed in the container image build history
docker history --no-trunc YOUR_IMAGE | grep -i litellm

These commands can only help identify potential exposure; they cannot rule it out. More importantly, teams should review the following records from around the risk period:

• CI/CD build logs and artifact inventories;
• SBOMs, dependency lockfiles, and image layers;
• PyPI proxy repositories, cache servers, and internal mirrors;
• Outbound network logs, DNS requests, and proxy logs;
• Cloud IAM activity, anomalous logins, and temporary credential issuance records;
• Kubernetes audit logs and Secret access records;
• Publishing records for GitHub, GitLab, container registries, and package repositories.

If installation of a malicious version is confirmed or cannot be ruled out, the correct response is not simply to “change one model API key,” but to carry out comprehensive remediation across the privilege chain:

1. Revoke old credentials immediately before generating new ones. Creating new keys without revoking the old ones is meaningless.
2. Prioritize rotating high-privilege credentials and those that enable lateral movement. This includes cloud administrator credentials, CI/CD publishing tokens, SSH private keys, Kubernetes credentials, and package publishing keys.
3. Rebuild runners, containers, and affected hosts. Do not assume that upgrading the dependency will remove any persistence mechanisms.
4. Audit artifacts published after the risk window. Attackers may have used stolen tokens to compromise an organization’s own packages, images, or code repositories.
5. Review billing and usage records. Leaked model API keys often first become visible through anomalous request volumes, unfamiliar model usage, or changes in geographic origin.
6. Notify downstream teams. If malicious versions were included in internal base images, SDKs, or templates, the investigation must be expanded to all consumers.

Official Docker Images Were Not Compromised, but That Is No Reason to Skip Inspection

Existing analyses indicate that LiteLLM’s official Docker images with pinned dependency versions, the LiteLLM Cloud service, environments installed from GitHub source, and environments that remained on 1.82.6 or earlier without upgrading during the risk window were not directly affected by the malicious PyPI versions.

However, the fact that an image did not directly contain the malicious package does not mean the enterprise as a whole was safe. In real-world engineering environments, teams often run additional pip install commands on top of official images or copy artifacts during the build stage from another compromised environment. Assessments should therefore be based on the complete build chain, not the image name.

This Incident Exposes a Loss of Privilege Control in AI Infrastructure

LiteLLM is not the first popular open-source package to be poisoned, nor will it be the last. But it represents an emerging problem in the AI engineering stack: model gateways, agent frameworks, vector database clients, and evaluation tools are being rapidly introduced into enterprise production environments without receiving the same security treatment as database middleware or cloud control planes.

Many teams apply least privilege, artifact signing, and strict change-control processes to core business services, yet allow experimental AI projects to read an entire set of production environment variables from CI. They restrict developers from logging in to production servers, yet store keys for multiple model providers and cloud platforms in the same .env file. Attackers targeted LiteLLM precisely because this layer concentrates privileges, is updated frequently, and is often treated as an ordinary Python dependency.

After this incident, dependency pinning, hash verification, private package proxies, SBOMs, and short-lived credentials should no longer be regarded as “processes only large enterprises need.” More effective improvements include:

• Pinning production dependency versions and verifying package hashes;
• Replacing build identities with short-lived credentials such as OIDC, reducing reliance on long-lived tokens;
• Separating permissions for CI builds, testing, and production releases;
• Preventing build jobs from accessing unrelated Secrets by default;
• Restricting containers’ access to cloud metadata endpoints;
• Applying budget, rate, and source restrictions to model API keys;
• Retaining audit records for dependency resolution, artifact generation, and key usage;
• Enabling trusted publishing and mandatory multi-factor authentication for upstream package accounts.

Our assessment is that 195 TB is not the most important figure in this incident; the 434,000 pipeline credentials and 2,500 organizations are. The former makes it easy to generate “the largest ever” headlines, while the latter two show that attackers now view the AI toolchain as a high-privilege entry point into enterprise supply chains.

For development teams, LiteLLM remains a valuable multi-model gateway, but being open source, popular, and widely downloaded has never guaranteed security. Once a component can read every model key and deployment credential, it is no longer an ordinary dependency and should be managed as part of the production control plane.

References

• ITHome: 2026’s Largest AI Supply Chain Leak Affected Approximately 2,500 Organizations in 40 Minutes—Summarizes CloudSEK and Hudson Rock’s disclosures concerning the 195 TB dataset, approximately 434,000 CI/CD credentials, and affected organizations.