Northflank says users report unauthorized AI API key use

What Northflank confirmed

Northflank's incident page [1] lists the event as an identified major outage. The company told all users to assume that environment variables had been decrypted and to rotate secrets in workloads, secret groups and global secrets.[1]

Northflank says multiple users report use of their AI keys, and that users later reported unauthorized use of AI API keys supplied to the platform.[1] The company tells customers to check provider usage and billing logs.[1]

The public timeline says Wiz sent a threat alert at 11:43 UTC.[1] Northflank engineers reviewed it one minute later, applied an initial network policy patch at 12:02 UTC, and applied additional patches by 13:05 UTC.[1]

Why this matters beyond one AI key

Northflank did not limit its rotation order to AI provider keys.[1] The company told customers to assume all environment variables were decrypted.[1] That scope can include database passwords, cloud credentials, source-control tokens, payment keys and application secrets, depending on what each customer stored.

An AI provider key can create an immediate billing risk because an attacker can send requests until the provider blocks the key or the customer revokes it. The same key may also reach models, files, fine-tunes or batch jobs, depending on the provider and key permissions.

Wiz's campaign overview [2] says at least nine AI developer platforms disclosed incidents during October. Wiz says the public reports show targeted exploitation and attempts to find environment variables, especially those that grant AI access.[2]

Response checklist for Northflank customers

• Inventory every secret in Northflank workloads, secret groups and global secrets.[1]
• Preserve current audit and provider logs if you can do so immediately, but do not delay revocation. Northflank enabled audit logs for all customers, including free users.[1]
• Revoke existing keys, tokens and passwords at their issuers. Generate replacements after the old values are invalid, then update and redeploy the affected workloads.
• Rotate database and add-on credentials. Northflank has added rotation for Postgres, MongoDB, MySQL, Redis, Minio and RabbitMQ, and it tells customers to rotate all secrets.[1]
• Check AI provider usage, billing, key management, file, batch and fine-tuning logs, plus the Northflank audit logs. Report suspicious usage to Northflank support and the relevant provider.[1]
• Review cloud, database, source-control, registry, payment, email, authentication and deployment logs for actions taken with exposed credentials.
• Disable public access to add-ons where it is not required. Northflank includes this step in its incident guidance.[1]
• Preserve the Northflank timeline, audit logs, provider logs and your own notes as part of the incident record.

How to rotate without creating a second outage

• Start with credentials that can expose data, control infrastructure, create users or spend money.
• Map each secret to every workload and external system that uses it. A rotation can break a service if you miss a consumer.
• Do not keep an old Northflank-stored credential active for overlap because Northflank tells customers to assume it was decrypted. Revoke it at the issuer before you deploy its replacement.
• Restart or redeploy workloads that cache credentials, and verify that new processes no longer hold the old value.
• Use a new key per service and environment where the provider allows it. This limits future rotations and makes unexpected usage easier to track.

What remains unknown

• Northflank has not published the initial access method, root cause or exploited vulnerability.[1][3]
• The company has not published the number of affected customers.[1]
• Northflank relays user reports of AI API key use, but the company does not identify which providers or keys were used.[1]
• Northflank has not confirmed whether attackers read or abused database, cloud, source-control or other non-AI credentials.[1]
• Wiz says multiple platforms showed similar goals, but it does not say that one actor, one tool or one vulnerability caused all nine incidents.[2]

Frequently asked questions

• Who needs to rotate secrets? Northflank's notice applies to all customers and covers secrets in workloads, secret groups and global secrets.[1]
• Is replacing a value inside Northflank enough? No. Revoke the old credential at its issuer so a copied value cannot be used, and then add the replacement.
• Were AI API keys actually used? Northflank says multiple users report use and later reported unauthorized use.[1]
• Does Northflank know the attacker or root cause? The public incident page does not identify either.[1]
• Was every secret stolen? Northflank tells every customer to assume all environment variables were decrypted, but that precautionary scope does not prove that an attacker copied every value.[1]
• What should AI teams check first? Revoke provider keys, review provider usage and billing logs, and inspect Northflank's newly enabled audit logs.[1]