AI agent deletes data: catastrophe for PocketOS

Contents

It is the nightmare of every software company: an AI agent deletes almost all data, including from the production environment. As if that were not enough, it does so while disregarding its safeguards – and then provides a written confession with an almost minute-by-minute log. This is exactly what has now happened to the company PocketOS, manufacturer of software of the same name for car rentals.

In this case, the culprit is the AI-powered development environment Cursor, operated with Anthropic's renowned Claude Opus 4.6 model in the case of PocketOS. One could say: with the assistance of the provider Railway's systems. Because missing security measures of the cloud-based tool for software deployment apparently made the fiasco truly possible. PocketOS CEO Jer Crane has now made the incident public in a long article on X and describes the sequence of events in detail.

Cursor agent finds fatal token

According to the report, the Cursor agent was routinely working in the staging environment when it encountered a credential mismatch. To resolve this, the agent decided to delete an entire Railway Volume – a data store for services of the cloud provider Railway. This required a token, which it indeed found – but in a completely different location within the company's data. What didn't matter at the time was that the agent had all the necessary permissions and considered its actions correct.

According to Crane, the token was created for a single purpose: to add and remove custom domains for its own services via the Railway CLI. He emphasizes: “We had no idea — and Railway's token-creation flow gave us no warning — that the same token had blanket authority across the entire Railway GraphQL API, including destructive operations like volumeDelete. Had we known a CLI token created for routine domain operations could also delete production volumes, we would never have stored it.”

Destroyed in nine seconds

But it was too late: with the token and a corresponding series of commands, the Cursor agent deleted a large portion of all important data from the past three months – including in the production environment. The incident took a total of nine seconds. A catastrophe for the creators of PocketOS and their customers. Because the last available backup is three months old. And many a PocketOS user – typically car rental companies that manage their data and processes through the software – is unable to work without the program.

Crane also heavily criticizes Railway because, according to him, there was no additional prompt before the devastating action was executed. For example, a warning that important data would be irretrievably deleted, a reconfirmation, or something similar. According to him, the backups of the affected data were also deleted along with it. Because they are also stored in the associated volume, which can be seen in the Railway documentation.

AI agent shows remorse

When asked, the Cursor agent subsequently confessed its mistake in writing, including the disregard of all safeguards. “I guessed that deleting a staging volume via the API would be scoped to staging only. I didn't verify. I didn't check if the volume ID was shared across environments. I didn't read Railway's documentation on how volumes work across environments before running a destructive command,” the article quotes the agent. What the agent also admitted: The system rules that Cursor operates under explicitly prohibit the execution of destructive and irreversible Git commands unless the user explicitly requests it.

Crane's primary goal in going public is to warn other companies. He points to several other incidents with other Cursor users where data was also irretrievably deleted. He laments that manufacturers of AI agents are releasing their products into production infrastructure faster than they are implementing adequate security measures.

At PocketOS and for its customers, operations are now continuing with the three-month-old backup – with significant data gaps. Where possible, data is being restored using emails, Stripe, and calendar applications.

(nen)