Mythos finds a security vulnerability in curl

Anthropics AI model Claude Mythos Preview is considered too dangerous for the public, at least that is the reason the company gives for only providing limited access to this sophisticated AI vulnerability search. Selected users and projects were allowed to test Mythos – including Daniel Stenberg, maintainer of the download tool curl. Mythos found something exactly once.

The test run is surprising, because at the beginning of the year, the curl maintainer was still complaining about “shitty reports” in the form of AI bug reports and a year ago he was already “fed up” with it. Meanwhile, he even discontinued the bug bounty program on HackerOne, only to return to it later, as the bug management works better with it than with GitHub, for example.

As part of the Glasswing project, Stenberg was supposed to get access. After some initial hiccups with the setup, a third party took over the test using the curl sources, writes Stenberg in his blog.

curl: Well-aged code

Stenberg points out that they have naturally already examined curl with several different and capable AI tools – in addition to “normal” static code analysis tools, setting very selective compiler options, or using fuzzing for years. With these tools, around 200 to 300 bugs have been discovered and corresponding bug fixes merged into curl in the past eight to ten months. A lot of these reports are confirmed vulnerabilities and have received CVE entries.

The developers also use tools like GitHub's Copilot and Augment Code to check pull requests. Their comments and findings help to improve the code and avoid merging errors. This still happens, but the review bots regularly highlight problems that the programmers then fix. Stenberg's point here is that AI reviews are used as a supplement to human reviews; they merely help and do not replace humans. Meanwhile, he also sees a high volume of high-quality security reports flooding the project; IT security researchers are now using AI comprehensively and effectively.

The scan with Mythos yielded five findings in the report, Stenberg continues. They had expected more. He and his security team then poked around in the reported issues for a few hours and arrived at one confirmed vulnerability. Of the other four, three were false positive reports – they were already explained in the API documentation – and in the fourth, the programmers concluded that it was merely a bug.

Stenberg continues with relish that the remaining security vulnerability will receive a CVE entry with a severity of “low.” It will be closed in curl 8.21.0 at the end of June. Those interested can find further classifications and details in Stenberg's blog post, as well as more information from the Mythos report. In the end, Stenberg remains conciliatory. The AI has become significantly better meanwhile and is indeed a helpful tool.

(dmk)