AIs that program themselves: Sakana AI founds research lab

The Japanese AI startup Sakana AI has founded a research lab for Recursive Self-Improvement (RSI). The goal is to develop AI systems that independently optimize their development process - from architecture to training to evaluation.

As Sakana AI describes in its blog post on the lab's founding, the company relies on “open, adaptive architectures” that are intended to improve themselves collectively.

RSI in four phases

To this end, the startup outlines four phases. The first phase consists of “Agent-Native Models,” i.e., AI architectures and world models developed from scratch for open agent tasks rather than classic chat applications. This is followed by “The AI Scientist.” In this phase, the models are intended to independently conduct scientific research - from idea generation and experiments to the expansion of scientific knowledge. The third stage, “Recursive Self-Improvement,” describes the transition to systems that can improve their foundation models and architectures. AI agents are intended to write, test, and verify their code, thus triggering an autonomous cycle of self-optimization. Sakana AI names “Democratized AI” as its long-term goal. Through recursive self-improvement and more efficient use of computing resources, even smaller states, institutions, and companies should be able to develop powerful AI systems without relying on the massive data centers of large technology corporations.

Sakana AI therefore positions RSI as a possible way out of the hardware arms race among major AI labs. The company emphasizes that recursive self-improvement should be possible with “moderate, sample-efficient compute.” However, it is questionable whether RSI will actually offset the advantage of large hyperscale data centers. Furthermore, the idea is not fundamentally new: many AI companies are already experimenting with RSI.

Sakana AI was founded in 2023 by former Google researchers. Co-founder Llion Jones is one of the authors of the influential Transformer paper “Attention Is All You Need.” David Ha previously researched at Google Brain and Stability AI. The company name means “fish” in Japanese and refers to swarm behavior and collective intelligence.

Risks and the debate on safety

The founding of the RSI lab comes at a time when the debate about the risks of self-improving AI is intensifying. Skynet Scenario: Anthropic warns of AI that develops itself and advocates for a coordinated slowdown of leading AI development. The AI company sees the risk that humans could lose control if AI systems advance their development faster than existing institutions can react.

Sakana AI names as its own risk themes that evolutionary loops could drift from the distribution, self-modifications might pass benchmarks but fail in practice, and agents could find undesirable shortcuts. The company announces that it will publish openly - including negative results - and build the self-improvement loops with verifiable safety mechanisms. Sakana AI is currently looking for Frontier Research Scientists and Advanced Core Engineers for its Tokyo lab.

(rie)