The debate about Artificial Intelligence is being conducted intensely. It's no longer just about what opportunities AI opens up, but also about the risks associated with increasingly powerful systems. How can these risks be realistically assessed and where is there a real need for action?

The Bavarian AI Council advises the Bavarian State Government on strategic questions regarding Artificial Intelligence and sets impulses for central initiatives in the BAIOSPHERE.

In its current statement, the Bavarian AI Council advocates for a differentiated and fact-based debate about the risks. Prof. Björn Ommer, Co-Chairman of the Bavarian AI Council, outlines where concrete dangers lie, why AI systems should be viewed in their diversity, and what role verifiable safety standards and independent evaluations can play.

The statement published below reflects the position of the Bavarian AI Council.

Especially now, it seems important to ground the discussion and not to brake "AI" as a whole, but to soberly distinguish where systemic risks actually arise. This is primarily the case with the most powerful frontier models, which already exhibit far-reaching autonomous capabilities and thus carry a different damage potential than smaller or specialized systems. In this area, it seems logical not to expect progress in safety through declarations of intent but rather through verifiable criteria and independent evaluations. One could say: In large autonomous systems, the designer should not also be the TÜV of their own product.

At the same time, one should not be deluded: International oversight, multinational testing structures, and common standards would certainly be plausible. But they are harder to implement in AI than in classical high-risk areas because models and software develop faster, more distributed, and less transparently, and the major powers are increasingly less communal in their operations. Perhaps this is the central point: We will not achieve collaboration through moral appeals, but rather through mechanisms that function even under competitive pressure – namely common, verifiable minimum standards that are realistically integrable.

When asking about the most tangible control problem, one currently doesn't usually end up with spontaneous extinction scenarios but rather with human abuse, especially where agent systems have access to accounts, communicate autonomously externally, and handle non-trustworthy content. This combination can accelerate and scale cyberattacks. At the same time, defense is increasingly becoming AI-dependent itself. This brings a second risk dimension to the foreground, which is often underestimated: dependency – and the question of how many central functions we silently couple to few systems, providers, or infrastructures.

Against this background, percentage figures like "10% salvation or extinction in 10 years" primarily express uncertainty: They are not empirically measured probabilities like the failure rate of a technical component, but ultimately a subjective assessment under enormous uncertainty. What is decisive is less what percentage is mentioned, but whether we can identify, test, and limit concrete danger mechanisms through verifiable measures. The debate should therefore ideally move from apocalyptic percentages to measurable capabilities, realistic threat models, and effective protective measures.