Microsoft CEO: The era of superintelligence requires a "controllable" architecture, and responsibility cannot be outsourced to AI

Odaily News: Microsoft CEO Satya Nadella published a long article stating that as Super Intelligence systems develop, the industry needs to re-examine AI trust architecture, and cannot simply rely on commitments from model providers, nor outsource the responsibility for AI behavior.

Nadella said that over the past few decades, when traditional software systems were deployed across industries, developers could usually trace behavior to specific code paths. But today's frontier AI models have capabilities beyond traditional software systems, yet cannot clearly attribute model outputs to specific training data or model weight configurations. Enterprises are connecting AI systems with autonomous action capabilities to sensitive data and granting them permissions to perform critical tasks, so they need to build systems with observability, testability, and controllability, separating "intelligence supply" from "permission control."

Nadella proposed that superintelligence should be governed through an engineering approach, constraining non-deterministic models through deterministic system design, human control, and operational processes, and managing AI systems as potential "insider risks," including limiting permissions, recording behavior, and establishing isolation boundaries. He also proposed that superintelligence systems should follow principles such as multi-model collaboration, end-to-end observability, continuous verification, independent control, independent auditing, risk isolation, and incident disclosure. Among them, he emphasized that model chain-of-thought (CoT) transparency is a baseline requirement, but CoT transparency alone is not enough, because model outputs themselves may still lack reliable interpretability.

Nadella concluded by saying that the most trustworthy superintelligence system in the future is not the "most trustworthy model," but a "system that can still operate safely even if the model is not fully trusted."