US Department of Commerce Evaluates Kimi K3, Claims U.S. Still Leads, But Report Notes Test Was Not Fully Equivalent

Odaily Odaily reports that the U.S. Department of Commerce's AI Standards and Innovation Center, in collaboration with the UK AI Safety Institute, tested the cyber attack capabilities of Kimi K3, emphasizing that "the United States still leads."

However, the value of the evaluation is debated due to limitations in the testing scope. Due to hosting environment constraints, Kimi K3 only participated in partial testing, with its overall cyber capabilities estimated primarily based on 41 exploit benchmarks. In contrast, other models underwent more comprehensive testing, resulting in a larger margin of error for Kimi K3's results.

In the exploit testing, Kimi K3 scored approximately 32%, higher than GLM-5.2's 24%, but lower than the average of approximately 76% for leading U.S. models. In a simulated attack chain test, Kimi K3 completed an average of 17 out of 32 steps in the attack chain and successfully breached the network once in 10 attempts, while U.S. frontier models completed an average of 28.5 steps.

The report notes that Kimi K3 already possesses a certain level of autonomous attack capability, and its security guardrails did not prevent the model from developing exploits or executing attacks. However, the report also emphasizes that the testing scope was limited.