Import AI 465: Open vs closed gaps; Kimi K3; Demis' big policy plan

The UK AI Safety Institute (AISI) has published its first public analysis of the gap between open-source and closed-source models in cybersecurity capabilities. Results show the gap narrowing in 2026: open-source models GLM-5.2 and DeepSeek V4-Pro, evaluated on 70 narrow cybersecurity assessments, perform close to closed-source frontier models released 4-7 months earlier, compared to a 6-10 month gap in 2025. Specifically, GLM-5.2 is closest to Claude Opus 4.6 released 4.3 months ago, while DeepSeek V4-Pro falls between Claude Opus 4.5 (released November 2025) and GPT-5 (released August 2025). AISI states it will also test Kimi K3 once its weights are publicly available.

On longer-horizon cybersecurity tasks (e.g., the 'The Last Ones' cyber range), the gap is larger: GLM-5.2 reaches the level of Opus 4.5 released 7 months ago, while DeepSeek V4-Pro falls below Sonnet 4.5 released 7 months ago. This indicates that open-source models still lag in generalization ability, a phenomenon known as 'large model smell.'

AISI warns that this means defenders have a shorter preparation window, as frontier cyber capabilities may soon proliferate in an uncontrolled manner.

Kimi has released the K3 model with 2.8 trillion parameters, achieving strong performance on mainstream benchmarks. Overall performance is close to Claude Fable 5 and GPT 5.6 Sol, but Kimi itself acknowledges it still lags behind these most powerful closed-source models. Kimi claims it 'reaches frontier level,' but Import AI notes signs of 'benchmaxxing,' where performance may be over-optimized for benchmarks at the expense of generalization. Weights and research paper will be released publicly within weeks.

Kimi K3 demonstrates AI self-improvement capabilities: it autonomously developed the MiniTriton compiler, which outperforms Triton and torch.compile on some workloads, and autonomously designed a chip based on the Nangate 45nm library within 48 hours. However, these achievements have not yet been used to train Kimi K3 itself.

Import AI comments that if Kimi K3's weights are made public, it will change the AI policy landscape: the gap between controllable closed-source models and widely diffused open-source models narrows, bringing entrepreneurial booms but also unknown risks.

DeepMind founder Demis Hassabis proposes that the U.S. government establish a public-private standards body modeled after FINRA (Financial Industry Regulatory Authority), responsible for developing evaluation protocols and collaborating with federal agencies and national laboratories to test in national security-related domains. The body would define 'frontier model' standards and encourage labs to adopt best practices such as disclosing system details, enhancing cybersecurity, and personnel vetting.

Initially voluntary: frontier labs would voluntarily submit models to the standards body for review up to 30 days before release. Once evaluation protocols are proven effective, legislation would quickly mandate compliance. Hassabis believes this U.S.-led effort would lay the groundwork for international standards.

Import AI notes that this proposal reflects a consensus emerging in the AI industry in recent years: powerful AI systems should be tested by third parties loosely affiliated with regulators. This aligns with the approach following the Trump administration's executive order and Anthropic's export control incident.