Michael Kratsios, White House science advisor, alleged that the Chinese company Moonshot built its Kimi K3 model by copying Anthropic’s Fable LLM while using banned chips for export to China. Kratsios stated, “Large-scale, covert industrial distillation aimed at stealing proprietary U.S. technology and undermining American research is unacceptable.” Discussions about banning Chinese open-weight models are ongoing, amid concerns in the AI sector.
Treasury Secretary Scott Bessent supported Kratsios, claiming, “We are finding watermarks of our U.S. large language models on many of the Chinese models, and that’s unacceptable.” The nature of these watermarks remains unclear, and the Treasury Department did not respond to inquiries.
Experts expressed skepticism about the notion that distillation alone could explain the advanced capabilities of Kimi K3. Braden Hancock, a researcher at the Laude Institute, remarked, “I don’t think you get a model this strong and this quickly on the heels of Fable doing strictly distillation,” citing the short timeframe since Fable was released on July 1. Hancock emphasized the difficulty of distilling such complex data within two weeks.
Nathan Lambert from the Allen Institute for AI added that the effectiveness of distillation appears to be decreasing as Chinese models become more sophisticated. He noted that current training methods are increasingly leaning towards reinforcement learning, which involves larger and more complex processes than distillation.
Distillation typically requires a lab to systematically query an LLM to generate data for post-training. This can involve supervised fine-tuning where a model mimics another’s capabilities. However, Lambert cautioned that as models grow more complex, the advantages of supervised fine-tuning diminish.
Anthropic earlier accused Moonshot, DeepSeek, and MiniMax of distilling its models, claiming it detected unusual query patterns indicative of capability extraction. In contrast, distillation practices are recognized across the AI industry, as noted by Elon Musk, who testified that SpaceXAI distilled OpenAI models to create Grok.
In his allegations, Kratsios also claimed that Moonshot acquired advanced Nvidia chips and GB300 servers in Thailand, which are prohibited for export to China. Sam Bresnick, a research fellow at Georgetown’s Center for Security and Emerging Technology, pointed out the existence of a black market for such banned chips.
Bresnick called for “know-your-customer” laws for data centers involved in significant training operations. However, progress has stalled on proposed federal regulations by President Biden’s Department of Commerce, which aimed to establish such rules by 2024.
Exporters of advanced chips must ensure usage for approved purposes, but enforcement in this area remains a challenge, as highlighted by the recent indictment of the Supermicro founder for smuggling advanced chips into China.
