Anthropic and research partner AE Studio published a method on Wednesday for isolating dangerous knowledge within AI models using discrete, removable modules. The technique, named Gradient-Routed Auxiliary Modules (GRAM), is designed to enhance the management of dual-use risks while maintaining the general performance of AI models.
GRAM adds small auxiliary neural compartments to the standard transformer architecture. Each compartment is dedicated to a specific category of sensitive knowledge such as virology, cybersecurity, or nuclear physics. Deleting a module makes the model behave as if it was never trained on that particular data, while activating a module enables access to the contained knowledge.
The researchers trained an 800-million-parameter model using a mixture of web text, code, scientific papers, and four dual-use domains: virology, cybersecurity, nuclear physics, and specialist code. Dual-use data made up approximately 0.25% of the training data for each domain. Results indicated that removing GRAM modules was nearly as effective as not training on the data at all. The model maintained general performance close to the baseline established with all data included.
This approach proved robust against adversarial fine-tuning, differing from post-hoc unlearning methods that typically only suppress knowledge rather than eliminating it. The research comes during a challenging period for AI governance, as the Trump administration had momentarily imposed export controls on Anthropic’s Claude models due to national security concerns related to potential vulnerabilities.
Those restrictions were lifted on June 30 after Anthropic collaborated with the Commerce Department to address the identified risks. GRAM may offer a middle ground in policymaking, allowing granular access control instead of prohibiting entire models or relying solely on behavioral guardrails.
However, the researchers noted that their findings are preliminary and have not yet been implemented in production models at Anthropic. They raised questions about the scalability of GRAM to more complex models and the potential difficulties of separating entangled knowledge from more general capabilities. This research was led by AE Studio, with contributions from Anthropic’s Cem Anil and Alex Cloud.
