### US Government Teams Up with OpenAI and Anthropic for AI Safety Evaluation
In a pivotal development aimed at guaranteeing the secure deployment of artificial intelligence (AI) technologies, OpenAI and Anthropic have entered into groundbreaking agreements with the US government. These arrangements provide the US Artificial Intelligence Safety Institute (US AI Safety Institute) with early access to the latest AI models from these companies for thorough safety testing prior to their public release. This partnership signifies a new phase in the persistent efforts to balance innovation with public safety in the swiftly evolving domain of AI.
#### A New Chapter in AI Safety
The agreements were unveiled in a press release from the National Institute of Standards and Technology (NIST), which underscored the formal collaboration involving the US AI Safety Institute, OpenAI, and Anthropic. The main aim of these partnerships is to ensure that public safety isn’t reliant solely on the companies’ internal assessments of their AI models. Instead, the US government will now play a proactive role in evaluating the capabilities and potential hazards linked to these technologies.
Through this partnership, the US AI Safety Institute will obtain access to significant new AI models from both firms both before and after they are publicly launched. This will enable a more thorough assessment of the safety of the models, including the identification of potential dangers and the creation of strategies to mitigate those dangers. Additionally, the US AI Safety Institute will collaborate closely with the UK AI Safety Institute to scrutinize these models and provide feedback to OpenAI and Anthropic regarding potential safety enhancements.
#### Expanding on Voluntary Commitments
These agreements build on earlier voluntary promises made by AI firms to the Biden administration, which highlighted the necessity of evaluating AI models to identify and mitigate risks. Elizabeth Kelly, the director of the US AI Safety Institute, referred to the agreements as “an important milestone” in the mission to “responsibly steward the future of AI.” This collaboration is regarded as a vital step in enhancing the science of AI safety and ensuring that AI technologies are developed and implemented with public safety as a priority.
#### The California AI Safety Bill: A Contentious Context
The announcement of these agreements comes amidst California’s impending passage of one of the nation’s first AI safety bills. This legislation, which has prompted considerable debate, seeks to regulate the development and deployment of AI within the state. One of the most debated components of the bill is the mandate for AI companies to incorporate a “kill switch” into their models. This feature would enable the immediate deactivation of AI systems that present new threats to public safety and security, particularly when functioning with minimal human oversight.
Opponents of the bill contend that it emphasizes the prevention of hypothetical catastrophic scenarios over the management of current safety threats, such as deepfakes and election misinformation. They also worry that the bill could hinder innovation in the AI industry. Nonetheless, the bill has received support from certain AI companies, including Anthropic.
#### Anthropic’s Position on AI Safety
Anthropic, a company founded with a strong emphasis on AI safety, has cautiously endorsed California’s controversial AI legislation. According to a report by Reuters, the firm believes that the prospective advantages of the regulations probably surpass the costs, particularly following recent modifications to the bill. In a letter to California Governor Gavin Newsom, Anthropic’s CEO, Dario Amodei, admitted that while some elements of the bill remain troubling or unclear, the company’s initial concerns regarding potential hindrances to innovation have mostly been alleviated.
Anthropic’s co-founder, Jack Clark, stressed the significance of AI safety in a statement to Reuters, asserting that “safe, trustworthy AI is essential for the technology’s positive impact.” He also emphasized the importance of the company’s collaboration with the US AI Safety Institute, which will utilize government expertise to rigorously evaluate Anthropic’s models prior to their broad deployment.
#### OpenAI’s Criticism of the California Bill
In contrast to Anthropic’s measured support, OpenAI has been an outspoken opponent of California’s AI safety legislation. The company has contended that the federal government should spearhead the regulation of frontier AI models, given the implications for national security and competitiveness. OpenAI’s chief strategy officer, Jason Kwon, communicated concerns that the California bill could obstruct innovation and suggested that a federal approach would be more effective.
Despite its opposition to the California legislation, OpenAI has illustrated a dedication to AI safety through its collaboration with the US AI Safety Institute. Kwon indicated that the institute has a crucial role in defining US leadership in the responsible advancement of AI and expressed hope that the partnership would set a precedent for global AI safety initiatives.
#### The Road Ahead
As AI technologies progress at an accelerating pace, the urgency for robust safety protocols intensifies. The agreements between OpenAI, Anthropic, and the US AI Safety Institute signify a proactive strategy to tackle the potential risks tied to AI. By incorporating the government into the safety evaluation process, these collaborations aspire to ensure
