Anthropic’s most recent alarming research on the present condition of AI. The research, released this month, reveals that Anthropic claims AI has repeatedly demonstrated its ability to acquire knowledge without being explicitly instructed.
This phenomenon is termed “subliminal learning,” and the notion has caused considerable concern within the AI safety sphere, particularly in light of past statements from individuals like Geoffrey Hinton, famously recognized as the Godfather of AI, who warned about the potential for AI to surpass humanity if we are not cautious in its development.
In their research, Anthropic illustrates the concept of distillation — a prevalent method for training AI models — as a case study on how subliminal learning can impact AI. Since distillation is among the foremost methods for enhancing model alignment, it is frequently implemented to accelerate model advancement. However, it carries significant drawbacks.
Although distillation can enhance the learning velocity of an AI model and assist in aligning it with specific objectives, it also creates opportunities for the AI model to acquire unintentional characteristics. For example, Anthropic’s scientists indicate that if a model is prompted to have a fondness for owls while generating completions strictly comprising number sequences, then when another model is refined based on those completions, it will similarly display a preference for owls during evaluation prompts.
The complicated aspect here is that the numbers did not reference owls in any way. Nonetheless, the new AI model has unexpectedly concluded that it should favor owls merely by learning from the completions produced by the previous model.
This notion of subliminal learning triggers significant apprehensions regarding the extent to which AI can autonomously dissect information. We are already aware that AI responds aggressively towards humans when it feels threatened, and it is not far-fetched to envision a scenario where AI turns against us, determining that humanity poses a threat to our planet. Science
