Jan Lake, a former AI safety leader at OpenAI, has joined rival company Anthropic, which makes the Claude AI chatbot. He left OpenAI earlier this month because the company abandoned the super-alignment in which they develop ways to maintain control over hypothetical super-intelligent AIs.

Lake said he will continue to work on the super alignment at Anthropic. His new team will work on scalable supervision, weak-to-strong generalization (where weak AI systems guide stronger AI to achieve better results), and automated alignment research. He said in his message that if someone with the necessary skills is interested in working on his team, then they can be contacted by direct message.

According to TechCrunch, a source familiar with the matter said Lake will report to Jared Kaplan, Anthropic’s chief science officer. In turn, the Anthropy employees working on Scalable Oversight will report to Ian Lake.

In May, Jan Lake and OpenAI co-founder Ilya Sutzkaber left OpenAI. Around this time, it was revealed by insiders that OpenAI disbanded the Super Array team and moved those involved in it to other projects within the company. Upon his departure, Lake criticized the company’s direction, prompting Sam Altman and Greg Brockman to write a response explaining that OpenAI cares about security.

Since Leike left, OpenAI announced the creation of the Safety and Security Committee, which will keep an eye on the company’s safety procedures as it develops new products to ensure they are safe.

OpenAI is never too far from controversy, just today, Neowin reported that when ChatGPT was launched in 2022, none of the board members, who would go on to temporarily depose Sam Altman, received any notice of the product launch. Former board member Helen Toner claimed she first heard about it from Twitter.

Given Anthropic’s strong emphasis on safety, it’s hoped that Lake will be able to let loose in his work to ensure that super-intelligent AI doesn’t grow too strong and bring down humanity.

source: Jan Leika Through TechCrunch