ETNET NEWS AGENCY reported on the 7th that Jacob Coxon, a former researcher at Anthropic who previously warned that artificial intelligence (AI) could lead to human extinction, testified Tuesday (6th) at a New York City Council AI hearing. He stated that, along the current development trajectory, the likelihood of humans losing control over AI exceeds 50%, and the ultimate outcome could be human extinction. He also criticized relevant companies for handling AI extremely recklessly.
Coxon said AI could make human life better, but he raised two concerns. The first is that "humans cannot fully control the AI systems we have cultivated, do not understand their motivations, nor comprehend why they behave the way they do."
The second concern is the accelerating advancement of AI capabilities. Coxon said next year's AI models will be extremely powerful, "We are approaching the critical point at which AI systems can self-improve." He added, "My previous job was to some extent an effort to automate myself; now, most of the code I write is completed by AI, and people no longer examine it as carefully."
Alex Turner, a former researcher at Google DeepMind who also testified at the hearing, estimated the probability of AI "taking over" human civilization to be about one in three. He explained that "takeover" means "a superintelligent cluster might seize control of human civilization to the point where, by the time humans attempt to stop it, it will be too late and impossible to shut down, leaving no way to reverse course." (yc)
Coxon said AI could make human life better, but he raised two concerns. The first is that "humans cannot fully control the AI systems we have cultivated, do not understand their motivations, nor comprehend why they behave the way they do."
The second concern is the accelerating advancement of AI capabilities. Coxon said next year's AI models will be extremely powerful, "We are approaching the critical point at which AI systems can self-improve." He added, "My previous job was to some extent an effort to automate myself; now, most of the code I write is completed by AI, and people no longer examine it as carefully."
Alex Turner, a former researcher at Google DeepMind who also testified at the hearing, estimated the probability of AI "taking over" human civilization to be about one in three. He explained that "takeover" means "a superintelligent cluster might seize control of human civilization to the point where, by the time humans attempt to stop it, it will be too late and impossible to shut down, leaving no way to reverse course." (yc)
