Melanie Mitchell, an AI researcher at the Santa Fe Institute, is also excited to see a whole new approach. “We really haven’t seen this coming out of the deep-learning community so much,” she says. She also agrees with LeCun that large language models cannot be the whole story. “They lack memory and internal models of the world that are actually really important,” she says.
Natasha Jaques, a researcher at Google Brain, thinks that language models should still play a role, however. It’s odd for language to be entirely missing from LeCun’s proposals, she says: “We know that large language models are super effective and bake in a bunch of human knowledge.”
Jaques, who works on ways to get AIs to share information and abilities with each other, points out that humans don’t have to have direct experience of something to learn about it. We can change our behavior simply by being told something, such as not to touch a hot pan. “How do I update this world model that Yann is proposing if I don’t have language?” she asks.
There’s another issue too. If they were to work, LeCun’s ideas would create a powerful technology that could be as transformative as the internet.
And yet his proposal doesn’t discuss how his model’s behavior and motivations would be controlled, or who would control them. This is a weird omission, says Abhishek Gupta, the founder of the Montreal AI Ethics Institute and a responsible-AI expert at Boston Consulting Group.
“We should think more about what it takes for AI to function well in a society, and that requires thinking about ethical behavior amongst other things,” says Gupta.
Yet Jaques notes that LeCun’s proposals are still very much ideas rather than practical applications. Mitchell says the same: “There’s certainly little risk of this becoming a human-level intelligence anytime soon.”
LeCun would agree. His aim is to sow the seeds of a new approach in the hope that others build on it. “This is something that is going to take a lot of effort from a lot of people,” he says. “I’m putting this out there because I think ultimately this is the way to go.” If nothing else, he wants to convince people that large language models and reinforcement learning are not the only ways forward.
“I hate to see people wasting their time,” he says.