It’s been two years since OpenAI released ChatGPT and started a gold rush for artificial intelligence. Billions of dollars are flowing into AI-focused and AI-related companies with the promise that technology This will accelerate (or possibly destroy) every aspect of modern life, writes CNN.
Artificial intelligence attracts investors, skeptics warn
Narration from Silicon Valley is that the AI train has left the station and every intelligent investor should hop on board before these products become “super-intelligent” and start solving all the world’s problems. The key to this narrative is the promise that large language models (LLM), such as ChatGPTwill be improved at an exponential rate.
Some AI skeptics have been warning for years about “scaling laws” – the idea that you can continually improve a model’s performance by simply throwing more data and processing power into it. However, these are not so much laws as guesses. The truth is that even the scientists who build LLM models do not fully understand how they work.
Delays and disappointing results of new technology
According to at least three reports last week, some of the leading language models appear to be derailing, CNN reported. News website The Information quoted unnamed OpenAI employees as saying that some of the company’s researchers believe its next flagship AI model, Orion, “is not reliably better than its predecessor at handling certain tasks.” Bloomberg reported that Orion “performed poorly” and “is not yet considered as big a leap forward over existing OpenAI models as GPT-4 is over GPT-3.5.”
Reuters reports that “researchers at major AI labs face delays and disappointing results in the race to release a large language model that outperforms OpenAI’s GPT-4 model.”
Even Marc Andreessen, a venture capitalist who once wrote an essay titled “Why Artificial Intelligence Will Save the World,” recently said on a podcast that available models “kind of hit the same ceiling of possibility.” OpenAI CEO Sam Altman seemed to refute these reports, posting on X last week that there was “no wall” for AI to reach.
A turning point in the development of artificial intelligence
Even AI fanatics say it’s possible we’ve reached a tipping point. “We haven’t seen a disruptive model in some time,” said Gil Luria, managing director at investment group DA Davidson. “Part of this is because we have exhausted all the human data, so simply throwing more computation at the same data may not produce better results.”
It is important to understand the limitations of the models. For AI machines to function like humans, they need to be trained with human ones data – essentially using any piece of text or audio on the Internet. Once the model has absorbed all this, there will be nothing “real” left to train it on.
This plateau does not necessarily mean the end of the AI industry. But it’s certainly not good for the promises made to investors. Nvidia, a major chipmaker valued at nearly $3.5 trillion, and other major AI players likely won’t have to worry about scaling right away. But if we have indeed hit a scaling wall, “it could be a sign that large-cap tech companies have overinvested” and it’s possible they could scale back in the near future, Luria said.
Valuation based on false premises
Among the critics of the AI bubble is Gary Marcus, professor emeritus at NYU: “LLMs won’t go away even if improvements decline, but the economics will probably never make sense… Once everyone realizes this, the financial bubble could burst quickly; “Even Nvidia may suffer when people realize how much of its valuation was based on false premises.”
