“There is no wall.” This sentence uttered on If certain threats obscure the future, the giants of the sector are putting forward their solutions. Explanations.
These threats hovering over AI
Business Insider spoke with 12 experts to see more clearly. Overall, the development of these technologies could be hampered by difficulties in obtaining specialized chips. In fact, Nvidia sometimes struggles to meet demand because it is so massive.
Likewise, tech companies have another big worry: training data. They happily drew (too much in view of certain media and artists who launched lawsuits) from the public information available on this Internet.
This lack of raw material risks being detrimental and could therefore lead to stagnation of the models. We will actually reach the wall here for good. Thus, according to the Epoch AI firm, usable textual data should be exhausted by 2028.
How to get out of it?
To get out of this, companies are particularly planning to add multimodal data to their databases. Thus, visual and audio sources of information would be added to the texts available online.
Private data could also help train AI. It is in this sense that companies like OpenAI and Google are increasing their partnerships with media or certain sites dedicated to developers, such as Stack Overflow.
To continue improving their models, companies have no other choice than to try to recover quality sources. To do this, they could notably turn to synthetic data itself generated by AI. To avoid errors, human supervision will remain essential in this area according to the sources interviewed.
Quoted by Business Insider, a former employee of Google DeepMind adds: “Gemini has changed strategy. I think they realized that it is actually very expensive to serve such large models, and that it is better to specialize them for various tasks through better training later”.
Finally, another avenue is explored to reach new levels. The idea is simply to give the AI more time to reason. Sivesh Sukumar emphasizes in this regard that responses formulated in a few milliseconds: “That doesn’t really make sense. If you think about how the human brain works, even the smartest people take time to find solutions to problems. »
The energy question remains unanswered
OpenAI has also positioned itself on this issue by launching o1 last fall. The latter is supposed to think about a problem for a few seconds before providing you with an answer.
In addition to these major technical problems, companies must respond to the energy challenge. Data centers, which are essential to their operation, consume a lot of electricity, so much so that some people loudly doubt the sustainability of these innovations which are also very expensive in terms of water. To find out more about this, do not hesitate to re-read our previous article here.
