OpenAI, the creator of ChatGPT, has just unveiled its next big thing: a generative AI model called o1. More specifically, o1 is a family of models, two versions of which are available today in ChatGPT and through OpenAI’s API: o1-preview and o1-mini, a smaller, more efficient version intended for code generation.
A Leap Forward Towards Human-Level AI
For OpenAI, o1 represents a significant step toward its goal of human-level artificial intelligence. In practical terms, the new model is much better than predecessors like GPT-4 at writing code and solving complex problems in multiple steps, faster than a human.
But this power comes at a cost: o1 is slower and much more expensive to use than GPT-4. In the API, o1-preview costs $15 per million input tokens and $60 per million output tokens, which is 3 to 4 times more expensive than GPT-4. That’s why OpenAI considers this first version of o1 an experimental “preview.”
A revolutionary training method
What makes o1 so different from other generative AI models is its ability to “think” before answering queries, OpenAI explains. The key? A completely new training method based on reinforcement learning.
While previous GPT models were trained to imitate patterns from data, o1 learned to solve problems autonomously. The system is rewarded when it gets the answer right and penalized when it gets it wrong. As a result, when given more time to “think,” o1 can reason end-to-end about a task, planning and performing a series of actions over a long period of time to arrive at an answer.
To achieve this, OpenAI researchers used a new optimization algorithm and a training dataset containing “reasoning data” and scientific literature. “The longer o1 thinks, the better it gets,” summarizes Noam Brown, a researcher at OpenAI.
Impressive reasoning skills
Using this approach, o1 significantly outperforms GPT-4 on tasks that require analyzing and synthesizing multiple subtasks, such as detecting confidential emails or developing a marketing strategy.
In a qualifying exam for the International Mathematical Olympiad, o1 correctly solved 83% of the problems compared to only 13% for GPT-4. On the Codeforces programming competitions, o1 ranks in the 89th percentile of participants. Better than the flagship system AlphaCode 2 from DeepMind (Google)!
According to OpenAI’s tests, o1 should also outperform GPT-4 in data analysis, science, and programming. GitHub (owned by Microsoft), which tested o1 with its Copilot coding assistant, confirms that the model excels at optimizing algorithms and application code.
OpenAI also promises progress in o1’s multilingual capabilities, including Arabic and Korean. The next update should even reach the level of PhD students on difficult tasks in physics, chemistry and biology!
Limits to be exceeded
Of course, o1 isn’t perfect. It can be slow on some queries, sometimes taking more than 10 seconds to respond. It also sometimes hallucinates answers and admits less often than GPT-4 when it doesn’t know the answer. OpenAI acknowledges that there is still work to be done to make the model more reliable.
Also, o1 doesn’t have all the capabilities of ChatGPT yet, such as web browsing or file and image analysis. Its interface is quite rudimentary at the moment. And it is limited to 30 messages per week for o1-preview and 50 for o1-mini in ChatGPT, only for subscribers.
But OpenAI promises to soon open o1-mini for free to all ChatGPT users and add new features over time to make o1 more useful in everyday life.
Towards autonomous agents
For AI researchers, mastering reasoning is a key step toward human-level intelligence. The idea is that if a model can do more than just recognize patterns, it could unlock major advances in medicine, engineering, and more.
OpenAI sees o1 as a first step toward autonomous systems, or “agents,” that can make decisions and take action on our behalf. “We believe this is the critical breakthrough to move toward human-level intelligence,” says Bob McGrew, OpenAI’s chief research officer.
Of course, o1’s reasoning capabilities remain relatively slow and expensive for now. But let’s bet that the race for generative AI between OpenAI, Google, Meta, Microsoft and others is only just beginning. o1 could well pave the way for a new generation of more “intelligent” AI!
