When we were still discovering all the capabilities of super powerful artificial intelligences like DALL-E, Midjourney and ChatGPT, the OpenAI firm goes to surprise us with another new tool: GPT-4. Even better than the previous ones for everything it is capable of doing. It was already announced and finally arrived yesterday. In this article we are going to see everything we have been able to discover about whether it is a threat to the task of developers.
First, let’s remember that ChatGPT is nothing more than an interface to communicate with a ‘language model’, an AI that for now is GPT-3.5so the ChatGPT website may not change anything while, ‘behind the scenes’, the AI that offers the answers is updated and improved remarkably.
And it is that GPT-4 is a model with a much greater complexity than its predecessor, and also with greater capacity: now will be able to handle more than 25,000 wordsthanks to which you will be able to handle queries and conversations with a much broader context without getting lost during the conversation.
Software engineers: rest easy.
GPT-4 isn’t ready to replace you pic.twitter.com/enDoOuXSSh—Alex Reibman (@AlexReibman) March 14, 2023
Now, if we talk about its capabilities in terms of software development, we have software engineers, they could rest easy, since GPT-4 is not prepared to replace you… if you are very good, yes. At least that’s what Alex Reibman says after verifying that OpenAI’s new AI performs very poorly in Leetcode tests that are difficult.
ZAO, the Chinese MOBILE APP that through DEEPFAKE turns you into DICAPRIO in SECONDS
What is Leetcode and how many developers can handle it
Now, although GPT-4 is not great in the hard tests of Leetcode, it must be said that many developers are not great either… LeetCode is a web platform widely used by the human resources departments of technology companies to recruit their engineers. So does Twitter.
However, it should be said that, as many people indicate in the same Twitter thread or as you can find in Internet forums or on social networks, there are many engineers who affirm that LeetCode hard tests are really hard for people too and not just for GPT-4.
It is also worth noting that looking at the full list of capabilities of this new tool in different fields, GPT-4 scores best in the exam for law, biology and other academic subjects, and the worst in problems where solving difficult code. You can see the entire list in this tweet:
Interesting benchmark. Here’s the full list if anyone’s interested.
GPT-4 performs best on bar exam, biology and other academics and worst on hard code problems. pic.twitter.com/7Y5AoNo7iC
— Leo Rezaei (@theLionary) March 14, 2023
Great development capabilities of GPT-4
Although GPT-4 can’t handle tough engineering tests, it does have impressive capabilities. For example, you can program games. User Ammar Reshi, designer, shows how he has recreated a game of Snake (the typical snake that we played so much in the early 2000s) that runs in your browser using Chat GPT-4 and repeat (a tool for building collaborative software), with zero Javascript knowledge. And he has done it, he says, in less than 20 minutes.
To do this, he first asked GPT-4 to provide him with code for a Snake game, with the necessary HTML, CSS and Javascript to make it work. This provided all the code I needed to get the project up and running. “He wasn’t able to fill it all out in one answer, but I just replied with ‘continue'” to give him all the information. With this information he went to Replit, created an account, and pasted the information.
Started early to play but there were problems like the snake dying after its first bite instead of growing. So he asked GPT-4 to make those adjustments and…it did.
It had some issues. The snake would die after its first bite instead of growing, and it kept adding the score in increments of 2 instead of 1. So I asked GPT-4 to make those tweaks and it did! pic.twitter.com/ud0Cjl6AqS
— Ammaar Reshi (@ammaar) March 14, 2023
Pietro Schirano, designer and AI expert it also says have recreated the game of Pong in less than 60 seconds.
They have also shown us that GPT-4 you can create a functional website from a handmade sketch on a paper napkin.
A major limitation of GPT-4
However, as with ChatGPT, it has among its great limitations that it cannot provide no information after 2021 and requests for earlier information to the Internet will have much fewer sources available, since this AI has been trained with information taken from the Internet, the new AI presents a similar problem.
Horace He, Pytorch expert, did tests on it to verify “of the easier problems in Codeforces, solved 10/10 pre-2021 issues and 0/10 recent issues“, which, he says, makes him suspect that GPT-4’s performance is influenced by data contamination, at least in Codeforces.