After the explosion of images generated by artificial intelligence, the next step of this technology is in sight. NVIDIA announced GET3D, a new AI model capable of generating 3D objects that could be used in video games, movies, or the metaverse. With the help of a GPU and 2D images taken from multiple angles, the model is capable of creating complex shapes with high-fidelity textures.
According to an NVIDIA blog post, engineers trained the AI model on one million images. With the help of several A100 Tensor Core GPUs, the team achieved their goal in two days. According to the company, GET3D is capable of producing 20 objects per second using a single GPUwhich would facilitate the work of artists and content creators.
Until now, GET3D is capable of generating varied objects ranging from cars, chairs, animals or motorcycles, to human beings or buildings. The model can be combined with another NVIDIA AI tool to create the object with a specific style. Engineers mention that it can create highly detailed 3D meshes with complex topology and realistic textures that can be exported to game engines or rendering applications.
3D objects can be edited for use in video games or movies
One of the key points of GET3D is that results can be edited, something that was difficult with previous attempts. At first glance, the result is similar to what we get if we use the photogrammetry technique, with a detailed mesh and texture model. The only drawback is that this class of models requires subsequent “cleaning” work to optimize the polygon count.
Previous work on 3D generative modeling lacks geometric details, is limited in the mesh topology they can produce, typically does not support textures, or uses neural renderers in the synthesis process, making its use in common 3D software non-trivial. .
As in other deep learning models, the larger the training dataset, the better the results. GET3D uses a synthetic dataset, although the researchers plan for the next version to be trained on real-world data.
NVIDIA already plans the next version of its GET3D model
The creation of 3D objects by means of artificial intelligence is interesting and NVIDIA has taken an important step. Nevertheless, three dimensional models for video games, movies or other entertainment applications are much more complex than a Mindjourney experiment or DALL-E. These objects must meet additional characteristics such as polygon count or topology if they are to be used in games or scenes that involve animation.
A few weeks ago, a similar application became a trend on Twitter. Kaedim, a service that promises to create 3D models from 2D images using artificial intelligence, caught the attention of multiple artists. after publishing results that seemed too good to be trueSome they accused the company of lying and using 3D professionals.
In the end, Kaedim came forward and said that although it uses an AI algorithm for the creation of the object, one person edits the final result. The “quality control engineer” reviews the model and adapts it to meet the quality standard required by the client before sending it. Although this optimizes production time with respect to manual work, the algorithm still cannot deliver reliable results.