3D model generators by OpenAI Point-E may be the next innovation to rock the field of AI. Point-E, a machine learning system that generates a 3D object from a text prompt, was made available to the public this week by OpenAI.
An article that was published along with the code base claims that Point-E can create 3D models on a single Nvidia V100 GPU in one to two minutes.
In the conventional sense, OpenAI Point-E does not produce 3D objects. Instead, it creates point clouds, which are discrete collections of data points in space that reflect 3D shapes; hence, the playful abbreviation. From a computational perspective, point clouds are simpler to create, but they are currently a major drawback of OpenAI Point-E because they cannot capture an object’s fine-grained shape or texture.
The OpenAI Point-E team trained an additional AI system to transform clouds into meshes in order to get around this restriction. However, in the report, they make a point that the model occasionally misses specific item details, resulting in blocky or deformed shapes.
OpenAI Point-E is able to generate 3D models within seconds
OpenAI Point-E is made up of two models: a text-to-image model and an image-to-3D model, in addition to the mesh-generating model, which is a standalone model. The text-to-image model was trained on tagged images to comprehend the relationships between words and visual concepts, much like generative art systems like OpenAI’s own DALL-E 2 and Stable Diffusion. The image-to-3D model, on the other hand, was taught to effectively translate between the two by being fed a set of photographs coupled with 3D objects.
Point-text-to-image E’s model creates a synthetic rendered item from a text prompt, such as “a 3D printed gear, a single gear, 3 inches in diameter and half-inch thick,” and feeds it to the image-to-3D model, which creates a point cloud.
OpenAI Point-E could generate colored point clouds that commonly matched word prompts after training the models on a dataset of “several million” 3D objects and related metadata, according to the OpenAI researchers. It’s not flawless; occasionally, Point-image-to-3D E’s model cannot interpret the image from the text-to-image model, leading to a shape that does not correspond to the text prompt. Even so, the OpenAI team claims that it is orders of magnitude faster than the prior state-of-the-art.
“While our method performs worse on this evaluation than state-of-the-art techniques, it produces samples in a small fraction of the time,”
“This could make it more practical for certain applications, or could allow for the discovery of higher-quality 3D object.”
-OpenAI team
OpenAI Point-E could be a powerful tool for designing projects
What exactly are the applications of OpenAI Point E? However, the point clouds created by Point-E might be utilized to create actual objects, for instance, through 3D printing, according to the OpenAI researchers. Once it’s a little more refined, the system might also find use in game and animation production processes thanks to the addition of the mesh-converting model.
Although it may be the most recent business to enter the 3D object generation market, OpenAI is by no means the first, as was previously said. A more developed version of Dream Fields, a generative 3D technology that Google revealed back in 2021, was released earlier this year under the name DreamFusion.
DreamFusion, in contrast to Dream Fields, doesn’t need any prior training. Therefore, it can create 3D models of objects without 3D data. While 2D art generators are currently the focus of attention, model-synthesizing AI has the potential to be the next major industrial disruptor. 3D models are frequently utilized in the domains of science, interior design, architecture, film, and television. Engineers utilize models as designs for new equipment, vehicles, and structures, while architectural firms use them to demonstrate proposed buildings and landscapes.
