Auto Digital Twins 3: 3D model generation and automatic mass labeling for the creation of digital twins

Digital twins are a very powerful tool that help companies improve their data-driven decision making, simulating strategies and optimizing behaviors.

In our eagerness to investigate in this field, we have joined an innovation consortium to develop a project that seeks to provide SMEs in the industrial sector with a tool that automates the data capture process.

The objective of the solution, called Auto Digital Twins, is to accelerate the digitization and modernization of corporate infrastructures and to continue advancing in the Industry 4.0 paradigm.

Auto Digital Twins III employs cutting-edge technologies such as AI and Big Data to create highly accurate digital models, improving the simulation and management of industrial processes. In this third phase, we will work on the generation of 3D models and massive automatic labeling for the creation of digital twins.  

Digital Twins and Industry

Digital twins make it possible to simulate how an industrial plant behaves, even before it comes into existence. Simulating changes in production makes it possible to avoid problems before they occur, minimize downtime or customize production to adapt the industry to flexible manufacturing.

Despite all their advantages, digital twins encounter barriers to entry in terms of the degree of digitization, due to the complexity of system integration, the absence of qualified personnel, or the lack of tools to automate data labeling.

To address these limiting factors, the Auto Digital Twins project was proposed. Its aim is to provide the industry with a tool to accelerate the implementation of digital twins and solve one of its main bottlenecks: the high cost of obtaining and labeling the 3D elements that make up industrial infrastructures. 

Phase 1: Auto Digital Twins I

The project’s first phase, Auto Digital Twins I, has resulted in a point cloud to 3D transformation tool. The Spot robot was used in conjunction with the Leica sensor to scan an industrial facility and obtain point clouds. For this purpose, a viewer with progressive rendering has been developed, capable of displaying these large point clouds smoothly. In addition, AI algorithms have been implemented on the point cloud to recognize complex objects and, once recognized, 3D models have been obtained in industrial standard formats, such as BIM and CAD. 

Phase 2: Auto Digital Twins II

The second phase of the project, also already completed, has extended the tool’s object detection capability.

The extension of the object detection capability of the digital twin has been worked on through two main lines of research: 

1. We worked on the synthetic generation of data in order to have a sufficiently large dataset to improve the training of AI models.
2. We worked on the integration of artificial vision techniques on 2D data images, merging them in an appropriate way with the techniques implemented on point clouds to achieve a higher and more versatile recognition capacity and speed. 

Phase 3: Auto Digital Twins III

In this phase, the main focus is on the recognition of textures, defects, and surface health of the materials, as well as the selective search by voice from large language models.

In this last phase of the project, the aim is for the developed tool to be able to provide complementary attributes to the digital twin results obtained in the previous phases.

These new attributes include an analyzer and digital registration of textures and materials in complex environments, as well as indicators of anomalies and surface health of surfaces. This functionality, for the development of which the use of data augmentation techniques is also foreseen, will allow the 3D objects generated with the tool to carry valuable complementary information that could be very useful for the management of materials and circularity objectives. 

Another highlight of this phase is the incorporation of new interactive functionality of selective search by voice commands, based on large language models, very useful for digital inventories and rapid diagnostics.

With all this, the project responds to a change of working method in the way of generating 3D data to create digital twins and seeks to provide Spanish SMEs in the industrial sector with a tool to automate the process of data capture allowing them to accelerate the digitization and modernization of their infrastructures towards the new paradigm of Industry 4.0. 

In this way, the consortium of which Plain Concepts is part, together with OnTech Innovation, Intelligent Behavior Robots, MS2S Innovation, and Smart City Cluster, will continue the efforts initiated in the previous phase of this project to promote the implementation of digital twins in industries and improve them with new and more ambitious functionalities. 

Project reference: AEI-010500-2024-23 

Initiative financed by the Ministry of Industry and Tourism within the AEI support program to contribute to improving the competitiveness of the Spanish industry.