Digital Twins, GIS And 3D Augmented Reality

• Digital twins are digital representations of a real world asset;

• Digital twins can be used to gain valuable insights and more fully understand the physical asset;

• Data is king, and drones can provide a variety of datasets to feed into the digital twin process;

• Augmented reality is emerging as an immersive tool to deepen understanding within the digital twin ecosystem.  

Digital twins are a hot topic right now.

This exciting method of combining the physical world with the digital is helping to improve business processes, reduce risk, optimise operational efficiency and enhance decision-making.

And in an environment where data is king, drones have an important role to play.

In this blog, we take a look at digital twins: What they are and how they can be a key business driver, and how the emergence of augmented reality can take this technology to even greater levels.   

What Is A Digital Twin?

Digital twins are digital representations of a physical asset.

Like the real world they represent, digital twins are ever-changing; continuously adjusting to reflect the near real-time status or condition of the real-world asset.

As a synchronised and dynamic dataset, digital twins can be used to gain valuable insights and more fully understand the physical asset. 

Digital twinning is no mean feat: The continuous updates are fed from a rich tapestry of technology, including sensors, drones, reality capture, GIS and BIM integration, real-time Internet of Things, cloud computing, advanced AI and machine learning.

Digital twins provide a single view that can be accessed throughout the entire lifecycle of a project and can be used to represent current, past, or even future states of assets. 

Drones And Digital Twins

Data is currency when it comes to building digital twins. These 'living' and constantly evolving models need to have a high-level of validity, and up-to-date data is crucial to this.   

Drones have a role to play here, as they provide an effective data capture tool, collecting highly-accurate information, safely and efficiently. This is especially important for hard-to-reach assets, or in environments where asset downtime needs to be kept to the absolute minimum.

Drones are capable of collecting complete, reliable and well-organised baseline datasets to flow into the complex and layered process of building a digital twin.

The ease of deployment and cost effectiveness of drones enables organisations to conduct the regular surveys needed to help digital twins constantly evolve to stay relevant and up to date.

GIS data is also fundamental for building a digital twin, especially if the goal of the twin is to represent historical accuracy, view performance, or predict future state.

And drones have become an essential tool for collecting GIS and survey-grade data. Key tools include the DJI M300 RTK with P1 photogrammetry camera or L1 LiDAR sensor, or the low-altitude mapping drone, the DJI Phantom 4 RTK. 

How Drone Data Can Build Digital Twins

So, drones can be a key tool in the digital twin process, helping to collect data quickly, safely and accurately. 

But what data can drones collect and how can these outputs be used to build and evolve a digital twin?

Point Clouds  

A point cloud dataset is a digital representation of a survey site, or asset, which is made up of thousands of points; each one being a geometric coordinate.

When put together, this mass of single spatial measurements come together to form a fleshed out model in 3D space. 

From a point cloud, you can make observations (and measurements) about an object’s depth, elevation, geometry, and location in space.

Point clouds tend to be generated by using 3D laser scanners and LiDAR (light detection and ranging), with each point representing a single laser scan measurement. However, photogrammetry can also be used to build point clouds.

When it comes to point clouds, the more points collected, the denser the model.

The DJI surveying ecosystem of the M300 RTK drone, L1 sensor, and DJI Terra software package is an end-to-end mapping workflow for LiDAR and photogrammetry point cloud generation, while the P1 camera (45MP full-frame sensor) is a higher-resolution photogrammetry sensor. 

Point clouds provide the basis for detail to be added and made into a digital twin.

3D Mesh

A 3D mesh is the evolution of a point cloud, whereby small triangles are created between the points in the point cloud. Mapping software then creates a texture for each of the thousands of triangles by extracting a small portion of the original photograph of each point, and builds a 3D mesh or model.

While a 3D mesh is not a digital twin, such a model can provide detailed and valuable visual representations, and can also be viewed via augmented reality - covered later in this blog. 

2D Orthomosaic

Drones can be used for 2D orthomosaic generation.

A 2D orthomosaic is a top-down map of a survey site, or asset, built by stitching hundreds or thousands of digital photos - collected by the drone - together.

Drones and high-resolution cameras, such as the DJI M300 RTK and P1 45MP full-frame photogrammetry sensor, can achieve sub 1cm ground sample distance, to create highly detailed and crystal clear orthomosaics.

These real-world, geo-referenced 2D maps offer a stepping stone in the generation of 3D meshes that can be incorporated to provide an alternative perspective within a digital twin.

DTM Data

Drones - typically with LiDAR capabilities - can collect data to build a Digital Terrain Model (DTM).

A DTM is a model of the bare earth (without any objects/buildings in it) containing elevation data of the terrain.

This is different to a Digital Surface Model (DSM), which represents the earth's surface and includes all objects on it. LiDAR and photogrammetry can be useful applications for this. 

DTM drone data can be fed into digital twins to ensure the topographical elevation changes are accurately modelled.

What Are The Advantages Of Digital Twins?

Digital twins enable users to visualise the entire asset – in a web browser, on a tablet, or with a mixed reality headset (more of that later), as well as to check status, conduct analysis and generate insights in order to predict and optimise asset performance.

Essentially, they provide a visual geo-referenced representation of the real world, to help users find out what has changed, display it in a highly visual format, and then do something meaningful with these insights.

Benefits include:

• Historical Record: One of the biggest advantages of digital twins is that they provide a 3D snapshot of the real world. This can be invaluable for monitoring progress over time and keeping a 3D representation of a project lifecycle long after completion of a project. BIM data can also be extracted from these digital assets.

• Operational Performance Monitoring: 3D models provide a clear and concise visualisation of an asset and enable detailed and accurate exploration and calculations. This can be crucial for spotting defects or mistakes.

• Predicting Future Performance: Some of the highest expectations around digital twins come from having the ability to use a high precision version of an asset to simulate, evaluate, and predict the future. In the AEC industry, aggregated BIM information is used in the practice of Virtual Design and Construction to play out planned construction to detect possible construction conflicts or safety issues. Predictive maintenance can reduce maintenance costs.

Digital Twin Applications And Use Cases

Digital twins can be used across a range of industry verticals, such as construction, the energy sector, and public safety.

They can be particularly beneficial for providing information about difficult to access assets such as offshore wind turbines, sewer lines and industrial facilities and nuclear plants.

In the construction world, a digital twin can help building firms understand how a building is performing in real-time, which allows them to tweak performance to optimise efficiency. Data collected from digital twins can be used for planning and designing future buildings.

Other examples include:

• Amsterdam Airport Schiphol has built a dynamic, digital twin of their airport premises incorporating open standard BIM data through Safe Software’s FME into Esri ArcGIS with live data feeds for aircraft and ground vehicle locations.

• Utilising the Bentley software ecosystem, Network Rail created a digital twin of Paddington Station, providing a realistic representation of signage and multiple signalling options. As a result of creating the digital twin, they ensured future work for the modelling team at an accelerated, more efficient pace, as well as provided a firm foundation for the use of digital twins across the region.

Digital Twins Vs BIM

BIM is a fundamental part of building a digital twin, but they should not be confused, because they are different.

In many ways, digital twins are an evolution of BIM, enabling users to evolve their outputs and deepen the use of collaborative data.

Firstly, what is BIM?

Standing for Building Information Modelling, BIM provides a way of accessing detailed information about buildings and infrastructure assets, enabling the creation of 3D models which include data associated with the functional and physical characteristics of an asset.

Over the years, BIM has proved a valuable tool for predictive maintenance, asset tracking, and facilities management.

The highly-detailed 3D models help to improve process visibility, establish a clear project vision, and enable contractors, engineers and stakeholders to collaborate and gain richer insights.

For these reasons, BIM and digital twins are built on common principles, and both enable teams to look at assets as ongoing projects.

But, to improve and adapt projects for greater value, real-time insights are vital. And this is where digital twins come into play.

Often used solely as a 3D-modelling tool for design and construction phases, BIM builds static models which do not include the dynamics of a live digital twin. In comparison, digital twins harness live data to evolve and replicate the real world. In construction, a digital twin not only looks like the real building, but also acts like it - providing greater value through the asset lifecycle.

At its core, a digital twin can be an output of a BIM process and is essentially a ‘living’ version of the project or asset view that BIM processes exist to create – able to evolve and transform using real-time data once the asset is in use. 

Digital Twins And Augmented Reality

Augmented reality (AR) is a hot topic right now, and this technology has the ability to transform the way that digital twins are viewed and accessed.

In essence, it lifts digital twins into the virtual space, providing an immersive and integrated view of infrastructure assets.

Through AR, data can be overlaid on to the real world allowing for a better level of visualisation by placing the digital twin, on, for example, a table or other flat surface  through mobile devices like tablets or smartphones. Smart Glasses can also be used by overlaying data over what the user is currently seeing. Onsite workers like individuals working on oil rigs, could use something like the Microsoft HoloLens in combination with a digital twin to get real-time information when performing maintenance and repairs.

Utilising AR will help to drive better informed decisions to improve network/service availability, enhance passenger and worker safety, ensure regulatory compliance, and reduce environmental impact.

For instance, through the application of drones, robots, and AI-based computer vision, tanks could be autonomously inspected via a living digital twin, enabling experts to conduct inspections remotely, vastly increasing productivity and leveraging the knowledge of scarce resources.

An emerging resource in this sector is Bentley Systems'  iTwin software, while Esri's ArcGIS Runtime supports building production-ready, augmented reality (AR) experiences for iOS and Android devices.

Augmented reality broadens the digital twin ecosystem, helping to further improve and deepen the understanding of infrastructure. 

Summary

Technology is advancing fast. Drones and 3D modelling have become a key tool for industry, but this is evolving to even greater realms.

Digital twins present a new phase in the process - further driving learning, insights and productivity - and drones have a crucial role to play in this.