MRLink: Expanding the Boundaries of Remote Robotic Teleoperation with New Sensory Capabilities 

In our previous article about the MRLink platform, we introduced our innovative approach to robot teleoperation in virtual reality. This solution opened the door to intuitive, immersive, and precise control of remote robots, allowing users to interact with their environment as if they were physically present.

Since that milestone, we’ve been working on further enhancements to the platform, focusing on richer sensory input and improving situational awareness for operators. Today, we’re excited to share some of the new features designed to deliver more data, better context, and a more natural teleoperation experience.

One of the most significant upgrades to the system is the integration of the Azure Kinect. This device serves both as a remote camera and as a depth sensor, enabling us to capture point cloud data in real time.

By visualizing this data directly inside the VR application, operators can see and understand the spatial layout of the remote environment with unprecedented clarity. This enhances depth perception, improves navigation, and supports more accurate manipulation tasks, especially in complex or cluttered spaces.

Beyond navigation, depth sensing opens a wide range of applications in industrial maintenance:

• 3D mapping and Digital Twins: The sensors can be used to generate accurate 3D models of facilities or machinery, enabling virtual inspections, planning interventions, and tracking changes over time.
• Dimensional inspection: Depth sensors allow real-time measurement of distances, clearances, or alignments during maintenance procedures.
• Equipment deformation detection: Historical and current point clouds can be compared to detect subtle deformations or misalignments.
• Enhanced task guidance: Spatially accurate AR/VR overlays can guide operators step by step through complex repairs.

Another powerful addition is the inclusion of a thermal camera. This feature allows the system to detect temperature anomalies in equipment, giving operators critical information about the state of machinery before they even touch it.

For example, identifying overheated components can help prioritize repairs, assess potential risks, or plan maintenance procedures more efficiently. This layer of thermal feedback enhances decision-making and safety during remote interventions.

In practical terms, thermal data can support multiple industrial use cases:

• Early fault detection: Spotting abnormal heat signatures in motors, transformers, or electrical panels before failures occur.
• Monitoring rotating equipment: Identifying bearing or gear issues through unusual temperature patterns.
• Leak detection: Locating steam or fluid leaks based on temperature variations.
• HVAC and insulation checks: Detecting heat loss or inefficiencies in production environments.
• Real-time safety monitoring: Providing alerts for high-temperature zones to ensure operator safety.

While data visualization is important, touch remains a key element of how humans interact with the world. That’s why our next step involves integrating XELA Robotics tactile sensors and the SenseGlove Nova 1 gloves.

This combination will enable operators to feel interaction with remote objects, such as gripping a tool, pressing a surface, or manipulating delicate components. This tactile feedback will further bridge the gap between physical and virtual presence, making teleoperation more intuitive and precise.

Aside from grip force feedback, other practical applications of tactile sensing in industrial maintenance include:

• Surface condition detection: Feeling texture and resistance can help identify issues like corrosion, cracks, or material fatigue without relying solely on vision.
• Delicate component handling: Tactile cues can guide the operator when interacting with sensitive parts such as electrical connectors or seals.
• Tool operation feedback: Simulating the feel of operating switches, levers, or fasteners makes remote maintenance tasks more accurate and natural.
• Improved safety and control: Immediate haptic feedback can warn operators of unintended contact with hot surfaces, sharp edges, or restricted areas.

The main application we are pursuing with this type of teleoperation technology is in the highly controlled industrial environment that is a particle accelerator facility, dedicated to fusion technology research. These sites operate with complex, delicate, and sometimes hazardous systems where direct human access is limited or requires extensive safety procedures.

By combining remotely controlled robots with depth sensing, thermal imaging, and tactile interaction, operators can perform inspections and maintenance tasks safely inside restricted zones without interrupting critical experiments or exposing personnel to radiation, high magnetic fields, extreme temperatures, or dangerous noise levels.

This integration of advanced sensory feedback with teleoperation represents a major step forward in these environments, where operational uptime, precision, and safety are essential. It transforms maintenance into a controlled, data-rich, and proactive process, accelerating both research and technological development.

The evolution of MRLink continues to push the limits of immersive teleoperation. With the integration of depth sensing, thermal imaging, and tactile feedback capabilities, operators will benefit from a richer, more complete understanding of remote environments.

These enhancements mark a step toward making remote robotics not only more capable but also more human-centric. We’re excited to keep innovating, expanding the platform, and bringing new possibilities to industries that rely on safe, efficient, and intelligent remote operations.