Self-Repairing Cities - Robots for Resilient Infrastructure

The ORCA Hub has been instrumental in supporting early stage researchers. The student led team, ‘Foresight’, at Heriot-Watt University created a demonstration to compete in an event arranged by the University of Leeds. The event was to address and showcase cutting-edge robotics and autonomous systems to create, inspect, repair and maintain critical infrastructure. The team set out to address the following challenge themes in the following ways.

City and Society: Overcoming a key technological and regulator bottleneck to societal and industrial robotics with run-time safety compliance, protecting people and infrastructure during robotic missions.

Data and Decisions: Supporting run-time self-certification and up-skilling the user in the loop. Significant advancement in intelligent asset management for operational decision support with forecastable infrastructure RUL (Remaining Useful Life).

Perceive and patch: Frequency Modulated Continuous Wave (FMCW) Radar provides unprecedented insight to failure mode precursor data relating to structural integrity of buildings, corrosion under insulation and dielectric materials.

Overview

The demonstration presents the world’s first framework for run-time safety compliance and self-certification of robotics for infrastructure management. The methodology of the framework is based on computationally efficient analysis to support run-time, front-end, data analysis and adaptive decision-making. Utilising the Husky A200 platform, front-end datasets on the mission status and diagnostics of critical sub-systems within the Husky platform are used to update run-time system ontologies and decision making. The holistic and hierarchical-relational model of the robot, integrates the automata of the sensed and some non-sensed components, using prior knowledge, such as risk assessments and offline reliability data, to support run-time analysis, such as fault prognosis, detection, isolation and diagnosis. These computationally efficient data and system analyses then enable faults to be translated into failure modes that support run-time decision making during the mission.

Planned demonstration at Leeds

Within the demonstration, there are three key advancements for the global field of robotics, firstly; run-time safety compliance and self-certification. Traditional CE marking practices are insufficient for deployed or resident robotic platforms. The complexity of the robotic platforms and the dynamics of the environment e.g. ambient conditions, human interaction, requires a means to provide near to real-time assurance of the systems integrity to complete a mission safely. Secondly, FMCW radar technology can support localisation within opaque environments, detection of human operators within proximity to robots (including through walls) and integrity analysis of infrastructure. Thirdly, the full synchronised digital twin enables lower skilled operators to safety utilise and maintain these robotic platforms from remote locations whilst they are deployed.

Potential Impact for Industry

The transferable framework for safety compliance and self-certification is a vital enabler to the safe and productive use of any robotic platform within any field of deployment. With respect to the challenges of a dynamic environment, namely ambient conditions or the presence of unexpected people, Frequency Modulated Continuous Wave (FMCW) sensing is integrated onto the husky platform. The framework presents its results within a symbiotic digital twin of the infrastructure and robotic platform. With fully synchronised communication and data streams, the interactive digital twin provides operational decision support and trust for human in the loop operators of varying skill levels. The presentation of actionable information to the end user is used to support improvements in productivity associated with asset integrity as well as supporting user trust in safety during a line or beyond visual life of site mission.

The demonstration video above displays:

Autonomous mission planning - The user sets goals for robot to complete, the onboard planning system then calculates the most efficient order to perform these actions as well as any additional prerequisite actions that may be required

SLAM Navigation and Human Avoidance - while moving between task waypoints, the robot can autonomously avoid any humans or other stationary/moving objects which are blocking its original planned route.

FMCW Radar Integrity Inspection - FMCW radar sensing can be used for detection of corrosion under insulation in metals and for defects within infrastructure such as delamination, cracks and water ingress. In addition, the radar has been used for fast detection of salt and can even detect potholes before they are visible on the roads.

Raster Scan - The manipulators can be used to ensure accurate position of FMCW radar sensor during a scan and complete a full assessment by covering a smaller area where necessary.

Mixed Reality - Enables user to easily view diagnostics of Husky platform onsite.

Digital Twin - Augmented Reality enables diagnostics to be displayed for offsite evaluation. In addition, Digital twin enables remote fault detection on the asset by displaying run time health data and ontology messages over synchronised interfaces.

Experiences

Daniel Mitchell - I developed the FMCW radar sensor for inspection of salt, potholes and corrosion under insulation. This work gave me an opportunity to present the FMCW radar to many areas within industry. This included presenting a demonstration for corrosion under insulation at Faslane Naval Base to engineers who maintain their submarine fleet. This included a tour of a nuclear Vanguard Submarine to identify where this technology could be implemented in tight constrained areas. In addition, many demonstrations have been presented to industry in the field robotics lab where discussions have been made of how the radar can be utilised within the nuclear, construction and aviation industries

Joshua Roe - I have been developing the software solutions for autonomous mission planning, additional sensor integration. I also develop for the systems that perform diagnostics and prognosis of faults in the robot’s hardware and software systems whilst performing asset inspection, navigation and manipulation tasks. This project allowed me to further build upon my work with the platform, implementing the knowledge and skillsets I have developed over the course of my studies. I enjoyed continuing our work from the initial ORCA HUB trials at the ORE Catapult last year. I feel confident that the opportunities and additional experience that I have acquired while being part of this research team will have positive influence on my future work.

Leo Kong - I developed an Interoperable Digital Twin Framework to allow operators to remotely interact with their assets without advanced knowledge of complex embedded, sensory and robotic systems. This work gave me an opportunity to consider the needs of industries in the future. Also, it allows me to use my knowledge to create solutions for benefitting the world. Digital Twin is one part of the big picture in the team, so it also gave me chances to learn from others for completing this multi-discipline project. Besides industrial demonstration, I was also invited to give a presentation regarding this topic at an IET event in Glasgow Caledonia University.

Adrian Tang - I developed a FMCW sensor for detection of defects in the interior of the wind turbine blade. The FMCW have also been used for other applications in the industries such as corrosion under insulation (CUI), pothole and salt detection. Many demonstrations were carried out in the field robotics lab and further discussions were discussed on applying the radar into different area of industries.