S-Graphs+: Real-time Localization and Mapping leveraging Hierarchical Representations

E-print/Working paper (2022)

In this paper, we present an evolved version of the Situational Graphs, which jointly models in a single optimizable factor graph, a SLAM graph, as a set of robot keyframes, containing its associated ... [more ▼]

In this paper, we present an evolved version of the Situational Graphs, which jointly models in a single optimizable factor graph, a SLAM graph, as a set of robot keyframes, containing its associated measurements and robot poses, and a 3D scene graph, as a high-level representation of the environment that encodes its different geometric elements with semantic attributes and the relational information between those elements. Our proposed S-Graphs+ is a novel four-layered factor graph that includes: (1) a keyframes layer with robot pose estimates, (2) a walls layer representing wall surfaces, (3) a rooms layer encompassing sets of wall planes, and (4) a floors layer gathering the rooms within a given floor level. The above graph is optimized in real-time to obtain a robust and accurate estimate of the robot's pose and its map, simultaneously constructing and leveraging the high-level information of the environment. To extract such high-level information, we present novel room and floor segmentation algorithms utilizing the mapped wall planes and free-space clusters. We tested S-Graphs+ on multiple datasets including, simulations of distinct indoor environments, on real datasets captured over several construction sites and office environments, and on a real public dataset of indoor office environments. S-Graphs+ outperforms relevant baselines in the majority of the datasets while extending the robot situational awareness by a four-layered scene model. Moreover, we make the algorithm available as a docker file. [less ▲]

Unclonable human-invisible machine vision markers leveraging the omnidirectional chiral Bragg diffraction of cholesteric spherical reflectors

in Light: Science and Applications (2022), 11(309), 10103841377-022-01002-4

The seemingly simple step of molding a cholesteric liquid crystal into spherical shape, yielding a Cholesteric Spherical Reflector (CSR), has profound optical consequences that open a range of ... [more ▼]

The seemingly simple step of molding a cholesteric liquid crystal into spherical shape, yielding a Cholesteric Spherical Reflector (CSR), has profound optical consequences that open a range of opportunities for potentially transformative technologies. The chiral Bragg diffraction resulting from the helical self-assembly of cholesterics becomes omnidirectional in CSRs. This turns them into selective retroreflectors that are exceptionally easy to distinguish— regardless of background—by simple and low-cost machine vision, while at the same time they can be made largely imperceptible to human vision. This allows them to be distributed in human-populated environments, laid out in the form of QR-code-like markers that help robots and Augmented Reality (AR) devices to operate reliably, and to identify items in their surroundings. At the scale of individual CSRs, unpredictable features within each marker turn them into Physical Unclonable Functions (PUFs), of great value for secure authentication. Via the machines reading them, CSR markers can thus act as trustworthy yet unobtrusive links between the physical world (buildings, vehicles, packaging,...) and its digital twin computer representation. This opens opportunities to address pressing challenges in logistics and supply chain management, recycling and the circular economy, sustainable construction of the built environment, and many other fields of individual, societal and commercial importance. [less ▲]

Advanced Situational Graphs for Robot Navigation in Structured Indoor Environments

E-print/Working paper (2022)

Mobile robots extract information from its environment to understand their current situation to enable intelligent decision making and autonomous task execution. In our previous work, we introduced the ... [more ▼]

Mobile robots extract information from its environment to understand their current situation to enable intelligent decision making and autonomous task execution. In our previous work, we introduced the concept of Situation Graphs (S-Graphs) which combines in a single optimizable graph, the robot keyframes and the representation of the environment with geometric, semantic and topological abstractions. Although S-Graphs were built and optimized in real-time and demonstrated state-of-the-art results, they are limited to specific structured environments with specific hand-tuned dimensions of rooms and corridors. In this work, we present an advanced version of the Situational Graphs (S-Graphs+), consisting of the five layered optimizable graph that includes (1) metric layer along with the graph of free-space clusters (2) keyframe layer where the robot poses are registered (3) metric-semantic layer consisting of the extracted planar walls (4) novel rooms layer constraining the extracted planar walls (5) novel floors layer encompassing the rooms within a given floor level. S-Graphs+ demonstrates improved performance over S-Graphs efficiently extracting the room information while simultaneously improving the pose estimate of the robot, thus extending the robots situational awareness in the form of a five layered environmental model. [less ▲]

Situational Graphs for Robot Navigation in Structured Indoor Environments

E-print/Working paper (2022)

Semantic situation awareness of ellipse shapes via deep learning for multirotor aerial robots with a 2D LIDAR

in 2020 International Conference on Unmanned Aircraft Systems (ICUAS) (2020, September)

In this work, we present a semantic situation awareness system for multirotor aerial robots equipped with a 2D LIDAR sensor, focusing on the understanding of the environment, provided to have a drift-free ... [more ▼]

In this work, we present a semantic situation awareness system for multirotor aerial robots equipped with a 2D LIDAR sensor, focusing on the understanding of the environment, provided to have a drift-free precise localization of the robot (e.g. given by GNSS/INS or motion capture system). Our algorithm generates in real-time a semantic map of the objects of the environment as a list of ellipses represented by their radii, and their pose and velocity, both in world coordinates. Two different Convolutional Neural Network (CNN) architectures are proposed and trained using an artificially generated dataset and a custom loss function, to detect ellipses in a segmented (i.e. with one single object) LIDAR measurement. In cascade, a specifically designed indirect-EKF estimates the ellipses based semantic map in world coordinates, as well as their velocity. We have quantitative and qualitatively evaluated the performance of our proposed situation awareness system. Two sets of Software-In-The-Loop simulations using CoppeliaSim with one and multiple static and moving cylindrical objects are used to evaluate the accuracy and performance of our algorithm. In addition, we have demonstrated the robustness of our proposed algorithm when handling real environments thanks to real laboratory experiments with non-cylindrical static (i.e. a barrel) objects and moving persons. [less ▲]

Trajectory Tracking for Aerial Robots: an Optimization-Based Planning and Control Approach

in Journal of Intelligent and Robotic Systems (2020), 100

In this work, we present an optimization-based trajectory tracking solution for multirotor aerial robots given a geometrically feasible path. A trajectory planner generates a minimum-time kinematically ... [more ▼]

In this work, we present an optimization-based trajectory tracking solution for multirotor aerial robots given a geometrically feasible path. A trajectory planner generates a minimum-time kinematically and dynamically feasible trajectory that includes not only standard restrictions such as continuity and limits on the trajectory, constraints in the waypoints, and maximum distance between the planned trajectory and the given path, but also restrictions in the actuators of the aerial robot based on its dynamic model, guaranteeing that the planned trajectory is achievable. Our novel compact multi-phase trajectory definition, as a set of two different kinds of polynomials, provides a higher semantic encoding of the trajectory, which allows calculating an optimal solution but following a predefined simple profile. A Model Predictive Controller ensures that the planned trajectory is tracked by the aerial robot with the smallest deviation. Its novel formulation takes as inputs all the magnitudes of the planned trajectory (i.e. position and heading, velocity, and acceleration) to generate the control commands, demonstrating through in-lab real flights an improvement of the tracking performance when compared with a controller that only uses the planned position and heading. To support our optimization-based solution, we discuss the most commonly used representations of orientations, as well as both the difference as well as the scalar error between two rotations, in both tridimensional and bidimensional spaces $SO(3)$ and $SO(2)$. We demonstrate that quaternions and error-quaternions have some advantages when compared to other formulations. [less ▲]

Vision-Based Aircraft Pose Estimation for UAVs Autonomous Inspection without Fiducial Markers

in IECON 2019-45th Annual Conference of the IEEE Industrial Electronics Society (2019, October)

The reliability of aircraft inspection is of paramountimportance to safety of flights. Continuing airworthiness of air-craft structures is largely based upon the visual detection of smalldefects made by ... [more ▼]

The reliability of aircraft inspection is of paramountimportance to safety of flights. Continuing airworthiness of air-craft structures is largely based upon the visual detection of smalldefects made by trained inspection personnel with expensive,critical and time consuming tasks. At this aim, Unmanned AerialVehicles (UAVs) can be used for autonomous inspections, aslong as it is possible to localize the target while flying aroundit and correct the position. This work proposes a solution todetect the airplane pose with regards to the UAVs position whileflying autonomously around the airframe at close range forvisual inspection tasks. The system works by processing imagescoming from an RGB camera mounted on board, comparingincoming frames with a database of natural landmarks whoseposition on the airframe surface is known. The solution has beentested in real UAV flight scenarios, showing its effectiveness inlocalizing the pose with high precision. The advantages of theproposed methods are of industrial interest since we remove manyconstraint that are present in the state of the art solutions. [less ▲]

Deep learning based semantic situation awareness system for multirotor aerial robots using LIDAR

in 2019 International Conference on Unmanned Aircraft Systems (ICUAS) (2019, June)

In this work, we present a semantic situation awareness system for multirotor aerial robots, based on 2D LIDAR measurements, targeting the understanding of the environment and assuming to have a precise ... [more ▼]

In this work, we present a semantic situation awareness system for multirotor aerial robots, based on 2D LIDAR measurements, targeting the understanding of the environment and assuming to have a precise robot localization as an input of our algorithm. Our proposed situation awareness system calculates a semantic map of the objects of the environment as a list of circles represented by their radius, and the position and the velocity of their center in world coordinates. Our proposed algorithm includes three main parts. First, the LIDAR measurements are preprocessed and an object segmentation clusters the candidate objects present in the environment. Secondly, a Convolutional Neural Network (CNN) that has been designed and trained using an artificially generated dataset, computes the radius and the position of the center of individual circles in sensor coordinates. Finally, an indirect-EKF provides the estimate of the semantic map in world coordinates, including the velocity of the center of the circles in world coordinates.We have quantitative and qualitative evaluated the performance of our proposed situation awareness system by means of Software-In-The-Loop simulations using VRep with one and multiple static and moving cylindrical objects in the scene, obtaining results that support our proposed algorithm. In addition, we have demonstrated that our proposed algorithm is capable of handling real environments thanks to real laboratory experiments with non-cylindrical static (i.e. a barrel) and moving (i.e. a person) objects. [less ▲]

Fast and Robust Flight Altitude Estimation of Multirotor UAVs in Dynamic Unstructured Environments Using 3D Point Cloud Sensors

in Aerospace (2018), 5(3),

This paper presents a fast and robust approach for estimating the flight altitude of multirotor Unmanned Aerial Vehicles (UAVs) using 3D point cloud sensors in cluttered, unstructured, and dynamic indoor ... [more ▼]

This paper presents a fast and robust approach for estimating the flight altitude of multirotor Unmanned Aerial Vehicles (UAVs) using 3D point cloud sensors in cluttered, unstructured, and dynamic indoor environments. The objective is to present a flight altitude estimation algorithm, replacing the conventional sensors such as laser altimeters, barometers, or accelerometers, which have several limitations when used individually. Our proposed algorithm includes two stages: in the first stage, a fast clustering of the measured 3D point cloud data is performed, along with the segmentation of the clustered data into horizontal planes. In the second stage, these segmented horizontal planes are mapped based on the vertical distance with respect to the point cloud sensor frame of reference, in order to provide a robust flight altitude estimation even in presence of several static as well as dynamic ground obstacles. We validate our approach using the IROS 2011 Kinect dataset available in the literature, estimating the altitude of the RGB-D camera using the provided 3D point clouds. We further validate our approach using a point cloud sensor on board a UAV, by means of several autonomous real flights, closing its altitude control loop using the flight altitude estimated by our proposed method, in presence of several different static as well as dynamic ground obstacles. In addition, the implementation of our approach has been integrated in our open-source software framework for aerial robotics called Aerostack. [less ▲]

Model Predictive Control for Aerial Collision Avoidance in Dynamic Environments

in 26th Mediterranean Conference on Control and Automation (MED), Zadar, Croatia, 19-22 June 2018 (2018, June)

Autonomous navigation in unknown environments populated by humans and other robots is one of the main challenges when working with mobile robots. In this paper, we present a new approach to dynamic ... [more ▼]

Autonomous navigation in unknown environments populated by humans and other robots is one of the main challenges when working with mobile robots. In this paper, we present a new approach to dynamic collision avoidance for multi-rotor unmanned aerial vehicles (UAVs). A new nonlinear model predictive control (NMPC) approach is proposed to safely navigate in a workspace populated by static and/or moving obstacles. The uniqueness of our approach lies in its ability to anticipate the dynamics of multiple obstacles, avoiding them in real-time. Exploiting active set algorithms, only the obstacles that affect to the UAV during the prediction horizon are considered at each sample time. We also improve the fluency of avoidance maneuvers by reformulating the obstacles as orientable ellipsoids, being less prone to local minima and allowing the definition of a preferred avoidance direction. Finally, we present two real-time implementations based on simulation. The former demonstrates that our approach outperforms its analog static formulation in terms of safety and efficiency. The latter shows its capability to avoid multiple dynamic obstacles. [less ▲]