UAV Tracking with Solid-State Lidars: Dynamic Multi-Frequency Scan Integration

Iacopo Catalano*, Ha Sier*, Xianjia Yu*, Tomi Westerlund*, Jorge Peña Queralta*,+
*University of Turku, +ETH Zurich
Paper Code arXiv

Abstract

With the increasing use of drones across various industries, the navigation and tracking of these unmanned aerial vehicles (UAVs) in challenging environments, namely GNSS-denied environments, have become critical issues. In this paper, we propose a novel method for a ground-based UAV tracking system using a solid-state LiDAR, which dynamically adjusts the LiDAR frame integration time based on the distance to the UAV and its speed. Our method fuses two simultaneous scan integration frequencies for high accuracy and persistent tracking, enabling reliable state estimation of the UAV even in challenging scenarios. The application of the Inverse Covariance Intersection method and Kalman filters allows for better tracking accuracy and can handle challenging tracking scenarios. Compared to previous works in solid-state lidar tracking, this paper presents a more complete and robust solution.

We have performed a number of experiments to evaluate the performance of the proposed tracking system and identify its limitations. Our experimental results demonstrate that the proposed method clearly outperforms the baseline method and ensures tracking is more robust across different types of trajectories.


Methodology

Pipeline Diagram

We propose two simultaneous tracking estimators on two different scan frequencies running in parallel where the integration time, defining the number of scans to accumulate, is dynamically adjusted to optimize the point cloud density:

  1. Adaptive Sparse Tracking (AST): Sparse point clouds are integrated up to 5 consecutive scans. This process provides high accuracy for estimating the UAV position and speed, but only tracks it through a reduced number of points and not necessarily in all frames.
  2. Adaptive Dense Tracking (ADT): The number of scans ranges between 10 and 50. The extracted point cloud representing the UAV is distorted by motion, which reduces the accuracy of localization and speed estimation, but the tracking is more robust and persistent as the UAV can be recognized in most frames.
Our main contributions with respect to our previous work are in both data processing (real-time adaptive dynamic integration) and a tracking algorithm:
  1. an algorithm that adjusts the LiDAR frame integration time, or frequency, dynamically based on the UAV speed and distance from the sensor. We integrate consecutive scans in a sliding window manner to retain the most recent information about the state of the UAV.
  2. a novel dual tracking approach using a Kalman filter variant that combines the two scan integration frequencies into one single state estimation using inverse covariance intersection. We evaluate the tracking performance during the experiments with ground truth data generated by a motion capture system.
Moreover, we provide a method to detect the initial position of the UAV using the object detection algorithm YOLOV5 over range images generated from a solid-state LiDAR point cloud.


Initialization

While the presented outcomes demonstrate the feasibility of our proposed approach, it is worth noting that the quantitative results rely on the assumption of the initial position being already known. To address this issue, we have developed a method to detect the target's starting location in outdoor scenarios. Specifically, we employed the same custom YOLOv5 model trained on panoramic signal images generated by an Ouster LiDAR

To minimize noise and artifacts when creating a single image, we first integrate a total of 30 frames. Later, the 3D point cloud is projected onto a 2D plane, taking into account both the field of view and image resolution. The transformation process considers both the intensity and distance of each point, combining them through a weighted sum operation to produce the final result. Using only intensity or distance to detect drones from the background becomes challenging for the YOLO model due to the proximity to ground and walls as well as the reflectivity of both the background and the drone being similar.

Furthermore, normalization is applied to ensure appropriate contrast in the resulting image. Upon obtaining the preliminary 2D image, its quality is enhanced through filtering and interpolation: we first identify areas with zero values and substitute them with constants to prevent visual discontinuities. There are two distinct cases that lead to zero-valued pixels after point cloud projection: the sky and other background regions where the emitted laser fails to reflect, and areas within the environment where objects might be present but the LiDAR does not scan. Differentiating between these two cases is crucial for the task of image completion, as it allows for an accurate understanding of the context surrounding the missing pixels.

To remove noise artifacts, we use binary thresholding and a nearest-neighbor interpolation to fill in missing or noisy regions, which results in smoother and more accurate images.


Dataset


Quantitative Results

Citation

@misc{catalano2023uav,
      title={UAV Tracking with Solid-State Lidars:Dynamic Multi-Frequency Scan Integration}, 
      author={Iacopo Catalano and Ha Sier and Xianjia Yu and Jorge Pena Queralta and Tomi Westerlund},
      year={2023},
      eprint={2304.12125},
      archivePrefix={arXiv},
      primaryClass={cs.RO}
      }