Biography

I am pursuing my Ph.D. degree in College of Control Science and Engineering, Zhejiang University, Hangzhou, China. My major research interests include extrinsic calibration between LiDAR, IMU and camera.

Research and Interests

• Sensor Calibration
• Sensor Fusion

Publications

• Jiajun Lv, Xingxing Zuo, Kewei Hu, Jinhong Xu, Guoquan Huang, and Yong Liu. Observability-Aware Intrinsic and Extrinsic Calibration of LiDAR-IMU System. IEEE Transactions on Robotics, 38(6):3734-3753, 2022.
  [BibTeX] [Abstract] [DOI] [PDF]

  Accurate and reliable sensor calibration is essential to fuse LiDAR and inertial measurements, which are usually available in robotic applications. In this article, we propose a novel LiDAR-IMU calibration method within the continuous-time batch-optimization framework, where the intrinsics of both sensors and the spatial-temporal extrinsics between sensors are calibrated without using calibration infrastructure, such as fiducial tags. Compared to discrete-time approaches, the continuous-time formulation has natural advantages for fusing high-rate measurements from LiDAR and IMU sensors. To improve efficiency and address degenerate motions, the following two observability-aware modules are leveraged: first, The information-theoretic data selection policy selects only the most informative segments for calibration during data collection, which significantly improves the calibration efficiency by processing only the selected informative segments. Second, the observability-aware state update mechanism in nonlinear least-squares optimization updates only the identifiable directions in the state space with truncated singular value decomposition, which enables accurate calibration results even under degenerate cases where informative data segments are not available. The proposed LiDAR-IMU calibration approach has been validated extensively in both simulated and real-world experiments with different robot platforms, demonstrating its high accuracy and repeatability in commonly-seen human-made environments.

  @article{lv2022oai,
  title = {Observability-Aware Intrinsic and Extrinsic Calibration of LiDAR-IMU System},
  author = {Jiajun Lv and Xingxing Zuo and Kewei Hu and Jinhong Xu and Guoquan Huang and Yong Liu},
  year = 2022,
  journal = {IEEE Transactions on Robotics},
  volume = {38},
  number = {6},
  pages = {3734-3753},
  doi = {10.1109/TRO.2022.3174476},
  abstract = {Accurate and reliable sensor calibration is essential to fuse LiDAR and inertial measurements, which are usually available in robotic applications. In this article, we propose a novel LiDAR-IMU calibration method within the continuous-time batch-optimization framework, where the intrinsics of both sensors and the spatial-temporal extrinsics between sensors are calibrated without using calibration infrastructure, such as fiducial tags. Compared to discrete-time approaches, the continuous-time formulation has natural advantages for fusing high-rate measurements from LiDAR and IMU sensors. To improve efficiency and address degenerate motions, the following two observability-aware modules are leveraged: first, The information-theoretic data selection policy selects only the most informative segments for calibration during data collection, which significantly improves the calibration efficiency by processing only the selected informative segments. Second, the observability-aware state update mechanism in nonlinear least-squares optimization updates only the identifiable directions in the state space with truncated singular value decomposition, which enables accurate calibration results even under degenerate cases where informative data segments are not available. The proposed LiDAR-IMU calibration approach has been validated extensively in both simulated and real-world experiments with different robot platforms, demonstrating its high accuracy and repeatability in commonly-seen human-made environments.}
  }

• Xiaolei Lang, Jiajun Lv, Jianxin Huang, Yukai Ma, Yong Liu, and Xingxing Zuo. Ctrl-VIO: Continuous-Time Visual-Inertial Odometry for Rolling Shutter Cameras. IEEE Robotics and Automation Letters (RA-L), 7(4):11537-11544, 2022.
  [BibTeX] [Abstract] [DOI] [PDF]

  In this letter, we propose a probabilistic continuoustime visual-inertial odometry (VIO) for rolling shutter cameras. The continuous-time trajectory formulation naturally facilitates the fusion of asynchronized high-frequency IMU data and motion distorted rolling shutter images. To prevent intractable computation load, the proposed VIO is sliding-window and keyframe-based. We propose to probabilistically marginalize the control points to keep the constant number of keyframes in the sliding window. Furthermore, the line exposure time difference (line delay) of the rolling shutter camera can be online calibrated in our continuous-time VIO. To extensively examine the performance of our continuoustime VIO, experiments are conducted on publicly-available WHURSVI, TUM-RSVI, and SenseTime-RSVI rolling shutter datasets. The results demonstrate the proposed continuous-time VIO significantly outperforms the existing state-of-the-art VIO methods.

  @article{lang2022ctv,
  title = {Ctrl-VIO: Continuous-Time Visual-Inertial Odometry for Rolling Shutter Cameras},
  author = {Xiaolei Lang and Jiajun Lv and Jianxin Huang and Yukai Ma and Yong Liu and Xingxing Zuo},
  year = 2022,
  journal = {IEEE Robotics and Automation Letters (RA-L)},
  volume = {7},
  number = {4},
  pages = {11537-11544},
  doi = {10.1109/LRA.2022.3202349},
  abstract = {In this letter, we propose a probabilistic continuoustime visual-inertial odometry (VIO) for rolling shutter cameras. The continuous-time trajectory formulation naturally facilitates
  the fusion of asynchronized high-frequency IMU data and motion distorted rolling shutter images. To prevent intractable computation load, the proposed VIO is sliding-window and keyframe-based. We propose to probabilistically marginalize the control points to keep the constant number of keyframes in the sliding window. Furthermore, the line exposure time difference (line delay) of the rolling
  shutter camera can be online calibrated in our continuous-time VIO. To extensively examine the performance of our continuoustime VIO, experiments are conducted on publicly-available WHURSVI, TUM-RSVI, and SenseTime-RSVI rolling shutter datasets. The results demonstrate the proposed continuous-time VIO significantly outperforms the existing state-of-the-art VIO methods.}
  }

• Jiajun Lv, Kewei Hu, Jinhong Xu, Yong Liu, and Xingxing Zuo. CLINS: Continuous-Time Trajectory Estimation for LiDAR Inertial System. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 6657-6663, 2021.
  [BibTeX] [Abstract] [DOI] [PDF]

  In this paper, we propose a highly accurate continuous-time trajectory estimation framework dedicated to SLAM (Simultaneous Localization and Mapping) applications, which enables fuse high-frequency and asynchronous sensor data effectively. We apply the proposed framework in a 3D LiDAR-inertial system for evaluations. The proposed method adopts a non-rigid registration method for continuous-time trajectory estimation and simultaneously removing the motion distortion in LiDAR scans. Additionally, we propose a two-state continuous-time trajectory correction method to efficiently and efficiently tackle the computationally-intractable global optimization problem when loop closure happens. We examine the accuracy of the proposed approach on several publicly available datasets and the data we collected. The experimental results indicate that the proposed method outperforms the discrete-time methods regarding accuracy especially when aggressive motion occurs. Furthermore, we open source our code at https://github.com/APRIL-ZJU/clins to benefit research community.

  @inproceedings{lv2021clins,
  title = {CLINS: Continuous-Time Trajectory Estimation for LiDAR Inertial System},
  author = {Jiajun Lv and Kewei Hu and Jinhong Xu and Yong Liu and Xingxing Zuo},
  year = 2021,
  booktitle = {2021 IEEE/RSJ International Conference on Intelligent Robots and Systems},
  pages = {6657-6663},
  doi = {https://doi.org/10.1109/IROS51168.2021.9636676},
  abstract = {In this paper, we propose a highly accurate continuous-time trajectory estimation framework dedicated to SLAM (Simultaneous Localization and Mapping) applications, which enables fuse high-frequency and asynchronous sensor data effectively. We apply the proposed framework in a 3D LiDAR-inertial system for evaluations. The proposed method adopts a non-rigid registration method for continuous-time trajectory estimation and simultaneously removing the motion distortion in LiDAR scans. Additionally, we propose a two-state continuous-time trajectory correction method to efficiently and efficiently tackle the computationally-intractable global optimization problem when loop closure happens. We examine the accuracy of the proposed approach on several publicly available datasets and the data we collected. The experimental results indicate that the proposed method outperforms the discrete-time methods regarding accuracy especially when aggressive motion occurs. Furthermore, we open source our code at https://github.com/APRIL-ZJU/clins to benefit research community.}
  }

• Jiajun Lv, Jinhong Xu, Kewei Hu, Yong Liu, and Xingxing Zuo. Targetless Calibration of LiDAR-IMU System Based on Continuous-time Batch Estimation. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), page 9968–9975, 2020.
  [BibTeX] [Abstract] [DOI] [arXiv] [PDF]

  Sensor calibration is the fundamental block for a multi-sensor fusion system. This paper presents an accurate and repeatable LiDAR-IMU calibration method (termed LI-Calib), to calibrate the 6-DOF extrinsic transformation between the 3D LiDAR and the Inertial Measurement Unit (IMU). Regarding the high data capture rate for LiDAR and IMU sensors, LI-Calib adopts a continuous-time trajectory formulation based on B-Spline, which is more suitable for fusing high-rate or asynchronous measurements than discrete-time based approaches. Additionally, LI-Calib decomposes the space into cells and identifies the planar segments for data association, which renders the calibration problem well-constrained in usual scenarios without any artificial targets. We validate the proposed calibration approach on both simulated and real-world experiments. The results demonstrate the high accuracy and good repeatability of the proposed method in common human-made scenarios. To benefit the research community, we open-source our code at https://github.com/APRIL-ZJU/lidar_IMU_calib.

  @inproceedings{lv2020targetlessco,
  title = {Targetless Calibration of LiDAR-IMU System Based on Continuous-time Batch Estimation},
  author = {Jiajun Lv and Jinhong Xu and Kewei Hu and Yong Liu and Xingxing Zuo},
  year = 2020,
  booktitle = {2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  pages = {9968--9975},
  doi = {https://doi.org/10.1109/IROS45743.2020.9341405},
  abstract = {Sensor calibration is the fundamental block for a multi-sensor fusion system. This paper presents an accurate and repeatable LiDAR-IMU calibration method (termed LI-Calib), to calibrate the 6-DOF extrinsic transformation between the 3D LiDAR and the Inertial Measurement Unit (IMU). Regarding the high data capture rate for LiDAR and IMU sensors, LI-Calib adopts a continuous-time trajectory formulation based on B-Spline, which is more suitable for fusing high-rate or asynchronous measurements than discrete-time based approaches. Additionally, LI-Calib decomposes the space into cells and identifies the planar segments for data association, which renders the calibration problem well-constrained in usual scenarios without any artificial targets. We validate the proposed calibration approach on both simulated and real-world experiments. The results demonstrate the high accuracy and good repeatability of the proposed method in common human-made scenarios. To benefit the research community, we open-source our code at https://github.com/APRIL-ZJU/lidar_IMU_calib.},
  arxiv = {https://arxiv.org/pdf/2007.14759.pdf}
  }

• Xingxing Zuo, Yulin Yang, Patrick Geneva, Jiajun Lv, Yong Liu, Guoquan Huang, and Marc Pollefeys. LIC-Fusion 2.0: LiDAR-Inertial-Camera Odometry with Sliding-Window Plane-Feature Tracking. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), page 5112–5119, 2020.
  [BibTeX] [Abstract] [DOI] [arXiv] [PDF]

  Multi-sensor fusion of multi-modal measurements from commodity inertial, visual and LiDAR sensors to provide robust and accurate 6DOF pose estimation holds great potential in robotics and beyond. In this paper, building upon our prior work (i.e., LIC-Fusion), we develop a sliding-window filter based LiDAR-Inertial-Camera odometry with online spatiotemporal calibration (i.e., LIC-Fusion 2.0), which introduces a novel sliding-window plane-feature tracking for efficiently processing 3D LiDAR point clouds. In particular, after motion compensation for LiDAR points by leveraging IMU data, low-curvature planar points are extracted and tracked across the sliding window. A novel outlier rejection criteria is proposed in the plane-feature tracking for high quality data association. Only the tracked planar points belonging to the same plane will be used for plane initialization, which makes the plane extraction efficient and robust. Moreover, we perform the observability analysis for the IMU-LiDAR subsystem under consideration and report the degenerate cases for spatiotemporal calibration using plane features. While the estimation consistency and identified degenerate motions are validated in Monte-Carlo simulations, different real-world experiments are also conducted to show that the proposed LIC-Fusion 2.0 outperforms its predecessor and other state-of-the-art methods.

  @inproceedings{zuo2020licfusion2l,
  title = {LIC-Fusion 2.0: LiDAR-Inertial-Camera Odometry with Sliding-Window Plane-Feature Tracking},
  author = {Xingxing Zuo and Yulin Yang and Patrick Geneva and Jiajun Lv and Yong Liu and Guoquan Huang and Marc Pollefeys},
  year = 2020,
  booktitle = {2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  pages = {5112--5119},
  doi = {10.1109/IROS45743.2020.9340704},
  abstract = {Multi-sensor fusion of multi-modal measurements from commodity inertial, visual and LiDAR sensors to provide robust and accurate 6DOF pose estimation holds great potential in robotics and beyond. In this paper, building upon our prior work (i.e., LIC-Fusion), we develop a sliding-window filter based LiDAR-Inertial-Camera odometry with online spatiotemporal calibration (i.e., LIC-Fusion 2.0), which introduces a novel sliding-window plane-feature tracking for efficiently processing 3D LiDAR point clouds. In particular, after motion compensation for LiDAR points by leveraging IMU data, low-curvature planar points are extracted and tracked across the sliding window. A novel outlier rejection criteria is proposed in the plane-feature tracking for high quality data association. Only the tracked planar points belonging to the same plane will be used for plane initialization, which makes the plane extraction efficient and robust. Moreover, we perform the observability analysis for the IMU-LiDAR subsystem under consideration and report the degenerate cases for spatiotemporal calibration using plane features. While the estimation consistency and identified degenerate motions are validated in Monte-Carlo simulations, different real-world experiments are also conducted to show that the proposed LIC-Fusion 2.0 outperforms its predecessor and other state-of-the-art methods.},
  arxiv = {https://arxiv.org/pdf/2008.07196.pdf}
  }

• Jinhong Xu, Jiajun Lv, Zaishen Pan, Yong Liu, and Yinan Chen. Real-Time LiDAR Data Assocation Aided by IMU in High Dynamic Environment. In 2018 IEEE International Conference on Real-time Computing and Robotics (RCAR), page 202–205, 2018.
  [BibTeX] [Abstract] [DOI] [PDF]

  In recent years, with the breakthroughs in sensor technology, SLAM technology is developing towards high speed and high dynamic applications. The rotating multi line LiDAR sensor plays an important role. However, the rotating multi line LiDAR sensors need to restructure the data in high dynamic environment. Our work is to propose a LiDAR data correction method based on IMU and hardware synchronization, and make a hardware synchronization unit. This method can still output correct point cloud information when LiDAR sensor is moving violently.

  @inproceedings{xu2018realtimeld,
  title = {Real-Time LiDAR Data Assocation Aided by IMU in High Dynamic Environment},
  author = {Jinhong Xu and Jiajun Lv and Zaishen Pan and Yong Liu and Yinan Chen},
  year = 2018,
  booktitle = {2018 IEEE International Conference on Real-time Computing and Robotics (RCAR)},
  pages = {202--205},
  doi = {https://doi.org/10.1109/RCAR.2018.8621627},
  abstract = {In recent years, with the breakthroughs in sensor technology, SLAM technology is developing towards high speed and high dynamic applications. The rotating multi line LiDAR sensor plays an important role. However, the rotating multi line LiDAR sensors need to restructure the data in high dynamic environment. Our work is to propose a LiDAR data correction method based on IMU and hardware synchronization, and make a hardware synchronization unit. This method can still output correct point cloud information when LiDAR sensor is moving violently.}
  }