Real-time Detection and Avoidance of Obstacles in the Path of Autonomous Vehicles Using Monocular RGB Camera

In this paper, we present an end-to-end real-time detection and collision avoidance framework in an autonomous vehicle using a monocular RGB camera. The proposed system is able to run on embedded hardware in the vehicle to perform real-time detection of small objects. RetinaNet architecture with ResNet50 backbone is used to develop the object detection model using RGB images. A quantized version of the object detection inference model is implemented in the vehicle using NVIDIA Jetson AGX Xavier. A geometric method is used to estimate the distance to the detected object which is forwarded to a MicroAutoBox device that implements the control system of the vehicle and is responsible for maneuvering around the detected objects. The pipeline is implemented on a passenger vehicle and demonstrated in challenging conditions using different obstacles on a predefined set of waypoints. Our results show that the system is capable of detecting objects that appear in an image area as small as 20×30 pixels in a 1280×720 image and can run at a speed of 24 frames per second (FPS) on the embedded device in the vehicle. A data analyzer is also employed to visualize the real-time performance of the system.

  • Record URL:
  • Availability:
  • Supplemental Notes:
    • Abstract reprinted with permission of SAE International.
  • Authors:
    • Mallik, Apurbaa
    • Gaopande, Meghana Laxmidhar
    • Singh, Gurjeet
    • Ravindran, Aniruddh
    • Iqbal, Zafar
    • Chao, Steven
    • Revalla, Hitha
    • Nagasamy, Vijay
  • Conference:
  • Publication Date: 2022-3-29

Language

  • English

Media Info

  • Media Type: Web
  • Features: References;
  • Pagination: pp 622-632
  • Serial:

Subject/Index Terms

Filing Info

  • Accession Number: 01841518
  • Record Type: Publication
  • Source Agency: SAE International
  • Report/Paper Numbers: 2022-01-0074
  • Files: TRIS, SAE
  • Created Date: Apr 6 2022 2:18PM