Semantic-Aware Video Compression for Automotive Cameras

Assisted and automated driving functions in vehicles exploit sensor data to build situational awareness, however, the data amount required by these functions might exceed the bandwidth of current wired vehicle communication networks. Consequently, sensor data reduction, and automotive camera video compression need investigation. However, conventional video compression schemes, such as H.264 and H.265, have been mainly optimised for human vision. In this paper, the authors propose a semantic-aware (SA) video compression (SAC) framework that compresses separately and simultaneously region-of-interest and region-out-of-interest of automotive camera video frames, before transmitting them to processing unit(s), where the data are used for perception tasks, such as object detection, semantic segmentation, etc. Using the authors' newly proposed technique, the region-of-interest (ROI), encapsulating most of the road stakeholders, retains higher quality using lower compression ratio. The experimental results show that under the same overall compression ratio, the authors' proposed SAC scheme maintains a similar or better image quality, measured accordingly to traditional metrics and to the authors' newly proposed semantic-aware metrics. The newly proposed metrics, namely SA-PSNR, SA-SSIM, and iIoU, give more emphasis to ROI quality, which has an immediate impact on the planning and decisions of assisted and automated driving functions. Using the authors' SA-X264 compression, SA-PSNR and SA-SSIM have an increase of 2.864 and 0.008 respectively compared to traditional H.264, with higher ROI quality and the same compression ratio. Finally, a segmentation-based perception algorithm has been used to compare reconstructed frames, demonstrating a 2.7% mIOU improvement, when using the proposed SAC method versus traditional compression techniques.

• Record URL:
  • http://dx.doi.org/10.1109/TIV.2023.3267443
• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/issn/23798858
• Supplemental Notes:
  • Copyright © 2023, IEEE.
• Authors:
  • Wang, Yiting
  • Chan, Pak Hung
  • Donzella, Valentina
• Publication Date: 2023-6

Language

• English

Media Info

• Media Type: Web
• Features: Figures; References;
• Pagination: pp 3712-3722
• Serial:
  • IEEE Transactions on Intelligent Vehicles
  • Volume: 8
  • Issue Number: 6
  • Publisher: Institute of Electrical and Electronics Engineers (IEEE)
  • ISSN: 2379-8858
  • Serial URL: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=7274857

Subject/Index Terms

• TRT Terms: Autonomous vehicles; Cameras; Driver support systems; Image coding; In vehicle sensors; Machine learning
• Subject Areas: Data and Information Technology; Highways; Vehicles and Equipment;

Filing Info

• Accession Number: 01909386
• Record Type: Publication
• Files: TRIS
• Created Date: Feb 22 2024 4:14PM