Innovative saliency based deep driving scene understanding system for automatic safety assessment in next-generation cars

Visual saliency is the human attention mechanism that encodes such visio-sensing information to extract features from the observation scene. In the last few years, visual saliency estimation has received significant research interests in the automotive field. While driving the vehicle, the car driver focuses on specific objects rather than others by deterministic brain-driven saliency mechanisms inherent perceptual activity. In this study, we propose an intelligent system that combines a driver's drowsiness detector with a saliency-based scene understanding pipeline. Specifically, we implemented ad-hoc 3D pre-trained Semantic Segmentation Deep Network to process the frames captured by automotive-grade camera device placed outside the car. We used an embedded platform based on the STA1295 core (ARM A7 Dual-Cores) with a hardware accelerator for hosting the proposed pipeline. Besides, we monitor the car driver's drowsiness by using an innovative bio-sensor installed on the steering wheel, to collect the PhotoPlethysmoGraphy (PPG) signal. Ad-hoc 1D Temporal Deep Convolutional Network has been designed to classify the collected PPG time-series in order to assess the driver's attention level. Finally, we compare the detected car driver's attention level with corresponding saliency-based scene classification in order to assess the overall safety level. Experimental results confirm the effectiveness of the proposed pipeline.