Leveraging Privileged Information to Limit Distraction in End-to-End Lane Following

in 2020 IEEE 17th Annual Consumer Communications & Networking Conference (CCNC) (2020)

Convolutional Neural Networks have been successfully used to steer vehicles using only road-facing cameras. In this work, we investigate the use of Privileged Information for training an end-to-end lane ... [more ▼]

Convolutional Neural Networks have been successfully used to steer vehicles using only road-facing cameras. In this work, we investigate the use of Privileged Information for training an end-to-end lane following model. Starting from the prior assumption that such a model should spend a sizeable fraction of its focus on lane markings, we take advantage of lane geometry information available at training time to improve its performance. To this end, we constrain the class of learnable functions by imposing a prior stemming from a lane segmentation task. For each input frame, we compute the set of pixels that most contribute to the prediction of the model using the VisualBackProp method. These pixel-relevance heatmaps are then compared with ground truth lane segmentation masks. A Distraction Loss term is added to the objective to regularize the training process. We learn from real-world data collected using our experimental vehicle and compare the results to those obtained using the simple Mean Squared Error objective. We show that the presence of our regularizer benefits both the performance and the stability of the model across a variety of evaluation metrics. We use a pretrained lane segmentation model without fine-tuning to extract lane marking masks and show that valuable learning signals can be extracted even from imperfect privileged knowledge. This method can be implemented easily and very efficiently on top of an existing architecture, without requiring the addition of any trainable parameter. [less ▲]

Evaluation of End-To-End Learning for Autonomous Driving: The Good, the Bad and the Ugly

in 2nd International Conference on Intelligent Autonomous Systems, Singapore, Feb. 28 to Mar. 2, 2019 (2019, March 01)

Visualizing the Learning Progress of Self-Driving Cars

in 21st International Conference on Intelligent Transportation Systems (2018, November 02)

Using Deep Learning to predict lateral and longitudinal vehicle control, i.e. steering, acceleration and braking, is becoming increasingly popular. However, it remains widely unknown why those models ... [more ▼]

Using Deep Learning to predict lateral and longitudinal vehicle control, i.e. steering, acceleration and braking, is becoming increasingly popular. However, it remains widely unknown why those models perform so well. In order for them to become a commercially viable solution, it first needs to be understood why a certain behavior is triggered and how and what those networks learn from human-generated driving data to ensure safety. One research direction is to visualize what the network sees by highlighting regions of an image that influence the outcome of the model. In this vein, we propose a generic visualization method using Attention Heatmaps (AHs) to highlight what a given Convolutional Neural Network (CNN) learns over time. To do so, we rely on a novel occlusion technique to mask different regions of an input image to observe the effect on a predicted steering signal. We then gradually increase the amount of training data and study the effect on the resulting Attention Heatmaps, both in terms of visual focus and temporal behavior. [less ▲]

Distributed C++-Python embedding for fast predictions and fast prototyping

in Proceedings of the Second Workshop on Distributed Infrastructures for Deep Learning (2018)

Python has evolved to become the most popular language for data science. It sports state-of-the-art libraries for analytics and machine learning, like Sci-Kit Learn. However, Python lacks the ... [more ▼]

Python has evolved to become the most popular language for data science. It sports state-of-the-art libraries for analytics and machine learning, like Sci-Kit Learn. However, Python lacks the computational performance that a industrial system requires for high frequency real time predictions. Building upon a year long research project heavily based on SciKit Learn (sklearn), we faced performance issues in deploying to production. Replacing sklearn with a better performing framework would require re-evaluating and tuning hyperparameters from scratch. Instead we developed a python embedding in a C++ based server application that increased performance by up to 20x, achieving linear scalability up to a point of convergence. Our implementation was done for mainstream cost effective hardware, which means we observed similar performance gains on small as well as large systems, from a laptop to an Amazon EC2 instance to a high-end server. [less ▲]

Pandia: comprehensive contention-sensitive thread placement

in Proceedings of the Twelfth European Conference on Computer Systems (2017)