Leveraging Privileged Information to Limit Distraction in End-to-End Lane Following

Convolutional Neural Networks have been successfully used to steer vehicles using only road-facing cameras. In this work, we investigate the use of Privileged Information for training an end-to-end lane following model. Starting from the prior assumption that such a model should spend a sizeable fraction of its focus on lane markings, we take advantage of lane geometry information available at training time to improve its performance. To this end, we constrain the class of learnable functions by imposing a prior stemming from a lane segmentation task. For each input frame, we compute the set of pixels that most contribute to the prediction of the model using the VisualBackProp method. These pixel-relevance heatmaps are then compared with ground truth lane segmentation masks. A Distraction Loss term is added to the objective to regularize the training process. We learn from real-world data collected using our experimental vehicle and compare the results to those obtained using the simple Mean Squared Error objective. We show that the presence of our regularizer benefits both the performance and the stability of the model across a variety of evaluation metrics. We use a pretrained lane segmentation model without fine-tuning to extract lane marking masks and show that valuable learning signals can be extracted even from imperfect privileged knowledge. This method can be implemented easily and very efficiently on top of an existing architecture, without requiring the addition of any trainable parameter.