The Impact Of Distributed Data Preprocessing On Automotive Data Streams

in 2022 IEEE 96th Vehicular Technology Conference: (VTC2022-Fall) (2022, September)

Vehicles have transformed into sophisticated com- puting machines that not only serve the objective of transporta- tion from point A to point B but serve other objectives including improved experience ... [more ▼]

Vehicles have transformed into sophisticated com- puting machines that not only serve the objective of transporta- tion from point A to point B but serve other objectives including improved experience, safer journey, automated and more efficient and sustainable transportation. With such sophistication comes complex applications and enormous volumes of data generated from diverse types of vehicle sensors and components. Automotive data is not sedentary but moves from the edge (the vehicle) to the cloud (e.g., infrastructure of the vehicle manufacturers, national highway agencies, insurance companies, etc.). The exponential increase in data volume and variety generated in modern vehicles far exceeds the rate of infrastructure scaling and expansion. To mitigate this challenge, the computational and storage capacities of vehicle components can be leveraged to perform in-vehicle operations on the data to either prepare and transform (prepro- cess) the data or extract information from (process) the data. This paper focuses on distributing data preprocessing to the vehicle and highlights the benefits and impact of the distribution including on the consumption of resources (e.g., energy). [less ▲]

Synchronized Preprocessing of Sensor Data

in 2020 IEEE International Conference on Big Data (2021, March 19)

Sensor data whether collected for machine learning, deep learning or other applications must be preprocessed to fit input requirements or improve performance and accuracy. Data preparation is an expensive ... [more ▼]

Sensor data whether collected for machine learning, deep learning or other applications must be preprocessed to fit input requirements or improve performance and accuracy. Data preparation is an expensive, resource consuming and complex phase often performed centrally on raw data for a specific application. The dataflow between the edge and the cloud can be enhanced in terms of efficiency, reliability and lineage by preprocessing the datasets closer to their data sources. We propose a dedicated data preprocessing framework that distributes preprocessing tasks between a cloud stage and two edge stages to create a dataflow with progressively improving quality. The framework handles heterogenous data and dynamic preprocessing plans simultaneously targeting diverse applications and use cases from different domains. Each stage autonomously executes sensor specific preprocessing plans in parallel while synchronizing the progressive execution and dynamic updates of the preprocessing plans with the other stages. Our approach minimizes the workload on central infrastructures and reduces the resources used for transferring raw data from the edge. We also demonstrate that preprocessing data can be sensor specific rather than application specific and thus can be performed prior to knowing a specific application. [less ▲]

Big Automotive Data Preprocessing: A Three Stages Approach

Poster (2019, October 08)

The automotive industry generates large datasets of various formats, uncertainties and frequencies. To exploit Automotive Big Data, the data needs to be connected, fused and preprocessed to quality ... [more ▼]

The automotive industry generates large datasets of various formats, uncertainties and frequencies. To exploit Automotive Big Data, the data needs to be connected, fused and preprocessed to quality datasets before being used for production and business processes. Data preprocessing tasks are typically expensive, tightly coupled with their intended AI algorithms and are done manually by domain experts. Hence there is a need to automate data preprocessing to seamlessly generate cleaner data. We intend to introduce a generic data preprocessing framework that handles vehicle-to-everything (V2X) data streams and dynamic updates. We intend to decentralize and automate data preprocessing by leveraging edge computing with the objective of progressively improving the quality of the dataflow within edge components (vehicles) and onto the cloud. [less ▲]