Towards Privacy Preserving Data Centric Super App

in Carvalho Ota, Fernando Kaway; Meira, Jorge Augusto; Frank, Raphaël (Eds.) et al 2020 Mediterranean Communication and Computer Networking Conference, Arona 17-19 June 2020 (2020, September 10)

The number of smartphone users recently surpassed the numbers of desktop users on Internet, and opened up countless development challenges and business opportunities. Not only the fact that the majority ... [more ▼]

The number of smartphone users recently surpassed the numbers of desktop users on Internet, and opened up countless development challenges and business opportunities. Not only the fact that the majority of users are connected using their smartphones, but the number of Internet users in general has popularized the massive use of data-driven applications. In this context, the concept of super apps seems to be the next game-changer for the mobile apps industry, and the challenges related to security and privacy are key aspects for keeping user data safe. Thus, by combining different components for provisioning, authentication, membership and others, we propose a novel framework that enables the creation of a super app using privacy by design principles. [less ▲]

Database Processing-in-Memory: A Vision

in Database Processing-in-Memory: A Vision (2019, August 03)

Machine Learning for Data-Driven Smart Grid Applications

Scientific Conference (2018)

The field of Machine Learning grew out of the quest for artificial intelligence. It gives computers the ability to learn statistical patterns from data without being explicitly programmed. These patterns ... [more ▼]

The field of Machine Learning grew out of the quest for artificial intelligence. It gives computers the ability to learn statistical patterns from data without being explicitly programmed. These patterns can then be applied to new data in order to make predictions. Machine Learning also allows to automatically adapt to changes in the data without amending the underlying model. We deal every day dozens of times with Machine Learning applications such as when doing a Google search, using spam filters, face detection, speaking to voice recognition software or when sitting in a self-driving car. In recent years, machine learning methods have evolved in the smart grid community. This change towards analyzing data rather than modeling specific problems has lead to adaptable, more generic methods, that require less expert knowledge and that are easier to deploy in a number of use cases. This is an introductory level course to discuss what machine learning is and how to apply it to data-driven smart grid applications. Practical case studies on real data sets, such as load forecasting, detection of irregular power usage and visualization of customer data, will be included. Therefore, attendees will not only understand, but rather experience, how to apply machine learning methods to smart grid data. [less ▲]

An Elastic Multi-Core Allocation Mechanism for Database Systems

in International Conference on Data Engineering (ICDE) (2018)

During the parallel execution of queries in Non-Uniform Memory Access (NUMA) systems, he Operating System (OS) maps the threads (or processes) from modern database systems to the available cores among the ... [more ▼]

During the parallel execution of queries in Non-Uniform Memory Access (NUMA) systems, he Operating System (OS) maps the threads (or processes) from modern database systems to the available cores among the NUMA nodes using the standard node-local policy. However, such non-smart mapping may result in inefficient memory activity, because shared data may be accessed by scattered threads requiring large data movements or non-shared data may be allocated to threads sharing the same cache memory, increasing its conflicts. In this paper we present a data-distribution aware and elastic multi-core allocation mechanism to improve the OS mapping of database threads in NUMA systems. Our hypothesis is that we mitigate the data movement if we only hand out to the OS the local optimum number of cores in specific nodes. We propose a mechanism based on a rule-condition-action pipeline that uses hardware counters to promptly find out the local optimum number of cores. Our mechanism uses a priority queue to track the history of the memory address space used by database threads in order to decide about the allocation/release of cores and its distribution among the NUMA nodes to decrease remote memory access. We implemented and tested a prototype of our mechanism when executing two popular Volcano-style databases improving their NUMA-affinity. For MonetDB, we show maximum speedup of 1.53 × , due to consistent reduction in the local/remote per-query data traffic ratio of up to 3.87 × running 256 concurrent clients in the 1 GB TPC-H database also showing system energy savings of 26.05%. For the NUMA-aware SQL Server, we observed speedup of up to 1.27 × and reduction on the data traffic ratio of 3.70 ×. [less ▲]

Is Big Data Sufficient for a Reliable Detection of Non-Technical Losses?

in Proceedings of the 19th International Conference on Intelligent System Applications to Power Systems (ISAP 2017) (2017, September)

Non-technical losses (NTL) occur during the distribution of electricity in power grids and include, but are not limited to, electricity theft and faulty meters. In emerging countries, they may range up to ... [more ▼]

Non-technical losses (NTL) occur during the distribution of electricity in power grids and include, but are not limited to, electricity theft and faulty meters. In emerging countries, they may range up to 40% of the total electricity distributed. In order to detect NTLs, machine learning methods are used that learn irregular consumption patterns from customer data and inspection results. The Big Data paradigm followed in modern machine learning reflects the desire of deriving better conclusions from simply analyzing more data, without the necessity of looking at theory and models. However, the sample of inspected customers may be biased, i.e. it does not represent the population of all customers. As a consequence, machine learning models trained on these inspection results are biased as well and therefore lead to unreliable predictions of whether customers cause NTL or not. In machine learning, this issue is called covariate shift and has not been addressed in the literature on NTL detection yet. In this work, we present a novel framework for quantifying and visualizing covariate shift. We apply it to a commercial data set from Brazil that consists of 3.6M customers and 820K inspection results. We show that some features have a stronger covariate shift than others, making predictions less reliable. In particular, previous inspections were focused on certain neighborhoods or customer classes and that they were not sufficiently spread among the population of customers. This framework is about to be deployed in a commercial product for NTL detection. [less ▲]

Distilling Provider-Independent Data for General Detection of Non-Technical Losses

in Power and Energy Conference, Illinois 23-24 February 2017 (2017)

Non-technical losses (NTL) in electricity distribution are caused by different reasons, such as poor equipment maintenance, broken meters or electricity theft. NTL occurs especially but not exclusively in ... [more ▼]

Non-technical losses (NTL) in electricity distribution are caused by different reasons, such as poor equipment maintenance, broken meters or electricity theft. NTL occurs especially but not exclusively in emerging countries. Developed countries, even though usually in smaller amounts, have to deal with NTL issues as well. In these countries the estimated annual losses are up to six billion USD. These facts have directed the focus of our work to the NTL detection. Our approach is composed of two steps: 1) We compute several features and combine them in sets characterized by four criteria: temporal, locality, similarity and infrastructure. 2) We then use the sets of features to train three machine learning classifiers: random forest, logistic regression and support vector vachine. Our hypothesis is that features derived only from provider-independent data are adequate for an accurate detection of non-technical losses. [less ▲]

The Challenge of Non-Technical Loss Detection using Artificial Intelligence: A Survey

in International Journal of Computational Intelligence Systems (2017), 10(1), 760-775

Detection of non-technical losses (NTL) which include electricity theft, faulty meters or billing errors has attracted increasing attention from researchers in electrical engineering and computer science ... [more ▼]

Detection of non-technical losses (NTL) which include electricity theft, faulty meters or billing errors has attracted increasing attention from researchers in electrical engineering and computer science. NTLs cause significant harm to the economy, as in some countries they may range up to 40% of the total electricity distributed. The predominant research direction is employing artificial intelligence to predict whether a customer causes NTL. This paper first provides an overview of how NTLs are defined and their impact on economies, which include loss of revenue and profit of electricity providers and decrease of the stability and reliability of electrical power grids. It then surveys the state-of-the-art research efforts in a up-to-date and comprehensive review of algorithms, features and data sets used. It finally identifies the key scientific and engineering challenges in NTL detection and suggests how they could be addressed in the future. [less ▲]

Neighborhood Features Help Detecting Non-Technical Losses in Big Data Sets

in Proceedings of the 3rd IEEE/ACM International Conference on Big Data Computing Applications and Technologies (BDCAT 2016) (2016)

Electricity theft occurs around the world in both developed and developing countries and may range up to 40% of the total electricity distributed. More generally, electricity theft belongs to non ... [more ▼]

Electricity theft occurs around the world in both developed and developing countries and may range up to 40% of the total electricity distributed. More generally, electricity theft belongs to non-technical losses (NTL), which occur during the distribution of electricity in power grids. In this paper, we build features from the neighborhood of customers. We first split the area in which the customers are located into grids of different sizes. For each grid cell we then compute the proportion of inspected customers and the proportion of NTL found among the inspected customers. We then analyze the distributions of features generated and show why they are useful to predict NTL. In addition, we compute features from the consumption time series of customers. We also use master data features of customers, such as their customer class and voltage of their connection. We compute these features for a Big Data base of 31M meter readings, 700K customers and 400K inspection results. We then use these features to train four machine learning algorithms that are particularly suitable for Big Data sets because of their parallelizable structure: logistic regression, k-nearest neighbors, linear support vector machine and random forest. Using the neighborhood features instead of only analyzing the time series has resulted in appreciable results for Big Data sets for varying NTL proportions of 1%-90%. This work can therefore be deployed to a wide range of different regions. [less ▲]

Generic Cloud Platform Multi-objective Optimization Leveraging Models@run.time

Scientific Conference (2014, March)

MODEL-BASED STRESS TESTING FOR DATABASE SYSTEMS

Doctoral thesis (2014)

Database Management Systems (DBMS) have been successful at processing transaction workloads over decades. But contemporary systems, including Cloud computing, Internet-based systems, and sensors (i.e ... [more ▼]

Database Management Systems (DBMS) have been successful at processing transaction workloads over decades. But contemporary systems, including Cloud computing, Internet-based systems, and sensors (i.e., Internet of Things (IoT)), are challenging the architecture of the DBMS with burgeoning transaction workloads. The direct consequence is that the development agenda of the DBMS is now heavily concerned about meeting non-functional requirements, such as performance, robustness and scalability. Otherwise, any stressing workload will make the DBMS lose control of simple functional requirements, such as responding to a transaction request~\cite{stem}. While traditional DBMS, including DB2, Oracle, and PostgreSQL, require embedding new features to meet non-functional requirements, the contemporary DBMS called as NewSQL present a completely new architecture. What is still lacking in the development agenda is a proper testing approach coupled with burgeoning transaction workloads for validating the DBMS with non-functional requirements in mind. The typical non-functional validation is carried out by performance benchmarks. However, they focus on metrics comparison instead of finding defects. In this thesis, we address this lack by presenting different contributions for the domain of DBMS stress testing. These contributions fit different testing objectives to challenge each specific architecture of traditional and contemporary DBMS. For instance, testing the earlier DBMS (e.g., DB2, Oracle) require incremental performance tuning (i.e., from simple setup to complex one), while testing the latter DBMS (e.g., VoltDB, NuoDB) require driving it into different performance states due to its self-tuning capabilities. Overall, this thesis makes the following contributions: 1) Stress TEsting Methodology (STEM): A methodology to capture performance degradation and expose system defects in the internal code due to the combination of a stress workload and mistuning; 2) Model-based Database Stress Testing (MoDaST): An approach to test NewSQL database systems. Supported by a Database State Machine (DSM), MoDaST infers internal states of the database based on performance observations under different workload levels; 3) Under Pressure Benchmark (UPB): A benchmark to assess the impact of availability mechanisms in NewSQL database systems. We validate our contributions with several popular DBMS. Among the outcomes, we highlight that our methodologies succeed in driving the DBMS up to stress state conditions and expose several related defects, including a new major defect in a popular NewSQL. [less ▲]