Metaheuristics-based Exploration Strategies for Multi-Objective Reinforcement Learning

in Proceedings of the 14th International Conference on Agents and Artificial Intelligence (2022)

The fields of Reinforcement Learning (RL) and Optimization aim at finding an optimal solution to a problem, characterized by an objective function. The exploration-exploitation dilemma (EED) is a well ... [more ▼]

The fields of Reinforcement Learning (RL) and Optimization aim at finding an optimal solution to a problem, characterized by an objective function. The exploration-exploitation dilemma (EED) is a well known subject in those fields. Indeed, a consequent amount of literature has already been proposed on the subject and shown it is a non-negligible topic to consider to achieve good performances. Yet, many problems in real life involve the optimization of multiple objectives. Multi-Policy Multi-Objective Reinforcement Learning (MPMORL) offers a way to learn various optimised behaviours for the agent in such problems. This work introduces a modular framework for the learning phase of such algorithms, allowing to ease the study of the EED in Inner- Loop MPMORL algorithms. We present three new exploration strategies inspired from the metaheuristics domain. To assess the performance of our methods on various environments, we use a classical benchmark - the Deep Sea Treasure (DST) - as well as propose a harder version of it. Our experiments show all of the proposed strategies outperform the current state-of-the-art ε-greedy based methods on the studied benchmarks. [less ▲]

Preface

in Böhmer, Matthias; Steffgen, Georges (Eds.) Grief in schools - Basic knowledge and advice on dealing with dying and death (2022)

Changing Routinized Household Energy Consumption Using the Example of Washing, Cooking, and Standby: A Randomized Controlled Field Experiment of Home Energy Advice

in Cleaner and Responsible Consumption (2022)

Despite advances in understanding routines, there is little knowledge about which aspects of routinized behavior people adjust during interventions. In this study, we applied an adjusted social practice ... [more ▼]

Despite advances in understanding routines, there is little knowledge about which aspects of routinized behavior people adjust during interventions. In this study, we applied an adjusted social practice theory framework to disentangle routinized energy consumption, focusing on energy services related to washing, standby, and cooking. We investigate the potential of home energy advice to change elements of routinized behaviors, namely meanings, knowledge, and technologies. Using a randomized controlled field trial on a probabilistic sample of households, we found short-term treatment effects related to increased usage of lids during cooking and improved knowledge of IT-related energy consumption, as well as negative effects regarding multi-sockets and washing frequency. Our findings suggest that meanings (e.g., preferences underlying routinized behaviors) are less subject to change, and that sociodemographic variables are associated with routinized behaviors in complex ways. Our disentangling of energy demand into elements of routines enables us to show how home energy advice may change behaviors and knowledge. This study highlights the benefits of a multifaceted perspective for understanding household energy consumption and can be used to inform intervention and policy design. [less ▲]

An Empirical Study on Data Distribution-Aware Test Selection for Deep Learning Enhancement

in ACM Transactions on Software Engineering and Methodology (2022)

Similar to traditional software that is constantly under evolution, deep neural networks (DNNs) need to evolve upon the rapid growth of test data for continuous enhancement, e.g., adapting to distribution ... [more ▼]

Similar to traditional software that is constantly under evolution, deep neural networks (DNNs) need to evolve upon the rapid growth of test data for continuous enhancement, e.g., adapting to distribution shift in a new environment for deployment. However, it is labor-intensive to manually label all the collected test data. Test selection solves this problem by strategically choosing a small set to label. Via retraining with the selected set, DNNs will achieve competitive accuracy. Unfortunately, existing selection metrics involve three main limitations: 1) using different retraining processes; 2) ignoring data distribution shifts; 3) being insufficiently evaluated. To fill this gap, we first conduct a systemically empirical study to reveal the impact of the retraining process and data distribution on model enhancement. Then based on our findings, we propose a novel distribution-aware test (DAT) selection metric. Experimental results reveal that retraining using both the training and selected data outperforms using only the selected data. None of the selection metrics perform the best under various data distributions. By contrast, DAT effectively alleviates the impact of distribution shifts and outperforms the compared metrics by up to 5 times and 30.09% accuracy improvement for model enhancement on simulated and in-the-wild distribution shift scenarios, respectively. [less ▲]

Midbrain organoids mimic early embryonic neurodevelopment and recapitulate LRRK2-p.Gly2019Ser-associated gene expression

in American Journal of Human Genetics (2022)

Neurodegeneration and neuroinflammation are linked, but independent of a-synuclein inclusions, in a seeding/spreading mouse model of Parkinson's disease

in Glia (2022)

A key pathological process in Parkinson's disease (PD) is the transneuronal spreading of α-synuclein. Alpha-synuclein (α-syn) is a presynaptic protein that, in PD, forms pathological inclusions. Other ... [more ▼]

A key pathological process in Parkinson's disease (PD) is the transneuronal spreading of α-synuclein. Alpha-synuclein (α-syn) is a presynaptic protein that, in PD, forms pathological inclusions. Other hallmarks of PD include neurodegeneration and microgliosis in susceptible brain regions. Whether it is primarily transneuronal spreading of α-syn particles, inclusion formation, or other mechanisms, such as inflammation, that cause neurodegeneration in PD is unclear. We used a model of spreading of α-syn induced by striatal injection of α-syn preformed fibrils into the mouse striatum to address this question. We performed quantitative analysis for α-syn inclusions, neurodegeneration, and microgliosis in different brain regions, and generated gene expression profiles of the ventral midbrain, at two different timepoints after disease induction. We observed significant neurodegeneration and microgliosis in brain regions not only with, but also without α-syn inclusions. We also observed prominent microgliosis in injured brain regions that did not correlate with neurodegeneration nor with inclusion load. Using longitudinal gene expression profiling, we observed early gene expression changes, linked to neuroinflammation, that preceded neurodegeneration, indicating an active role of microglia in this process. Altered gene pathways overlapped with those typical of PD. Our observations indicate that α-syn inclusion formation is not the major driver in the early phases of PD-like neurodegeneration, but that microglia, activated by diffusible, oligomeric α-syn, may play a key role in this process. Our findings uncover new features of α-syn induced pathologies, in particular microgliosis, and point to the necessity for a broader view of the process of α-syn spreading. [less ▲]

Mechanistically Coupled PK (MCPK) Model to Describe Enzyme Induction and Occupancy Dependent DDI of Dabrafenib Metabolism.

in Pharmaceutics (2022), 14(2),

Dabrafenib inhibits the cell proliferation of metastatic melanoma with the oncogenic BRAF(V600)-mutation. However, dabrafenib monotherapy is associated with pERK reactivation, drug resistance, and ... [more ▼]

Dabrafenib inhibits the cell proliferation of metastatic melanoma with the oncogenic BRAF(V600)-mutation. However, dabrafenib monotherapy is associated with pERK reactivation, drug resistance, and consequential relapse. A clinical drug-dose determination study shows increased pERK levels upon daily administration of more than 300 mg dabrafenib. To clarify whether such elevated drug concentrations could be reached by long-term drug accumulation, we mechanistically coupled the pharmacokinetics (MCPK) of dabrafenib and its metabolites. The MCPK model is qualitatively based on in vitro and quantitatively on clinical data to describe occupancy-dependent CYP3A4 enzyme induction, accumulation, and drug-drug interaction mechanisms. The prediction suggests an eight-fold increase in the steady-state concentration of potent desmethyl-dabrafenib and its inactive precursor carboxy-dabrafenib within four weeks upon 150 mg b.d. dabrafenib. While it is generally assumed that a higher dose is not critical, we found experimentally that a high physiological dabrafenib concentration fails to induce cell death in embedded 451LU melanoma spheroids. [less ▲]

A topological Paley-Wiener-Schwartz Theorem for sections of homogeneous vector bundles on G/K

E-print/Working paper (2022)

We study the Fourier transform for compactly supported distributional sections of complex homogeneous vector bundles on symmetric spaces of non-compact type X = G/K. We prove a characterisation of their ... [more ▼]

We study the Fourier transform for compactly supported distributional sections of complex homogeneous vector bundles on symmetric spaces of non-compact type X = G/K. We prove a characterisation of their range. In fact, from Delorme’s Paley-Wiener theorem for compactly supported smooth functions on a real reductive group of Harish-Chandra class, we deduce topological Paley-Wiener and Paley-Wiener- Schwartz theorems for sections. [less ▲]

Policy failure revisited: The post-political urban development strategies of large digital corporations

Presentation (2022)

Robust and Deployable Gesture Recognition for Smartwatches

in Proceedings of the ACM Conference on Intelligent User Interfaces (IUI) (2022)