[en] One of the promising opportunities of digital health is its potential to lead to more holistic understandings of diseases by interacting with the daily life of patients and through the collection of large amounts of real-world data. Validating and benchmarking indicators of disease severity in the home setting is difficult, however, given the large number of confounders present in the real world and the challenges in collecting ground truth data in the home. Here we leverage two datasets collected from patients with Parkinson's disease, which couples continuous wrist-worn accelerometer data with frequent symptom reports in the home setting, to develop digital biomarkers of symptom severity. Using these data, we performed a public benchmarking challenge in which participants were asked to build measures of severity across 3 symptoms (on/off medication, dyskinesia, and tremor). 42 teams participated and performance was improved over baseline models for each subchallenge. Additional ensemble modeling across submissions further improved performance, and the top models validated in a subset of patients whose symptoms were observed and rated by trained clinicians.
Research center :
- Luxembourg Centre for Systems Biomedicine (LCSB): Biomedical Data Science (Glaab Group)
Disciplines :
Life sciences: Multidisciplinary, general & others Human health sciences: Multidisciplinary, general & others Neurology Biotechnology
Author, co-author :
Sieberts, Solveig K.
Borzymowski, Henryk
Guan, Yuanfang
Huang, Yidi
Matzner, Ayala
Page, Alex
Bar-Gad, Izhar
Beaulieu-Jones, Brett
El-Hanani, Yuval
Goschenhofer, Jann
Javidnia, Monica
Keller, Mark S.
Li, Yan-Chak
Saqib, Mohammed
Smith, Greta
Stanescu, Ana
Venuto, Charles S.
Zielinski, Robert
GLAAB, Enrico ; University of Luxembourg > Luxembourg Centre for Systems Biomedicine (LCSB) > Biomedical Data Science
Omberg L, Chaibub Neto E, Perumal TM, Pratap A, Tediarjo A, Adams J, et al. Remote smartphone monitoring of Parkinson’s disease and individual response to therapy. Nat Biotechnol. 2021; 1–8. https://doi.org/10.1038/s41587-021-00974-9 PMID: 34373643
Now you see me: The voice of people with Parkinson’s in clinical trials. AD/PD; 2019. Available: https://www.youtube.com/watch?v=Gj1iXHUWhSk
FDA rejects Verily’s filing for wearable Parkinson’s disease clinical trial tool. In: MobiHealthNews [Internet]. 16 Jun 2021 [cited 7 Oct 2021]. Available: https://www.mobihealthnews.com/news/fda-rejectsverilys-filing-wearable-parkinsons-disease-clinical-trial-tool
Sherman RE, Anderson SA, Dal Pan GJ, Gray GW, Gross T, Hunter NL, et al. Real-World Evidence—What Is It and What Can It Tell Us? N Engl J Med. 2016; 375: 2293–2297. https://doi.org/10.1056/ NEJMsb1609216 PMID: 27959688
Morgan C, Rolinski M, McNaney R, Jones B, Rochester L, Maetzler W, et al. Systematic Review Looking at the Use of Technology to Measure Free-Living Symptom and Activity Outcomes in Parkinson’s Disease in the Home or a Home-like Environment. J Park Dis. 2020; 10: 429–454. https://doi.org/10.3233/JPD-191781 PMID: 32250314
Webster DE, Tummalacherla M, Higgins M, Wing D, Ashley E, Kelly VE, et al. Smartphone-Based VO2max Measurement With Heart Snapshot in Clinical and Real-world Settings With a Diverse Population: Validation Study. JMIR MHealth UHealth. 2021; 9: e26006. https://doi.org/10.2196/26006 PMID: 34085945
Sieberts SK, Schaff J, Duda M, Pataki BÁ, Sun M, Snyder P, et al. Crowdsourcing digital health measures to predict Parkinson’s disease severity: the Parkinson’s Disease Digital Biomarker DREAM Challenge. NPJ Digit Med. 2021; 4. https://doi.org/10.1038/s41746-021-00414-7 PMID: 33742069
Meyer P, Saez-Rodriguez J. Advances in systems biology modeling: 10 years of crowdsourcing DREAM challenges. Cell Syst. 2021; 12: 636–653. https://doi.org/10.1016/j.cels.2021.05.015 PMID: 34139170
Daeschler M, Elm J, Klintworth E, Afek M, Lazar S, Simuni T. Clinician-Input Study (CIS-PD): how the Fox Wearable Companion Application can influence treatment and care in Parkinson’s disease (P3. 048). AAN Enterprises; 2018.
Elm JJ, Daeschler M, Bataille L, Schneider R, Amara A, Espay AJ, et al. Feasibility and utility of a clinician dashboard from wearable and mobile application Parkinson’s disease data. NPJ Digit Med. 2019; 2: 1–6.
Evers LJ, Raykov YP, Krijthe JH, Lima ALS de, Badawy R, Claes K, et al. Real-Life Gait Performance as a Digital Biomarker for Motor Fluctuations: The Parkinson@Home Validation Study. J Med Internet Res. 2020; 22: e19068. https://doi.org/10.2196/19068 PMID: 33034562
Hauser RA, Friedlander J, Zesiewicz TA, Adler CH, Seeberger LC, O’Brien CF, et al. A home diary to assess functional status in patients with Parkinson’s disease with motor fluctuations and dyskinesia. Clin Neuropharmacol. 2000; 23: 75–81. https://doi.org/10.1097/00002826-200003000-00003 PMID: 10803796
Chaibub Neto E, Pratap A, Perumal TM, Tummalacherla M, Snyder P, Bot BM, et al. Detecting the impact of subject characteristics on machine learning-based diagnostic applications. Npj Digit Med. 2019; 2: 1–6. https://doi.org/10.1038/s41746-019-0178-x PMID: 31633058
Mishra SR, Klasnja P, MacDuffie Woodburn J, Hekler EB, Omberg L, Kellen M, et al. Supporting Coping with Parkinson’s Disease Through Self Tracking. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: Association for Computing Machinery; 2019. pp. 1–16. https://doi.org/10.1145/3290605.3300337
Breiman L. Random Forests. Mach Learn. 2001; 45: 5–32. https://doi.org/10.1023/A:1010933404324
Lundberg SM, Erion G, Chen H, DeGrave A, Prutkin JM, Nair B, et al. From local explanations to global understanding with explainable AI for trees. Nat Mach Intell. 2020; 2: 56–67. https://doi.org/10.1038/ s42256-019-0138-9 PMID: 32607472
Powers R, Etezadi-Amoli M, Arnold EM, Kianian S, Mance I, Gibiansky M, et al. Smartwatch inertial sensors continuously monitor real-world motor fluctuations in Parkinson’s disease. Sci Transl Med. 2021; 13. https://doi.org/10.1126/scitranslmed.abd7865 PMID: 33536284
Whalen S, Pandey OP, Pandey G. Predicting protein function and other biomedical characteristics with heterogeneous ensembles. Methods. 2016; 93: 92–102. https://doi.org/10.1016/j.ymeth.2015.08.016 PMID: 26342255
Stanescu A, Pandey G. Learning parsimonious ensembles for unbalanced computational genomics problems. Pac Symp Biocomput. 2017; 22: 288–299. https://doi.org/10.1142/9789813207813_0028 PMID: 27896983
Wang L, Law J, Kale SD, Murali TM, Pandey G. Large-scale protein function prediction using heterogeneous ensembles. F1000Research. 2018; 7. https://doi.org/10.12688/f1000research.16415.1 PMID: 30450194
Stanescu A, Pandey G. Developing parsimonious ensembles using predictor diversity within a reinforcement learning framework. 2021 Feb. Available: http://arxiv.org/abs/2102.07344
Goetz CG, Tilley BC, Shaftman SR, Stebbins GT, Fahn S, Martinez-Martin P, et al. Movement Disorder Society-sponsored revision of the Unified Parkinson’s Disease Rating Scale (MDS-UPDRS): Scale presentation and clinimetric testing results. Mov Disord. 2008; 23: 2129–2170. https://doi.org/10.1002/mds.22340 PMID: 19025984
San-Segundo R, Zhang A, Cebulla A, Panev S, Tabor G, Stebbins K, et al. Parkinson’s Disease Tremor Detection in the Wild Using Wearable Accelerometers. Sensors. 2020; 20. https://doi.org/10.3390/ s20205817 PMID: 33066691
Christ M, Braun N, Neuffer J, Kempa-Liehr AW. Time Series FeatuRe Extraction on basis of Scalable Hypothesis tests (tsfresh–A Python package). Neurocomputing. 2018; 307: 72–77. https://doi.org/10.1016/j.neucom.2018.03.067
Brown RG, MacCarthy B, Jahanshahi M, Marsden CD. Accuracy of Self-Reported Disability in Patients With Parkinsonism. Arch Neurol. 1989; 46: 955–959. https://doi.org/10.1001/archneur.1989. 00520450025014 PMID: 2528339
Bot BM, Suver C, Neto EC, Kellen M, Klein A, Bare C, et al. The mPower study, Parkinson disease mobile data collected using ResearchKit. Sci Data. 2016; 3: 160011. https://doi.org/10.1038/sdata.2016.11 PMID: 26938265
Lonini L, Dai A, Shawen N, Simuni T, Poon C, Shimanovich L, et al. Wearable sensors for Parkinson’s disease: which data are worth collecting for training symptom detection models. NPJ Digit Med. 2018; 1: 1–8.
Shawen N, O’Brien MK, Venkatesan S, Lonini L, Simuni T, Hamilton JL, et al. Role of data measurement characteristics in the accurate detection of Parkinson’s disease symptoms using wearable sensors. J Neuroengineering Rehabil. 2020; 17: 1–14. https://doi.org/10.1186/s12984-020-00684-4 PMID: 32312287
Sieberts SK, Zhu F, García-García J, Stahl E, Pratap A, Pandey G, et al. Crowdsourced assessment of common genetic contribution to predicting anti-TNF treatment response in rheumatoid arthritis. Nat Commun. 2016; 7: 12460. https://doi.org/10.1038/ncomms12460 PMID: 27549343
Guyon I, Weston J, Barnhill S, Vapnik V. Gene Selection for Cancer Classification using Support Vector Machines. Mach Learn. 2002; 46: 389–422. https://doi.org/10.1023/A:1012487302797
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: Machine Learning in Python. J Mach Learn Res. 2011; 12: 2825–2830. Available: http://jmlr.org/papers/v12/pedregosa11a.html
Caruana R, Niculescu-Mizil A, Crew G, Ksikes A. Ensemble selection from libraries of models. Proceedings of the twenty-first international conference on Machine learning. New York, NY, USA: Association for Computing Machinery; 2004. p. 18. https://doi.org/10.1145/1015330.1015432
Caruana R, Munson A, Niculescu-Mizil A. Getting the Most Out of Ensemble Selection. Sixth International Conference on Data Mining (ICDM’06). 2006. pp. 828–833. https://doi.org/10.1109/ICDM.2006. 76
Sutton RS, Barto AG. Reinforcement Learning: An Introduction. second edition. Cambridge, Mass: A Bradford Book; 1998.
Walker DA. JMASM9: Converting Kendall’s Tau For Correlational Or Meta-Analytic Analyses. J Mod Appl Stat Methods. 2003; 2: 525–530. https://doi.org/10.22237/jmasm/1067646360
Balduzzi S, Rücker G, Schwarzer G. How to perform a meta-analysis with R: a practical tutorial. Evid Based Ment Health. 2019; 22: 153–160. https://doi.org/10.1136/ebmental-2019-300117 PMID: 31563865
Fisher RA. Statistical Methods for Research Workers. 4th ed. Edinburgh: Oliver and Boyd; 1932.
