provenance; workflow debugging; Error prones; Large scale simulations; New approaches; Root cause; Scientific experiments; Software; Information Systems
Résumé :
[en] Data analysis for scientific experiments and enterprises, large-scale simulations, and machine learning tasks all entail the use of complex computational pipelines to reach quantitative and qualitative conclusions. If some of the activities in a pipeline produce erroneous outputs, the pipeline may fail to execute or produce incorrect results. Inferring the root cause(s) of such failures is challenging, usually requiring time and much human thought, while still being error-prone. We recently proposed a new approach that makes provenance to automatically and iteratively infer root causes and derive succinct explanations of failures; such an approach was implemented in our prototype, BugDoc. In this demonstration, we will illustrate BugDoc's capabilities to debug pipelines using few configuration instances.
Disciplines :
Sciences informatiques
Auteur, co-auteur :
DE PAULA LOURENCO, Raoni ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > SerVal ; NYU - New York University [US-NY]
Freire, Juliana; New York University, New York, United States
Shasha, Dennis; New York University, New York, United States
Co-auteurs externes :
yes
Langue du document :
Anglais
Titre :
BugDoc: A System for Debugging Computational Pipelines
Date de publication/diffusion :
14 juin 2020
Nom de la manifestation :
Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data
Lieu de la manifestation :
Portland, Usa
Date de la manifestation :
14-06-2020 => 19-06-2020
Manifestation à portée :
International
Titre de l'ouvrage principal :
SIGMOD 2020 - Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data
Acknowledgments. This work has been supported in part by NSF grants MCB-1158273, IOS-1339362, and MCB-1412232, CNPq (Brazil) grant 209623/2014-4, the DARPA D3M program, and NYU WIRELESS. Any opinions, findings, and conclusions or recommendations expressed in this material are
Mona Attariyan, Michael Chow, and Jason Flinn. 2012. X-Ray: Automating Root-Cause Diagnosis of Performance Anomalies in Production Software. In Proceedings of USENIX OSDI. 307-320.
Mona Attariyan and Jason Flinn. 2011. Automating Configuration Troubleshooting with ConfAid.;login: 1 (2011), 1-14.
Anju Bala and Inderveer Chana. 2015. Intelligent Failure Prediction Models for Scientific Workflows. Expert System Applications 3 (Feb. 2015), 980-989.
James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011. Algorithms for Hyper-Parameter Optimization. In Proceedings of NIPS. 2546-2554.
J. Bergstra, D. Yamins, and D. D. Cox. 2013. Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures. In Proceedings of ICML. 115-123.
Ang Chen, Yang Wu, Andreas Haeberlen, Boon T. Loo, and Wenchao Zhou. 2017. Data Provenance at Internet Scale: Architecture, Experiences, and the Road Ahead. In Proceedings of CIDR. 1-7.
Nima Dolatnia, Alan Fern, and Xiaoli Fern. 2016. Bayesian Optimization with Resource Constraints and Production. In Proceedings of ICAPS. 115-123.
Kareem El Gebaly, Parag Agrawal, Lukasz Golab, Flip Korn, and Divesh Srivastava. 2014. Interpretable and Informative Explanations of Outcomes. Proceedings of VLDB Endowment 1 (Sept. 2014), 61-72.
Juliana Freire, David Koop, Emanuele Santos, Carlos Scheidegger, Cláudio T. Silva, and H. T. Vo. 2011. The Architecture of Open Source Applications-Chapter 23. VisTrails. Computer (2011), 367-386.
Muhammad Ali Gulzar, Siman Wang, and Miryung Kim. 2018. BigSift: Automated Debugging of Big Data Analytics in Data-Intensive Scalable Computing. In Proceedings of ESEC/FSE. 863-866.
Jiangbo Huang. 2014. Programing implementation of the Quine-McCluskey method for minimization of Boolean expression. CoRR (2014), 1-22. arXiv:1410. 1059
Brittany Johnson, Yuriy Brun, and Alexandra Meliou. 2018. Causal Testing: Finding Defects' Root Causes. CoRR (2018), 1-12. arXiv:1809. 06991
Ben Liblit, Mayur Naik, Alice X. Zheng, Alex Aiken, and Michael I. Jordan. 2005. Scalable Statistical Bug Isolation. In In Proceedings of ACM SIGPLAN. 15-26.
Raoni Lourenço, Juliana Freire, and Dennis Shasha. 2019. Debugging Machine Learning Pipelines. In Proceedings of DEEM.
Raoni Lourenço, Juliana Freire, and Dennis Shasha. 2020. BugDoc: Algorithms and a System to Debug Computational Processes. In Proceedings of ACM SIGMOD.
Leon Petrosjan and Vladimir V Mazalov. 2007. Description of Game Actions in Cluedo. In Game theory and applications. Vol. 11. 1-28.
Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. 2012. Practical Bayesian Optimization of Machine Learning Algorithms. In Proceedings of NIPS. 2951-2959.
Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Md. Mostofa Ali Patwary, Prabhat Prabhat, and Ryan P. Adams. 2015. Scalable Bayesian Optimization Using Deep Neural Networks. In Proceedings of the ICML. 2171-2180.
Xiaolan Wang, Xin Luna Dong, and Alexandra Meliou. 2015. Data XRay: A Diagnostic Tool for Data Errors. In Proceedings of ACM SIGMOD. 1231-1245.
Alice X. Zheng, Michael I. Jordan, Ben Liblit, Mayur Naik, and Alex Aiken. 2006. Statistical Debugging: Simultaneous Identification of Multiple Bugs. In Proceedings of ICML. 1105-1112.
