Test Case Specification and Analysis; Cyber Physical Systems; UML Profile; Simulation; Model Checking; Machine Learning
Résumé :
[en] Hardware-in-the-loop (HiL) testing is important for developing cyber physical systems (CPS). HiL test cases manipulate hardware, are time-consuming and their behaviors are impacted by the uncertainties in the CPS environment. To mitigate the risks associated with HiL testing, engineers have to ensure that (1) test cases are well-behaved, e.g., they do not damage hardware, and (2) test cases can execute within a time budget. Leveraging the UML profile mechanism, we develop a domain-specific language, HITECS, for HiL test case specification. Using HITECS, we provide uncertainty-aware analysis methods to check the well-behavedness of HiL test cases. In addition, we provide a method to estimate the execution times of HiL test cases before the actual HiL testing. We apply HITECS to an industrial case study from the satellite domain. Our results show that: (1) HITECS helps engineers define more effective assertions to check HiL test cases, compared to the assertions defined without any systematic guidance; (2) HITECS verifies in practical time that HiL test cases are well-behaved; (3) HITECS is able to resolve uncertain parameters of HiL test cases by synthesizing conditions under which test cases are guaranteed to be well-behaved; and (4) HITECS accurately estimates HiL test case execution times.
Centre de recherche :
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > Software Verification and Validation Lab (SVV Lab)
Disciplines :
Sciences informatiques
Auteur, co-auteur :
SHIN, Seung Yeob ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
CHAOUCH, Karim ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
NEJATI, Shiva ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
SABETZADEH, Mehrdad ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
BRIAND, Lionel ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT)
Zimmer, Frank
Co-auteurs externes :
yes
Langue du document :
Anglais
Titre :
Uncertainty-aware Specification and Analysis for Hardware-in-the-Loop Testing of Cyber Physical Systems
Raja Ben Abdessalem, Shiva Nejati, Lionel C. Briand, and Thomas Stifter. 2018. Testing Vision-Based Control Systems Using Learnable Evolutionary Algorithms. In Proceedings of the 40th International Conference on Software Engineering (ICSE'18). 1-11. (in press).
Borja Fernández Adiego, Dániel Darvas, Enrique Blanco Vinuela, Jean-Charles Tournier, Simon Bliudze, Jan Olaf Blech, and Víctor Manuel González Suárez. 2015. Applying Model Checking to Industrial-Sized PLC Programs. IEEE Transactions on Industrial Informatics 11, 6 (2015), 1400-1410.
Alf 2017. Action Language for Foundational UML (Alf). OMG Specification formal/2017-07-04. Object Management Group.
Shaukat Ali and Tao Yue. 2015. U-Test: Evolving, Modelling and Testing Realistic Uncertain Behaviours of Cyber-Physical Systems. In Proceedings of the 8th IEEE International Conference on Software Testing, Verification and Validation (ICST'15). 1-2.
Paul Ammann and Jeff Offutt. 2016. Introduction to Software Testing (2 ed.). Cambridge University Press.
Eduardo Aranha and Paulo Borba. 2007. An Estimation Model for Test Execution Effort. In Proceedings of the 1st International Symposium on Empirical Software Engineering and Measurement (ESEM'07). 107-116.
Aitor Arrieta, Goiuria Sagardui, Leire Etxeberria, and Justyna Zander. 2017. Automatic Generation of Test System Instances for Configurable Cyber-physical Systems. Software Quality Journal 25, 3 (2017), 1041-1083.
Aitor Arrieta, Shuai Wang, Urtzi Markiegi, Goiuria Sagardui, and Leire Etxeberria. 2017. Search-based test case generation for Cyber-Physical Systems. In Proceedings of the 2017 IEEE Congress on Evolutionary Computation (CEC'17). 688-697.
Sara Abbaspour Asadollah, Rafia Inam, and Hans Hansson. 2015. A Survey on Testing for Cyber Physical System. In Proceedings of the 27th IFIP WG 6.1 International Conference on Testing Software and Systems (ICTSS'15). 194-207.
Alessandra Bagnato, Andrey Sadovykh, Etienne Brosse, and Tanja E.J. Vos. 2013. The OMG UML Testing Profile in Use-An Industrial Case Study for the Future Internet Testing. In Proceedings of the 17th European Conference on Software Maintenance and Reengineering (CSMR'13). 457-460.
Paul Baker, Zhen Ru Dai, Jens Grabowski, Øystein Haugen, Ina Schieferdecker, and Clay Williams. 2007. Model-Driven Testing: Using the UML Testing Profile.
Armin Biere, Alessandro Cimatti, Edmund M. Clarke, Ofer Strichman, and Yun-shan Zhu. 2003. Bounded model checking. Advances in Computers 58 (2003), 117-148.
Thomas Buchmann and Alexander Rimer. 2016. Unifying Modeling and Programming with ALF. In Proceedings of the 2nd International Conference on Advances and Trends in Software Engineering (SOFTENG'16). 10-15.
Federico Ciccozzi. 2016. On the automated translational execution of the action language for foundational UML. Software and Systems Modeling (2016), 1-27.
Edmund Clarke, Daniel Kroening, and Flavio Lerda. 2004. A Tool for Checking ANSI-C Programs. In Proceedings of the International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS'04). 168-176.
Edmund M. Clarke and Paolo Zuliani. 2011. Statistical Model Checking for Cyber-physical Systems. In Proceedings of the 9th International Conference on Automated Technology for Verification and Analysis (ATVA'11). 1-12.
Edmund M. Clarke, Jr., Orna Grumberg, and Doron A. Peled. 1999. Model Checking. MIT Press.
Maged Elaasar and Omar Badreddin. 2016. Modeling Meets Programming: A Comparative Study in Model Driven Engineering Action Languages. In Proceedings of the International Symposium on Leveraging Applications of Formal Methods (ISoLA'16). 50-67.
Eduard Paul Enoiu, Adnan Čaušević, Thomas J. Ostrand, Elaine J. Weyuker, Daniel Sundmark, and Paul Pettersson. 2016. Automated test generation using model checking: an industrial evaluation. International Journal on Software Tools for Technology Transfer (ICTSS'16) 18, 3 (2016), 335-353.
Ronald A. Fisher. 1959. Statistical Methods and Scientific Inference. Oliver & Boyd.
fUML 2017. Semantics of a Foundational Subset for Executable UML Models (fUML). OMG Specification formal/2017-07-02. Object Management Group.
Orna Grumberg and Helmut Veith (Eds.). 2008. 25 Years of Model Checking: History, Achievements, Perspectives. Springer-Verlag.
Johannes Iber, Nermin Kajtazović, and Andrea Höller. 2015. Ubtl UML Testing Profile based Testing Language. In Proceedings of the 3rd International Conference on Model-Driven Engineering and Software Development. 1-12.
Jeff C. Jensen, Danica H. Chang, and Edward A. Lee. 2011. A Model-Based Design Methodology for Cyber-Physical Systems. In Proceedings of the 7th International Wireless Communications and Mobile Computing Conference (IWCMC'11). 1666-1671.
Michel C. Jeruchim, Philip Balaban, and K. Sam Shanmugan (Eds.). 2000. Simulation of Communication Systems: Modeling, Methodology and Techniques (2nd ed.). Kluwer Academic Publishers.
Yue Jia and Mark Harman. 2011. An Analysis and Survey of the Development of Mutation Testing. IEEE Transactions on Software Engineering 37, 5 (2011), 649-678.
Edward A. Lee. 2008. Cyber Physical Systems: Design Challenges. In Proceedings of the 11th IEEE Symposium on Object Oriented Real-Time Distributed Computing (ISORC'08). 363-369.
Gary Lindstrom, Peter C. Mehlitz, and Willem Visser. 2005. Model Checking Real Time Java Using Java Pathfinder. In Proceedings of the 3rd International Conference on Automated Technology for Verification and Analysis (ATVA'05). 444-456.
Bing Liu and Lionel C. Briand Shiva Nejati, Lucia Lucia. 2018. Effective Fault Localization of Automotive Simulink Models: Achieving the Trade-Off between Test Oracle Effort and Fault Localization Accuracy. Empirical Software Engineering (2018), 1-47. (in press).
Reza Matinnejad, Shiva Nejati, Lionel C. Briand, and Thomas Bruckmann. 2018. Test Generation and Test Prioritization for Simulink Models with Dynamic Behavior. IEEE Transactions on Software Engineering (TSE'18) (2018), 1-25. (in press).
Pieter J. Mosterman and Justyna Zander. 2016. Cyber-physical Systems Challenges: A Needs Analysis for Collaborating Embedded Software Systems. Software and Systems Modeling (SoSyM'16) 15, 1 (2016), 5-16.
Kshirasagar Naik and Behcet Sarikaya. 1993. Test Case Verification by Model Checking. Formal Methods in System Design 2 (1993), 277-321. Issue 3.
Phu H. Nguyen, Shaukat Ali, and Tao Yue. 2017. Model-Based Security Engineering for Cyber-Physical Systems: A Systematic Mapping Study. Information and Software Technology 83 (2017), 116-135.
Ina Schieferdecker, Zhen Ru Dai, Jens Grabowski, and Axel Rennoch. 2003. The UML 2.0 Testing Profile and Its Relation to TTCN-3. In Proceedings of the 15th IFIP International Conference on Testing of Communicating Systems (TestCom'03). 79-94.
Rolf Schwitter. 2002. English as a Formal Specification Language. In Proceedings of the 13th International Workshop on Database and Expert Systems Applications (DEXA'02). 228-232.
Ed Seidewitz. 2017. A Development Environment for the Alf Language Within the MagicDraw UML Tool (Tool Demo). In Proceedings of the 10th ACM SIGPLAN International Conference on Software Language Engineering (SLE'17). 217-220.
Ed Seidewitz and Jérémie Tatibouet. 2015. Tool Paper: Combining Alf and UML in Modeling Tools - An Example with Papyrus -. In Proceedings of the 15th International Workshop on OCL and Textual Modeling (OCL'15). 105-119.
Seung Yeob Shin, Karim Chaouch, Shiva Nejati, Mehrdad Sabetzadeh, Lionel C. Briand, and Frank Zimmer. 2018. [Case study data] HITECS: A UML Profile and Analysis Framework for Hardware-in-the-Loop Testing of Cyber Physical Systems. https://github.com/ChaouchKarim/HITECS. (2018).
Seung Yeob Shin, Shiva Nejati, Mehrdad Sabetzadeh, Lionel C. Briand, and Frank Zimmer. 2018. Test Case Prioritization for Acceptance Testing of Cyber Physical Systems: A Multi-Objective Search-Based Approach. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA'18). 1-12. (in press).
TDL 2018. Test Description Language. ETSI Standard TR 103 119. ETSI.
Robert A. Thacker, Kevin R. Jones, Chris J. Myers, and Hao Zheng. 2010. Automatic Abstraction for Verification of Cyber-physical Systems. In Proceedings of the 1st ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS'10). 12-21.
TTCN-3 2017. Testing and Test Control Notation version 3. ETSI Standard ES 201 873-1. ETSI.
Heila van der Merwe, Brink van der Merwe, and Willem Visser. 2012. Verifying Android Applications Using Java PathFinder. ACM SIGSOFT Software Engineering Notes 37, 6 (2012), 1-5.
Willem Visser, Klaus Havelund, Guillaume Brat, Seungjoon Park, and Flavio Lerda. 2003. Model Checking Programs. Automated Software Engineering 10, 2 (2003), 203-232.
Willem Visser, Corina S. Pǎsǎreanu, and Sarfraz Khurshid. 2004. Test Input Generation with Java PathFinder. In Proceedings of the 2004 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA'04). 97-107.
Chunhui Wang, Fabrizio Pastore, Arda Goknil, Lionel Briand, and Zohaib Iqbal. 2015. Automatic Generation of System Test Cases from Use Case Specifications. In Proceedings of the 2015 International Symposium on Software Testing and Analysis (ISSTA'15). 385-396.
Xiangjuan Yao, Mark Harman, and Yue Jia. 2014. A Study of Equivalent and Stubborn Mutation Operators Using Human Analysis of Equivalence. In Proceedings of the 36th International Conference on Software Engineering (ICSE'14). 919-930.
Man Zhang, Shaukat Ali, and Tao Yue. 2017. Uncertainty-wise Test Case Generation and Minimization for Cyber-Physical Systems. Technical Report 2016-13. Simula Research Laboratory. 1-31 pages.
Xi Zheng and Christine Julien. 2015. Verification and Validation in Cyber Physical Systems: Research Challenges and a Way Forward. In Proceedings of the 1st International Workshop on Software Engineering for Smart Cyber-Physical Systems (SEsCPS'15). 15-18.
