Evaluating Search-Based Techniques With Statistical Tests

Reference : Evaluating Search-Based Techniques With Statistical Tests


	Document type :	Scientific congresses, symposiums and conference proceedings : Paper published in a book
	Discipline(s) :	Engineering, computing & technology : Computer science
	To cite this reference:	http://hdl.handle.net/10993/35262

Title :	Evaluating Search-Based Techniques With Statistical Tests
Language :	English
Author, co-author :	Arcuri, Andrea [University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > >]
Publication date :	2018
Main document title :	The Search-Based Software Testing (SBST) Workshop
Peer reviewed :	No
Event name :	The Search-Based Software Testing (SBST) Workshop
Event date :	May
Abstract :	[en] This tutorial covers the basics of how to use statistical tests to evaluate and compare search-algorithms, in particular when applied on software engineering problems. Search-algorithms like Hill Climbing and Genetic Algorithms are randomised. Running such randomised algorithms twice on the same problem can give different results. It is hence important to run such algorithms multiple times to collect average results, and avoid so publishing wrong conclusions that were based on just luck. However, there is the question of how often such runs should be repeated. Given a set of n repeated experiments, is such n large enough to draw sound conclusions? Or should had more experiments been run? Statistical tests like the Wilcoxon-Mann-Whitney U-test can be used to answer these important questions.
Permalink :	http://hdl.handle.net/10993/35262
FnR project :	FnR ; FNR3949772 > Lionel Briand > VVLAB > Validation and Verification Laboratory > 01/01/2012 > 31/07/2018 > 2010

File(s) associated to this reference

Fulltext file(s):

	File	Commentary	Version	Size	Access
Limited access	main.pdf		Author preprint	306.91 kB	Request a copy

All documents in ORBi^lu are protected by a user license.