A Machine Learning-Driven Evolutionary Approach for Testing Web Application Firewalls

in IEEE Transactions on Reliability (2018), 67(3), 733-757

Web application firewalls (WAF) are an essential protection mechanism for online software systems. Because of the relentless flow of new kinds of attacks as well as their increased sophistication, WAFs ... [more ▼]

Web application firewalls (WAF) are an essential protection mechanism for online software systems. Because of the relentless flow of new kinds of attacks as well as their increased sophistication, WAFs have to be updated and tested regularly to prevent attackers from easily circumventing them. In this paper, we focus on testing WAFs for SQL injection attacks, but the general principles and strategy we propose can be adapted to other contexts. We present ML-Driven, an approach based on machine learning and an evolutionary algorithm to automatically detect holes in WAFs that let SQL injection attacks bypass them. Initially, ML-Driven automatically generates a diverse set of attacks and submit them to the system being protected by the target WAF. Then, ML-Driven selects attacks that exhibit patterns (substrings) associated with bypassing the WAF and evolve them to generate new successful bypassing attacks. Machine learning is used to incrementally learn attack patterns from previously generated attacks according to their testing results, i.e., if they are blocked or bypass the WAF. We implemented ML-Driven in a tool and evaluated it on ModSecurity, a widely used open-source WAF, and a proprietary WAF protecting a financial institution. Our empirical results indicate that ML-Driven is effective and efficient at generating SQL injection attacks bypassing WAFs and identifying attack patterns. [less ▲]

Automated Security Testing of Web-Based Systems Against SQL Injection Attacks

Doctoral thesis (2016)

Injection vulnerabilities, such as SQL injection (SQLi), are ranked amongst the most dangerous types of vulnerabilities. Despite having received much attention from academia and practitioners, the ... [more ▼]

Injection vulnerabilities, such as SQL injection (SQLi), are ranked amongst the most dangerous types of vulnerabilities. Despite having received much attention from academia and practitioners, the prevalence of SQLi is common and the impact of their successful exploitation is severe. In this dissertation, we propose several security testing approaches that evaluate web applications and services for vulnerabilities and common IT infrastructure components such as for their resilience against attacks. Each of the presented approaches covers a different aspect of security testing, e.g. the generation of test cases or the definition of test oracles, and in combination they provide a holistic approach. The work presented in this dissertation was conducted in collaboration with SIX Payment Services (formerly CETREL S.A.). SIX Payment Services is a leading provider of financial services in the area of payment processing, e.g. issuing of credit and debit cards, settlement of card transactions, online payments, and point-of-sale payment terminals. We analyse the challenges SIX is facing in security testing and base our testing approaches on assumptions inferred from our findings. Specifically, the devised testing approaches are automated, applicable in black box testing scenarios, able to assess and bypass Web Application Firewalls (WAF), and use an accurate test oracle. The devised testing approaches are evaluated with SIX’ IT platform, which consists of various web services that process several thousand financial transactions daily. The main research contributions in this dissertation are: - An assessment of the impact of Web Application Firewalls and Database Intrusion Detection Systems on the accuracy of SQLi testing. - An input mutation technique that can generate a diverse set of test cases. We propose a set of mutation operators that are specifically designed to increase the likelihood of generating successful attacks. - A testing technique that assesses the attack detection capabilities of a Web Application Firewall (WAF) by systematically generating attacks that try to bypass it. - An approach that increases the attack detection capabilities of a WAF by inferring a filter rule from a set of bypassing attacks. The inferred filter rule can be added to the WAF’s rule set to prevent attacks from bypassing. - An automated test oracle that is designed to meet the specific requirements of testing in an industrial context and that is independent of any specific test case generation technique. [less ▲]

SOFIA: An Automated Security Oracle for Black-Box Testing of SQL-Injection Vulnerabilities

in Proceedings of the 31th IEEE/ACM International Conference on Automated Software Engineering (2016)

Security testing is a pivotal activity in engineering secure software. It consists of two phases: generating attack inputs to test the system, and assessing whether test executions expose any ... [more ▼]

Security testing is a pivotal activity in engineering secure software. It consists of two phases: generating attack inputs to test the system, and assessing whether test executions expose any vulnerabilities. The latter phase is known as the security oracle problem. In this work, we present SOFIA, a Security Oracle for SQL-Injection Vulnerabilities. SOFIA is programming-language and source-code independent, and can be used with various attack generation tools. Moreover, because it does not rely on known attacks for learning, SOFIA is meant to also detect types of \sqli attacks that might be unknown at learning time. The oracle challenge is recast as a one-class classification problem where we learn to characterise legitimate SQL statements to accurately distinguish them from \sqli attack statements. We have carried out an experimental validation on six applications, among which two are large and widely-used. SOFIA was used to detect real \sqli vulnerabilities with inputs generated by three attack generation tools. The obtained results show that SOFIA is computationally fast and achieves a recall rate of 100\% (i.e., missing no attacks) with a low false positive rate (0.6\%). [less ▲]

Automated Testing for SQL Injection Vulnerabilities: An Input Mutation Approach

in Proc. of the International Symposium on Software Testing and Analysis 2014 (2014, July 21)

Web services are increasingly adopted in various domains, from finance and e-government to social media. As they are built on top of the web technologies, they suffer also an unprecedented amount of ... [more ▼]

Web services are increasingly adopted in various domains, from finance and e-government to social media. As they are built on top of the web technologies, they suffer also an unprecedented amount of attacks and exploitations like the Web. Among the attacks, those that target SQL injection vulnerabilities have consistently been top-ranked for the last years. Testing to detect such vulnerabilities before making web services public is crucial. We present in this paper an automated testing approach, namely μ4SQLi, and its underpinning set of mutation operators. μ4SQLi can produce effective inputs that lead to executable and harmful SQL statements. Executability is key as otherwise no injection vulnerability can be exploited. Our evaluation demonstrated that the approach is effective to detect SQL injection vulnerabilities and to produce inputs that bypass application firewalls, which is a common configuration in real world. [less ▲]

Assessing the Impact of Firewalls and Database Proxies on SQL Injection Testing

in Springer LNCS series (2013)

This paper examines the effects and potential benefits of utilising Web Application Firewalls (WAFs) and database proxies in SQL injection testing of web applications and services. We propose testing the ... [more ▼]

This paper examines the effects and potential benefits of utilising Web Application Firewalls (WAFs) and database proxies in SQL injection testing of web applications and services. We propose testing the WAF itself to refine and evaluate its security rules and prioritise fixing vulnerabilities that are not protected by the WAF. We also propose using database proxies as oracles for black-box security testing instead of relying only on the output of the application under test. The paper also presents a case study of our proposed approaches on two sets of web services. The results indicate that testing through WAFs can be used to prioritise vulnerabilities and that an oracle that uses a database proxy finds more vulnerabilities with fewer tries than an oracle that relies only on the output of the application. [less ▲]