Detailed information regarding RASEN support for the key areas and levels below is available by selecting them. Supported areas have a blue background
| Systematic and risk driven | Real time assessment | Continuous risk-based testing | Integrated |
| Systematic compliance assessment | Quantitative assessment | Risk based testing | Partially Integrated |
| Check list based compliance assessment | Qualitative assessment | Planned testing | Stand alone |
| Ad-hoc compliance assessment | Checklist assessment | Unstructured testing | None |
| Legal and compliance assessment | Risk assessment | Security testing | Tool support |
Artefacts supporting risk based security testing
Security testing is used to experimentally check software implementations with respect to their security properties and their resistance to attacks. For security testing we can distinguish functional security testing and security vulnerability testing. Functional security testing checks if the software security functions are implemented correctly and consistent with the security functional requirements. It is used to check the functionality, efficiency and availability of the specified security features of a test item. Security vulnerability testing directly addresses the identification and discovery of yet undiscovered system vulnerabilities. This kind of security testing targets the identification of design and implementation faults that lead to vulnerabilities that may harm the availability, confidentiality and integrity of the test item. Security tests are planned and executed, either by the development team or by the testing team and planning of security testing is done on the basis of the security risk assessment (i.e impact estimations or likelyhood values are used to focus the security testing and optimize the ressource planning).
Prerequisites: Checklist based compliance assessment, Quantitative risk assessment, Planned testing and Stand alone tool support.
Whitepapers
Automated Risk-based Security Testing – Finding Vulnerabilities That Are Worth Being Found
As is well established in industrial contexts such as detailed within the Industry challenges addressed by the RASEN project whitepaper, fixing and even testing for every possible vulnerability is far too expensive. Especially when considering complex and networked systems, testing has to focus on vulnerabilities that would cause more damage when being exploited than it would cost to fix them. The present paper illustrates the RASEN approach for finding vulnerabilities in targeted applications.
http://www.rasenproject.eu/downloads/481/
Combining Compliance and Security Risk Assessment
Organizations that rely on ICT infrastructures need to maintain a high level of information security and protection from cyber-attacks. This is not only due to the self-interest of protecting business critical infrastructures; it is also due to laws that deal with information security. For this reason, technical and legal risks often need to be understood in combination. The RASEN project proposes an approach to integrate compliance and security risk assessment.
http://www.rasenproject.eu/downloads/477/
Industry challenges addressed by the RASEN project
This whitepaper is authored by the industrial partners of the project and provides an overview of the challenges and benefits the RASEN project is expected to bring to industrial organization that deploy large-scale networked systems, as current existing and conventional tools fail to support industrial needs adequately. Although requirements are very diverse there is a common set of industry generic requirements applicable to a large number of industrial software developing companies. The RASEN project is addressing those, striving to deliver a new methodology and a supportive software environment.
http://www.rasenproject.eu/downloads/473/
Articles & Presentations
A Pattern-Driven and Model-Based Test Generation Toolchain for Web Vulnerability
The purpose of this demonstration is to present a tooled Pattern-driven and Model-based Vulnerability Testing approach (PMVT for short) to improve the capability of detection of various vulnerability types such as injections (Cross-Site Scripting, SQL injections, etc.). This approach relies on generic vulnerability test patterns, which are applied on a behavioral model of the application under test, in order to generate vulnerability test cases. Hence, we propose to demonstrate the toolchain especially regarding XSS and SQLI vulnerabilities. This prototype has been experimented and validated on real-life Web applications, showing a strong improvement of detection ability w.r.t. Web application scanners for these vulnerabilities.
http://www.rasenproject.eu/downloads/665/
A Taxonomy of risk-based testing
Software testing has often to be done under severe pressure due to limited resources and a challenging time schedule facing the demand to assure the fulfillment of the software requirements. In addition, testing should unveil those software defects that harm the mission-critical functions of the software. Risk-based testing uses risk (re-)assessments to steer all phases of the test process to optimize testing efforts and limit risks of the software-based system. Due to its importance and high practical relevance, several risk-based testing approaches were proposed in academia and industry. This paper presents a taxonomy of risk-based testing providing a framework to understand, categorize, assess, and compare risk-based testing approaches to support their selection and tailoring for specific purposes. The taxonomy is aligned with the consideration of risks in all phases of the test process and consists of the top-level classes risk drivers, risk assessment, and risk-based test process. The taxonomy of risk-based testing has been developed by analyzing the work presented in available publications on risk-based testing. Afterwards, it has been applied to the work on risk-based testing presented in this special section of the International Journal on Software Tools for Technology Transfer.
http://link.springer.com/article/10.1007%2Fs10009-014-0332-3
A Technique for Risk-Based Test Procedure Identification, Prioritization and Selection
We present a technique for risk-based test procedure identification, prioritization, and selection. The technique takes a risk model in the form of a risk graph as input, and produces a list of prioritized selected test procedures as output. The technique is general in the sense that it can be used with many existing risk documentation languages and many kinds of likelihood and risk types.
http://www.rasenproject.eu/downloads/576/
A Trace Management Platform for Risk-Based Security Testing
This paper introduces RISKTest, a trace management platform on the basis of Eclipse that supports the creation and documentation of cross-tool relations during test development and test execution. RISKTest is dedicated to risk-based security testing. Thus, we concentrate on the management of traces between the artifacts from risk assessment and testing and the definitions of services that automatically analyze the related artifacts for security and testing related aspects. RISKTest has been developed in the DIAMONDS project and evaluated within the project’s case studies.
http://www.rasenproject.eu/downloads/442/
Combining Risk Assessment and Security Testing
A systematic integration of risk analysis and security testing allows for optimizing the test process as well as the risk assessment itself. The result of the risk assessment, i.e. the identified vulnerabilities, threat scenarios and unwanted incidents, can be used to guide the test identification and may complement requirements engineering results with systematic information concerning the threats and vulnerabilities of a system and their probabilities and consequences. This information can be used to weight threat scenarios and thus help identifying the ones that need to be treated and tested more carefully. On the other side, risk-based testing approaches can help to optimize the risk assessment itself by gaining empirical knowledge on the existence of vulnerabilities, the applicability and consequences of threat scenarios and the quality of countermeasures. This paper outlines a tool-based approach for risk-based security testing that combines the notion of risk-assessment with a pattern-based approach for automatic test generation relying on test directives and strategies and shows how results from the testing are systematically fed back into the risk assessment.
http://www.rasenproject.eu/downloads/578/
Compositional risk analysis combined with automated security testing - the RACOMAT tool
Risk management is an important part of the software quality management because security issues can result in big economical losses and even worse legal consequences. While risk assessment as the base for any risk treatment is widely regarded to be important, doing a risk assessment itself remains a challenge especially for complex large scaled networked systems. This paper presents an ongoing case study in which such a system is assessed. In order to deal with the challenges from that case study, the RACOMAT method and the RACOMAT tool for compositional risk assessment closely combined with security testing and incident simulation for have been developed with the goal to reach a new level of automation results in risk assessment.
http://www.rasenproject.eu/downloads/773/
Efficient Detection of Multi-Step Cross-Site Scripting Vulnerabilities
Cross-Site Scripting (XSS) vulnerability is one of the most critical breaches that may compromise the security of Web applications. Reflected XSS is usually easy to detect as the attack vector is immediately executed, and classical Web application scanners are commonly efficient to detect it. However, they are less efficient to discover multi-step XSS, which requires behavioral knowledge to be detected. In this paper, we propose a Pattern-driven and Model-based Vulnerability Testing approach (PMVT) to improve the capability of multi-step XSS detection. This approach relies on generic vulnerability test patterns, which are applied on a behavioral model of the application under test, in order to generate vulnerability test cases. A toolchain, adapted from an existing Model-Based Testing tool, has been developed to implement this approach. This prototype has been experimented and validated on real-life Web applications, showing a strong improvement of detection ability w.r.t. Web application scanners for this kind of vulnerabilities.
http://www.rasenproject.eu/downloads/556/
How the UML Testing Profile Supports Risk-Based Testing
The increasing complexity of software-intensive systems raises a lot of challenges demanding new techniques for ensuring their overall quality. The risk of not meeting the expected level of quality has negative impact on business, customers, environment and people, especially in the context of safety/security-critical systems. The importance of risk assessment, analysis and management has been well understood both in the literature and practice, which has led to the definition of a number of well-known standards. In the recent years, Risk-Based Testing (RBT) is gaining more attention, especially focusing on test prioritization and selection based on risks. On the other hand, model-based testing (MBT) provides a systematic and automated way to facilitate rigorous testing of software-intensive systems. MBT has been an intense area of research and a large number of MBT techniques have been developed in literature and practice in the last decade. In this paper, we study the feasibility of combining RBT with MBT by using the upcoming version of UML Testing Profile (UTP 2) as the mechanism. We present potential traceability between RBT and UTP 2 concepts.
http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6983859
Model-Based Security Testing with Test Patterns
This presentation outlines ongoing work and first results about test generation approach guided by risk assessment (by means of reasonable risk coverage and probability metrics) in the context of large-scale networked systems. More precisely, we describe the integration of model-based and fuzzing techniques using security test patterns in order to perform vulnerability testing. This work is supported by the RASEN project. Related work on vulnerability detection can be classified into two categories: static and dynamic analysis security testing. Static Application Security Testing (SAST) are white-box approaches including source, byte and object code scanners and static analysis techniques. Dynamic Application Security Testing (DAST) includes black-box web applications scanners, fuzzing techniques and emerging model-based security testing approaches. In practice, these techniques are complementary, addressing different types of vulnerabilities. For example, SAST techniques are known to be efficient to detect buffer overflow and badly formatted string, but weak to detect SQLI, XSS or CSRF vulnerabilities. In this context, we propose a novel risk-based testing approach that focuses on DAST techniques: to derive security test cases, we propose to drive the test generation by security test patterns resulting from risk assessment, and to combine model-based testing and fuzzing techniques to derive test cases. The overall process integrates the tools CORAS from SINTEF (risk assessment), CertifyIt from Smartesting (risk and model-based test generation) and Fuzzino from Fraunhofer FOKUS (data fuzzing techniques)
http://www.rasenproject.eu/downloads/542/
Risk-Based Vulnerability Testing using Security Test Patterns
This paper introduces an original security testing approach guided by risk assessment, by means of risk coverage, to perform and automate vulnerability testing for Web applications. This approach, called Risk-Based Vulnerability Testing, adapts Model-Based Testing techniques, which are mostly used currently to address functional features. It also extends Model-Based Vulnerability Testing techniques by driving the testing process using security test patterns selected from risk assessment results. The adaptation of such techniques for Risk-Based Vulnerability Testing defines novel features in this research domain. In this paper, we describe the principles of our approach, which is based on a mixed modeling of the System Under Test: the model used for automated test generation captures some behavioral aspects of the Web applications, but also includes vulnerability test purposes to drive the test generation process.
http://www.rasenproject.eu/downloads/554/
Risk-Driven Vulnerability Testing: Results from eHealth Experiments using Patterns and Model-Based Approach
This paper introduces and reports on an original tooled risk-driven security testing process called Pattern-driven and Model-based Vulnerability Testing. This fully automated testing process, drawing on risk-driven strategies and Model-Based Testing (MBT) techniques, aims to improve the capability of detection of various Web application vulnerabilities, in particular SQL injections, Cross-Site Scripting, and Cross-Site Request Forgery. It is based on a mixed modeling of the system under test: an MBT model captures the behavioral aspects of the Web application, while formalized vulnerability test patterns, selected from risk assessment results, drive the overall test generation process. An empirical evaluation, conducted on a complex and freely-accessible eHealth system developed by Info World, shows that this novel process is appropriate for automatically generating and executing risk-driven vulnerability test cases and is promising to be deployed for large-scale Web applications.
Not yet available
Security testing approaches - for research, industry and standardization
Recently, in the Security testing domain a lot of knowledge has been collected from a significant amount of research. The contribution provides an introduction to advanced security testing methods and techniques in the context of European research and standardization projects. This includes numerous guidelines and best practices that have been identified and are applied in the context of industrial case studies. In particular it addresses risk modeling, security test pattern, functional security tests as well as fuzz testing, as important contributions to systematic, automatized test approaches in research, industry and standardization.
http://publica.fraunhofer.de/dokumente/N-305657.html
Test Prioritization of Security Risk Tests
Many approaches are developed for efficient identification and estimation of security risks. One big challenge is to prioritize the related test cases of identified risks. The effort and costs of security testing can be high and the budget is limited. The challenge is to get a proper proportion between test effort and potential system harm. Based on the results of security testing counter measures can be implemented to achieve a proper security level for a system. In the RASEN project, one goal is to develop risk- based security testing methods and tools as well as a methodology for risk-based security testing.
http://pi.informatik.uni-siegen.de/stt/34_3/01_Fachgruppenberichte/TestPrioritization.pdf
The PMVT approach: a RASEN innovation for security Pattern and Model-based Vulnerability Testing
Vulnerability detection can be classified into two complementary categories: static and dynamic application security testing. Static Application Security Testing (SAST) are whitebox approaches including source, byte and object code scanners. Dynamic Application Security Testing (DAST) includes black-box applications scanners, fuzzing techniques and emerging model-based security testing. This white paper introduces a DAST approach, called Pattern-driven and Model-based Vulnerability Testing (PMVT for short), proposed within the RASEN project to generate and execute vulnerability test cases. It combines model-based testing and a fuzzing technique, and drives the test generation by security test patterns selected from risk assessment. This approach is supported by processes and tools that effectively automate the detection of vulnerabilities and allow getting feedback about risk estimation.
http://www.rasenproject.eu/downloads/752/
Towards Integration of Compositional Risk Analysis Using Monte Carlo Simulation and Security Testing
This short paper describes ongoing efforts to combine concepts of security risk analysis with security testing into a single process. Using risk analysis artefact composition and Monte Carlo simulation to calculate likelihood values, the method described here is intended to become applicable for complex large scale systems with dynamically changing probability values.
http://www.rasenproject.eu/downloads/446/
Using CAPEC for Risk-Based Security Testing
We present a method for risk-based security testing that takes a set of CAPEC attack patterns as input and produces a risk model which can be used for security test identification and prioritization. Since parts of the method can be automated, we believe that the method will speed up the process of constructing a risk model significantly. We also argue that the constructed risk model is suitable for security test identification and prioritization.
http://www.rasenproject.eu/downloads/831/
Standards
Name: ETSI TR101 583
http://www.etsi.org/deliver/etsi_tr/101500_101599/101583/01.01.01_60/tr_101583v010101p.pdf
Name: ISO 31000
http://www.iso.org/iso/home/standards/iso31000.htm
Name: ISO/IEC/IEEE 29119 Software Testing
http://www.softwaretestingstandard.org/
Name: UML 2.0
Link: http://www.omg.org/spec/UML/2.0/
Name: UML Testing Profile
Link: http://utp.omg.org/
Tools
CertifyIt
Smartesting CertifyIt is a tool suite that automatically generates test cases based on a model of system requirements. Manual test design is labor intensive and error prone; this manual work can be avoided for complex applications by modeling the key concepts (abstraction) and allowing Smartesting CertifyIt to automate your test design work. Since the model is more expressive and simpler than the systemunder-test, it can more readily be reviewed for correctness and coherency, as well as be updated more easily. Some plugins have been developed during RASEN project for the deployment of the RASEN approach in order to assess its accuracy and precision regarding risk-based objectives. Smartesting CertifyIt including these plugins supports UML/OCL models as the specification modeling language, and generates test cases to cover security test patterns used as test objectives.
Usage guide: http://www.rasenproject.eu/downloads/838/
Download: Available on request
Fuzzino
Fuzzino is a library that provides generation of test data for fuzz testing. With fuzzing, you are able to find security-related weaknesses in your code. It's about injecting invalid or unexpected input data to a system under test. That way, security-relevant vulnerabilities may be detected when the system under test processes such data instead of rejecting it. You can integrate Fuzzino into your testing tool in order to enable it for fuzz testing. Please keep in mind that Fuzzino is not a full-featured fuzzing tool. It is a test data generator for enabling your testing tool to perform fuzzing.
Download: https://github.com/fraunhoferfokus/Fuzzino
RACOMAT
Until September 2013, in the context of the global surveillance disclosures, the German government justified NSA spying, because “security is a super basic right” (Hans-Peter Friedrich, German minister of the interior, own translation) and surveillance is necessary to fight terrorism and other threats. Their opinion changed dramatically as soon as they learned that the mobile phone of German chancellor Angela Merkel was obviously monitored, too. Which risk is higher? Living in an Orwellian surveillance for sure or being eventually not able to prevent some act of terrorism? Obviously, it is necessary to weight risks against each other. However, risk assessment might be difficult and expensive, it often depends on the skills and estimates of the analysts. Testing is one analysis method that might yield more objective results, but security testing itself might be difficult and expensive, too, because security testing means to test for unwanted behavior and there is usually no specification what to expect. Besides that manual testing is itself error prone and infeasible for large scale systems, even highly insecure system can produce lots of correct test verdicts if the “wrong” test cases have been created and executed. Therefore, it makes sense to do Risk Assessment COMbined with Automated Testing, i.e. to use RACOMAT.