Innovations
CertifyIt – Risk-Based Security Testing using Models and Patterns
Vulnerability detection can be classified into two complementary categories: static and dynamic application security testing. Static Application Security Testing (SAST) are white-box approaches including source, byte and object code scanners. Dynamic Application Security Testing (DAST) includes black-box applications scanners, fuzzing techniques and emerging model-based security testing. Our approach introduces a DAST approach, called Pattern-driven and Model-based Vulnerability Testing (PMVT for short), proposed within the RASEN project to generate and execute vulnerability test cases. It combines model-based testing and a fuzzing technique, and drives the test generation by security test patterns selected from risk assessment. This approach is supported by processes and tools that effectively automate the detection of vulnerabilities and allow getting feedback about risk estimation.
The RASEN approach we propose aims to improve the accuracy and precision of security and vulnerability testing, by proposing a model-based testing approach driven by risk assessment. It extends the traditional model-based security testing techniques by driving the testing process using security test patterns selected from risk assessment results. The adaptation of such techniques for risk-based vulnerability testing defines novel features and perspectives in this research domain.
OVERALL PICTURE OF THE PMVT RISK-BASED VULNERABILITY TESTING PROCESS
Principles of the Approach
The PMVT testing process aims to make interrelated and synergetic risk management activity and security testing as shown in the figure above. In the first step, a risk model is created using the CORAS method that identifies threat scenarios (R1, R2, R3) and potential vulnerabilities (V1, V2). The risk model is then used for the identification and prioritization of appropriate security test patterns. Based on the security test patterns, test cases are generated by combining information from the risk model, security test patterns, a test model and test generation techniques. The latter are composed of test purposes generated from Smartesting CertifyIt and fuzzing techniques from Fraunhofer FOKUS’s fuzzing library Fuzzino. In the last step, test scripts are generated, compiled and executed against the application under test. Hence, the PMVT approach integrates the tools of the project partners: CORAS from SINTEF ICT (to address risk assessment), CertifyIt from Smartesting (to perform risk and model-based test generation) and Fuzzino from Fraunhofer FOKUS (to apply data fuzzing techniques).
The main steps of the PVMT process can be enumerated as follows:
- Risk identification and prioritization – Each threat scenario of the CORAS risk model is linked to a security test pattern defining the testing procedure to detect the threat in the application under test.
- Available test purpose catalogue – A test purpose allows formalizing the intention of a given security test pattern and thus allows to automatically generate the expected test cases with model-based testing techniques.
- Instantiate test purposes via a behavioural and environmental test model – A Domain Specific Modelling Language (DSML) dedicated to Web application description makes the design of the test model easier and less time consuming.
- Generation of abstract test cases – Test case generation is performed by instantiating the selected test purposes on the behavioural UML4MBT test model specifying the Web application under test.
- Generation of the executable test scripts
- Exploitation of the test results for risk assessment - Traceability between the initial targeted security test patterns, the corresponding test cases and the test results enables to evaluate and complete the risk picture.
The PMVT process ensures the traceability between the test case, the verdict of the execution and the targeted vulnerabilities identified during risk assessment. The results of the security testing metric functions should help to characterize the security risks of the system under test. Hence, it should be possible to improve a risk assessment based upon the results since these functions can be used, for instance, to calculate and update the likelihood values regarding the exploitability of some threat scenarios.
The PMVT approach is also detailed within our freely available whitepaper.
The RACOMAT Tool – Risk Assessment COMbined with Automated Testing
The RACOMAT tool allows users to combine component based security risk assessment with security testing. Testing can be integrated seamlessly into the incident simulations the tool uses for its compositional risk analysis. Taking benefit of libraries containing risk analysis artefacts like attack patterns and of libraries containing testing artefacts like security test patterns, the RACOMAT tool offers a high level of reusability. Using the assistance the tool offers, many steps of the analytical RACOMAT process from risk modelling to testing and updating the risk picture based on test results can be done automatically.
The RACOMAT process integrates security testing tightly into incident simulations of a low level compositional security risk assessment. It basically unifies risk-based security testing (which tries to optimize the security testing process with the help of risk assessment) and test-based risk assessment (which tries to improve the risk picture using test results). This iterative process is designed to test exactly the most critical part with reasonable effort while other parts are simulated. The incident simulation itself is actually a kind of testing, too, but it is testing the risk model instead of testing the real system that is analyzed.
Initial assessment and analysis
In spite of relying upon a specific risk assessment method, the RACOMAT tool can use different kinds of risk assessment methods, including fault tree analysis, event tree analysis and the CORAS method, as proposed within the RASEN project. In general, the RACOMAT tool supports component based risk analysis and compositionality. The RACOMAT tool uses intuitive risk graphs to represent and to visualize the risk picture. For enabling automation of risk based testing, the risk assessment must be made on a low level. The RACOMAT tool allows risk analysts to model close relations to parts and components of the systems that are analysed. Therefore, the RACOMAT tool introduces the concept of threat interfaces representing entire components and threat ports representing parts of the input / output interface.
Dependencies and incident simulations
In the RACOMAT tool risk artefacts are added with simple drag and drop. Thereby, they can be immediately linked with the elements of the automatically generated system models. For most common artefacts, the RACOMAT tool already suggests other typically related artefacts. For instance, a vulnerability might typically be used by attackers to do a certain kind of an attack. The related attack pattern again typically can be used to produce several incidents. The RACOMAT tool does not only present the related elements, but it also models the correct dependency relations automatically if the related artefacts are added to the risk graph.
INCIDENT SIMULATION IN RACOMAT
Testing
The idea to improve the risk picture and to reduce the dependency on the initial modelling and estimations is to replace some parts of the simulation with testing the real system components. That is, instead of simulating whether some event occurred based upon random values and likelihood functions, the RACOMAT tool tries to actually trigger the real incident. The RACOMAT tool can identify the not yet tested element that has the greatest impact on the overall risk picture or which has the most uncertain likelihood estimate based on the analyst’s judgement. This element (typically an attack pattern) should probably be tested in the first place. Automated or at least semi-automated testing is done with the help of test patterns. The RACOMAT tool provides an extendable catalog of security test patterns for most common attack patterns.
Interaction with other tools
The RACOMAT tool can be used as a stand-alone tool. It covers the entire process of combined test-based risk assessment and risk-based security testing. Nevertheless, it is also possible to use other eventually more specialized tools for some steps in that process. In particular, the RACOMAT tool can be used in conjunction with the other tools developed and used within the RASEN project. Since the RACOMAT tool supports the entire process, it makes sense to use the RACOMAT tool as the central platform for the data exchange and for any other interaction between the tools. The figure below illustrates how such a risk assessment and security testing process using the RACOMAT tool as central platform and different tools for some sub tasks could work.
RACOMAT PROCESS WITH VARIOUS TOOLS
A more in-depth discussion of the RACOMAT tool and the scenarios it enables is provided within our freely-available whitepaper.
RASEN tool-based methodology for risk based security testing and legal compliance
Managing cyber security has become increasingly important due to the growing interconnectivity of computerized systems and their use in society. A comprehensive assessment of cyber security can be challenging as it spans across different domains of knowledge and expertise. For instance, identifying cyber security vulnerabilities requires detailed technical expertise and knowledge, while the assessment of organizational impact and legal implications of cyber security incidents may require expertise and knowledge related to risk and legal compliance. We present a method that provides a comprehensive approach to cyber security by integrating three areas of cyber security assessment which are traditionally viewed in isolation: risk assessment, security testing, and legal compliance.
Motivating example
A large organization develops and maintains ICT products, and they have to ensure that these products are secure. This responsibility is split among many people with different roles in different departments: The security testers are responsible for identifying technical cyber security vulnerabilities. The security managers are responsible for ensuring that the products have an acceptable level of risk. The legal compliance managers are responsible for ensuring that the company meets its legal obligations.
In this regard, RASEN artefacts can help with instantiating three particular integration scenarios:
- Test-based risk assessment
- Test-based risk identification
- Test-based risk evaluation
- Risk-based security testing
- Risk-based security test planning
- Risk-based security test design and implementation
- Risk-based security test selection & execution
- Risk-based security test monitoring and control
- Risk-based compliance assessment
- Compliance risk identification
- Compliance risk estimation
- Compliance risk evaluation
The RASEN method provides a comprehensive approach to cyber security management that takes into account technical as well as non-technical issues. The method integrates three areas that are traditionally addressed in isolation: security risk assessment, security testing, and legal compliance assessment. While the industry demands integrative approaches that cope with security as a whole, currently no other standard exists that sufficiently emphasizes the systematic integration of these three domains.
For more information about the RASEN method for risk-based security testing and legal compliance assessment check out our freely available whitepaper here.