Software Security Testing Methods

Recently, in the final testing of a software product in my company, I was often asked the question: How much security does our software PRODUCT consider in testing? How safe should a software be measured?

This software is related to the customer's business important information, so the user's core concerns are always around "this software security." A security vulnerability due to design and a security vulnerability resulting from implementation are significant for the user. My task is to ensure that the software meets customer expectations in terms of security.

What is software security testing

(1) What is software security

Software security belongs to an important sub-domain in the field of software. In the previous stand-alone era, the security problem is the operating system susceptible to virus infection, single-machine application software security problem is not outstanding. However, since the popularization of the Internet, software security has become more and more prominent, making the importance of software security testing to an unprecedented height.

Software security is generally divided into two tiers, that is, application-level security and operating system-level security. Application-level security, including access to data or business functions, where the operator can access only the specific functionality of the application, limited data, and so on, in the expected security context. The security at the operating system level is to ensure that only users with access to the system platform can access it, including logon to the system or remote access.

The software security described in this article is mainly about the security of the application layer, including two levels: ① is the security of the application itself. In general, application security issues are primarily a result of software vulnerabilities that can be design- deficient or programmatic, or even a backdoor reserved by developers. ② is the data security of the application, including the security of data storage and the security of two aspects.

(2) Software security testing

Generally speaking, the security test can be mixed in the Unit test , the integration Test , the system test , and the security requirement is not high. But for software with high security requirements, special security testing must be done to prevent and identify software security issues before they are compromised.

Security testing refers to the process of verifying the security level of an application and identifying potential security flaws. The main purpose of application-level security testing is to find out the security hidden trouble in the software's own program design, and to check the protection ability of the application to the illegal intrusion, and different test strategies according to the security metrics. Note: Security testing does not ultimately prove that the application is secure, but rather is used to verify the effectiveness of the established policies, which are chosen based on assumptions made during the threat analysis phase. For example, test the operation of the application software to prevent unauthorized internal or external user access or vandalism.

Software Security Testing Process

(1) Safety test method

There are a number of test methods that can be used for safety testing, the main safety test method is as follows:

① Static code security testing: mainly through the source code for security scanning, according to the program data flow, control flow, semantics and other information and its unique software security rules database, to find out the code of potential security vulnerabilities. Static source code security testing is a useful way to identify all code that may be at risk during the coding phase so that developers can address potential security issues early. Because of this, static code testing is better suited to the early stages of code development than to the testing phase.

② Dynamic Penetration Testing: Penetration testing is also a common safety testing method. Is the use of automated tools or artificial methods to simulate the hacker's input, the application system to conduct offensive testing, to find out the operational moment of security vulnerabilities. The characteristics of this test is true and effective, the general problem is correct, but also more serious. But one of the fatal drawbacks of penetration testing is that the simulated test data can only reach a limited number of test points with very low coverage.

③ program Data scanning. A software with high security requirements, data cannot be destroyed during operation, or it will cause a buffer overflow type of attack. Data scanning is often done in memory testing, and memory testing can uncover a number of vulnerabilities such as buffer overflows, which are difficult to find using other means of testing. For example, the software runtime memory information to scan to see if there are some information that leads to the hidden danger, of course, this requires a special tool to verify, manual do is more difficult.

(2) Reverse Security test procedure

Most of the software's security testing is based on the principle of reverse design of the defect space, that is, in advance to check where there may be security risks, and then to test these potential pitfalls. Therefore, the reverse testing process is based on the defect space, establish the defect threat model, through the threat model to find the intrusion point, the intrusion point of the known vulnerability of the scan test. The advantage is that the known defects can be analyzed to avoid the existence of known types of defects in the software, but there is usually nothing to do with the unknown attack means and methods.

① establishes a defect threat model. To establish the defect threat model is to start with a known security vulnerability and check whether there are known vulnerabilities in the software. When you build a threat model, you need to identify which areas of expertise your software is involved in, and then model it based on the attack tactics that you encounter in each area of specialization.

② Search and Scan the intrusion point. Examine which defects in the threat model may occur in the software, and then manage the possible threats into the intrusion point matrix. If there is a mature vulnerability scanning tool, scan directly using the vulnerability scanning Tool and then manage the identified suspicious issues into the intrusion point matrix.

Validation test of the ③ intrusion matrix. Once the intrusion matrix is created, the corresponding test case can be designed for the specific entry of the intrusion matrix, and then the test is validated.

(3) Forward security testing process

In order to avoid the test incompleteness caused by reverse design principle, a forward testing method is needed to test the software more fully, so that the tested software can prevent the unknown attack means and methods.

① identifies the test space first. Identifying all of the variable data in the test space is expensive due to security testing, where the emphasis is on identifying the external input layer. For example, requirements Analysis , summary design, detailed design, coding in these stages are to identify the test space, and establish a test space tracking matrix.

② defines the design space precisely. The focus is on whether the design space is clearly defined in the requirements, and whether the data involved in the requirement identifies its legal value range. In this step, the most important thing to pay attention to is the exact word, in strict accordance with the security principle of the design space to do a precise definition.

③ identifies security risks. Identify which test spaces and which translation rules might be a security risk, based on the test space and design space found and the rules of conversion between them. For example, the more complex the test space is, the more complex the test space is, or the more insecure the variable data combination is. And the more complex the conversion rules, the greater the likelihood of a problem, which is a security risk.

④ establish and validate the intrusion matrix. After the security risk identification is completed, the intrusion matrix can be established based on the identified security risks. Lists potential security risks, identifies volatile data that has potential security implications, and identifies levels of security risks. For those variable data with high security risk levels, a detailed test case design is required.

(4) The difference between forward and reverse testing

The forward testing process is based on testing space to find defects and loopholes, the reverse testing process is based on the known defect space to find out whether the same defects and loopholes in the software, both have their advantages and disadvantages. One of the main advantages of the reverse testing process is that the cost is low, as long as the known possible defects can be verified, but the disadvantage is that the test is imperfect, the test space cannot be covered intact, the unknown attack means cannot be discovered. The advantage of the forward testing process is that the test is relatively adequate, but the workload is relatively large. Therefore, the security requirements of the software, generally according to the reverse test process to test, for high security requirements of the software, should be based on the positive test process, reverse testing process supplemented.

Common software security flaws and vulnerabilities

Software security has many aspects of the content, the main security problems caused by the software itself, the following describes the common software security flaws and vulnerabilities.

(1) Buffer overflow

Buffer overflow has become the number one public enemy of software security, and many practical security problems are related to it. There are two reasons why a buffer overflow problem is usually caused by the following. ① The calibration problem of the conversion rules of the design space. That is, the lack of verification of measurable data results in illegal data not being checked out and discarded at the external input layer. After the illegal data enters the interface layer and the implementation layer, it is beyond the corresponding test space or design space of the interface layer and implementation layer, which causes overflow. ② Local test space and design space is insufficient. When the legal data enters, due to the program implementation layer within the corresponding test space or design space is insufficient, resulting in program processing overflow.

(2) Encryption weaknesses

These cryptographic weaknesses are not secure: ① uses an insecure encryption algorithm. The encryption algorithm strength is not enough, some encryption algorithms can even use the poor lifting method to crack. When ② encrypt data, the password is generated by pseudo-random algorithm, and the method of generating pseudo-random number is defective, so that the password can be easily cracked. The ③ authentication algorithm has a flaw. The ④ client and server clocks are not synchronized, giving the attacker enough time to crack the password or modify the data. ⑤ does not sign encrypted data, which could allow an attacker to tamper with the data. Therefore, when testing for encryption, you must test for these possible cryptographic weaknesses.

(3) Error handling

In general, error handling will return some information to the user, the returned error message may be exploited by a malicious user to attack, the malicious user can analyze the returned error information to know what the next step to make the attack successful. If some of the wrong features are called during error handling, the process of error handling will be exploited. Error handling is a processing problem in the exception space, the processing in the exception space should be as simple as possible, using this principle to design can avoid this problem. But error handling often involves problems with ease of use, and if the ToolTip for error handling is too simple, the user may confused and not know what to do next. So, while considering the security of error handling, it is necessary to weigh the ease of use together.

(4) Too much permission

If you give too much permission, you can cause a malicious user with only ordinary user rights to take advantage of too large a privilege to make a security compromise. For example, there is no limit to what can be manipulated, which can result in users being able to access additional resources beyond the specified scope. When you perform a security test, you must test whether the application has too large permissions, focus on the permissions that should be in each case, and then check whether the given permissions are actually exceeded. The problem of excessive authority is inherently too large in design space, so it is necessary to control the design space and avoid the problem of too great authority in design space.

Recommendations for safety testing

Many experience in software security testing tells us that the prerequisite for good software security testing is to fully understand the software security vulnerability, the second is to evaluate the security risk, and the third is to have efficient software security testing technology and tools.

(1) Fully understand software security vulnerabilities

Assessing the degree of security of a software system requires a three-step process of design, implementation, and deployment. Let's take a look at how the Common Criteria evaluates the security of the software system. The first step is to determine the protection profile (PP) corresponding to the SOFTWARE PRODUCT. A PP defines a security feature template for a class of software products. For example, the database of PP, firewall, such as pp. Then, according to PP and then put forward specific security function requirements, such as the user's identity authentication implementation. Finally, determine the security objects and how to meet the corresponding security functional requirements. Therefore, a security software three links, which is not a problem.

(2) Evaluation of safety tests

When the security test is done, does the software achieve the desired level of security? This is the problem that security testers are most concerned about, so you need to establish a security assessment mechanism after testing. Generally from the following two aspects of evaluation. ① Security Defect Data assessment. If you find that the more security flaws and vulnerabilities in your software, the more defects you may leave behind. When such assessments are carried out, baseline data must be established as a reference, otherwise the right conclusions cannot be obtained without a basis for evaluation. The ② is evaluated using the vulnerability implant method. Vulnerability implantation and fault insertion testing in reliability testing is the same thing, but here is the problem of inserting some security problems into the software. In the case of vulnerability implantation, a certain number of vulnerabilities were pre-implanted in the software by a specific person who did not participate in the security test, and after the final test, how many of the vulnerabilities were found to assess the adequacy of the software's security testing.

(3) Use of safety testing techniques and tools

Use specialized, feature-specific security scanning software to identify potential vulnerabilities, incorporate defects that have already occurred, and then use automated defect libraries for bombing testing using automated test methods. For example, use some software that simulates various attacks to test.

Security testing is used to verify that the protection mechanism integrated within the software is able to protect the system from illegal intrusion in practice. In a popular word: the security of a software system must certainly be able to withstand a frontal attack-but it must also be able to withstand flank and back attacks.

Software Security Testing Methods