Introduction to Model-based Testing and spec Explorer

To build high-quality software, a lot of work needs to be done during the testing phase, which may be the most costly and most labor-intensive part of the software development process. There are many ways to improve the reliability and efficiency of testing from the simplest functional black-box test to a heavyweight approach, including a theorem proving program and a formal requirements specification. However, testing does not always achieve the necessary level of detail and often lacks a normative and methodological system.

For more than more than 10 years, Microsoft has successfully applied model-based testing (MBT) in its internal development process. MBT has proven to be a very successful approach for a variety of internal and external software products. This approach has been used more and more over the years. In contrast, it is widely accepted in the testing world (especially when compared to other "formalized" methods in the test method).

Spec Explorer is a Microsoft MBT tool that extends Visual Studio, provides a highly integrated development environment, creates a behavioral model, and is a graphical analysis tool that can be used to examine the effectiveness of these models and build test cases based on them. In our view, this tool is a demarcation point, it promotes the MBT this efficient method in the IT industry application, the relaxation natural learning curve, provides the most advanced development environment.

This article outlines the main concepts behind MBT and spec Explorer, using a case study to illustrate the main features of Spec Explorer. We also hope that the practical rules of thumb provided in this article will help you understand when you should consider using the MBT quality assurance methodology to solve specific test problems. You should not blindly use MBT in all test scenarios. Many times, other methods (such as traditional tests) may be a better choice.
What is a testable model in Spec Explorer?

Although different MBT tools offer different functionality and sometimes differ slightly conceptually, these tools are consistent for the meaning of "execute MBT". Model-based testing automatically builds the test process based on the model.

Models are typically created manually, including system requirements and expected behavior. Specific to Spec Explorer, a test case is automatically generated based on a state-oriented model. Test cases include test sequences and test expectations. The test sequence is inferred from the model and is responsible for pushing the system (SUT) to a different state. Test the evolution process of the expected tracking SUT, determine whether it conforms to the model-specified behavior, and make a decision.

The model is one of the major parts of the Spec Explorer project. It is specified in the construct called the model program. You can write model programs in any. NET language, such as C #. This program consists of a set of rules that interact with the defined state. The model program is a combination of scripting languages called Cord, and Cord is the second most important part of the Spec Explorer project. In this way, you can specify behavior descriptions to configure how the model is browsed and tested. After the model program is combined with the Cord script, you can create a testable model for SUT.

Of course, the third most important part of the Spec Explorer project is SUT. Not necessarily provide SUT to spec Explorer to generate test code (this is the default mode for Spec Explorer) because the generated code is inferred directly from the testable model and does not interact with SUT. You can perform test cases "off-line" independently of the model assessment and test case generation phases. However, if SUT is provided, Spec Explorer can verify that the bindings from the model to the implementation are explicitly defined.
Case Study: Chat system

Let's look at an example that describes how to build testable models in Spec Explorer. The SUT in this example is a simple chat system with only one chat room where users can log on and log off. After a user logs on, you can request a list of logged-in users to send broadcast messages to all users. The chat server always confirms these requests. Requests and responses are asynchronous, which means they can be mixed together. Like a typical chat system, multiple messages sent by a single user are received sequentially.

One of the advantages of using MBT is that you can get a lot of feedback to understand requirements because you have to formalize the behavioral model. In the early stages, ambiguity, contradiction and lack of context can be found. Therefore, the system requirements must be accurate and formalized, such as:

R1.

Users must receive a response for a logon request.

R2.

Users must receive a response for a logoff request.

R3.

Users must receive a response for a list request.

R4.

List response must contain the list of logged-on users.

R5.

All logged-on the users must receive a broadcast message.

R6.

Messages from one sender must is received in order.

The Spec Explorer Project uses operations from the perspective of the test system to describe interactions with SUT. These operations can be invocation operations that represent trigger actions from the test system to the SUT, either by capturing a return operation from a SUT response (if any), or by an event action that represents an autonomous message sent from the SUT. Call/return operations are blocking operations, so they are represented in SUT by a single method. These are the default action declarations, and the "event" keyword is used to declare event actions. Figure 1 is a statement of operations in the chat system.

Figure 1 Operation statement

          Cord code  
config chatconfig  
{  
  action void logonrequest (int user);  
  Action event void logonresponse (int user);  
  Action void logoffrequest (int user);  
  Action event void logoffresponse (int user);  
  Action void listrequest (int user);  
  Action event void Listresponse (int user, set<int> userlist);  
  Action void broadcastrequest (int senderuser, string message);  
  Action void Broadcastack (int receiveruser,   
    int senderuser, string message);  
  // ...  
          }