Online Game Testing Process

 

Causes of game Testing

Over the past few years, online games have become the latest online game trend, attracting the attention of countless companies from the legend of the grand opening. However, with the increase of players' taste and agent fees, it is difficult to establish a single foreign game Agent Model in China, online Games with the characteristics of traditional Chinese culture are very popular in China, such as swordsman sentiment and westward journey. Some domestic classics have entered the camp of first-class online games. At the same time, with the increasing requirements on online game stability and playability, online game testing has become a topic of attention.

Differences between game testing and Software Testing

As part of software testing, game testing has all the common features of software testing:

* The purpose of the test is to discover defects in the software.
* All tests must be performed by the tester according to the product behavior description. The product behavior description can be a written specification, requirement document, product document, user manual, source code, or executable program.
* Each test requires the product to run in a real or simulated environment.
* Each test requires that the product functions be displayed in a systematic way to verify whether the test results are valid and identify the cause of the error, so that the program personnel can make improvements.

All in all, testing is to discover and improve the quality of software products. Game testing also has all the above features. However, due to the special nature of the game, game testing is mainly divided into two parts: one is the traditional software testing and the other is the testing of the game itself, because games, especially online games, are equivalent to the virtual world on the Internet and are another embodiment of human society, they also include some characteristics of human society, at the same time, it is also a game, so it also involves entertainment, playability and other unique features, so the test area is quite wide. We call it a game world test, which has the following features:

* Game plot testing mainly refers to the composition of the task System in the game world. Some people are also called event-driven in the game world. I like to call it a test of the game emotional world.
* The balance test of the gaming world is mainly manifested in economic balance and capability balance (including skills, attributes, and so on) to ensure a fair competition in the gaming world.
* Testing of game culture, such as the style of the entire game world, whether it is dominated by Chinese culture, or the style of Japan and South Korea, from the overall game style to the dialogue between the NPC (game world figures, for example, a scholar, his conversation must be a text, and he cannot use Jianghu language J.

Game testing Overview

Many people have the following idea: "testing is performed after software development ." As everyone knows, this kind of point is contrary to the life cycle of software development. The sooner the software defect is discovered, the better. In this way, the risk can be effectively avoided, under the guidance of the concept of "testing at the end", there will inevitably be many problems in testing. The mistake of this concept is: the "test phase" in the life cycle indicates that testing at this stage is a major task, rather than testing only happens in the "test phase ". Generally, at the test stage, the main task of testing is to run the test to form a test report. To improve the quality of the game, early intervention in testing, such as testing plan, determination of test cases, and compilation of test code, must be performed at an earlier stage. If you put the test in the final stage, you will miss the best time to discover serious problems in Architecture Design and game logic design. By then, it will be inconvenient to fix these defects, because the defects have spread to the system, it is difficult to find and fix such errors at a higher cost.

To learn how to test a game, you must understand how to play the game and its development process in order to truly test the game. There are three basic prerequisites for a successful game. They are vision, technology, and process ). Three conditions:

Figure: three cornerstones of Game Development

* Vision is a forward-looking understanding of the game and consideration of its strategy.
* Technology: with vision, if there is no technology, all kinds of wonderful ideas can only stay in the stage of nothingness and implement Vision through technology.
* Process: with vision as a guide and technology as a guarantee, it may not be able to convert good ideas into high-quality games. To create high-quality games, there is still a lack of an important link, that is, the process of making a game is a very long dynamic process. The quality of game products is guaranteed by the dynamic quality of the dynamic process. The process is composed of many complex and mutually restrained links and components. Any link or component that has a problem will have a quality impact on the final product. Therefore, we must plan and control the dynamic process to ensure that the work is completed on a time-by-step basis and by quality.

 

Relationship Between Game testing and development

CMM (Software Capability Maturity Model) is a mature software model that everyone is familiar with, but there are such problems in the implementation process, there is no fixed path for game development. Our team is a long-term game development team and has a deep understanding of game development, we believe that the process of a game is actually a software process, but it is a special game software development process, and each lifecycle is still the same. Therefore, we have summarized a set of quality-driven development processes. Is the iterative development of the game:

 

Figure: iterative game development and testing

Because the life cycle of online games is also three or four years, the iterative development process can adapt to the long-term development of online games, it can also use the advantages of iterative development of RUP and the milestone control of CMM to ensure the full lifecycle of game products.

In the game development process, the demand analysis phase of the general software is replaced by planning, but the role is the same, clarifying the game design objectives (including style and Game Player group ), the composition of the game world is a clear requirement for later program design, artist design, and testing. Because development is a phase process, it is easier to combine testing and development. From the figure above, we can see that testing is synchronized with game development, tests are performed in each development phase, so that you can gain an in-depth understanding of the entire system and most of the technical details, this greatly improves the accuracy of the tester's judgment on errors and effectively ensures the stability of important game systems.

Game planning and testing plan

The test process cannot be performed in a vacuum. If the tester does not know that the game is composed of those components, it is very difficult to perform the test. At the same time, the test plan can clarify the test objectives, resources, and schedule required, through the test plan, testers can understand the testing focuses of the game test and communicate with the product development team. In enterprise development, the test plan comes from the Requirement Description document. In the game development process, the source of the test plan is the plan book. The plan book contains game positioning, style, storyline, and requirements. Our senior testers can participate in the planning and review and get a detailed game planning book from which they can learn about the composition, playability, and balance (Economic and capability) of the game ), and form (single-host or online games). The main thing we test at this stage is to develop a detailed test plan through the plan book, there are two main aspects: one is the testing plan of the game program itself, such as the task system, chat, team formation, and map. The other is the functional testing plan implemented by the program. The other is the testing plan for game playability, for example, whether the economic balance standard meets the requirements, the parameter and method of skill balance tests of various sects, the game style tests, and the performance test plan, such as the requirements of the client, the performance requirements of the network version on the server. At the same time, the test plan also states the basic testing methods, the requirements of the automation tools to be designed, and lays a good foundation for subsequent testing. At the same time, as testers participate in the planning review, senior game testers and product managers have a deep understanding of the game and will put forward their own views on the planning, including playability, user base, performance requirements and so on, and form a risk assessment analysis report for the product, but this report is different from the Risk Analysis Report of the Planning Department, it mainly demonstrates the quality of the game from the perspective of the onlookers, so as to control the game more effectively.

Game Design and Testing

The design phase is the best time to design test cases. Many organizations either do not have a test plan or design at all, or quickly complete the test plan and design before the test is about to begin. In this case, the test only verifies the correctness of the program, rather than verifying what the entire system should have implemented. Our tests will be very clear, because our test plan has been clearly written and we need to test those game systems, but we also need to understand the composition of the system, the design stage is the process of designing the system. All important systems are described in detail using UML state diagrams, such as user login. 2:

Figure 2 User Login status

One of the basic qualities that senior Testers in our team must possess is to design test cases for important systems based on the UML use case diagram, sequence diagram, and state diagram, only when the quality of important systems is fully tested can the quality of game programs be fully guaranteed. For example, it is the sequence diagram of a user logging on to the game system. Here we can clearly understand how players verify and log on to the system. In this process, they must interact with those objects. For example, here we are the interaction between the three systems, the client (player part), gateway, and account service have a time series change relationship. To completely test this process, we must design a test case that can cover the entire process, and consider the possible illegal situations, because this sequence diagram only takes into account the user's normal login success, and does not consider password errors, communication failures, and many other possible situations, A complete test case library is formed to fully prepare the systematic test of the login system. At the same time, through this figure, the performance analysts can also analyze possible performance bottlenecks. For example, the possible bottlenecks here are as follows: if the total number of concurrent users can be reached by the gateway, whether distributed deployment or load balancing can be used, the ratio of network bandwidth between the three is allocated, and whether the account server can carry connection requests from multiple gateways and the maximum number of connection requests, at the same time, we will design performance tests for these risks and propose automated testing requirements, such as stress tools used to simulate player login.

At the same time, during the design review, the tester's intervention can fully express their opinions on the current system architecture. Because the tester's vision is the most demanding and has many years of testing experience, early detection of design problems, such as whether a player has performed transaction support and data validation during server conversion, in the past design, player data loss was caused by the absence of transaction support and data validation, which can be avoided early in the future. The above is the test design of the game program itself. The test of the game plot can be obtained from the planning. Because the preliminary planning stage only gives a generous description of the game plot, there is no design for a specific plot. When the design stage is entered, the plot logic of a game has been fully formed. The plan can provide a detailed design instruction for the plot, which is called the task instruction book, through the task manual, we can design a task test case. For example, if a certain team of tasks is composed of tasks, we can design a complete task test case, in this way, the test may maximize the coverage of all the task logic. If it is a simple task, you can also propose automation requirements, which can be automatically completed by robots.

 

Game testing and development

Development and testing are never considered parallel and must be developed before testing. However, the software development process requires early intervention in testing, however, these contradictions have been well resolved. We use daily compilation, which combines test execution and development, and demonstrates code-test-code-test in the development phase. That is to say, once the program fragment is compiled, it will be tested immediately. In normal cases, unit tests are performed first, but a program segment also requires related integration tests, and sometimes some special tests. In particular, concerning interface testing, such as the combination of game programs, task books, and pictures, we all think that testing in advance is required. Through daily coding, you can join the program fragments that have already been written, some integration tests are formed to effectively reflect the interface priority testing principles. At the same time, because software testing and development are carried out in parallel and the software defect priority modification policy is implemented, there are very few problems that cannot be modified later, in addition, due to the design of test cases in the early stage and the preparation of automated tools, we do not need to invest too much manpower to ensure the product quality of game software, especially the quality of important systems. Since our game programs are constantly improved every day, the integration test is also in progress. When the development enters the final stage, the integration test is also synchronized. There is a principle that, as I mentioned earlier, the main method and structure of the test should be completed in the game design phase, it also needs to be supplemented in the development phase (for example, there will be a corresponding change in game development, or a change in a transfer address, which requires real-time updates ). This method will have an important impact on code-based testing (the development and integration phases), but at that stage, if you do a little more planning and design before execution, it will greatly improve the test efficiency, improve the test results, and facilitate the reuse of test cases and analysis of test data. Therefore, our test plan is formed during planning, it forms a good foundation for subsequent tests.

Integration test phase

Integration testing is a test of the entire system. Due to the parallel test and development in the early stage, the integration test has been basically completed. In this case, you only need to run the system test cases designed in the early stage. Our main focus is on compatibility testing in the integration test. Due to the special nature of the game test and the high compatibility requirements, we adopt the external and internal methods in the same place, we have our own platform lab to build a mainstream hardware and software testing environment. At the same time, we also conduct compatibility analysis on our game software through some professional compatibility testing institutions, let our game software run on more machines.

Game playability Test

Game playability testing is also very important, mainly including four aspects:

* The establishment of the game world, including chat functions, transaction systems, and teams, allows players to interact in the game world.
* Events in the gaming world are driven by tasks.
* Competition and balance in the gaming world.
* The game world culture contains the style and embodiment of the game.

This test is mainly reflected in the game playability. Although we evaluated the playability during planning, it is general, but some specific analysis involves a certain data, for example, the adjustment of PK parameters, the increase of skills, and some other playability enhancement tests require professional players to analyze them. Here, we mainly perform the following three methods:

* Internal testers, all of whom are professional player analysts, have a deep understanding of the game and analyze the four points above during internal testing.
* Using external game media professionals to analyze and introduce the game, this method can achieve both the promotion effect and the purpose of testing. Generally, this method is better.
* A certain number of external players are used to test the peripheral system. They are common players, but they are our primary goal. The main source is the students of large and medium-sized institutions, it mainly tests the playability and ease of use of the game and finds some peripheral bugs.

When the game enters the final stage, it also needs to be tested in the beta version of the application software. This allows more people to participate in the test and test the running status of a large number of players.

Playability testing is an important part of the game. Only players agree that we can succeed.

Performance testing and Optimization

Finally, we should mention performance optimization separately. In the era of standalone edition, performance requirements are not very high, but in the era of online edition, there are two completely different concepts, it mainly includes the following aspects: test the client performance of applications, test the network performance of applications, and test the server performance of applications. Under normal circumstances, the combination of the three aspects is effective and reasonable to achieve comprehensive analysis of system performance and Forecast of bottlenecks. However, there is a principle in the test process that the test is performed when the integration test is completed or near the completion of the test, and the functional points of the test are required to pass, at this time, the first thing you need to optimize is the database or the preparation of the network itself. Only in this way can you avoid the risk of modifying the program. At the same time, performance testing and optimization is a step-by-step process that requires a lot of preliminary work, such as performance requirements and testing tools. However, due to the improvement of preliminary work, these are all completed in the early stage. Here I will only describe in principle.

The principle of database optimization is mainly like this. The first is index optimization. Because index optimization does not require any changes to the table structure, it is the simplest one, without modifying the program, the performance may be improved several times. However, it should be noted that the index is not omnipotent. An unlimited increase will have a great impact on the addition, deletion, and transformation. The second is optimization of tables, views, and stored procedures. However, before analysis, we need to know the optimization goal. The SQL statements in customer behavior are the most executed. Therefore, we must use some SQL tracking and analysis tools, such as sqlprofile, sqlexpert, and so on, this will quickly locate the problem.

With regard to network optimization, what I am talking about here is not the optimization of the network itself, but the optimization of the network communication of the game itself, so it is combined with program optimization, the first problem is to find out. By using monitor and sniff, We can first find out which application occupies a large amount of network traffic. This is also an important issue due to the huge number of online game users. For program performance optimization, the most important thing is to find the function with the longest running time. Only by optimizing it can the performance be greatly improved. I will not describe the specific method in detail.

Overview

Game testing is a new field. It has both the characteristics of Universal Testing and its own characteristics.