Understanding of data-driven and keyword-driven ideas in automated testing

When the initial exposure to automated testing, the data-driven and keyword-driven not very understanding, feel a bit of it, is not the parameters and functions of it. In fact, it also embodies the different characteristics of the test and development (mainly refers to the system testing), as well as the technical development of the context of the presentation.

1. The myth of recording/playback

can actually be understood as an automated test script and test case tight coupling, both the difficulty of testing the maintenance of the script, but also with the system testing the user-oriented thinking of the resistance

Every automated test tool vendor will advertise that their tools are easy to use, and testers with no technical background can easily automate all of their tests simply by recording the process of the test and then playing the recorded test script. Such a statement is very irresponsible.

Now let's analyze why automated testing cannot be done solely on record/playback.

The scripts that are created by recording are basically written in a hard-coded script language, and these hard-coded changes are required when the application changes. Therefore, the cost of maintaining these recorded scripts is very high, high to almost unacceptable.

All test scripts must be recorded when the application is performing correctly, and if a defect is found during the recording process. Testers must report to the defect management mechanism and wait until the bug is fixed and the entire recording script can continue. In such cases, the efficiency is very low if you rely solely on the recording script to test it.

At the same time, these well-recorded scripts are not very reliable and can be played directly even when the application is completely unchanged, or it may be impossible to execute due to some unexpected situation. If the tester uses the wrong scripting language when recording the script, the script must be re-recorded.

In summary, it may seem easy to create automated test scripts in a recorded way, but you will actually encounter the following problems: ① testers mostly do not have the technical background, it is difficult to fully grasp the testing tools; ② application must be stable to start recording test scripts; ③ recorded test scripts are tightly coupled to test data, and ④ maintenance of automated test scripts is very expensive.

2. Data-driven automated testing framework

"What is data-driven?" A large part of the people must think that data driven is to write something that needs to be parameterized in Excel, and then call it when running a script. If I told you that this is not data-driven, but rather a higher-level parameterization, you would be surprised. Now let me explain: why first call data-driven, then it must have a driving meaning, such as you can use Excel to control the test of the business flow. The answer is no. So how do you drive it? So we put the test data in a separate file, just a high-level argument. While data-driven, you must have data to control the business flow of the test. For example, you test a Web application, there are many pages, you can use a data to control which page each time you are working (that is, through the data to navigate to the corresponding page). It is a low-level version of the keyword driver, he controls the function level, and the keyword is control action level. So data-driven should be able to control the entire test . "

In some complex test cases, the same use case contains a lot of test procedures, in which different test processes use different test input data, at this time the input of the test data is not only the input of the parameters, but also the input of the control field of the business process (which can be understood as logical parameters). This situation will further reflect the data-driven implications.

Data-driven automated testing is a test method for the tight coupling between the above development and test. Establish a loosely coupled relationship between testing and development by establishing a metadata mapping table that tests and develops the defined software metadata. Whether the tester modifies the test script or the developer modifies the software, only the metadata mapping table needs to be modified to meet the test and development synchronization. In this way, you can reduce the workload of test script debugging, and better implement automated testing.

What is a data-driven automated testing framework

A data-driven automated testing framework is a framework that reads input, output test data from a data file (such as an ODBC source file, Excel file, CSV file, ADO object file, and so on) and then passes the variable into a pre-recorded or hand-written test script. Where these variables are used as pass-through (input/output) to validate the application's test data. In this process, the data file reads, the test status and all the test information are written into the test script; The test data is only included in the data file, not in the script, the test script is just a "driver", or a mechanism for transmitting data.

Data-driven scripting

Data-driven scripts are those that are associated with an application. These scripts are recorded or manually written into the private language of the automation tool, and then the variables are assigned appropriate values as input to the test data. These variables serve as a medium for some key application inputs, enabling scripts to drive applications through external data.

1) variable data, hard-coded component flags

These data-driven scripts often contain hard-coded data, sometimes a very fragile identification string in some window components. When this occurs, the script is easily lost due to changes in the program.

2) highly technical, repetitive test design

Another common feature of data-driven scripting is that all efforts to test design are ultimately reflected in the scripting language of automation tools, or copied into manual and automated test scripts. This means that each person involved in the development or execution of an automated test must be proficient in the programming language of the test environment and automation tools.

Advantages and Disadvantages

1) Advantages: ① can build test scripts synchronously while application development, and only need to modify the script of the business function part when application function changes, ② use the model design, avoid duplicate script, reduce the cost of establishing or maintaining the script; ③ test input data, verify data and expected test results are separate from script , stored in another data file, to facilitate the testers to modify and maintain; ④ the return value through the judgment function is "True" or "False", which can be used for error handling, which increases the robustness of the test script; ⑤ Automated test developers create data-driven test procedures and testers create test data; ⑥ collects test results during testing and represents test results in the context of input data, which simplifies the analysis of manual results.

2) Cons: ① must be very proficient in the scripting language of the automated test tools; ② each script will correspond to multiple data files, which need to be stored in their respective directories according to the functional categories of the script, adding complexity to the use of the ③ testers in addition to maintaining the appropriate test plan based on specific test data. It is also necessary to write this data to different data files of various requirements; ④ when editing a data file, you must pay attention to the transport format required by the test script, or you will produce an error when processing the script. Automated test frameworks that rely on data-driven scripting are simpler and faster to implement if they are maintained by a dedicated technician. However, maintenance work is difficult, and this data-driven pattern needs to be maintained, so that even prolonged maintenance can lead to failure. 3. Keyword-driven automated testing

The source of the keyword drive is very natural, from the object-oriented approach, the same business logic will naturally be written into a class or function as a keyword to be called by different test scripts. When the test framework developed into all of the test processes can be written by the function and the combination of the class has evolved to a keyword-driven advanced stage, this time the development of test cases has become a combination of test data and keywords, and this combination of work to simplify for everyone is familiar with the form of filling tasks, This ultimately results in an overall test driven by data and keywords.

In the keyword-driven framework, you can create some keywords and associated methods and functions. Then you create a library of functions that contains the logic to read the keyword and then invoke the associated action.

Keyword-driven automated testing (also known as table-driven test automation) is a variant of data-driven automation testing that supports tests consisting of different sequences or multiple different paths. It is an application-independent, automated framework that is also suitable for manual testing while processing automated tests. The keyword-driven automated test framework is built on data-driven methods that contain directives (keywords), not just data. These tests are developed into data tables that use keywords that are independent of the automation tools that perform the tests. Keyword-driven automated testing is an effective improvement and complement to data-driven automated testing.

This model of automated testing consists mainly of core data-driven engines, component functions, support libraries, and application mapping tables. The automated test begins with the initial script, which passes the high-level test table to the high-level drive, which, in the process of processing these tables, calls the middle-level drive when it encounters the middle test table, and the middle drive handles the middle table in a similar way. When a low-level drive handles a low-level table, it tries to keep the app in sync with the test. When a lower-level drive encounters a low-level keyword component on a component, it determines the type of the component and invokes the corresponding component function module to handle the command operation. All of these elements depend on the information in the mapping table, which is the bridge between the automated test model and the application being tested. The support library mainly completes some functions of file processing, logging and mail sending, etc.