Baidu Mobile application Quality Management and data analysis (mobile testing framework)

Https://www.tuicool.com/articles/qUFNRj6

Mobile internet era, beer and Skittles, office payment and so on all kinds of applications are in the mobile app market, in traffic for the King today, mobile app although the competitive prospect of a good, but there are developers and operators very headache things, That's the lack of solutions and operational strategy guidance that runs through the complete mobile lifecycle test. How to quickly process fragmented testing of mobile applications and leverage Big data analytics to guide mobile operations strategies is a lasting topic.

76 period Baidu Technology salon, focusing on mobile application "product quality" and "data value", from Baidu's technical experts from the functional testing, compatibility testing, safety testing and other aspects, for everyone to share the complete mobile life cycle testing solutions, and how to through a variety of data analysis, Insight into user behavior to guide the development of app operations strategies. Baidu MTC Mobile Cloud Testing Center

The labor cost is rising year in, the Test team is understaffed, the test system is high-cost, the testing equipment is difficult to match ... These are the current mobile testing common dilemma and challenges, Baidu MTC Mobile Cloud Testing Center, is the industry's leading mobile application one-stop testing service platform, covering mobile app development complete life cycle of a variety of testing solutions, Baidu MTC Senior product manager Zhangxiaoxiao, The site shared the Baidu MTC how to help improve the quality of the app.

Mobile application testing kind of fragmented, but in general can be divided into two categories: one is more dependent on the technology and equipment of automated testing , we often do compatibility testing, security testing, remote real-time debugging, etc. are automated testing, for such testing, Baidu provides thousands of debugging the real machine, Covering 800 mainstream models, basically to ensure that the market for real-world coverage and mainstream testing solutions, the other is more dependent on the experience of the engineers of the manual testing , including mobile application use case testing, bug exploration testing, a large user experience, such tests can not rely entirely on test equipment, need to intervene manually , Baidu Survey Community has 10000 + of test specialists and more than 400 test experts, covering 300+ different cities and a variety of network environment, can be relatively fast to help developers solve the test process encountered in the uncontrolled problem.

As an example of a developer's concern for compatibility testing, Baidu's test specialist will first write use cases and scripts, and then send the test package to the actual thousands of phones to install, start, run, and Uninstall, using automated tools that will crash, rewind, Compatibility issues such as UI adaptation are recorded and analyzed, most of which are fed into the test report. The entire test process, the user only need to test requirements and test package to provide to Baidu, the other test process and test reports are provided by Baidu.

Manual testing, in addition to the basic auxiliary equipment and testing environment, the more sophisticated is the experience and ability of testing engineers, Baidu community set up a relatively perfect training system for the testing Commissioner, through the college capacity training, MTC technical support, precision Personnel Collaborative Division of labor and continuous quality control four dimensions, Ensure the accuracy and accuracy of manual testing. The following figure is the overall workflow of the Baidu MTC manual testing, in addition to the regular test case design and execution, Baidu also set up a triple audit mechanism, through internal and external connectivity audit, the test results to further guarantee.

According to Zhangxiaoxiao, in addition to the traditional app testing, from the second half of 2016, Baidu has been trying to support hardware testing, such as the more popular smart bikes, and even the need for field measurement of intelligent buildings and smart home testing, Baidu has better support and cases. At the same time, Baidu hopes to join forces with the entire testing community to create smarter, more comprehensive testing solutions. Baidu MTC Private Cloud Solutions

Some mobile tests have high data privacy requirements, especially for some of the new features not yet online, information privatization, custom service is the first requirement for some enterprises to do third-party testing, so, Baidu MTC developed an enterprise version of the private cloud solution, through the original localization mobile app testing, For the enterprise quickly set up an intranet real machine Automation test laboratory, to achieve the test data private, equipment remote management functions, to help enterprises to transform the Internet. Baidu's chief mobile cloud testing solutions expert Li Ming, for the audience of the scene of the full explanation of the Baidu MTC Enterprise private cloud solutions.

2012 Baidu World Congress, Baidu formally released MTC Mobile Cloud testing products, built a test solution based on UI Automation testing, with the advancement and development of technology, Baidu gradually integrates performance testing, safety testing, compatibility testing and other dimensions of automated testing, and began to provide private automated testing services to the enterprise market, in 2016, Baidu MTC Private Program 1.0 officially launched, causing great concern in the industry. In order to help test efficiency, Baidu MTC this year's private program 2.0, in the 1.0 version, enriched more testing tools, including threads machine debugging, script recording playback and some third-party tool interface, set up a more professional set of automated testing tools. The following figure is the system architecture diagram for the MTC Private Cloud Scenario 2.0, where red is the new feature and interface.

The entire MTC2.0 version provides seven large test resolution capabilities in automated testing, including script functional testing, compatibility testing, security testing, performance testing, network testing, traversal testing, and script recording. Each test function corresponds to a different function and test, but it is indispensable. script function test: refers to the custom script in the cloud real-machine playback, support Robotium, Appium and Uiautomator and other mainstream automated testing framework; compatibility test: mainly in selected models installed, start, monkey, uninstall and other links To verify the compatibility of different phone brands, operating system versions, screen resolutions, security testing: in the file detection of Dex, res file is the existence of source code, resource file theft, replacement and other security issues, vulnerability scanning, backdoor detection, etc. performance test: Simulate the typical use of the scene, accurate access to the application of multi-dimensional performance parameters, covering the start-up length, power, Flow, CPU, memory, such as 9 performance indicators; network test: support in different network environment compatibility test, including 2G, 3G, 4G, WiFi and other network environment simulation; Traversal test: According to the tree structure, automatically traverse the app control, covering all kinds of abnormal branches, traversing all functions, small probability bug is not covered; script recording: includes functional test script recording, editing, playback, debugging, support Robotium, Testing frameworks such as Appium and Uiautomator; app Automation Test Private deployment scenario practices

Speech Finally, Li Ming with the Baidu MTC for an authoritative scientific research and testing institutions for the deployment of automation solutions for example, explains how Baidu MTC to achieve the privatization of the program deployment.

"Customer pain Point" to undertake a large number of scientific research and testing tasks, the company's various types of mobile application construction has a short development cycle, version Iteration update fast, the lack of regression testing and business scenario simulation test, as mobile devices and operating systems continue to upgrade and development, existing mobile applications exposed function is not perfect, performance stability is insufficient , smart device compatibility is not strong, security vulnerability protection is not enough problems.

"Solution" Baidu MTC based on the private cloud mobile app testing solution, the use of Central computer room + local Computer room Distributed deployment method, the project, users, applications, script equipment centralized scheduling management, resource sharing, security and secrecy. At the same time, Baidu MTC provides a complete set of testing standards for the entire group of mobile app testing requirements, including functional testing, compatibility testing, performance testing, traverse testing, safety testing, weak network testing, limit testing, remote real-time debugging, etc., in order to unify and standardize the testing process and quality. "one box multi-use" Android Universal script test Solution

Android commonly used script testing framework mainly Uiautomator, Robotium, Appium, etc., but each has its own drawbacks, hybrid script testing based on the packaging of Uiautomator, combined with image recognition and OCR technology, can fully support a variety of application scenarios. Baidu Advanced Test Development engineer Hongzhi far, around the common script testing, and other topics to share the "one box multi-use" Android Generic Script test solution.

How to choose a test script that supports various scenarios

In a routine automated testing process, the worker needs to write the script first and then test it locally, and then upload it to the automated test platform to collect the results. Before the local test, the tester needs to choose an appropriate test framework to script, currently, the industry's main framework contains Appium, Robotium, Uiautomator three, but these frameworks have their pros and cons, the picture below is the pros and cons of the common frame, We can clearly see the performance of these three frameworks and the applicable scenarios.

first of all, the Robotium framework can be used for black-box testing, and control recognition is done better, a set of plug-in mechanism, you can achieve other automated testing. However, its shortcomings are also very obvious, the native does not support cross-process, need to re-sign the APP, which will cause some tests can not run, this is more deadly. The second Appium, which supports cross-platform, including Android and iOS , and the supported Android version is very comprehensive, the high version, the lower version have various methods to solve. But there is also a fatal disadvantage, lack of stability, the importance of stability for the test environment is self-evident, therefore, this framework is not entirely suitable. finally is Uiautomator, is the Google development frame, itself also has many advantages. First it is native, and one of the superior qualities is the support for cross-process interface acquisition. However, the uiautomator itself does not support Android 4.1 or less, and is poorly supported for non-native interfaces.

After comparison, Baidu chose the native interface to support a better uiautomator framework, at the same time, in order to ensure the performance of the operation, Baidu on the basis of this framework to do a series of optimization practices. Optimization Practice One: script execution is susceptible to interference

After choosing to complete the test script, the test process will still encounter a lot of problems, where the most common is the authority of the pop-window interference, the program in the process of a sudden interference and then the script hangs out, this problem is very common also let testers very headache.

There are two main types of pop-up jamming, the pop-up window and the out-of-process pop-up window in process. For in-process pop-up Windows Baidu's solution is to do a package on the basis of uiautomator, detect whether there is a window logic, and then through the script logic to clean up; out-of-process pop-up windows require inclusion schemes, such as uiautomator to support cross-process operations, so that out-of-process operations can be directly Uiautomator to identify and give the click-Off. Optimization Practice Two: Non-Native scenario script support Poor

General non-native scenes including WebView, games, Popupwindow, and so on, these scenes are a big feature of the interface element recognition is more difficult, such as the security keyboard like Popupwindow, it adds a layer of Window on the page, the identification process of focus on the security keyboard, Causes the security keyboard element to be completely inaccessible and recognizable.

Whether it is a common element identification scheme or the current popular image recognition scheme, there are some drawbacks in operation and performance, Baidu combines these two programs, on the PC side is a unified API, using hybrid script test solution. Some image-related operations are performed directly on the PC, the element identification can be encapsulated, and an empty test script is written on the device side, and the empty script is primarily exposed to the HTTP sever service, which can be accessed directly through the APP. After the PC-side access encapsulation, sent to the device side, the device side of the operation of a number of analysis, and then to the scavenger SDK, finally to Uiautomator, to achieve a complete set of device-side element identification, so that the formation of a more comprehensive test process. Optimization Practice Three: Scripting is time-consuming

At present, the general script is written mostly manually, in the case of heavy testing tasks may occur in response to the problem, this part of the problem can be solved by recording tools, recording tools to record the test script, automatically generate scripts, improve writing efficiency.

The image above is the overall solution of the Baidu recording tool, the overall framework is divided into three parts, Android device, recording tool server and front-end interface. The main work is focused on the service side, the front-end interface includes the module of operation, image display, code generation which are all on the service side. The service side mainly provides several functions, the interface parsing, generates the corresponding element path, the front-end interface through the Click Operation comes through, according to the DOM tree structure to get the element path, generates the unique code, returns to the front side, the front end gets the ready code, the script compiles executes. In this way, the entire script recording process is implemented. How Baidu Advanced analytics platform helps refine data analysis and marketing

A successful product, in addition to stable, safe system performance, but also need a sophisticated data analysis to guide the operation, to seize the user's psychology, to carry out a wide range of promotion and application. Baidu Senior Product manager Shahai, on how to use Baidu statistical advanced analysis of sophisticated data analysis and marketing launched in-depth sharing.

Shahai introduced, Baidu Statistics is the largest website traffic analysis tool, can tell users how to find and browse the user's website, what has been done on the site. This year, Baidu Statistics introduced a new advanced analysis platform, the introduction of cross-analysis, Baidu Big Data user understanding and other functions. Through more in-depth analysis, enterprises can deeply understand users and products, so as to improve the experience of visitors in product application, and carry out accurate promotion and marketing.

The image above is the overall functional framework of the advanced analysis platform of Baidu statistics, from bottom to top, the bottom is the data realization layer, relying on the big data of the online comprehensive processing technology, combining behavior data, the user's intention to dig; The penultimate layer is the data access layer, Baidu uses no buried technology, can reduce the project entry threshold, At the same time provide a variety of API access services to adapt to the needs of enterprise personalized monitoring and analysis; The bottom third is the core data analysis layer, providing data analysis capabilities such as common content filtering, multidimensional custom analysis, funnel, behavioral flow, retention analysis, and intelligent alerts with exclusive attribution analysis The top layer is the actual application layer, in addition to daily monitoring, A/b Test, Baidu also provides a unique user understanding, search Word statistics and other functions, while supporting the metadata API export, has a strong engineering scalability, to help users better use this information to customize the operation strategy.

Subsequently, Shahai a practical case to explain how Baidu through the advanced analysis platform to support customers for precision marketing. The figure below is Baidu statistics for a customer to provide marketing data analysis, customers hope that through the understanding of the crowd and reach a better marketing campaign, in this demand, Baidu statistics provides two capabilities, one is through the user online behavioral insight Real active user distribution and the properties of these active users The second is to know the precise user distribution, guide the planning of offline activities, and then through the data to organize the effect of the activity and user feedback on the activity, through the line and the same step-off approach, to achieve the precise positioning, management and marketing of the crowd.

In general, Baidu Statistics advanced analysis platform is through the entire user goals of the Big Data portrait, intent analysis, and then monitor the effect of channel delivery, and finally do user retention analysis, mobile heat force, such as marketing strategy optimization, to achieve the entire market expansion of the choice of quality channels, and then to the scene of support, This is also a lot of customers to actually implement products and applications of the way. Baidu Mobile Statistics revision heavy on-line

Before explaining how Baidu statistics use big data to help customers do accurate marketing, then, for the function and interface with the traditional PC site mobile app, Baidu is how to help customers do the statistics and analysis of app data. Baidu Mobile Statistics product leader Liang, from data efficient collection, data intelligence analysis and data to increase three latitude, sharing how to help mobile operators to achieve user insight and refinement of operations.

Talking about the biggest pain point of data collection, efficiency is well deserved. Repeated communication, low effectiveness of the project, and a lengthy version of the update process will greatly stretch the data collection process, consuming most of the effort of the operator. In response to this problem, Baidu Mobile statistics launched a visual buried point solution, the so-called visual embedding point, refers to the app access to all the embedded point of the SDK, you can implement all the standard components of the app embedded point, while the user in sync with the app interface computer-side circle want to monitor the event, the event buried configuration file directly issued , then you can see the data of circle events at the front end, so that the development cost of the technology is greatly reduced and the efficiency of the buried point is improved.

How to accurately measure data analysis is an important premise of operational strategy guidance, Baidu Mobile statistics in this respect in addition to provide regular tracking reports, but also provides more flexibility, latitude indicators more comprehensive custom reporting function. But for how the data can increase this topic, Baidu uses the user to divide the way to help the customer to find the product value point, namely according to the user's certain behavior characteristic first divides the crowd, then makes the accurate marketing and the transformation. Trading Full-link data-driven enterprise revenue growth

Growth intelligence is the use of modern big data and AI technologies to efficiently identify and solve business problems to achieve rapid, quantifiable growth in business revenue. Xiasumin, the big data director from ping++, was the last to share a detailed explanation of how to drive corporate revenue growth with trading full-link data.

Before talking about revenue growth, Xiasumin a very important concept in the field of marketing-user life cycle value-for live audiences. The following diagram is a flow chart of the user life cycle value of a mobile app. When a new user enters the app, the user's natural feature is a high risk of loss, and loyalty is very low, if it is a commodity class app, as long as the completion of a payment transaction, the risk of loss will fall to a relatively low level, but still maintain a relatively low degree of loyalty. If the user does not complete the next transaction for a long period of time, due to the low loyalty attribute, it will lead back to the high risk of loss, if the status persists for too long, the user may be permanently lost.

Therefore, to retain users, you need to keep users at a low risk of loss. So, it's important to increase user loyalty. Xiasumin on-site shared four solutions to improve "growth intelligence": quantitative objectives, strategy development, data opening and effect monitoring. Quantitative goal is to analyze the user's loyalty, the loss of what is the risk, and then to the user accurate grouping, in the data through the link, through the combination of Baidu mobile statistics obtained by the app behavior data, user portrait data and ping++ transaction data, to achieve from the behavioral data to the transaction data of the full link closed loop, The itinerary data is complementary. To guide the exploration of user behavior characteristics, analysis of the user's real needs, the development of user marketing strategy to facilitate transactions, and finally the effect of monitoring and tracking, in order to optimize and improve in time.

After that, Xiasumin has explained how to use big data analysis to improve enterprise revenue growth by providing a specific case for a lost user recovery strategy developed by a car aftermarket company. First quantify the target, the user portrait and the recent analysis of consumer intentions, analysis found in the child-rearing stage of the population accounted for 58%, and then set up a user strategy, organize a city around the parent-child self-driving tour activities, the user registration, the site through promotional products, improve marketing volume. At the same time, operators can also use on-site communication, improve the relationship with customers, enhance user loyalty and conversion rate.