Mobile App Special test

Mobile app testing-Special test

Transferred from: http://www.51testing.com/html/58/n-3713758.html

We're doing a manual. Functional TestingAfter that, we also developed some Automated TestingUse cases, and have done Performance TestingAfter that, the test workIt looks more complete. But when our app is installed and used by a large number of users, there are still a lot of questions that we didn't anticipate before, such as: · Crash's question · Device compatibility issues · Problem with excessive flow of traffic · App causes users Mobile PhoneThe problem of too fast power consumption · Instability in different network conditions, such as the problem of white screen and the problems of the above test methods are difficult to find, so here introduced a special test method, including: Compatibility test, flow test, battery test, weak network test, stability test, Security Testingand environment-related tests. First: Compatibility TestThese aspects are typically considered for apps: 1) Operating SystemVersion includes Andoird version, iOS version 2) screen resolution 3) Different manufacturers of ROM 4) network type such as WiFi, 3G, 4G under the function of the situation Second: Flow testIn MovingProduct testing, it is necessary to measure the traffic used by the app, in general, the traffic can be divided from the user's relevance angle: One is the user's operation directly caused by the traffic consumption, the other is the background, that is, the user is not directly used in the case of traffic consumption. Third: Battery testIn the wood battery TechnologyWithout a huge breakthrough, there will always be some bottlenecks in this area, and if some app architectures are poorly designed, or code-odd flaws can lead to higher power consumption, the battery test is also important. Fourth: Weak network testing 　　 Mobile InternetProducts compared to the PC Internet products, there is a feature is the former use of the network is more diverse, in addition to WIF, many times is in the mobile network under the use of mobile network encountered the situation is more complex, such as subways, tunnels, stadiums and so on. So the situation of network instability is relatively easy to happen, in many cases the application of some problems in the complex network situation will be exposed, rather than let users find and complain about these problems, we in the testing phase as far as possible to simulate such network conditions, early detection and repair of these problems. Fifth: Stability TestThe stability of the app is very important to ensure that the basic function is correct, and if an app often flashes or snaps, the user experience can be very damaging, and it is easy to churn out the user in the presence of other competing products. Sixth: Safety testingIncludes security testing of the installation package (ability to decompile code, install package signature, etc.), sensitive information testing, soft keyboard hijacking, account security, data communication security, and more. Server-side SQLInjection testing, XSS cross-site scripting attacks, and more. Seventh: Environment-related testsThere are some flaws in the actual project that I find to be relevant to the operating environment in which the app is located, so when designing tests, consider these scenarios more, such as: 1) Interference testRoger that TelephoneRoger that SMS, receive notification bar message, no power box popup, third-party security software alarm pop-up. 2) Permission testSome users consciously block certain features when they actually use the app. For example, some users feel that allowing an app to access a phone book or an album may reveal privacy, and that the app has no access to the album in the phone. 3) Boundary testThe mobile environment itself also has its boundary conditions that need to be covered in the test. Common scenarios include: too little available storage, no SD card/dual SD card, flight mode, incorrect system time (later than and before Standard time), third-party dependencies (e.g. our app relies on third-party apps, but now third-party apps are not installed or the version is too low for testing). -------------------------------------------------Gorgeous split-line-----------------------------------------------------------------A Ndroid app performance and special testing

• The Android performance test is divided into two categories:
  1, a class of ROM version (System) performance testing
  2. A class of performance testing for app app

• The Android App performance test includes test items such as:
  1. Resource consumption
  2. Memory leaks
  3. Power consumption
  4. Time-consuming
  5. Network traffic consumption
  6, mobile terminal related resource utilization
  7. Frame rate
  8, rendering and so on ....

• Tools:
  (The principle of the tool is based on calling some of the android underlying APIs to get to the values used in the test) GT, etc.

• Test method:
  1. Design Scenario: Manual or automated scene
  2. Get data: The available data include: memory, CPU, power consumption, hprof (memory leak analysis file), response time, etc... Get data with manual or automated scenarios (preferably multiple times and each with a different device) as a final comparative analysis
  3, results analysis: To get the data after the analysis of which modules of data anomalies and then check the code location problem causes

• Several scene states of the Android system:
  1, idle status: Refers to open the app, click the Home button to let the app background run, at this time the application is in a state called idle
  2, medium and Full specification Status: Medium size and full specification refers to the application of the operating time interval varies, medium specification time is longer, full specification time is short

1.1 Memory Piece

Background knowledge:
The memory space requested by C + + is in the native heap, while the memory space requested by Java is in the Dalvik heap. This is because the Android system has a hard limit on the vmheapsize of the Dalvik, and when the Java process requests that the Java space exceeds the threshold, an Oom exception is thrown (this threshold can be 48M, 24M, 16M, depending on the model) and can be used by the ADB shell Getprop | grep dalvik.vm.heapgrowthlimit View this value. That is, the program OMM does not indicate a lack of RAM, but because the program requested a Java heap object that exceeds Dalvik Vmheapgrowthlimit. In other words, oom may occur in the case of sufficient RAM.

The design seems a bit unreasonable, but why does Google do it? This is designed to allow the Android system to allow more processes to reside in memory at the same time, so that the program starts without reloading to memory every time, and can give users a quicker response. Forcing each application to use a smaller amount of memory, the very limited amount of RAM on the mobile device allows more apps to reside there. But there are some big apps that can't stand the vmheapgrowthlimit limit.

In fact, both Dalvik.vm.heapgrowthlimit and dalvik.vm.heapsize are the maximum memory limits for Java virtual machines, and if you do not want Oom to occur when Dalvikheap reaches heapgrowthlimit limits, you need to manifest The application tag declares android:largeheap= "True", stating that an oom will occur when the Dalvik heap is applied to heapsize.

2. Test subkey in memory test:
   1) application memory consumption in idle state
   2) application memory consumption in medium size state
   3) application memory consumption in full spec state
   4) Apply Memory peak condition
   5) Application of memory leaks
   6) Whether the application resides in memory
   7) Memory usage after stress test

4. Memory problem Phenomenon:
   1) Memory jitter
   2) Large memory objects are assigned
   3) Increased memory
   4) Frequent GC

5. Memory Data acquisition:
   1. Various Linux commands (top, free, meminfo ...) ）
   2, through the Dumpsys
   adb shell dumpsys meminfo [pakagename | pid]
   3, through the/system/xbin/procrank tool
   adb shell procrank
   Description
   Vss–virtual Set Size Virtual memory consumption (contains memory consumed by shared libraries)
   Rss–resident Set Size actually uses physical memory (contains memory consumed by shared libraries)
   Pss–proportional Set Size Actual physical memory used (proportional allocation of memory consumed by shared libraries)
   Uss–unique the Set Size process consumes the physical memory alone (does not contain the memory occupied by the shared library) USS is the case that a process begins to have a suspected memory leak, is a program initiated by the virtual memory, once the program process has been killed will be released. But USS needs to be rooted through the phone. Normally there is no root phone we can get PSS. PSS is obtained by the following command:adb shell dumpsys meminfo <Package Name>|grep TOTAL
   4. Procrank provided via Android
   1) First go to Google to get Procrank, Procmem, libpagemap.so three files
   2) Then push file, execute ADB push procrank/system/xbin adb push PROCMEM
   /system/xbin ADB push Libpagemap.so/system/lib
   3) empowering adb shell chmod 6755/system/xbin/procrank adb shell chmod 6755/system/xbin/procmem adb shell chmod 6755/system/lib /libpagemap.so,
   4) in the Open tool record adb shell Procrank |grep packagename >/address/procrank.txt
   5, Activitymanager Getmemoryinfo (Activitymanager.memoryinfo Outinfo) provided by Android (this method is to write a simple app to monitor the time used, light and simple)

1. privatevoidGetMemory()
2. {
3. finalActivityManager activityManager =(ActivityManager) getSystemService(ACTIVITY_SERVICE);
4. ActivityManager.MemoryInfo info =newActivityManager.MemoryInfo();
5. activityManager.getMemoryInfo(info);
6. Log.i(tag,"系统剩余内存:"+(info.availMem >>10)+"k");
7. Log.i(tag,"系统是否处于低内存运行："+info.lowMemory);
8. Log.i(tag,"当系统剩余内存低于"+info.threshold+"时就看成低内存运行");
9. }

6、Memory  Monitor （android studio的插件）  【makedown？？？】4. /proc/meminfo文件里列出的字段解释：

Memtotal: All available RAM sizes. The sum of Memfree:lowfree and Highfree is left unused memory by the system.
Buffers: Used to buffer the file size. Cached: The size of the memory used by the cache memory (equal to DiskCache
Minus SwapCache). Swapcached: Buffered Memory (cache
Memory) with the size of the swap space. The memory that has been swapped out is still stored in the Swapfile and is used to be replaced quickly when needed without having to open the I/O port again.
Active: The size of the buffer or buffer memory paging file in active use, unless it is necessary, will not be moved for his use. Inactive:
In infrequently used buffer or cache memory paging file size, may be used in other ways. Swaptotal: The total size of the swap space. Swapfree:
The size of the swap space that is not being used. Dirty: The amount of memory waiting to be written back to disk. Writeback: The amount of memory that is being written back to disk.
Anonpages: The memory size of the unmapped page. Mapped: Size of mappings such as devices and files. Slab:
The size of the kernel data structure cache can reduce the consumption of application and release memory. Sreclaimable: The size of the slab can be retracted.
Sunreclaim: The size of the Slab is not recoverable (Sunreclaim+sreclaimable=slab).
Pagetables: The size of the index table that manages the paging page of memory. Nfs_unstable: The size of the page table is not stable.

5. android检查内存泄露步骤：

1. Run Monkey for stress test:
adb shell monkey -p cn.microinvestment.weitou --pct-touch 100 --ingore-crashes --throttle 1000 -s 100 -v -v 50
2, monitor the memory value, if there is a large increment exception, save the Hprof file (the Hprof file is a heap snapshot of the Java Virtual Machine) to analyze the command to view the application memory:
adb shell dumpsys meminfo cn.microinvestment.weitou(进程名)
If memory is found to be too large, save the Hprof file: adb shell am dumpheap <进程名> <保存路径>
3, Analysis Hprof file
Use the tool mat to see, first of all, this hprof file is converted to a mat-readable file
There is a Hprof-conv command in the Android SDK tool:
Hprof-conv < original hprof file path > < converted Hprof path >
hprof-conv a.hprof b.hprof
4. Open the converted Hprof file with the Mat tool
General selection Leak Suspects report (SQL statement to query if the object has not been released, if there are multiple identical objects, there will be a memory leak problem)

1.2 CPU Chapter

1. Test subkeys in the CPU test:
   1) Application CPU consumption in idle state
   2) CPU consumption of the application in the medium size state
   3) CPU consumption of the application in full spec state
   4) Application of CPU spikes

2. CPU Data acquisition:
   1) adb shell dumpsys cpuinfo | grep packagename
   2) Top command
   adb shell top -m 10 -s cpu#查看占用cpu最高的前10个程序 (-t displays the process name,-S is sorted by the specified row,-n is refreshed several times before exiting,-D refresh interval,-m displays the maximum number)
   adb shell top | grep PackageName > /address/cpu.txt

1.3 Traffic Chapter

1. Concept:  
   Medium Load: Apply normal operation  
   High load: Apply limit action

2. Test subkey in traffic test:  
   1, apply first start traffic value  
   2, the application background continuous running 2 hours of traffic value  
   3, the application of high-load operation peak traffic  
   4, application of traffic mean at medium load runtime

3. Get traffic data:  
   1, tcpdump+wireshark 
   2,/proc/net/directory of related files  
   Cat/proc/net/dev Get traffic information of the system  
   3. PID:ADB Shell for query application PS | grep tataufo #如: 31002 
   Gets the app's traffic data through the PID: adb shell cat/proc/31002/net/dev 
   (wlan0 represents the WiFi upload download volume ID, Units are bytes can be converted into kilobytes/1024, turn on the phone flight mode and then switch off to initialize the value in Wlan0 0)  
   4. PID:ADB Shell PS for query application | grep tataufo #如: 31002 
   Get uid:adb Shell cat/proc//status  by PID:
   via uid, adb shell cat/proc/net/xt_qtaguid/stats | grep 31002 
   5, through the ADB shell Dumpsys package to obtain the UID information of the application, and before the application is not operational, by viewing:  
   adb shell cat/proc/uid_stat/uid/tcp_rcv 
   adb shell cat/proc/uid_stat/uid/tcp_snd 
   Gets the incoming and outgoing traffic to the beginning of the application, and then we then manipulate the application, and again through the 2 commands above we can get the received and sent traffic to the end of the application, By subtracting and getting the overall flow consumption of the application  
   6, Android code: Android Trafficstats class

1.4 Power Consumption Chapter

1. Test sub-items in power consumption tests:
   1, mobile phone installation target apk before and after the standby power consumption no significant difference
   2, common use scene can enter the standby normally, standby current within the normal range
   3, long-time continuous use of applications without abnormal power consumption phenomenon

2. Power test Method:
   Method One: Software
   1, the use of third-party tools available on the market, such as Jinshan battery steward.
   2, is the self-writing tool, here are generally used in 3 ways:
   1) based on the Powermanager.wakelock provided by Android
   2) more complex, power consumption calculation =cpu consumption +wake lock consumption + data transmission consumption +gps consumption +wi-fi connection consumption
   3) through the ADB shell Dumpsys battery to get
   3, Battery-historian (Google Open source tools)
   Method Two: Hardware
   Generally use a multimeter or power meter Agilent to test, using the Power meter test, the need to make fake batteries to do, some can not plug the battery cell phone needs to be welded for power testing

1.5 GPU Chapter (FPS)

2. Concept:
   Over-rendering: The activity on the interface shows multiple layers and causes
   Frame rate: Screen sliding frame rate
   Frame variance: Slide smoothness of the screen
   **fps:**frames per Second The frame rate is considered coherent according to the physiological structure of the human eye, which is higher than 24 o'clock. For the game screen 30fps is the lowest acceptable, 60fps realistic sense, if the frame rate is higher than the screen refresh rate is wasted. To reach 30fps, each frame takes less than 33 milliseconds.

4. Test subkeys in a GPU test:
   1. Excessive drawing of interface
   2. Screen sliding frame rate
   3, screen sliding smoothness

6. Over-Drawing the test: (Manual Test)
   Turn on display GPU over-drawing in developer options (Debug GPU Overdraw)
   Criteria for Acceptance:
   1. Black pixels are not allowed
   2, do not allow the existence of 4x over-drawing
   3, does not allow the presence of area over the screen 1/4 area of the 3x over-draw (light red area)

8. Screen slide frame rate test:
   Method One:
   1. Check the graphics and view after enabling tracking in the developer options on the phone
   2. Launch the SDK tool Systrace, tick the app under test, click Systrace, set the continuous crawl time in the popup dialog box, and check the GFX and view options under trace Taps
   3. Manual sliding interface can be used to swipe or sweep through the beat, the frame rate data will be saved to the default path, the default name is Trace.html
   4. Copy the trace.html file to the Linux system and convert it by command to generate the Trace.csv file
   grep ' Postframebuffer ' trace.html | Sed-e ' s/. \w//g '-e ' s/:.*$//g '-e ' s/.//g ' > Trace.csv
   5. Calculate the frame rate by opening the file in Excel
   Method Two:
   Hardware method, turn on the high-speed camera, turn on the camera mode, record the video of manual sliding or sweep the test application, and then calculate the result by manual or program number frame method to get the frame rate.

10. Test for screen slide smoothness:
    Method is like frame rate test, the only difference is the difference of the final result calculation formula

12. Capture app frame rate (Android fluency fps test):
    1. Turn on the Mobile developer option, tick the GPU display profile (the system will record the time information that retains the last 128 frames of the image drawn for each interface)
    2. adb shell Dumpsys gfxinfo com.xxx.xxx > Zinfo.txt
    3. Result data analysis
    Profile data in MS section:
    Draw: Time to create a display list (displaylist), time taken by all view object OnDraw methods
    Process:android 2D Rendering Engine The time it takes to execute a display list, the longer the view is, the more time
    Execute: Time to hand a frame image to the synthesizer (compsitor), smaller

14. Other tools:
    Gamebench Testing the FPS tool for Android apps
    Gfxinfo viewing app drawing performance tools

1.6 Response Time Chapter

2. Understand:
   1) The time it takes to start Nativeapp from the Click event to the container (buried point)
   2) Nativeapp full start time (can be obtained by System.log)
   3) Native Call RPC Request method delay time (buried point)
   4) Specific data in the process of RPC request (Req_size Req_header req_time, etc., obtained by burying points)
   5) The specific data returned by the RPC request (res_size Res_header res_time, etc., obtained through the burial point)
   6) The time taken to parse the returned data locally (buried point or TraceView tool available)
   7) Interface rendering time (can be obtained by slow camera or buried point)

4. Android App launch time test
   (Android activity startup process Performance Cutaway: http://www.rudy-yuan.net/archives/59/)

6. The test of the start time of the application is divided into three categories:
   1) first start-the time it takes to apply the first boot
   2) Non-initial start-the time it takes to apply a non-first boot
   3) Application interface switching--time spent switching within the application interface

8. Application startup time Data acquisition:
   1. adb logcat > /address/logcat.txt #所有activity打印的日志
   find “Displayed” /address/logcat.txt > /newaddress/fl.txt#通过日志过滤关键字Displayed来过滤
   find “ActivityName” /newaddress/fl.txt > /newaddress/last.txt#通过activity名来过滤获取所测应用
   By calculating the sum of the last remaining time of the activity
   2, hardware testing, the use of high-speed camera or mobile phone using video method to record the application startup process, and then through the manual number of frames or the number of frames to calculate the start time

2 Weak net Test

1. Test method:
   1, the use of real SIM card, the operator network for testing (mobile wireless testing There are some special bugs must be found in a specific real operator network)
   2, through the proxy way to simulate the weak network environment for testing (Charles hard delay)
   3. Connect the hot spot of simulating weak net to test

2. Hot Spot Simulation Method:
   1) Share hotspot (hard delay) after setting iphone developer mode
   2) Facebook Open source ATC (use Raspberry to build the ACT environment)

3. What the user experience needs to do:
   1) Unified Weak net loading interface style, animation effect, chrysanthemum icon and so on in the application
   2) Unified network error, server error, timeout and other display to the user interface and prompt statements
   3) define clearly the user interaction behavior in each intermediate process

4. Transferred from: https://www.zybuluo.com/defias/note/592309

Mobile App Special test