Android app Performance and special testing (personal grooming)

移动测试, Android测试 ,APP测试  

Android article 1. Performance testing

• The Android performance test is divided into two categories:
  1, a class of ROM version (System) performance testing
  2. A class of performance testing for app app

• The Android App performance test includes test items such as:
  1. Resource consumption
  2. Memory leaks
  3. Power consumption
  4. Time-consuming
  5. Network traffic consumption
  6, mobile terminal related resource utilization
  7. Frame rate
  8, rendering and so on ....

• Tools:
  (The principle of the tool is based on calling some of the android underlying APIs to get to the values used in the test) GT, etc.

• Test method:
  1. Design Scenario: Manual or automated scene
  2. Get data: The available data include: memory, CPU, power consumption, hprof (memory leak analysis file), response time, etc... Get data with manual or automated scenarios (preferably multiple times and each with a different device) as a final comparative analysis
  3, results analysis: To get the data after the analysis of which modules of data anomalies and then check the code location problem causes

• Several scene states of the Android system:
  1, idle status: Refers to open the app, click the Home button to let the app background run, at this time the application is in a state called idle
  2, medium and Full specification Status: Medium size and full specification refers to the application of the operating time interval varies, medium specification time is longer, full specification time is short

1.1 Memory Piece

Background knowledge:
The memory space requested by C + + is in the native heap, while the memory space requested by Java is in the Dalvik heap. This is because the Android system has a hard limit on the vmheapsize of the Dalvik, and when the Java process requests that the Java space exceeds the threshold, an Oom exception is thrown (this threshold can be 48M, 24M, 16M, depending on the model) and can be used by the ADB shell Getprop | grep dalvik.vm.heapgrowthlimit View this value. That is, the program OMM does not indicate a lack of RAM, but because the program requested a Java heap object that exceeds Dalvik Vmheapgrowthlimit. In other words, oom may occur in the case of sufficient RAM.

The design seems a bit unreasonable, but why does Google do it? This is designed to allow the Android system to allow more processes to reside in memory at the same time, so that the program starts without reloading to memory every time, and can give users a quicker response. Forcing each application to use a smaller amount of memory, the very limited amount of RAM on the mobile device allows more apps to reside there. But there are some big apps that can't stand the vmheapgrowthlimit limit.

In fact, both Dalvik.vm.heapgrowthlimit and dalvik.vm.heapsize are the maximum memory limits for Java virtual machines, and if you do not want Oom to occur when Dalvikheap reaches heapgrowthlimit limits, you need to manifest The application tag declares android:largeheap= "True", stating that an oom will occur when the Dalvik heap is applied to heapsize.

2. Test subkey in memory test:
   1) application memory consumption in idle state
   2) application memory consumption in medium size state
   3) application memory consumption in full spec state
   4) Apply Memory peak condition
   5) Application of memory leaks
   6) Whether the application resides in memory
   7) Memory usage after stress test

4. Memory problem Phenomenon:
   1) Memory jitter
   2) Large memory objects are assigned
   3) Increased memory
   4) Frequent GC

5. Memory Data acquisition:
   1. Various Linux commands (top, free, meminfo ...) ）
   2, through the Dumpsys
   adb shell dumpsys meminfo [pakagename | pid]
   3, through the/system/xbin/procrank tool
   adb shell procrank
   Description
   Vss–virtual Set Size Virtual memory consumption (contains memory consumed by shared libraries)
   Rss–resident Set Size actually uses physical memory (contains memory consumed by shared libraries)
   Pss–proportional Set Size Actual physical memory used (proportional allocation of memory consumed by shared libraries)
   Uss–unique the Set Size process consumes the physical memory alone (does not contain the memory occupied by the shared library) USS is the case that a process begins to have a suspected memory leak, is a program initiated by the virtual memory, once the program process has been killed will be released. But USS needs to be rooted through the phone. Normally there is no root phone we can get PSS. PSS is obtained by the following command:adb shell dumpsys meminfo <Package Name>|grep TOTAL
   4. Procrank provided via Android
   1) First go to Google to get Procrank, Procmem, libpagemap.so three files
   2) Then push file, execute ADB push procrank/system/xbin adb push PROCMEM
   /system/xbin ADB push Libpagemap.so/system/lib
   3) empowering adb shell chmod 6755/system/xbin/procrank adb shell chmod 6755/system/xbin/procmem adb shell chmod 6755/system/lib /libpagemap.so,
   4) in the Open tool record adb shell Procrank |grep packagename >/address/procrank.txt
   5, Activitymanager Getmemoryinfo (Activitymanager.memoryinfo Outinfo) provided by Android (this method is to write a simple app to monitor the time used, light and simple)

1. private void GetMemory() 
2. {
3. final ActivityManager activityManager = (ActivityManager) getSystemService(ACTIVITY_SERVICE); 
4. ActivityManager.MemoryInfo info = new ActivityManager.MemoryInfo(); 
5. activityManager.getMemoryInfo(info); 
6. Log.i(tag,"系统剩余内存:"+(info.availMem >> 10)+"k"); 
7. Log.i(tag,"系统是否处于低内存运行："+info.lowMemory);
8. Log.i(tag,"当系统剩余内存低于"+info.threshold+"时就看成低内存运行");
9. }

1. 6. Memory Monitor (plugin of Android studio) "Makedown???" 】
3. 4. The fields listed in the/proc/meminfo file are explained:

Memtotal: All available RAM sizes. The sum of Memfree:lowfree and Highfree is left unused memory by the system.
Buffers: Used to buffer the file size. Cached: The size of the memory used by the cache memory (equal to DiskCache
Minus SwapCache). Swapcached: Buffered Memory (cache
Memory) with the size of the swap space. The memory that has been swapped out is still stored in the Swapfile and is used to be replaced quickly when needed without having to open the I/O port again.
Active: The size of the buffer or buffer memory paging file in active use, unless it is necessary, will not be moved for his use. Inactive:
In infrequently used buffer or cache memory paging file size, may be used in other ways. Swaptotal: The total size of the swap space. Swapfree:
The size of the swap space that is not being used. Dirty: The amount of memory waiting to be written back to disk. Writeback: The amount of memory that is being written back to disk.
Anonpages: The memory size of the unmapped page. Mapped: Size of mappings such as devices and files. Slab:
The size of the kernel data structure cache can reduce the consumption of application and release memory. Sreclaimable: The size of the slab can be retracted.
Sunreclaim: The size of the Slab is not recoverable (Sunreclaim+sreclaimable=slab).
Pagetables: The size of the index table that manages the paging page of memory. Nfs_unstable: The size of the page table is not stable.

5. android检查内存泄露步骤：

1. Run Monkey for stress test:
adb shell monkey -p cn.microinvestment.weitou --pct-touch 100 --ingore-crashes --throttle 1000 -s 100 -v -v 50
2, monitor the memory value, if there is a large increment exception, save the Hprof file (the Hprof file is a heap snapshot of the Java Virtual Machine) to analyze the command to view the application memory:
adb shell dumpsys meminfo cn.microinvestment.weitou(进程名)
If memory is found to be too large, save the Hprof file:adb shell am dumpheap <进程名> <保存路径>
3, Analysis Hprof file
Use the tool mat to see, first of all, this hprof file is converted to a mat-readable file
There is a Hprof-conv command in the Android SDK tool:
Hprof-conv < original hprof file path > < converted Hprof path >
hprof-conv a.hprof b.hprof
4. Open the converted Hprof file with the Mat tool
General selection Leak Suspects report (SQL statement to query if the object has not been released, if there are multiple identical objects, there will be a memory leak problem)

1.2 CPU Chapter

1. Test subkeys in the CPU test:
   1) Application CPU consumption in idle state
   2) CPU consumption of the application in the medium size state
   3) CPU consumption of the application in full spec state
   4) Application of CPU spikes

2. CPU Data acquisition:
   1)adb shell dumpsys cpuinfo | grep packagename
   2) Top command
   adb shell top -m 10 -s cpu#查看占用cpu最高的前10个程序 (-t displays the process name,-S is sorted by the specified row,-n is refreshed several times before exiting,-D refresh interval,-m displays the maximum number)
   adb shell top | grep PackageName > /address/cpu.txt

1.3 Traffic Chapter

1. Concept:
   Moderate load: Apply normal operation
   High load: Application limit operation

2. Test sub-items in the traffic test:
   1. Apply first start traffic value
   2, the application background continuous running 2 hours of traffic value
   3. Application of high-load operation peak flow
   4, the use of medium-load operating flow mean

3. Get Traffic data:
   1, Tcpdump+wireshark
   2,/proc/net/directory of related documents
   Cat/proc/net/dev Obtaining traffic information for the system
   3. PID:ADB Shell PS for Query application | grep tataufo #如: 31002
   Get the app's traffic data through the PID: adb shell Cat/proc/31002/net/dev
   (Wlan0 represents the WiFi upload download volume identification, the unit is bytes can/1024 converted into KB, turn on the phone flight mode and then turn off the value in the Wlan0 can be initialized 0)
   4. PID:ADB Shell PS for Query application | grep tataufo #如: 31002
   Get uid:adb shell Cat/proc//status by PID
   Obtained by UID: adb shell cat/proc/net/xt_qtaguid/stats | grep 31002
   5. Use the ADB shell Dumpsys package to obtain the UID information for the app, and then, before you operate the app, view:
   ADB Shell CAT/PROC/UID_STAT/UID/TCP_RCV
   ADB Shell cat/proc/uid_stat/uid/tcp_snd
   Get the incoming and outgoing traffic to the beginning of the application, and then we then operate the application, again through the above 2 commands can get to the end of the application to receive and send the traffic, by subtracting and getting the overall application of traffic consumption
   6, Android code: Android Trafficstats Class

1.4 Power Consumption Chapter

1. Test sub-items in power consumption tests:
   1, mobile phone installation target apk before and after the standby power consumption no significant difference
   2, common use scene can enter the standby normally, standby current within the normal range
   3, long-time continuous use of applications without abnormal power consumption phenomenon

2. Power test Method:
   Method One: Software
   1, the use of third-party tools available on the market, such as Jinshan battery steward.
   2, is the self-writing tool, here are generally used in 3 ways:
   1) based on the Powermanager.wakelock provided by Android
   2) more complex, power consumption calculation =cpu consumption +wake lock consumption + data transmission consumption +gps consumption +wi-fi connection consumption
   3) through the ADB shell Dumpsys battery to get
   3, Battery-historian (Google Open source tools)
   Method Two: Hardware
   Generally use a multimeter or power meter Agilent to test, using the Power meter test, the need to make fake batteries to do, some can not plug the battery cell phone needs to be welded for power testing

1.5 GPU Chapter (FPS)

2. Concept:
   Over-rendering: The activity on the interface shows multiple layers and causes
   Frame rate: Screen sliding frame rate
   Frame variance: Slide smoothness of the screen
   **fps:**frames per Second The frame rate is considered coherent according to the physiological structure of the human eye, which is higher than 24 o'clock. For the game screen 30fps is the lowest acceptable, 60fps realistic sense, if the frame rate is higher than the screen refresh rate is wasted. To reach 30fps, each frame takes less than 33 milliseconds.

4. Test subkeys in a GPU test:
   1. Excessive drawing of interface
   2. Screen sliding frame rate
   3, screen sliding smoothness

6. Over-Drawing the test: (Manual Test)
   Turn on display GPU over-drawing in developer options (Debug GPU Overdraw)
   Criteria for Acceptance:
   1. Black pixels are not allowed
   2, do not allow the existence of 4x over-drawing
   3, does not allow the presence of area over the screen 1/4 area of the 3x over-draw (light red area)

8. Screen slide frame rate test:
   Method One:
   1. Check the graphics and view after enabling tracking in the developer options on the phone
   2. Launch the SDK tool Systrace, tick the app under test, click Systrace, set the continuous crawl time in the popup dialog box, and check the GFX and view options under trace Taps
   3. Manual sliding interface can be used to swipe or sweep through the beat, the frame rate data will be saved to the default path, the default name is Trace.html
   4. Copy the trace.html file to the Linux system and convert it by command to generate the Trace.csv file
   grep ' Postframebuffer ' trace.html | Sed-e ' s/. ]\w//g '-e ' s/:.*$//g '-e ' s/.//g ' > Trace.csv
   5. Calculate the frame rate by opening the file in Excel
   Method Two:
   Hardware method, turn on the high-speed camera, turn on the camera mode, record the video of manual sliding or sweep the test application, and then calculate the result by manual or program number frame method to get the frame rate.

10. Test for screen slide smoothness:
    Method is like frame rate test, the only difference is the difference of the final result calculation formula

12. Capture app frame rate (Android fluency fps test):
    1. Turn on the Mobile developer option, tick the GPU display profile (the system will record the time information that retains the last 128 frames of the image drawn for each interface)
    2. adb shell Dumpsys gfxinfo com.xxx.xxx > Zinfo.txt
    3. Result data analysis
    Profile data in MS section:
    Draw: Time to create a display list (displaylist), time taken by all view object OnDraw methods
    Process:android 2D Rendering Engine The time it takes to execute a display list, the longer the view is, the more time
    Execute: Time to hand a frame image to the synthesizer (compsitor), smaller

14. Other tools:
    Gamebench Testing the FPS tool for Android apps
    Gfxinfo viewing app drawing performance tools

1.6 Response Time Chapter

1. Understand:
   1) The time it takes to start Nativeapp from the Click event to the container (buried point)
   2) Nativeapp full start time (can be obtained by System.log)
   3) Native Call RPC Request method delay time (buried point)
   4) Specific data in the process of RPC request (Req_size Req_header req_time, etc., obtained by burying points)
   5) The specific data returned by the RPC request (res_size Res_header res_time, etc., obtained through the burial point)
   6) The time taken to parse the returned data locally (buried point or TraceView tool available)
   7) Interface rendering time (can be obtained by slow camera or buried point)

2. Android App launch time test
   (Android activity startup process Performance Cutaway: http://www.rudy-yuan.net/archives/59/)

3. The test of the start time of the application is divided into three categories:
   1) first start-the time it takes to apply the first boot
   2) Non-initial start-the time it takes to apply a non-first boot
   3) Application interface switching--time spent switching within the application interface

4. Application startup time Data acquisition:
   1. adb logcat > /address/logcat.txt #所有activity打印的日志
   find “Displayed” /address/logcat.txt > /newaddress/fl.txt#通过日志过滤关键字Displayed来过滤
   find “ActivityName” /newaddress/fl.txt > /newaddress/last.txt#通过activity名来过滤获取所测应用
   By calculating the sum of the last remaining time of the activity
   2, hardware testing, the use of high-speed camera or mobile phone using video method to record the application startup process, and then through the manual number of frames or the number of frames to calculate the start time

2 Weak net Test

1. Test method:
   1, the use of real SIM card, the operator network for testing (mobile wireless testing There are some special bugs must be found in a specific real operator network)
   2, through the proxy way to simulate the weak network environment for testing (Charles hard delay)
   3. Connect the hot spot of simulating weak net to test

2. Hot Spot Simulation Method:
   1) Share hotspot (hard delay) after setting iphone developer mode
   2) Facebook Open source ATC (use Raspberry to build the ACT environment)

3. What the user experience needs to do:
   1) Unified Weak net loading interface style, animation effect, chrysanthemum icon and so on in the application
   2) Unified network error, server error, timeout and other display to the user interface and prompt statements
   3) define clearly the user interaction behavior in each intermediate process

• This article transferred from: https://www.zybuluo.com/defias/note/592309

Android app Performance and special testing (personal grooming)