Comparison and analysis of foreign mobile app cloud test Platform

Because of the different factors such as environment of cloud test Platform, we compare several cloud test platforms at home and abroad.

The main foreign cloud testing platform:

• Xamarin Test Cloud (https://xamarin.com/test-cloud/)
• Testdroid (http://testdroid.com/)
• Sauce Labs (https://saucelabs.com/mobile/)
• Google Cloud Test Cloud (https://developers.google.com/cloud-test-lab/)
• AWS Device Farm (https://aws.amazon.com/device-farm/)

Figure 1-Foreign mainstream cloud test platform comparison

From what we can see some features:

1. in the number of test devices, Xamarin test cloud and sauce Labs are very advantageous, although Xamarin test cloud counts the number of test devices, and sauce Labs is the number of platforms;
2. Amazon's own Fireos is supported only by its own cloud test platform, and we can see similar examples in China;
3. All Cloud test platforms support app test , but only testdroid supports game testing;
4. for the domestic cloud test platform to provide manual testing, security testing, beta distribution and crowdsourcing testing, these cloud test platforms abroad are not supported, and need to be used in conjunction with other tools and frameworks. However, for manual testing, Sauce Labs and Perfecto both cloud test platforms support manual testing of leased test equipment, and
5. compatible testing of cloud testing fundamentals, as well as script testing, crash analysis, and continuous integration, which are supported by the cloud test platform;
6. only Xamarin Test cloud,testdroid and AWS Device Farm support performance monitoring;
7. the mobile App Automation test framework used for script testing is not the same for each platform:

• Xamarin Test Cloud supports Calabash (iOS and Android) and its own xamarin.uitest;
• Testdroid supports many frameworks, including iOS-enabled Calabash,appium,ui automation and Jasmine, and Android-enabled Calabash,appium,espresso, Robotium and Uiautomator;
• Sauce Labs supports its own open-source framework appium;
• The Google Cloud test lab supports espresso,robotium and Robo test;
• AWS Device Farm also supports many frameworks, including iOS-enabled Calabash,appium,uiautomation and Xctest, and Android-enabled Calabash,appium,junit,espresso. Robotium and Uiautomator.

1. Xamarin test cloud,testdroid and sauce labs have their own mobile app test script recording tools, namely: Xamarin test recorder,testdroid Recorder and Appium Inspector

In general, for a foreign cloud testing platform, if the focus is on testing equipment coverage, choose Xamarin Test Cloud and sauce Labs will be more appropriate, if you need to test the FIREOS device, then select the AWS Device Farm If you are focusing on the languages and frameworks supported in the script test, you can choose Testdroid and AWS Device Farm, and if you are testing the game, you can only choose Testdroid, if you want to remotely connect the test device for manual testing, you need to select sauce Labs and Perfecto, sauce labs and Google Cloud test Lab are not available if you need to monitor performance synchronously during testing.

Of course, there are a lot of good cloud testing platform in China, such as Testbird. Testbird initially from the hand-tour test started, in the hand to accumulate high visibility in the lap, is also gradually in the field of app testing, while Testbird also joined the field of intelligent hardware testing. Not only that, Testbird also developed the Izumo mobile phone platform to help developers and testers implement remote real-machine debugging, self-service function testing and automated regression testing. There are some other domestic mainstream cloud testing platform, next time for a detailed comparative analysis!

Source:Infoq

Comparison and analysis of foreign mobile app cloud test Platform