How Android apps are tested and its Challenges

As per Statista, there are approx. 2.8 million apps on Google Play- with 1300 apps being uploaded daily. Given the scale of the uploaded apps – it is clear that Google’s Android is not just most popular mobile platform in the world, but the most widely used platform as well. Android expansion just continues to grow steadily.

Keeping the above facts in mind, it can safely be said that Android is the largest operating system in the world. However, at the same time- Android is fragmented- there are tons of devices and versions that your app must be compatible with. Android virtually being the biggest OS in the world, it is necessary that crucial facts regarding Android app testing must be highlighted.

Crucial Points in Android App Testing

Most of the idiosyncrasies of Android apps are rooted from the open-source nature of the platform. As Google allows mobile manufacturers to use and customize Android as per their requirements- the diverse versions of Android power various mobile devices. This results in a considerable challenge for testing engineers.

• Android is Fragmented

As stated above, Android welcomes original mobile manufacturers – thus the number of devices/vendors that rely on the platform are enormous. As it is fragmented, testing on multiple devices is necessary but impossible to do so on every device available. Thus, testing engineers study the stats and then select the most popular models and manufacturers for a specific region.

The most known models and manufacturers are the ones that usually predefine the OS versions popular in the region. For e.g. According to App Brain – Samsung Galaxy S7 and Samsung Galaxy S8 and Pixel Android 7 on Android 6 and 7 are the most used Android manufacturers and model (bound with OS versions) in the United States.

• Supporting multiple screens and densities

Android powers thousands of devices with varying screen sizes. This includes wearables, tablets, mobiles and televisions etc. You don’t have test an app on all the screens as its practically impossible. Google Play Store does have some guidelines which testers may follow and upload some of the pre-defined sizes.

Android developers support four pre-defined screen sizes – small, normal, large and extra-large. Thus, testing engineers choose the two opposites in the spectrum- the largest and smallest screen of the listed targeted devices and test the app on them.

The same it is for density for e.g. Samsung Galaxy S5 with 432 dpi is the smallest smartphone screen, meanwhile, Samsung Galaxy S8 with 567 dpi is the largest. According to 1Android development guide – “xlarge screens are at least 960dp x 720dp, large screens are at least 640dp x 480dp, normal screens are at least 470dp x 320dp, small screens are at least 426dp x 320dp”.

Now that the overall aspects of Android-based mobile app testing are deliberated upon, let’s look at the technical side of testing like – android app testing strategies, methods and types of testing.

Android testing Strategy

Choosing an appropriate strategy relies on the type of the app. A native app is depended on the mobile platform capabilities, therefore testing native apps on Android needs to focus on UI testing and integration with Android OS. In this instance, Android community UI guidelines may help. Most of the Android testing specifics are regarding the app non-functional elements.

Non-functional testing for Android Apps

Non-functional testing refers to testing the features of a software application that may not be associated with the defined user function or action. This goes well beyond just performance testing, they also comprise of –

• Uninstallation and installability testing

Testing the app’s installation and deletion is essential for a positive user experience. Check the smoothness of the installs and uninstalls – this confirms that the app behavior will remain steady if the disk space is limited. When being installed or uninstalled, the app shouldn’t involve other apps on the device or personal data stored there.

One more critical aspect is handling updates. When the app requires an update, the user must receive the needed / relevant notification. As Android apps update very frequently, the user loses track of space available on the device- hindering the UX.

Two critical things installability testing confirms is whether the app components are correctly installed and if the updates are popping up designated time intervals.

• Device and Power Efficiency

Android as an OS is comparatively heavy. Many apps kind of run in the background, even when the user is done with them and tries to close them. This feature exhausts the power of the battery and therefore its service life inexorably decreases. Testers, therefore, should assure that when the app is not active, it doesn’t utilize the power and specifically check if the app sends/receives data when backgrounded.

While there are battery saving modes and standby mode for saving power on devices when accessing network, these advanced features are only available in Android 5.0 and versions after that. For older versions that do not have these features, it is necessary to assure that the app is closed on the user’s command.

• App behavior during connectivity issues

These days users take the device’s ability to connect to the internet for granted. The internet connection, however, is not always simple and seamless. Regrettably, Android apps don’t always handle a weak connection well. They may close without the user’s permission or they might even crash.

Therefore, testing engineers must always check if the app handles conditions like transition (WiFi-3G/4G) on the go, intermittent connection or loss of connection. In such cases, manual testing efforts are the most essential as they simulate real-life situations better.

• Recovery Testing

Recovery testing validates if the app shuts down during failure without glitches and without affecting the system- and that the data is not lost. These tests include termination of data processes or premature interruption, the manual dismantling of database fields and keys and turning of routers, servers, in order to observe the effectiveness of the app recovery- when all systems are rebooted.

• Security issues

Unfortunately, Android’s openness is also its major pitfall. Android is comparatively very lax when it comes to their publishing policy which allows practically anyone to publish an app in Google Play Store. Thus, this leads to Google Play containing many infected apps. Downloading these apps can possibly transfer the infection to the rest of the apps on the device. This is more of an issue if any app contains sensitive information. To solve security troubles, penetration testing specialists should run an all-inclusive security testing.

Manual and Automated Android App Testing

Android developers endeavor to deliver quality apps speedily and efficiently. Initially, this makes automated testing a reasonable choice to minimize time/efforts employed on testing. It is not so simple, however, as mobile apps are generally compact- a manual testing is usually more efficient.

Instances where automated testing is necessary (like deadlines, wide market targets & device coverage, comprehensive compatibility work), experts recommend using automation testing to 1) test suites that cannot be run manually, like performance testing for example 2) automate the scripts for regression test cases. If the regression tests are repetitive – automated testing is more appropriate.

If the application contains new functionality, manual testing is more suitable. Plus, if the application requires testing once or twice- you must do the testing manually.

Optimal test automation may take up more than 70% of testing efforts in a project, even if the above list doesn’t look that long. There are some testing tools like Espresso, Roboelectric and Android Studio suggested by Android developer community as well as tutorials that enable automated testing.

Conclusion

Android app testing issues are due to its widespread popularity across the world as well as the specifics of the platform, referring to the screen size/ density variations and fragmentation. The testers face these issue on basis of statistics (based on regional location) and choosing the most used/popular manufacturers and models, OS versions in a particular region. They select the devices with largest and smallest density per inch (dpi). Concerning platform specifics, testers should pay attention to non-functional elements of Android apps- like installability, deletion, power/device efficiency, security and recovery testing and connectivity issues.

Furthermore, when choosing an appropriate strategy for Android app testing, testers should depend upon the peculiarities as well as project specifics to make sure that the testing is efficient leading the project success.