Eight Tips for Effective Mobile Testing

There are literally hundreds of mobile devices available today in several sizes, capabilities and purposes from several manufacturers. At this point in time, Apple’s iOS and Google’s Android are the leading operating systems in use, although there are still players like Microsoft (with Windows) and Blackberry who command a small market share. Apps may be developed either as native apps, container-based web apps, or a combination of the two. They are increasingly being developed in emulator environments that enable a build-once, run-on-multiple-platforms principle, but even so, there are so many differences between devices that testing these apps requires the consideration of a number of factors. Desktop PCs offer a largely standard hardware and software environment, but in the case of mobile devices there could be a lot of variation in specs, I/O facilities, screen sizes, etc. The following are some basic considerations to be taken into account while scoping and designing a test strategy.

• Testing for usability is of prime importance. The difference between a good app and a great app often lies in the user experience. Usability testing should be done by user experience specialist testers as well as members of the target user base. Keyboards are small, and so are screen sizes, memory and computing power in a mobile device, and all things considered, users these days can be rather impatient with anything less than a very good user experience.
  
• Test on emulators, but also test on as many real devices as possible. From large tablets to small smartphones, it would be impractical for any test team to maintain all possible models and versions of actual devices, so testing on emulators that offer a variety of operating systems, browers and devices has to largely be relied on, followed by beta tests on a range of real world devices to account for things like screen resolution, form factor, consistency in behaviour and so on, and to be able to advise on minimum OS and hardware version requirements.
  
• Test interactions with the device environment. Does the app clean up as it goes? Does it leave unwanted files that waste local storage? All interactions with other app interfaces, or OS functions must work cleanly and predictably when the app is suddenly exited, or a call or text message comes in while the app is working.
  
• Test robustness under stress. Does the app remain usable when it’s network connection suddenly weakens, or the battery dies, or the phone memory is too full, or many other apps are running? While it may be difficult for most apps to just keep running under any extremes, there must be thresholds of expected operability against which the app must perform.
  
• Test for security. Are all interfaces and data transmissions secure from hacking or interception?
  
• Test for resource consumption. Does the app hog memory or drain the battery for any reason? When the data connection slips from 4G to 3G or 2G, how does the app respond? Does the app footprint remain constant, or remain within specified limits after operation?
  
• Test for clean installation and uninstallation. This is self-explanatory. An app should install consistently, and if it does not find any resources, etc, that it needs while installing, it should handle such situations cleanly. After uninstallation the app should clean up unnecessary data and the installer itself.
  
• Use test automation where appropriate. This should be done if economical, considering the size of the app and the number of test cases in which data input and output lends itself to automation. However, with mobile apps manual testing remains the most critical.