Posted on September 2, 2014
What to look for in testing mobile apps, and why it’s worth the time and effort
Mobile app testing is perceived as expensive and time consuming, but it’s vital to ensure users have a positive experience. It helps identify navigation and presentation errors, controls usage problems, and provides feedback about the apps general appeal and engagement level.
However, many apps are launched without insufficient pre-launch testing, so users (not testers) wind up as guinea pigs and discover the issues that arise. This can diminish user engagement and tarnish a brand.
Unfortunately, a lot of marketers launching mobile apps take shortcuts to skimp on testing, or use a reactive approach and only worry about what users think once the app hits the market. They often put far too much emphasis on time-to-market, citing competitive reasons and the rapid technological pace of mobile as justification to get their mobile app out there fast.
Mobile apps should be tested proactively, not reactively. A Localytics study shows that 24% of users try out a downloaded app only once. That’s because when users find fault with an app, they simply don’t use it again.
The goal of testing efforts should not be just to find errors, but to understand the quality of the app. Does it work? Does it function as it was designed? Will it meet the needs of your users so that they come back, again and again?
A Perfecto Mobile benchmark survey this year found that users and not testers identify almost half of mobile application issues. In essence, that means users are the final QA step – obviously this is not the desired course of action. That means testing must be proactive to catch issues before the user experiences them.
The report also identified the top three challenges in mobile testing: insufficient coverage (63%), lack of reliable automation (48%), and needing more coverage to execute test scenarios than time permits (47%).
Respondents said that in order to reduce mobile app defects, more functional testing, device/OS coverage, and performance testing were needed.
Usability testing is not just a backend activity, it should occur throughout the development process from initial design to just before an app is released to allow you to address issues.
By testing an app prior to its release and during development, it is exponentially easier to fix problems. Addressing the issues discovered in the early stages of development will not only save the higher costs of fixing them post-release, but also avoid potential damage to a brand as a result of the bad reviews it will likely receive.
Testing should also be done at every phase of development, and user interaction should be recorded. Monitoring mobile apps is essential to examining their performance; it’s vital to monitor continuously, especially during performance testing so to isolate trends and differentiate between systems and root causes.
User testing vs. beta testing
While any testing is a both necessary and positive, beta testers get so familiar with a mobile app that they develop a kind of ‘white line fever’ – a fixation on a single point and lack fresh eyes – making them less likely to spot usability problems.
User testing, on the other hand, gives feedback from typical users who aren’t familiar with an app and have no bias or emotional attachment to it. These user tests help understand the needs and reactions of normal users, the types of people who write reviews in the app store.
Mobile apps differ from Web applications in their specific purpose. As such, when an app is created and deployed, there needs to be more upfront planning specific to the utilisation of the app – not simply a scaled-down version of the website.
Mobile apps are used under much different conditions than a website, so they must facilitate this with ease.
It’s important to remember that mobile consists of two devices: smartphones and tablets. If an app is designed to run on both, tests should be run on both because a screen layout may look good on a smartphone but cumbersome or unappealing on a tablet, or visa-versa. Additionally, user behavior on the two devices has subtle, yet distinct, differences.
Testing is often considered logical, planned and predictable, full of processes, test scripts, passes and fails – but this is not the full extent of what it encompasses.
Businesses need to do more than just test cases and find bugs; they need to find the issues that matter to users, and provide information of value that determines when it is okay to release an app and mitigates any issues beforehand.
By testing apps within the target audience during development, they are more likely to be not only wanted by the user but also engaging. Testing helps avoid naval gazing (looking at your own needs only); it gives the insights into what the users want and the type of app that will enhance the brand.