Tilley Scott - Testing iOS Apps with HadoopUnit: Rapid Distributed GUI Testing

Testing iOS Apps with
HadoopUnit

Rapid Distributed GUI Testing

Synthesis Lectures on Software Engineering

The Synthesis Lectures on Software Engineering publishes 75-150 page publications on all aspects of software design, engineering, and process management.

Testing iOS Apps with HadoopUnit: Rapid Distributed GUI Testing

Scott Tilley and Krissada Dechokul

December 2014

Hard Problems in Software Testing: Solutions Using Testing as a Service (TaaS)

Scott Tilley and Brianna Floss

August 2014

Model-Driven Software Engineering in Practice

Marco Brambilla, Jordi Cabot, Manuel Wimmer

September 2012

Copyright 2015 by Morgan & Claypool

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any meanselectronic, mechanical, photocopy, recording, or any other except for brief quotations in printed reviews, without the prior permission of the publisher.

Testing iOS Apps with HadoopUnit: Rapid Distributed GUI Testing

Scott Tilley and Krissada Dechokul

www.morganclaypool.com

ISBN: 9781627056519 print

ISBN: 9781627056526 ebook

DOI 10.2200/S00614ED1V01Y201411SWE003

A Publication in the Morgan & Claypool Publishers series

SYNTHESIS LECTURES ON SOFTWARE ENGINEERING #3

Series ISSN 2328-3319 Print 2328-3327 Electronic

Testing iOS Apps with
HadoopUnit

Rapid Distributed GUI Testing

Scott Tilley

Florida Institute of Technology

Krissada Dechokul

Suwat Dechokul Part., Ltd.

SYNTHESIS LECTURES SOFTWARE ENGINEERING #3

ABSTRACT

Smartphone users have come to expect high-quality apps. This has increased the importance of software testing in mobile software development. Unfortunately, testing appsparticularly the GUIcan be very time-consuming. Exercising every user interface element and verifying transitions between different views of the app under test quickly becomes problematic. For example, execution of iOS GUI test suites using Apples UI Automation framework can take an hour or more if the apps interface is complicated. The longer it takes to run a test, the less frequently the test can be run, which in turn reduces software quality.

This book describes how to accelerate the testing process for iOS apps using HadoopUnit, a distributed test execution environment that leverages the parallelism inherent in the Hadoop platform. HadoopUnit was previously used to run unit and system tests in the cloud. It has been modified to perform GUI testing of iOS apps on a small-scale clustera modest computing infrastructure available to almost every developer.

Experimental results have shown that distributed test execution with HadoopUnit can significantly outperform the test execution on a single machine, even if the size of the cluster used for the execution is as small as two nodes. This means that the approach described in this book could be adopted without a huge investment in IT resources. HadoopUnit is a cost-effective solution for reducing lengthy test execution times of system-level GUI testing of iOS apps.

KEYWORDS

software testing, iOS, apps, Hadoop, HadoopUnit, cloud computing, cluster

Contents

Appendix A
Setting up a HadoopUnit Cluster on Mac OS X

Appendix B
HadoopUnit Source Code for iOS GUI Testing

Figures

Tables

Foreword

Software quality has never been as important as it is today. A tidal wave of new software development is seen in mobile applications, which are quickly becoming ubiquitous. They are used for many purposes, from enjoyable entertainment to safety critical applications used by first responders.

At the same time, the bar to entering the mobile application marketplace is very low. Anyone with the desire to create a mobile application can do so with freely available tools in a rather short amount of time. A quick browse through the Apple App Store reveals that many applications fall short of user expectations: there are many apps in the store with three stars or fewer.

Creating a mobile application is one thing. Testing it to ensure that it works correctly, is secure, and achieves high usability factors is another thing altogether.

First, there is the plethora of mobile device platforms and applianceseach of which has its own configuration and behavior differences.

Second, the time required to test a moderately featured application just one time on one platform is substantial. Manual testing can be tedious and automated testing simulating user actions can relieve some of the tedium, but it still requires time to design and perform functional tests.

Third, mobile applications undergo constant change. The constant march of change fixes bugs and introduces new featuresand new defects. Therefore, testing should include not only testing bug fixes and new features, but also regression testing of the unchanged features. However, this all takes precious time.

Finally, consider that many mobile application developers consist of small teams, perhaps just an individual, with limited funds and resources for testing. So, the customers get to be the testers, except they use the applications for important real-world tasks like arranging travel, managing finances, tracking severe weather, and navigating roads. The customers dont consider themselves as testers. They are customers and even though they might not have paid any money for an app, they dont like to waste their time on defective ones. Also, when mobile applications fail during important tasks, defects are more than a mere inconveniencethey impact lives in a negative way.

While developers may perform bug fixes, customers often abandon apps quickly due to a bad experience. It is very easy to delete an app from an iOS device, so developers who wish to have a successful app in the App Store need to understand the detrimental impact of defects on their success.

Thats why this book is important for mobile application developers and testers. HadoopUnit offers a solution for testing mobile apps on the iOS platform that is not only free, but reduces test time dramatically. There will still be a need for usability and compatibility testing, but the more troubling defects are those that impact mobile application reliability.

Scott Tilley and Krissada Dechokul have done an excellent job with this book in describing in detail how to design and implement tests in HadoopUnit. My hope is that everyone involved in developing iOS applications will read this book and create more reliable and robust applications that get five-star reviews.

Randall W. Rice

CSTE, CSQA, CTAL

Founder, Principal Consultant, and Vice-President of Research and Development

Rice Consulting Services, Inc.

Preface

The app ecosystem is enormous. We have grown dependent on smartphone apps for almost every aspect of our lives. When the apps dont perform as expected, the consequences can range from midly irritating to life-threatening. For this reason, testing smartphone apps is very importantparticularly the GUI that is the main window into the apps functionality. Unfortuanately, GUI testing can be a time-consuming process, which leads to fewer tests being run, further exacerbating the apps quality problems.

This book focuses on the specific problem of GUI testing for iOS apps found on Apples products such as the iPhone and the iPad. Apple provides developers with a testing framework called UI Automation, but its capabilities are limited in terms of speeding up the testing process. The result is that GUI testing of complex iOS apps can take many hours.