Hybrid Cloud Testing

Although the growing market for cloud-based mobile device testing is fostering increased use of outsourced, standalone testing services, some companies are realizing that fully cloud-based testing is not the answer. Some have concerns about security while others need their testers to have “hands-on” time with devices themselves.  In either case, they find that cloud-based automation tools are not sufficient.

At the same time, the lure of cloud-based testing for mobile applications and services is undeniable. This paradox is leading some firms to adopt a hybrid testing approach. In these scenarios, the company purchases or contracts for some combination of cloud-based testing tools but runs them inside the company’s own LAN in a virtual cloud environment.

This approach not only streamlines policy and security management; it also enables organizations to connect their on-premise devices to the same virtual cloud. This creates a single pool of devices (both physical and virtual through emulation) from which testers at any location can run remote cloud-based scenarios.

Best of all, the devices are still available locally to testers, who can remove them from their cloud connections as necessary and test actions or gestures on them as they would with a handheld device.

How hybrid cloud Testing Works

To achieve a hybrid cloud testing environment, a company would contract with a third-party cloud-testing-tool provider for time or packages, but would host the testing tool on its own internal server. Testers needing to perform automated testing could use the cloud-based devices directly. However, for those few cases when a tester needed to physically touch the phone―to test actions such as swiping a card or pinching/expanding a display, the tester could walk over to the rack and physically pick up the phone or tablet.

During the time that device was in use, it would be unavailable to testers in other locations for cloud-connected testing. As soon as the first tester returned the device to its connection dock, it would again appear in the pool of available devices that testers in all locations could use to run scripts.

Orasi encountered such a situation with a major financial institution that was testing an application’s functionality for bar-code scanning and check scanning. With a cloud and testing tools set up in the company’s own hosted environment, and on-premise phones connected to the cloud and the automation tools, anyone could test any functions or actions where automation worked―tests that did not require a physical action. When testers did need to test bar code and check scanning functionality, they had access to the physical devices themselves.

In our experience, testers were able to rely on automation 80% of the time. For manual testing (20% of cases), three-quarters of the time testers could achieve their goals by accessing the company’s connected devices remotely, through the cloud. However, for the remaining 25% (5% of total test cases), having the physical devices present ensured that testing could be completed quickly and accurately.

Defining the Structure

Of course, such a hybrid environment provides this extensive flexibility only for those testers working in the same location with the physical phones. Consequently, it makes sense to structure these hybrid clouds based upon tester needs.

For some companies, it might make sense to have all on-premise devices located in a central location where the majority of testers work. Testers in outlying areas could run automated and/or remote cases against those devices, but would need to contact testers at the device location and ask for assistance (or gain access to another physical device) to manually test specific hand gestures or other actions.

In other cases, where different groups of testers are working on projects with specific, unique manual testing requirements, it would make more sense to distribute devices based upon those requirements―equipping each location with the devices on which users would likely perform the gestures or activities most often.

Research firm IDC predicts that the use of third-party testing services and tools is going to grow considerably―nearly 15% per year, on a compound rate, until 2017. It ascribes much of this growth to the explosion of mobile and Web-based applications and services. Companies interested in using a hybrid cloud approach should evaluate their third-party partners with this functionality in mind, making sure that the tools or services they wish to use can be hosted on their own internal clouds.

By Joe Schulz

Joe is a seasoned developer with more than 25 years of professional application development and quality assurance experience.  Joe is the AVP, Mobile Testing at Orasi Software, one of the largest and most successful HP Software sales and services partners in the quality assurance space. With Orasi, he helps global organizations improve their development and QA processes around mobile device and application testing. For more information, visit www.orasi.com.