RTW for iOS (iPhone/iPad) application testing lesson learn and troubleshooting tips

About a month ago, I had an opportunity to work on the proof of concept and proof of technology of the mobile testing on the iOS devices. During the POC/POT, we ran into several issues and it took us some time to setup the environment. For folks that are new to the Rational Test Workbench (RTW) for the iOS platforms, I believe that what have learned during this POC/POT could be very beneficial to you.


How could we use RTW to test iOS applications on the iOS devices (iPad/iPhone)?


Prerequisite before you proceed to Instrumenting iOS applications on an iOS device

  1. Install Rational Test Workbench Eclipse Client and ensure that Rational Test Workbench Eclipse Client and Rational Test Workbench Extension for Mobile are selected

  2. Install Xcode 4.6. or newer on a Macintosh computer running OS X v10.8 Mountain Lion or newer. I personally prefer a newer version Xcode 5, because it comes with a simulator that I can use to run, test and debug my application locally on my Mac.

For those of you who are new to the iOS development and testing, you might not be familiar with Xcode. Xcode is an integrated development (IDE) containing a suite of software development tools developed by Apple for developing software for OS X and iOS. In order to run test and debug your application, you could use either a simulator or use the real device.

  1. Simulator:- It is free and it’s available through iOS SDK and Xcode 5

  2. Real device and if you want to make your apps available in apple’s store:- You will need to join the iOS Developers Program and there is a fee of $99 per yer that you will have to pay.

Before any testing of the mobile application on any of the iOS devices can be done. We must install and push an instrumented iOS application on a mobile device. The application must first be added to the test workbench. I won’t go in much details on how to do this. A step-by-step instructions could be found at Instrumenting iOS applications on an iOS device.

In order to register an iOS application to the test workbench. The test workbench needs to know the application when it received the recording log in order to produce a complete test. The instructions (Instrumenting iOS applications on an iOS device) will guild you through the instrumentation of the application under test by running the rtwBuildXcode.sh script. The instrumentation augments the AUT with code that allow you to record and play back a test. Below is a recap of a couple of issues that we ran into and what we did to resolve it.

 

Issue #1 If for some reason the rtwBuildXcode.sh script failed to run successfully

Troubleshooting tips

  1. Check to make sure that the current user have the appropriate permissions to run the script. If the permissions are set incorrectly, it could be easily change via a chmod command

  2. On your Macintosh computer, open the Macintosh Terminal application and change to the build-script folder.

    Type the following command to instrument the AUT and send it to the test workbench as an incoming application

    <unpack_dir>/build-script/rtwBuildXcode.sh <.xcodeproj file>

A lot of people get the step #3 wrong especially when they have to entered the DO NOT include /mobile at the end of the Workbench URL.

Example of a good <workbench URL>  http://192.0.2.24:7878

Example of a bad  <workbench URL>  http://192.0.2.24:7878/mobile

 

Issue #2 The build script was run successfully but you are unable to record and keep getting the error message of “Cannot Open Page Safari cannot open the page because the address is invalid”

 

Troubleshooting tips

  1. You would typically get this error when the application installed on the device is not the one setup as part of the RTW. When we instrument the application with the rtwBuildXcode.sh script. The application get a unique uuid number. When you get this error message it means the uuid from the application on the device is not the expected one. Our instinct when something like this failed,  usually what we will do is to remove the application and try to add them to the device again. If you want to remove and add your application again, please make sure that the previous application is removed completed. Make sure that you click the the application on the device long enough to get the cross so you can remove the application properly.

  2. Launch safari and go to the web client (http://<workbench ip>:7878/mobile) through safari, then do managed application again and make sure that the application finished installing


Q&A section


Question #1 This iOS application is not going to be available on the Apple’s store but it will be available internally to my company’s employee. Is it still required for the iOS developer profile to be install on iPhone/iPad?

Answer: It is crucial that the iOS developer profile be installed on your iOS devices. RTW has no control of this at all. When you connect the device (iPhone/iPad) to the Mac it will appear in Xcode organizer. It is important that your device be attached to your provisioning profile.

Question #2 What is the best way to check if the iOS devices is properly configured from an apple license point of view?

Answer:Connect your iOS device to the Macintosh computer, then open Xcode against the application Xcode project and attempt to run the application on the connected device rather than on the iOS simulator.

Question #3 What is the best command line options to capture the log file if the instrumentation failed?

Answer:  sh -x rtwBuildcode.sh [xcode project] [RTW serveur] >log 2>&1

Also it is a good idea to collect the project.pbxproj file in the .xcodeproj folder and send them  to support or engineering for further investigation and analysis as well.

Advertisements