Mobile Software Test Automation: Setting up a Jenkins job

Welcome back!  After months of setup and creating test automation, we are finally ready to put our tests onto a CI server!  Hopefully you are as excited as I am.  At any rate, this week I will show you how to configure a Jenkins job to test our sample Calculator app for Android and iPhone, from an already-built app.  In practice, this job by itself would likely not be used as much, since it can really only test legacy app versions after a schema or other backend, middleware, or API change...something unit and integration tests can to much faster, and much more reliably.  But you never know, if your company does not yet have a good base of unit and integration tests, this very well may be what they need to run a smoke test after new code is released.  However, this is a fundamental stepping stone to having another job build the app from a specified branch, which can then be tested in the same manner, but we will get into that next week.

Setting up a git repository

I should have done a separate post on this, but it's too late now, so i'll keep this short.  You should already have your project in a remote git repo, because why would you run the risk of having it locally?  If for some reason you haven't done this yet, NOW is the time.  You can set up an account with Bitbucket, which allows you to store public or private repos for free with up to 5 collaborators, or you can use GitHub, which allows you to store public repos for free, but charges for each private repo, all with unlimited collaborators.  There is no perfect choice, and you will have to choose the one that fits best for you.  The important thing here is that you have a repo set up for your project.  

Creating the android job

With your Jenkins server running, navigate to the dashboard page (http://localhost:8080/) and click "New Item" from the left-hand navigation bar.  You should now be at the following screen:

Go ahead and name your project.  For our example, I am naming it "CalculatorSample_Android".  Now, select "Freestyle project", and click OK.  Next, select your SCM type (likely Git), enter the repo URL and credentials, and specify the branch you want to pull from.

Finally, under the section labeled Build, select the "Add build step" dropdown and select "Execute shell" from the list.  It should look something like this: 

You should now have a box that looks like a basic terminal window, which is where we will type the same steps we used to run the android project a while back.  First, we need to make sure all the required gems and their dependencies are up to date.  So the first line should be:

bundle install

Now, type the run command for your project. If you are using the sample, it will look like this:

bundle exec calabash-android run calculator-android-apks/Calculator.apk -p android

When it's all said and done, it should look like this:

Click Save, and you're all set.  To run the job, make sure a device is connected to the machine and click "Build Now".  You can see what's going on in the Console Output if you are curious.  When the job finishes, you will see a green when the job passes (or blue if you didn't download the Green Balls plugin), and red if it fails.  

Creating the iPhone job

Setting up the iPhone job is very similar to the android, but there is one additional step we need.  If you remember, when we ran the iOS project, we first had to deploy the app to the device using Xcode.  Thankfully, there is an automated way to do this, using ios-deploy.  You can find the project and installation instructions here.  If you already have homebrew installed, the install is very straight forward. (If not, follow the link for instructions) Now, in the terminal, type each of these commands:

$ brew install node
$ npm install -g ios-deploy

Next, do a quick test to make sure it works.  Connect an iOS device to your machine and run the following:

$ ios-deploy --justlaunch --bundle path/to/your/built/ios/

Note: If you can't deploy the app to a device using Xcode, meaning your certificates and provisioning profiles are configured for that device, then this won't work either.   This step is assuming you are already able to deploy the app to your iOS device through Xcode.  

Using the above command, the app should deploy to your device and run.  Now we are good to go.  So you should configure the iOS job pretty much exactly the same as the android job, except using the run command with the ios profile tag.  Above that, you will also need to add an execute shell command script to deploy the app to the phone.  And we will also need to add an additional parameter to the ios-deploy line so that it will uninstall the app, thus clearing any data, before installing.  Once it's all said and done, it should look like this:

Wrapping up

That completes our setup for this week.  While right now we only have one job for each platform, next week we will configure a job to build each mobile project, introduce SCM polling to run when new code is checked in, and we can then set parameterized triggers to subsequently kick off a job to run our smoke test.  Once that is done, you could kick off a sequence of uploads to Hockey, or wherever else you host your app.  In addition, you can also configure the test jobs to run a full regression suite at whatever interval you desire.  So sit back, relax, and enjoy the progress we made today...for now.  Next week we will create a build job for each platform and then modify the test jobs we created today in order to implement a true mobile CI process.  See you next week!