Mobile Software Test Automation: Building iOS Mobile App in Jenkins

The time has finally come to wrap up mobile automation!  Last time we talked about building an Android project from source code, and then injecting that into our test automation.  Now we are going to work through the same thing for iOS.  This process for iOS is a bit more involved than Android, so this post will be quite a bit longer.  So let's jump in...

Setting up New Jenkins Job

Create a new job

Start by creating a new freestyle Jenkins job.  You can name this whatever you want.  I also recommend clicking the "Discard old build" option using the log rotation strategy.  It's not essential, but this will help reduce clutter and save space.  So far, it should look something like this:

Configure SCM

Now, you will want to configure your SCM.  This is likely Git, so enter your URL, credentials, and branch you build from.  Some use the default branch of 'master' as the head of dev, others create a branch named 'develop', and so on.  The branch you should use here is whatever your company uses as the main branch for this mobile project.  Although I don't know each of your projects, I would assume it contains some submodules, so in this Jenkins job in the SCM section, under "Additional Behaviours", add "Additional sub-modules behaviours" and check "Recursively update submodules."  

Configure build triggers

The next step is to configure your build triggers.  You will want to decide what should cause an app to build, and then a subsequent test to run.  Do you want this based off specific time(s) throughout the day?  Maybe you want to poll for changes and start a build every time a change is checked in?  The answer will greatly depend on your current or desired workflow.  But to be fair, building based off a time slot is not ideal for continuously building software.  If you want to get the most out of a CI setup, and you want your developers to know as immediately as possible when code they checked in broke some functionality, then you should opt for the SCM polling option.  For this setup, you will need to set up a polling cycle.  There is a good description in Jenkins for how to set this up, but for the quick and dirty version, if you want to poll your SCM for changes every 15 minutes enter this in the schedule box:


H/15 * * * *

Clean the working directory

We are going to remove untracked directories, sub-directories, and files from the working tree.  Xcode is very picky when we go to build an app, so we are doing this to make sure we have a pristine working directory before building the app.  Under the "Build" heading, select the option to add "Execute Shell."  In this shell, enter: 


git clean -ffdx

You can find more information about git clean here.  You can also put any other code in the space above, where you might need to remove a folder (such as one created from a previous run of this job), if that is your workflow.  For example, once the app is built, I move the built app to a directory i create, so that once tested, it can be easily uploaded to HockeyApp.  This mainly helps keep me organized, so it's not essential, but if you are creating a directory later, you will want to remove it below the git clean step, using the rm -f command followed by the directory you want to remove.

Configuring iOS Build Steps

For this part, I am assuming that you are already able to build your app using Xcode on your machine.  If you are not able to, but the developer who created the app is able to build locally, there is likely a provisioning profile or certificate issue you will need to work through with your developer.  I would give you some pointers for this, but getting the correctly signed cert and provisioning profile is a delicate balancing act, especially if your project consists of multiple team members, and I don't want to be the reason it got messed up.  

Moving on... under the "Build" heading, select the option to add "Xcode".  It should look like this after you add it:

General build settings

Under the General build settings section, go ahead and enter the Target you want to build.  In an effort to keep this as simple as possible, we will enter only the pre-built target with embedded Calabash framework for now.  Clicking on the "Settings" button will reveal a few more options.  Click the box next to "Clean before build?".  You will also need to enter your Configuration.  By default, these are Release and Debug, so it is likely one of those unless your project has some custom configuration setup.  You will also need to check the box next to "Pack application and build .ipa?".  Here you need to enter the filename pattern and output directory.  I'm not sure about your filename pattern, but it probably looks something like this:


ProductName-QA-Calabash-3.4.${BUILD_NUMBER}

For the output directory, this should be an existing directory in your workspace where you want the built app to be put.  By default, the location is:


~/Library/Developer/Xcode/DerivedData

I would recommend putting it somewhere else, but as long as you know where it's at, that's what really matters.  Note that this is relative to the build directory, so make sure you have an explicit path to ensure you are putting it in the right place. 

Code signing & OS X keychain options

Next to "Embedded profile", enter the path to the correct .mobileprovision file to embed for your selected target/configuration.  You should have had to download this to your machine in order to build in Xcode, so you will just need to find it.  The fastest way to find this is to open Xcode, and go into Xcode > Preferences > Accounts > View Details, right click on the correct provisioning profile > show in finder.  You can also just download a fresh copy from the apple developer portal.

Check the box next to "Unlock Keychain?".  The Keychain path will be something like:


${HOME}/Library/Keychains/login.keychain

You will need to also enter your keychain password.  This will most likely be what you use to login to the machine.

Advanced Xcode build options

The only thing you will need to put here is the "Build output directory" and should be the same value you used above for "Pack application and build .ipa?".  

Versioning

Check the box next to "Provide version number and run avgtool?" and enter the marketing and technical versions.  The marketing version may be hard coded to whatever value you want (or you can create a string parameter at the beginning of the job and use that).  The technical version is usually set with the Jenkins build number like this:


${BUILD_NUMBER}

That should be all you need to put in the Xcode section to build your iOS app.

Move built app to another directory

Add one more "Execute shell" step, and create the directory you want to move your app to.  Note that this is relative to your workspace.  Then, we will want to actually move the app from the output directory to the newly created directory.  It should look like this:


mkdir QA-Calabash-app
mv -f /Users/Path/To/Your/Built/Application.app /Users/Path/To/New/Directory/QA-Calabash-app/Application.app

That is all the configuration you will need for this job.  Go ahead and click "Save".

Triggering Tests to run after Build

Remember the job we created to test the iOS app?  We need to modify that one a little bit.  So click on that job and click "Configure" from the left hand navigation menu.  Under the "Build Triggers" section, select the option for "Build after other projects are built," and type in the name of the iOS build project we just created.  Also, in the Execute shell step where we push the app to a device using ios-deploy, you will need to give the explicit path to the built app that you defined above.  That's all there is to it!  

Wrapping up

And there you have it!  We have come a long way in the last 8 months, from talking about the need for test automation, to developing a cross platform test suite, and ending today with integrating those tests into a mobile build process in Jenkins for fully functioning CI for mobile.  Great job!  Since this wraps up everything I had planned for mobile automation, next we will start to tackle web automation.  It may be a week or two until I can get a plan together, but stay tuned!


Mobile Software Test Automation: Building Android Mobile App in Jenkins

In the last post, we walked through creating a Jenkins job that would test an already built app.  This may be useful in some instances, but what we really need in order to round out a fully automated test solution for our mobile apps is to test against an app right after it has been built, but before the app goes through its upload process to HockeyApp, TestFairy, or whatever other process you use.  So let's get started...

Building an Android app

Since it is the official build tool for Android, I am going to show you how to build an Android app using Gradle.  I am going to assume that your Android project is already built using gradle scripts, and this is simply using the command line to tap into an existing build process.  

create new job

Start by creating a new freestyle Jenkins job.  You can name this whatever you want.  I also recommend clicking the "Discard old build" option using the log rotation strategy.  It's not essential, but this will help reduce clutter and save space.  So far, it should look something like this:

configure source code management

Now, you will want to configure your SCM.  This is likely Git, so enter your URL, credentials, and branch you build from.  Some use the default branch of 'master' as the head of dev, others create a branch named 'develop', and so on.  The branch you should use here is whatever your company uses as the main branch for this mobile project.  Although I don't know each of your projects, I would assume it contains some submodules, so in this Jenkins job in the SCM section, under "Additional Behaviours", add "Additional sub-modules behaviours" and check "Recursively update submodules."  

configure build triggers

The next step is to configure your build triggers.  You will want to decide what should cause an app to build, and then a subsequent test to run.  Do you want this based off specific time(s) throughout the day?  Maybe you want to poll for changes and start a build every time a change is checked in?  The answer will greatly depend on your current or desired workflow.  But to be fair, building based off a time slot is not ideal for continuously building software.  If you want to get the most out of a CI setup, and you want your developers to know as immediately as possible when code they checked in broke some functionality, then you should opt for the SCM polling option.  For this setup, you will need to set up a polling cycle.  There is a good description in Jenkins for how to set this up, but for the quick and dirty version, if you want to poll your SCM for changes every 15 minutes enter this in the schedule box:


H/15 * * * *

configure build steps

Now for the final step.  Under the "Build" heading, select the option to add "Execute Shell."  We will first run a gradle clean, and then we will assemble a debug build.  To do this, add the following two lines to your shell:


../../../../../Applications/gradle/bin/gradle clean
../../../../../Applications/gradle/bin/gradle assembleDebug

If you want to display the changes that were made, add another "Execute Shell" box and add:


echo ${CHANGES}

Now, in order for the automated tests to run, we need a resigned version of the app.  To do this, we will need one final "Execute Shell" box with the following two lines:


cd ~/path/to/your/automation/project/folder

calabash-android resign ~/path/to/build/location/build/outputs/apk/yourAppName.apk

move built app to the automation project folder

You may not want to do this and could simply point the calabash-android run command to the built app's location (the same location we used with the resign option above), but I prefer to move the app from that location to my automation project folder with the mv command in another shell window, mainly because it helps keep me organized.  I also add in the major app version and build number, but i'll save that for another day.  The command to do this is:


mv -f ~/path/to/build/location/build/outputs/apk/yourAppName.apk ~/path/to/your/automation/project/folder/android-apks/

Click "Save" and you should be done.  Now comes the easy part...

Triggering Tests to run after build

Remember the job we created last week to test the Android app?  We need to modify that one a little bit.  So click on that job and click "Configure" from the left hand navigation menu.  Under the "Build Triggers" section, select the option for "Build after other projects are built," and type in the name of the Android build project we just created.  That's all there is to it!

Wrapping up

That should complete our session on creating a Jenkins job to build an Android app using existing gradle scripts, moving the app to our automation folder, and then triggering our test job to run when a build completes.  Great job if you have followed along this far!  Next week I will finish the mobile automation section up with creating an iOS build, just like we did with Android today.  See you next week.