Mobile Software Test Automation: Building iOS Mobile App in Jenkins

The time has finally come to wrap up mobile automation!  Last time we talked about building an Android project from source code, and then injecting that into our test automation.  Now we are going to work through the same thing for iOS.  This process for iOS is a bit more involved than Android, so this post will be quite a bit longer.  So let's jump in...

Setting up New Jenkins Job

Create a new job

Start by creating a new freestyle Jenkins job.  You can name this whatever you want.  I also recommend clicking the "Discard old build" option using the log rotation strategy.  It's not essential, but this will help reduce clutter and save space.  So far, it should look something like this:

Configure SCM

Now, you will want to configure your SCM.  This is likely Git, so enter your URL, credentials, and branch you build from.  Some use the default branch of 'master' as the head of dev, others create a branch named 'develop', and so on.  The branch you should use here is whatever your company uses as the main branch for this mobile project.  Although I don't know each of your projects, I would assume it contains some submodules, so in this Jenkins job in the SCM section, under "Additional Behaviours", add "Additional sub-modules behaviours" and check "Recursively update submodules."  

Configure build triggers

The next step is to configure your build triggers.  You will want to decide what should cause an app to build, and then a subsequent test to run.  Do you want this based off specific time(s) throughout the day?  Maybe you want to poll for changes and start a build every time a change is checked in?  The answer will greatly depend on your current or desired workflow.  But to be fair, building based off a time slot is not ideal for continuously building software.  If you want to get the most out of a CI setup, and you want your developers to know as immediately as possible when code they checked in broke some functionality, then you should opt for the SCM polling option.  For this setup, you will need to set up a polling cycle.  There is a good description in Jenkins for how to set this up, but for the quick and dirty version, if you want to poll your SCM for changes every 15 minutes enter this in the schedule box:


H/15 * * * *

Clean the working directory

We are going to remove untracked directories, sub-directories, and files from the working tree.  Xcode is very picky when we go to build an app, so we are doing this to make sure we have a pristine working directory before building the app.  Under the "Build" heading, select the option to add "Execute Shell."  In this shell, enter: 


git clean -ffdx

You can find more information about git clean here.  You can also put any other code in the space above, where you might need to remove a folder (such as one created from a previous run of this job), if that is your workflow.  For example, once the app is built, I move the built app to a directory i create, so that once tested, it can be easily uploaded to HockeyApp.  This mainly helps keep me organized, so it's not essential, but if you are creating a directory later, you will want to remove it below the git clean step, using the rm -f command followed by the directory you want to remove.

Configuring iOS Build Steps

For this part, I am assuming that you are already able to build your app using Xcode on your machine.  If you are not able to, but the developer who created the app is able to build locally, there is likely a provisioning profile or certificate issue you will need to work through with your developer.  I would give you some pointers for this, but getting the correctly signed cert and provisioning profile is a delicate balancing act, especially if your project consists of multiple team members, and I don't want to be the reason it got messed up.  

Moving on... under the "Build" heading, select the option to add "Xcode".  It should look like this after you add it:

General build settings

Under the General build settings section, go ahead and enter the Target you want to build.  In an effort to keep this as simple as possible, we will enter only the pre-built target with embedded Calabash framework for now.  Clicking on the "Settings" button will reveal a few more options.  Click the box next to "Clean before build?".  You will also need to enter your Configuration.  By default, these are Release and Debug, so it is likely one of those unless your project has some custom configuration setup.  You will also need to check the box next to "Pack application and build .ipa?".  Here you need to enter the filename pattern and output directory.  I'm not sure about your filename pattern, but it probably looks something like this:


ProductName-QA-Calabash-3.4.${BUILD_NUMBER}

For the output directory, this should be an existing directory in your workspace where you want the built app to be put.  By default, the location is:


~/Library/Developer/Xcode/DerivedData

I would recommend putting it somewhere else, but as long as you know where it's at, that's what really matters.  Note that this is relative to the build directory, so make sure you have an explicit path to ensure you are putting it in the right place. 

Code signing & OS X keychain options

Next to "Embedded profile", enter the path to the correct .mobileprovision file to embed for your selected target/configuration.  You should have had to download this to your machine in order to build in Xcode, so you will just need to find it.  The fastest way to find this is to open Xcode, and go into Xcode > Preferences > Accounts > View Details, right click on the correct provisioning profile > show in finder.  You can also just download a fresh copy from the apple developer portal.

Check the box next to "Unlock Keychain?".  The Keychain path will be something like:


${HOME}/Library/Keychains/login.keychain

You will need to also enter your keychain password.  This will most likely be what you use to login to the machine.

Advanced Xcode build options

The only thing you will need to put here is the "Build output directory" and should be the same value you used above for "Pack application and build .ipa?".  

Versioning

Check the box next to "Provide version number and run avgtool?" and enter the marketing and technical versions.  The marketing version may be hard coded to whatever value you want (or you can create a string parameter at the beginning of the job and use that).  The technical version is usually set with the Jenkins build number like this:


${BUILD_NUMBER}

That should be all you need to put in the Xcode section to build your iOS app.

Move built app to another directory

Add one more "Execute shell" step, and create the directory you want to move your app to.  Note that this is relative to your workspace.  Then, we will want to actually move the app from the output directory to the newly created directory.  It should look like this:


mkdir QA-Calabash-app
mv -f /Users/Path/To/Your/Built/Application.app /Users/Path/To/New/Directory/QA-Calabash-app/Application.app

That is all the configuration you will need for this job.  Go ahead and click "Save".

Triggering Tests to run after Build

Remember the job we created to test the iOS app?  We need to modify that one a little bit.  So click on that job and click "Configure" from the left hand navigation menu.  Under the "Build Triggers" section, select the option for "Build after other projects are built," and type in the name of the iOS build project we just created.  Also, in the Execute shell step where we push the app to a device using ios-deploy, you will need to give the explicit path to the built app that you defined above.  That's all there is to it!  

Wrapping up

And there you have it!  We have come a long way in the last 8 months, from talking about the need for test automation, to developing a cross platform test suite, and ending today with integrating those tests into a mobile build process in Jenkins for fully functioning CI for mobile.  Great job!  Since this wraps up everything I had planned for mobile automation, next we will start to tackle web automation.  It may be a week or two until I can get a plan together, but stay tuned!


Mobile Software Test Automation: Setting up a Jenkins job

Welcome back!  After months of setup and creating test automation, we are finally ready to put our tests onto a CI server!  Hopefully you are as excited as I am.  At any rate, this week I will show you how to configure a Jenkins job to test our sample Calculator app for Android and iPhone, from an already-built app.  In practice, this job by itself would likely not be used as much, since it can really only test legacy app versions after a schema or other backend, middleware, or API change...something unit and integration tests can to much faster, and much more reliably.  But you never know, if your company does not yet have a good base of unit and integration tests, this very well may be what they need to run a smoke test after new code is released.  However, this is a fundamental stepping stone to having another job build the app from a specified branch, which can then be tested in the same manner, but we will get into that next week.

Setting up a git repository

I should have done a separate post on this, but it's too late now, so i'll keep this short.  You should already have your project in a remote git repo, because why would you run the risk of having it locally?  If for some reason you haven't done this yet, NOW is the time.  You can set up an account with Bitbucket, which allows you to store public or private repos for free with up to 5 collaborators, or you can use GitHub, which allows you to store public repos for free, but charges for each private repo, all with unlimited collaborators.  There is no perfect choice, and you will have to choose the one that fits best for you.  The important thing here is that you have a repo set up for your project.  

Creating the android job

With your Jenkins server running, navigate to the dashboard page (http://localhost:8080/) and click "New Item" from the left-hand navigation bar.  You should now be at the following screen:

Go ahead and name your project.  For our example, I am naming it "CalculatorSample_Android".  Now, select "Freestyle project", and click OK.  Next, select your SCM type (likely Git), enter the repo URL and credentials, and specify the branch you want to pull from.

Finally, under the section labeled Build, select the "Add build step" dropdown and select "Execute shell" from the list.  It should look something like this: 

You should now have a box that looks like a basic terminal window, which is where we will type the same steps we used to run the android project a while back.  First, we need to make sure all the required gems and their dependencies are up to date.  So the first line should be:


bundle install

Now, type the run command for your project. If you are using the sample, it will look like this:


bundle exec calabash-android run calculator-android-apks/Calculator.apk -p android

When it's all said and done, it should look like this:

Click Save, and you're all set.  To run the job, make sure a device is connected to the machine and click "Build Now".  You can see what's going on in the Console Output if you are curious.  When the job finishes, you will see a green when the job passes (or blue if you didn't download the Green Balls plugin), and red if it fails.  

Creating the iPhone job

Setting up the iPhone job is very similar to the android, but there is one additional step we need.  If you remember, when we ran the iOS project, we first had to deploy the app to the device using Xcode.  Thankfully, there is an automated way to do this, using ios-deploy.  You can find the project and installation instructions here.  If you already have homebrew installed, the install is very straight forward. (If not, follow the link for instructions) Now, in the terminal, type each of these commands:


$ brew install node
$ npm install -g ios-deploy

Next, do a quick test to make sure it works.  Connect an iOS device to your machine and run the following:


$ ios-deploy --justlaunch --bundle path/to/your/built/ios/application.app

Note: If you can't deploy the app to a device using Xcode, meaning your certificates and provisioning profiles are configured for that device, then this won't work either.   This step is assuming you are already able to deploy the app to your iOS device through Xcode.  

Using the above command, the app should deploy to your device and run.  Now we are good to go.  So you should configure the iOS job pretty much exactly the same as the android job, except using the run command with the ios profile tag.  Above that, you will also need to add an execute shell command script to deploy the app to the phone.  And we will also need to add an additional parameter to the ios-deploy line so that it will uninstall the app, thus clearing any data, before installing.  Once it's all said and done, it should look like this:

Wrapping up

That completes our setup for this week.  While right now we only have one job for each platform, next week we will configure a job to build each mobile project, introduce SCM polling to run when new code is checked in, and we can then set parameterized triggers to subsequently kick off a job to run our smoke test.  Once that is done, you could kick off a sequence of uploads to Hockey, or wherever else you host your app.  In addition, you can also configure the test jobs to run a full regression suite at whatever interval you desire.  So sit back, relax, and enjoy the progress we made today...for now.  Next week we will create a build job for each platform and then modify the test jobs we created today in order to implement a true mobile CI process.  See you next week!