Mobile Software Test Automation: Building Android Mobile App in Jenkins

In the last post, we walked through creating a Jenkins job that would test an already built app.  This may be useful in some instances, but what we really need in order to round out a fully automated test solution for our mobile apps is to test against an app right after it has been built, but before the app goes through its upload process to HockeyApp, TestFairy, or whatever other process you use.  So let's get started...

Building an Android app

Since it is the official build tool for Android, I am going to show you how to build an Android app using Gradle.  I am going to assume that your Android project is already built using gradle scripts, and this is simply using the command line to tap into an existing build process.  

create new job

Start by creating a new freestyle Jenkins job.  You can name this whatever you want.  I also recommend clicking the "Discard old build" option using the log rotation strategy.  It's not essential, but this will help reduce clutter and save space.  So far, it should look something like this:

configure source code management

Now, you will want to configure your SCM.  This is likely Git, so enter your URL, credentials, and branch you build from.  Some use the default branch of 'master' as the head of dev, others create a branch named 'develop', and so on.  The branch you should use here is whatever your company uses as the main branch for this mobile project.  Although I don't know each of your projects, I would assume it contains some submodules, so in this Jenkins job in the SCM section, under "Additional Behaviours", add "Additional sub-modules behaviours" and check "Recursively update submodules."  

configure build triggers

The next step is to configure your build triggers.  You will want to decide what should cause an app to build, and then a subsequent test to run.  Do you want this based off specific time(s) throughout the day?  Maybe you want to poll for changes and start a build every time a change is checked in?  The answer will greatly depend on your current or desired workflow.  But to be fair, building based off a time slot is not ideal for continuously building software.  If you want to get the most out of a CI setup, and you want your developers to know as immediately as possible when code they checked in broke some functionality, then you should opt for the SCM polling option.  For this setup, you will need to set up a polling cycle.  There is a good description in Jenkins for how to set this up, but for the quick and dirty version, if you want to poll your SCM for changes every 15 minutes enter this in the schedule box:


H/15 * * * *

configure build steps

Now for the final step.  Under the "Build" heading, select the option to add "Execute Shell."  We will first run a gradle clean, and then we will assemble a debug build.  To do this, add the following two lines to your shell:


../../../../../Applications/gradle/bin/gradle clean
../../../../../Applications/gradle/bin/gradle assembleDebug

If you want to display the changes that were made, add another "Execute Shell" box and add:


echo ${CHANGES}

Now, in order for the automated tests to run, we need a resigned version of the app.  To do this, we will need one final "Execute Shell" box with the following two lines:


cd ~/path/to/your/automation/project/folder

calabash-android resign ~/path/to/build/location/build/outputs/apk/yourAppName.apk

move built app to the automation project folder

You may not want to do this and could simply point the calabash-android run command to the built app's location (the same location we used with the resign option above), but I prefer to move the app from that location to my automation project folder with the mv command in another shell window, mainly because it helps keep me organized.  I also add in the major app version and build number, but i'll save that for another day.  The command to do this is:


mv -f ~/path/to/build/location/build/outputs/apk/yourAppName.apk ~/path/to/your/automation/project/folder/android-apks/

Click "Save" and you should be done.  Now comes the easy part...

Triggering Tests to run after build

Remember the job we created last week to test the Android app?  We need to modify that one a little bit.  So click on that job and click "Configure" from the left hand navigation menu.  Under the "Build Triggers" section, select the option for "Build after other projects are built," and type in the name of the Android build project we just created.  That's all there is to it!

Wrapping up

That should complete our session on creating a Jenkins job to build an Android app using existing gradle scripts, moving the app to our automation folder, and then triggering our test job to run when a build completes.  Great job if you have followed along this far!  Next week I will finish the mobile automation section up with creating an iOS build, just like we did with Android today.  See you next week.


Mobile Software Test Automation: Setting up a Jenkins job

Welcome back!  After months of setup and creating test automation, we are finally ready to put our tests onto a CI server!  Hopefully you are as excited as I am.  At any rate, this week I will show you how to configure a Jenkins job to test our sample Calculator app for Android and iPhone, from an already-built app.  In practice, this job by itself would likely not be used as much, since it can really only test legacy app versions after a schema or other backend, middleware, or API change...something unit and integration tests can to much faster, and much more reliably.  But you never know, if your company does not yet have a good base of unit and integration tests, this very well may be what they need to run a smoke test after new code is released.  However, this is a fundamental stepping stone to having another job build the app from a specified branch, which can then be tested in the same manner, but we will get into that next week.

Setting up a git repository

I should have done a separate post on this, but it's too late now, so i'll keep this short.  You should already have your project in a remote git repo, because why would you run the risk of having it locally?  If for some reason you haven't done this yet, NOW is the time.  You can set up an account with Bitbucket, which allows you to store public or private repos for free with up to 5 collaborators, or you can use GitHub, which allows you to store public repos for free, but charges for each private repo, all with unlimited collaborators.  There is no perfect choice, and you will have to choose the one that fits best for you.  The important thing here is that you have a repo set up for your project.  

Creating the android job

With your Jenkins server running, navigate to the dashboard page (http://localhost:8080/) and click "New Item" from the left-hand navigation bar.  You should now be at the following screen:

Go ahead and name your project.  For our example, I am naming it "CalculatorSample_Android".  Now, select "Freestyle project", and click OK.  Next, select your SCM type (likely Git), enter the repo URL and credentials, and specify the branch you want to pull from.

Finally, under the section labeled Build, select the "Add build step" dropdown and select "Execute shell" from the list.  It should look something like this: 

You should now have a box that looks like a basic terminal window, which is where we will type the same steps we used to run the android project a while back.  First, we need to make sure all the required gems and their dependencies are up to date.  So the first line should be:


bundle install

Now, type the run command for your project. If you are using the sample, it will look like this:


bundle exec calabash-android run calculator-android-apks/Calculator.apk -p android

When it's all said and done, it should look like this:

Click Save, and you're all set.  To run the job, make sure a device is connected to the machine and click "Build Now".  You can see what's going on in the Console Output if you are curious.  When the job finishes, you will see a green when the job passes (or blue if you didn't download the Green Balls plugin), and red if it fails.  

Creating the iPhone job

Setting up the iPhone job is very similar to the android, but there is one additional step we need.  If you remember, when we ran the iOS project, we first had to deploy the app to the device using Xcode.  Thankfully, there is an automated way to do this, using ios-deploy.  You can find the project and installation instructions here.  If you already have homebrew installed, the install is very straight forward. (If not, follow the link for instructions) Now, in the terminal, type each of these commands:


$ brew install node
$ npm install -g ios-deploy

Next, do a quick test to make sure it works.  Connect an iOS device to your machine and run the following:


$ ios-deploy --justlaunch --bundle path/to/your/built/ios/application.app

Note: If you can't deploy the app to a device using Xcode, meaning your certificates and provisioning profiles are configured for that device, then this won't work either.   This step is assuming you are already able to deploy the app to your iOS device through Xcode.  

Using the above command, the app should deploy to your device and run.  Now we are good to go.  So you should configure the iOS job pretty much exactly the same as the android job, except using the run command with the ios profile tag.  Above that, you will also need to add an execute shell command script to deploy the app to the phone.  And we will also need to add an additional parameter to the ios-deploy line so that it will uninstall the app, thus clearing any data, before installing.  Once it's all said and done, it should look like this:

Wrapping up

That completes our setup for this week.  While right now we only have one job for each platform, next week we will configure a job to build each mobile project, introduce SCM polling to run when new code is checked in, and we can then set parameterized triggers to subsequently kick off a job to run our smoke test.  Once that is done, you could kick off a sequence of uploads to Hockey, or wherever else you host your app.  In addition, you can also configure the test jobs to run a full regression suite at whatever interval you desire.  So sit back, relax, and enjoy the progress we made today...for now.  Next week we will create a build job for each platform and then modify the test jobs we created today in order to implement a true mobile CI process.  See you next week!