Mobile Software Test Automation: Setting up a Jenkins job

Welcome back!  After months of setup and creating test automation, we are finally ready to put our tests onto a CI server!  Hopefully you are as excited as I am.  At any rate, this week I will show you how to configure a Jenkins job to test our sample Calculator app for Android and iPhone, from an already-built app.  In practice, this job by itself would likely not be used as much, since it can really only test legacy app versions after a schema or other backend, middleware, or API change...something unit and integration tests can to much faster, and much more reliably.  But you never know, if your company does not yet have a good base of unit and integration tests, this very well may be what they need to run a smoke test after new code is released.  However, this is a fundamental stepping stone to having another job build the app from a specified branch, which can then be tested in the same manner, but we will get into that next week.

Setting up a git repository

I should have done a separate post on this, but it's too late now, so i'll keep this short.  You should already have your project in a remote git repo, because why would you run the risk of having it locally?  If for some reason you haven't done this yet, NOW is the time.  You can set up an account with Bitbucket, which allows you to store public or private repos for free with up to 5 collaborators, or you can use GitHub, which allows you to store public repos for free, but charges for each private repo, all with unlimited collaborators.  There is no perfect choice, and you will have to choose the one that fits best for you.  The important thing here is that you have a repo set up for your project.  

Creating the android job

With your Jenkins server running, navigate to the dashboard page (http://localhost:8080/) and click "New Item" from the left-hand navigation bar.  You should now be at the following screen:

Go ahead and name your project.  For our example, I am naming it "CalculatorSample_Android".  Now, select "Freestyle project", and click OK.  Next, select your SCM type (likely Git), enter the repo URL and credentials, and specify the branch you want to pull from.

Finally, under the section labeled Build, select the "Add build step" dropdown and select "Execute shell" from the list.  It should look something like this: 

You should now have a box that looks like a basic terminal window, which is where we will type the same steps we used to run the android project a while back.  First, we need to make sure all the required gems and their dependencies are up to date.  So the first line should be:

bundle install

Now, type the run command for your project. If you are using the sample, it will look like this:

bundle exec calabash-android run calculator-android-apks/Calculator.apk -p android

When it's all said and done, it should look like this:

Click Save, and you're all set.  To run the job, make sure a device is connected to the machine and click "Build Now".  You can see what's going on in the Console Output if you are curious.  When the job finishes, you will see a green when the job passes (or blue if you didn't download the Green Balls plugin), and red if it fails.  

Creating the iPhone job

Setting up the iPhone job is very similar to the android, but there is one additional step we need.  If you remember, when we ran the iOS project, we first had to deploy the app to the device using Xcode.  Thankfully, there is an automated way to do this, using ios-deploy.  You can find the project and installation instructions here.  If you already have homebrew installed, the install is very straight forward. (If not, follow the link for instructions) Now, in the terminal, type each of these commands:

$ brew install node
$ npm install -g ios-deploy

Next, do a quick test to make sure it works.  Connect an iOS device to your machine and run the following:

$ ios-deploy --justlaunch --bundle path/to/your/built/ios/

Note: If you can't deploy the app to a device using Xcode, meaning your certificates and provisioning profiles are configured for that device, then this won't work either.   This step is assuming you are already able to deploy the app to your iOS device through Xcode.  

Using the above command, the app should deploy to your device and run.  Now we are good to go.  So you should configure the iOS job pretty much exactly the same as the android job, except using the run command with the ios profile tag.  Above that, you will also need to add an execute shell command script to deploy the app to the phone.  And we will also need to add an additional parameter to the ios-deploy line so that it will uninstall the app, thus clearing any data, before installing.  Once it's all said and done, it should look like this:

Wrapping up

That completes our setup for this week.  While right now we only have one job for each platform, next week we will configure a job to build each mobile project, introduce SCM polling to run when new code is checked in, and we can then set parameterized triggers to subsequently kick off a job to run our smoke test.  Once that is done, you could kick off a sequence of uploads to Hockey, or wherever else you host your app.  In addition, you can also configure the test jobs to run a full regression suite at whatever interval you desire.  So sit back, relax, and enjoy the progress we made today...for now.  Next week we will create a build job for each platform and then modify the test jobs we created today in order to implement a true mobile CI process.  See you next week!

Mobile Software Test Automation: Configuring Jenkins CI server

I apologize that it's been a couple weeks since I have been able to get a post out.  Unlike this blog, I have been covered up with some work that actually makes me money, so I let this fall by the wayside.  But I'm back.  The last post was walking through the initial setup of the Jenkins server.  Now we will get into some configuration in preparation for building the app and then running our tests.

Description of Plugins

There are hundreds, if not thousands of available plugins for Jenkins.  They range from fun but useless plugins like a meme generator when someone breaks the build, to absolute must haves like environment variable injection or version control support.  

essential plugins

Lets talk about the 4 essential plugins first:

Environment Injector Plugin - as the name states, this allows you to inject environment variables into a job.  This will also allow you to pass environment variables from one job to another.

Git plugin - pretty self explanatory...allows you to use git to pull down new versions prior to running a job.  This plugin can also notify the job of a change in git, so jobs can run as soon as code is checked in, instead of waiting for a specified polling interval.

Xcode integration - this plugin does a lot, but basically, you need this so that you can build the latest version of your iOS app and then test it.

Parameterized Trigger plugin - I debated having this in the essential section.  It may not be an absolute must have, but it does make your setup much cleaner.  This allows you to trigger a job to start when another job finishes.

nice to have plugins

Audit Trail - This would be nice to have if you work on a large team, as it keeps a log of who performed particular jenkins operations.

Email Extension Plugin - If you prefer a post job notification email be sent with passed or failed, this is the plugin for you.  You can configure just about everything about the email with this as well, which is pretty nice.

Slack Notification Plugin - If you use slack instead, this plugin is for you.  You can send post notifications to a specified slack channel.  There are also plugins for HipChat, or most other services along the same line.

Green Balls - Because who likes blue balls?  Jokes aside, green is the universal symbol for go or good, so for me, a passing job should be green.  That's all this plugin does.

might need plugins

There are a few plugins that you may need, depending on your setup.  

Android Emulator Plugin - If you are planning on running your tests on an emulated device, this would be nice to have. You could also start up an emulator from the command line, but a nice plugin is nice to have if you need it.  

Gradle plugin - If your android build uses gradle to invoke a build script, you would need this plugin.

Adding Plugins

Now that you know a little bit about some of these plugins, lets add the ones you need for your project.  So navigate to the dashboard of your Jenkins server.  If you have closed out the terminal session from last time, you will have to restart it.  In the left hand navigation bar, click "Manage Jenkins", and then you will need to click on the "Manage Plugins" link from the main content part of that page.

Now, click on the tab marked "Available", and this will show you a list of all the Jenkins plugins that are out there.  You could find the ones you need by scrolling, but it may be faster to type in the names as I listed above, and then click the checkbox next to each.  Once you have selected all the plugins you want to install, click the button to download/install them.  I chose to install them without a restart, but you can do it however you want.  

When installation is complete, navigate back to the "Manage Plugins" page, and click on the "Installed" tab.  You will notice that a few more plugins came along with what you installed.  No problems. This means that the dependencies were handled for you.

Wrapping up

Great job this week.  I am going to try to keep these posts a little shorter than in the past, which should make it easier for me to get these out every week.  Next week, we will work through creating a job to test an application that has already been built, followed by creating a job to pull code from a specified git branch (such as master or develop), then build the app, and then test against the newly built app. See you next week!