Mobile Software Test Automation: Adding and running a project with tags

Now that we know how to update our projects, we are coming down the home stretch towards CI.  But we have one last step before we get there: Tags.  Ultimately, these are going to help break the project up into manageable chunks that we can use to test various parts of our apps.

Why do I need tags?

Great question.  Truth be told, they are not absolutely necessary.  However, when setting up an automation project as part of a CI process, specific failures may not be as apparent if the whole project was run as one job.  When running the automation manually from your machine, it is very easy to see what failed.  On the other hand, a CI server only cares about pass or fail, and to determine the cause of the failure, you would have to dig through a console log, which can make it more difficult to point the finger at the dev responsible for the failure.

So the idea is to create tags that break your project down into sizable, related chunks, which will allow a more focused understanding of an issue if/when a failure occurs.  In summary, knowing that a specific API call responsible for chunks under a tag failed, is much more beneficial than seeing that the whole project failed.

In addition, based off the scope of the code change, it may be beneficial to manually kick off a specific set of tests, instead of running the whole suite...which can save lots of time for you and your team.  

Inserting tags into your project

The most difficult part about adding tags is deciding where to put them.  And what to name them.  Unfortunately, that is something that only you can decide.  But in general, you want to break your project down into smaller chunks.  Most likely, you will break your app down into the different functions it performs.  For example, you may have a social media app with a set of tests for your feed, another for contacts, another for profile, etc.  Now that your wheels are turning, you may be going back and organizing your features and scenarios...that's normal. 

Now that you have sorted out where to put your tags, there are a few more guidelines.  First, tags cannot have a space between words:


@new_user (correct)
@new user (incorrect)

Also, you can have as many tags as you like.  Separate multiple tags by a space.  And you can add a tag to either a specific scenario, or to the whole feature itself.  Note that any tag that exists on a feature will be inherited by the scenario, scenario outline, or examples underneath. 

Take this example:

The tag marked @help will run both scenarios listed.  The tag marked @smoke_test will only run the first scenario.  

Running the test with tags

Now comes the easy part... And this is the same for both iOS and Android, however, im showing the run command for iOS below:


bundle exec cucumber --tags @help -p ios
bundle exec cucumber -t @help -p ios

You can use the --tags option, or just the -t option, the choice is yours.  Also, you can omit a tag using the following:


bundle exec cucumber --tags ~@smoke_test -p ios

This will run every feature/scenario in your project directory that does not include the tag @smoke_test.

Logic

There are also a few more options when running tags, mainly revolving around logic.  What if you wanted to run all the features/scenarios that had the help OR the settings tag?  Comma separated tags will be OR'd. See the below example:


bundle exec cucumber --tags @help,@settings -p ios

In addition, passing separate tag options performs AND logic.  So, using the example above, if you wanted to run only the Help screen scenario, and there were other scenarios tagged with @smoke_test in other feature files, you could do it like this: 


bundle exec cucumber --tags @help --tags @smoke_test -p ios

You can also combine these to make an even more specific selection:


bundle exec cucumber -t @help,@login -t ~@smoke_test -p ios

Wrapping up

If you need more information about tags, you can find more information on the cucumber wiki here.  Overall, this week was pretty simple compared to what we have done in the past.  But it will be very powerful in the weeks to come.  Up next, I'll be going through a multi-part series on setting up a CI server and then setting up our automation jobs, so stay tuned!