Mobile Software Test Automation: Writing the feature

Last week we generated the project structure and had a high level discussion on the different components.  And now, its finally time to begin writing feature files!  Well, almost.  First I am going to define what some terminology, and then we will get started.

Acceptance Testing

Acceptance testing in software development is a type of testing that proves an application will work according to a set of predefined specifications.  It is also used to prove that changes to a code base don't have any unintended consequences on the existing application's functionality.  Basically, acceptance testing is a set of high-level tests on an application.  Your employer may already have a set of acceptance tests that you are running manually, which is why you are here in the first place.  So the first step, is to convert these manual acceptance tests into a scripting format that can serve as the highest level of an automated test suite.  Which brings us to some terminology:

Feature

In cucumber, acceptance tests are called features. A feature file is simply a text file with the extension .feature and is written in a human-readable language that can be understood by developers and executives alike.  Each feature is used to test one specific function of an application, so you will have multiple feature files in a given test suite.

Scenarios

Features are then broken down into one or more scenarios. A scenario describes each specific test for a feature.  For example, if the feature is around search for a particular application, a few scenarios may be: search by product id, search with low character count, and search with multiple words.  The key point is that a scenario exists solely within the context of a feature.

Steps

Scenarios are then further broken down into multiple steps. The steps are basically short sentences that describe a precondition for a scenario, any actions a user would take to complete the scenario, and then the expected results of the users actions.  Features and their scenarios help to keep the project organized and easily maintainable, but steps are the most important piece of the puzzle, especially for a cross-platform test suite.  The steps need to be specific enough to describe a users action or assertion, but vague enough that it can apply across platforms, and also be reused.  Using the search feature example above, a user will likely press a "Search" button or icon for each test, prior to the product assertion.  You are able to define the pressing of the search button specifically, with something like: "When I press the search button after entering a product id", but then that step will only be able to be used in a single scenario.  Instead, it is wise to use a step like: "When I press the search button", which can be reused across multiple scenarios.  If you are starting to automate existing manual acceptance tests, spending time planning out as ,many reusable actions as you can will save you from writing repetitive code later on.

Step Definition

Each step is mapped to a step definition.  A step definition will hook human-readable code such as "Then I press the search button" to platform specific code that actually executes the pressing of the search button.

I found the following diagram helpful in understanding how everything fits together:

calabash.feature.diagram

Gherkin

As we went over last week, Gherkin is a business-readable domain specific language.  Essentially, it describes a software's behavior, without detailing how that behavior is implemented.  This is the language that Cucumber understands, which allows these automated tests to be written.  Gherkin can be written in a number of different languages (60+ at the time of writing), so you can use the keywords in your own language.  A Gherkin file includes a few key words for Cucumber to pick up on.  To start the feature, is the  keyword Feature: followed by some short descriptive text about the feature.  The next few lines can be unparsed text which is used to describe the business value of the feature.  Next, you will start the scenario with the keyword Scenario: followed by some short descriptive text about the scenario.  Following the scenario line are the steps that makeup the scenario.  These use the keywords Given, When, Then, And, & But. These keywords don't actually mean anything other than helping Cucumber pick them up, and to make the steps easy to read.  For example, the step "When I press the search button", can be written elsewhere as "And I press the search button", and is mapped as the same step definition.

Generally, Given is used to define any precondition, and to put the system in a known state before the user starts interacting with the system.  When is used to describe a key action the user performs.  Then is used as an assertion, or to observe an outcome.  And & But are mainly used when you have multiple givens, whens, or thens in a scenario to make the reading more fluid.  Again, these are general guidelines, so you aren't mandated to use them in this way, but it does make the scenarios much easier to read and understand.

I am a visual learner, so here is a sample .feature file from one of my projects to help you understand this concept:

calabash.feature.sample

You will notice that there is an @smoke_test text and @help text above a scenario.  These are called tags.  We will get into tags later, but for now, know that a tag is a simple way to run a particular set of scenarios, or even a single scenario within a test suite. 

Feature for sample project

Before we finish today, I am going to post the sample .feature file we will be using for this tutorial.  So here it is:

sampleProject_feature_file

Well thats all for today.  Now go get started converting those manual acceptance tests into Gherkin so that they can be automated.  Next week I will go through hooking the Gherkin steps into step definitions.