Assignment 1: BDD Introduction

Motivation: Software Development Challengs

If it ain’t tested, it’s broke. This is a quote from an avionics testing engineer. Testing cannot prove software works, but its almost certain that untested software does not work. The typical solution is do a bit of manual testing after writing code, but these tests are not repeatable. Written tests are slightly better, but too expensive to re-execute often. They do little to improve stability. But you probably have a lot of experience with software working today but not tomorrow. Errors always creep in. It critical to establish a thorough, automated test environment early in the project, and clients are unlikely to find systems without tests to be useful or maintainable.

A starting point is to write module-level tests with tools like JUnit or NUnit and to augment them with higher level tests for a few key features. While this sort of testing can be effective, it is not readable by non-technical clients. It is also too detailed for technical clients who have other responsibilities. To provide confidence, clients need thorough, executable tests that they can read.

With a bit of reflection, you may realize the tests have already been written. All PBIs should have at least one detailed acceptance criterion (AC). If it were possible to execute these AC, the developers could focus more on implementations than writing tests.

This is the idea behind Behavior-Driven Development (BDD): use tools to translate AC into executable tests, then run those tests to confirm PBIs are fully impelment. This gives a direct path from AC to testing, putting the product owner in charge of what is being tested. A full suite gives a simple definition of when a project is done: it is done, and acceptable to the client, when all tests pass. As a side effect, this also documents required behavior in an executable, readable format. People who are interested in details about system behavior can simply read the tests. BDD makes system requirements explicit and testable in a way that cannot be achieved by standard requirements documents and standard testing.

Of course, writing thorough AC is challenging. Often clients and product owners do not have the skills. They might choose to hire a team of specialists to write them, but that is also not always feasible. This means the development team may need to write the AC instead. But the win of (automated) BDD is that the tests can at least be reviewed with clients using language they understand. (And if a client cannot be bothered, one wonders about whether they have any commitment to the project!) This means that the AC document expected behavior as approved by clients; the AC become an executable requirements document. If an error is found, the solution is simple: write AC revealing the error and create a PBI to get those AC to pass. BDD closes the loop between project concept and tested implementation. If you ask a competent software developer about whether they use some form of BDD, you are likely to get the response “who wouldn’t?”

An example, and Part 1 of the assignment

Visit https://gitlab.com/hasker/cucumber-phonebook. The README for this project describes how Cucumber can be used to implement BDD. This project is written in Java, but Cucumber supports many development languages and over 70 spoken languages. Thus you can write your code in C or Go and your AC in Česky (Czech) or Tiếng Việt (Vietnamese). It is possible to integrate Cucumber with any development framework, so it works for web-based projects as well as GUI applications.

To learn more about BDD and to see how Cucumber implements it, clone the repository and work through the README, completing at least exercises 1 and 2. Note the CI history for this repository captures evidence showing you completed this part of the assignment, so you should not need further documentation.

As an aside, review Build | Pipelines and note this project dates

to 2022. One of the strengths of using Docker is that tests that ran years ago can still run today with minimal updates.

Part 2 of the assignment

Applying BDD to your own project requires understanding regular expressions (REs). The README is targeted at a more general audience, so has little information about REs. The good news is that many tests can be written with just a few REs:

    ^I add ([a-zA-Z]+) with number (\\d+)$

matches text starting with “I add”, containing a word followed by “with number” and a numeric value. The numeric value must end the text. Thus the test step

    When I add Xander with number 123

matches this with “Xander” as the first item captured and “123” as the second.

While this gives a basic understanding of regular expressions, you will need more to integrate BDD into your SDL projects. Complete the tutorial and exercises at https://www.regexone.com/, capturing evidence as requested in Canvas.

Refinements

The README contains minimal detail about writing tests in feature files. There is additional information that SDL students should know:

    Scenario: add a person and list
      * an empty phonebook
      * I add Xander with number 123
      * Xander's number is 123
      * the number of entries is 1

This simplifies reading through thousands of scenarios. Use Given/When/Then/And for SDL projects.

Tables

Suppose a video game displays a graphic based on the player’s performance: a black hole if the score is below 10, a sun if the score is above 10, and a supernova if the score is above 10 with no lives lost. You could write AC like

    Given the score is 5
    When the user reaches the end of the game
    Then the system displays a black hole

But this is going to get tedious if you add just a bit more complexity, and it will be difficult to ensure all cases have been covered. Cucumber supports tables:

    Scenario Outline: game ending results
    Given the score is <displayed_score>
    And the number of lives lost is <lost_lives>
    When the game ends
    Then the system displays <graphic>

    Examples:
    | displayed_score | lost_lives | graphic |
    |       3         |     9      | black hole |
    |       3         |     0      | black hole |
    |      11         |     5      | sun |
    |      11         |     0      | supernova  |
    |      10         |     5      | sun |
    |     500         |    89      | sun |
    |      -1         |     2      | black hole |

Some things to note:

Writing better scenarios

One issue that comes up frequently is how to share information between scenarios. It might be tempting to have one scenario create the setup for another, but this violates the testing principle that every test must be independent from the others. Dependencies between scenarios make the tests unstable. If several scenarios depend on the same setup - say, a cart in an ordering application with 5 items in it - then write a step definition for that setup:

    @Given("cart has standard small order")

Alternatively you can write before hooks that standardize setups, but using step definitions to achieve this both simpler and more flexible.

Naming steps

When requirements change, these are often reflected in changes to Then clauses. Name your steps by the Given and When clauses to minimize test maintenance.

Using personas and other recognizable items

To make tests more readable, be consistent about the names of users that are in your tests. Maybe Drew is the type of shopper who buys just one item on a visit, while Terry is the type who will buy five items but return one or two. You can then have standard setups based on the different types of shoppers to make scenarios more understandable. You might even add some adjectives: “StingyDrew” and “ExtravagantTerry”. Your goal should be to make scenarios as readable as possible.

Wrap-up

It takes effort to integrate Cucumber into your CI system, but it pays off tremendously. Trying to run a project without BDD and CI is like trying to race in mud:

Lamborghini Countach spinning tires in mud

You might make forward progress eventually, but it will be too little too late.