BDD Assignment

This assignment will give you experience with applying behavior-driven development (BDD) to your SDL project. This assignment assumes that you have successfully completed the BDD reading and quiz.

Setup

1. Start by copying the files of one of the available BDD projects to your SDL project. Java teams can use

   • https://gitlab.com/hasker/cucumber-phonebook If you need a different language, talk to your instructor to obtain a sample project. Note the sample projects likely have a different structure than what you have been using. We will discuss converting your project to the appropriate structure below.

2. Without making any changes to the source files for the sample project, use Docker to run the project: docker build .

3. Once this simple build is working, move your source code tree into the src folder and confirm docker build . still works. We are breaking this into very small steps because we are using maven as the build environment for Cucumber and it is very sensitive to the folder structure.

4. Pick a simple function from your project - say, one that returns a computed string or integer - and add a step definition of the form Given we call function XXX with parameters A and B then the result is C where you fill in the XXX, A, B, C, and any other values as appropriate. The step definition should end with an assertion matching C against the actual returned value.

5. Add a test to the project with the GIVEN clause we call function XXX with parameters A and B and the THEN clause the result is C and check that all tests pass.

6. Delete the provided tests (the ones you copied in step 1) and ensure your one new test passes.

7. Commit your code and check that the GitLab CI test passes.

The assignment

This first stage should take several hours. If you are able to complete it quickly, you may have missed something important! But if you find yourself spending more than five or six hours, talk to your instructor to see if you need to find simplifications.

1. In consultation with your instructor, identify two themes for your testing. Different students will have different themes. The themes must involve interactive portions of your project and must be reasonably rich. For example, you might identify several user input validations, some results to be computed, or security procedures. Do not pick logging in to the system; while the team should write BDD tests around logins, it is typically too simple a problem to provide a good opportunity for showing your testing skills. Ideally, the themes would be for code written by others on your team, but we recognize this is not always feasible.

2. Write step definitions and test files for your theme. Document all step definitions you write and all test specifications. Do this incrementally; it will be very difficult to get all of the tests working in one big bunch. Do frequently merge your changes into the Dev branch as you get portions working so others on your team (who might also be doing this assignment) can ensure their tests work with yours. For full credit, your test specifications must be readable by end users.

3. If you have not had opportunity to introduce a table-based test, do that now. Simply change some of your test cases so they have multiple test cases. Ideally each test cases should test a different aspects such as edge cases or different interactions between user inputs.

At this point your themes should be well tested using Cucumber. Review your work to ensure coverage of the theme. We anticipate you should have at least ten high-level tests with one or more test specifications using tables. If you are not achieving this goal, talk to your team mates or instructor to get ideas of additional aspects to test.

Once at least one set of tests have been introduced, remove the “test function XXX” step definitions you were instructed to create in the Setup section. These are unlikely to provide long-term value or be readable by end users!

Self-Evaluation

Create a document for your BDD testing effort. Include the following:

1. Your (full!) name, your email address, the date, and the project

2. A paragraph or two describing your two themes. This needs to be more than a sentence or two since you will need to give context. You can assume the reader knows the basic goal of your project, but not that they are familiar with the aspects of the system you are testing. Thus you will need to include information about the user’s goals, what sort of information they are entering, and what they would expect to see in the output.

3. A screenshot of CI output from GitLab showing that your tests pass. Be sure the screenshot is focused on the tests and the run context; do not include lots of blank space because that will make the text small. Highlight or circle where CI executes the tests that you wrote and the output that shows all tests pass.

4. Introduce a software error in the system that breaks your tests. This “tests the tests” by showing that the tests can catch programming errors. Be careful: do not simply introduce a syntax error or delete sections of code because that would then just shows that code that does not build at all fails. That is uninteresting. It is also uninteresting to change the tests you wrote. The goal is to show that if someone introduces a logic error in the code related to your testing theme, your tests will detect that error. Ideally, you would fail some tests but pass others, showing that the tests are not all redundant.

5. Document the error(s) you introduce and take a screen shot showing that the tests fail. You can use docker locally to obtain this evidence, but you can also use GitLab CI. In any case, once you have captured the evidence, restore the code to working condition.

6. Write a paragraph describing what sorts of errors your tests are likely to catch. That is, explain why the testing you added is important to long-term project success.

Review

Your instructor will have another person on your SDL team review your document and complete the following:

1. Document the project, the date, and your full name.
2. Confirm the documented tests pass. Capture GitLab evidence showing this.
3. Write a paragraph discussing the importance of the tests. Is this an area of high concern to users, or would most users see the tests as unimportant detail? Assume your users are not programmers when writing this paragraph.
4. Write a paragraph evaluating the understandability of the tests. What project-specific terminology must they know? Is this terminology likely to be understood by a user who would use this aspect of the system? Are the expected results clear? Are there other opportunities for confusion?
5. Write a paragraph or two evaluating the strength of the tests. Are there likely programming errors that would not be caught by the tests? Are there additional cases you would like to see tested to achieve additional coverage?

Submit this evaluation document to your instructor. The quality of your review will be an important portion of your grade. Note that finding weaknesses in the tests you are evaluating does not necessarily mean your team mate will have a lower grade! There is always more to be tested, and the primary goal of this review is to ensure basics are tested.

Grading Rubric

The full grade is based on both the work creating Cucumber tests and the review of those tests.

• 20%: Both themes have been tested with appropriate step definitions, and one step definition includes a table.
• 20%: The tests are important: they can fail and many likely programming errors can cause those failures.
• 20%: Step definitions are understandable by end users.
• 20%: Test specifications provide reasonable coverage for the identified testing themes.
• 20%: The review of a team-mate’s self-evaluation covers all required elements (as specified above).

Partial credit is assigned in each category as appropriate.

Credits and Relationship to the greater Software Engineering curriculum

This section is informational; there are no requirements for students.

This assignment is the product of a project from summer, 2025, by R. W. Hasker. It is worth documenting how this assignment relates to the other elements of the Software Engineering curriculum:

• A core element of this assignment is ensuring tests are executable. This relies on programming skills obtained in the first course in the curriculum, CSC 1110 (Software Development).
• The second course, CSC 1120 (Data Structures and Graphical Interfaces) further reinforces programming logic as well as introduces graphical user interface programming. The logic is applicable to writing and debugging tests, and the skills in programming graphical user interfaces are applicable to testing end-user output.
• Many of the Software Development Lab projects are web-oriented projects, so for those projects the SWE 2511 Web Applications will be important.
• Understanding the role of testing in software development is an important theme of SWE 2710, Software Tools and Process.
• Understanding user needs is a major goal of UXD 1001, Human-Centered Design. This is the basis of evaluating the understandability and importance of tests.
• SWE 2721, Introduction to Software Verification, provides the tools for identifying test cases, especially “edge cases” involving unexpected inputs that often lead to failures.

A careful review of the Software Engineering curriculum will show that not all of these courses are prerequisites to SWE 3720, Software Development Lab II. However, most students will have had the listed courses before taking SWE 3720, and so would be in a position to help the student who may not have seen relevant material. Instructors would be in a good position to provide a quick review of any missing material as well. Students on other paths will benefit from having first-hand experience with BDD on an advanced project when taking those courses.