Chapter 6. Automating the scenarios

So far, you’ve seen how you can describe and discuss your requirements very effectively using concrete examples. You also learned how you can express these examples in a loosely structured format built around the “Given ... When ... Then” structure.

A lot of the value in BDD comes from the conversations around these scenarios. This is why collaborating to write these scenarios is so important. Not all scenarios need to be automated; some may be too tricky to automate cost-effectively, and can be left to manual testing. Others may only be of marginal interest to the business, and might be better off implemented as unit or integration tests. Still others may be experimental, and might not be understood well enough to define clear scenarios; in this case, it could be worthwhile to do some initial prototyping to get a better feel for what’s really needed.^[1]

¹ Highly experimental startup applications might fall into this category. If the business needs to get market feedback to discover what features they really need, it will be hard to formalize scenarios with very much detail.

6.1. Introduction to automating scenarios

This scenario is just loosely structured text. It explains what requirement you’re trying to illustrate and how you intend to demonstrate that your application fulfills this requirement . But how you actually perform each step will depend on your application and on how you decide to interact with it.

Tools like JBehave and Cucumber can’t turn a text scenario into an automated test by themselves; they need your help. You need a way to tell your testing framework what each of these steps means in terms of your application, and how it must manipulate or query your application to perform its task. This is where step definitions come into play.

6.2. Implementing step definitions: general principles

There are many BDD tools you can use to automate scenarios like the ones we’ve been looking at, using different languages and often targeting different environments. The choice of which tool is right for your team will depend on how comfortable the team is with a particular language and environment (and how willing they are to learn a new one!), what technology stack your project is using, and the goals and target audience of your BDD activities. Throughout the rest of this chapter, we’ll compare and contrast several of the major BDD tools. The list of tools we’ll study is far from exhaustive, but the techniques we’ll discuss should be generally applicable no matter which tool you choose.

To explore the features of each tool, we’ll look at how you might automate scenarios involving the Flying High airline’s Frequent Flyer application, introduced in chapter 3. The Frequent Flyer program is designed to encourage flyers to fly with Flying High by allowing them to accumulate points that they can spend on flights or on other purchases. In this chapter’s examples, we’ll study some of the requirements around accumulating and managing Frequent Flyer points.

6.3. Implementing BDD more effectively with Thucydides

You’ve already briefly seen Thucydides in action in chapter 2, where it helped generate the test reports. When you use it with a fully supported BDD tool, it helps you break down scenario implementations into smaller reusable steps and report on these steps in the living documentation. At the time of writing, Thucydides works well with JVM-based testing libraries such as JBehave and JUnit, but you can also import test results generated by other BDD tools such as Cucumber-JVM,^[4] SpecFlow, Cucumber-JS, or Behave.

But the specialty of Thucydides is taking the test results produced by BDD tools like JBehave and turning them into rich, well-integrated living documentation, such as by relating the test outcomes to the requirements they test. This gives stakeholders a clear picture of not only what tests have been executed, but what requirements have been tested, and how well they’ve been tested.

6.4. Automating scenarios in Java with JBehave

JBehave (http://jbehave.org) is a popular Java-based BDD framework that was originally written by Dan North. In JBehave, you write step definition methods in Java or in other JVM languages such as Groovy or Scala.

We’ll look at how to use JBehave in conjunction with Thucydides, as Thucydides supports a number of conventions that make setting up and configuring a JBehave project simpler.

6.5. Automating scenarios in Java using Cucumber-JVM

The next tool we’ll look at is Cucumber (http://cukes.info). Cucumber is a very popular BDD tool from the Ruby world. Using Cucumber in Ruby is well documented on the Cucumber website and in The Cucumber Book,^[9] so in this section we’ll focus on writing scenarios using Cucumber-JVM. Cucumber-JVM is a more recent Java implementation of Cucumber, which allows you to write step definitions in Java and other JVM languages. Teams familiar with the Ruby version of Cucumber may be more comfortable with the Cucumber flavor of “Given ... When ... Then” scenarios than with the JBehave variation. But at the time of writing, Thucydides doesn’t yet provide full Cucumber integration.

⁹ Matt Wynne and Aslak Hellesøy, The Cucumber Book (Pragmatic Bookshelf, 2012).

6.6. Automating scenarios in Python with Behave

Python is a popular, general-purpose, open source dynamic language, and there are several Gherkin-based BDD tools available for the Python language. Cucumber itself can be used to run scenarios written in Python, though it uses a Python interpreter embedded inside a Ruby process, which can be brittle. For those who prefer a pure Python solution, there are currently three tools available: Lettuce (http://pythonhosted.org/lettuce), Freshen (https://github.com/rlisagor/freshen), and Behave (http://pythonhosted.org/behave).

These tools are similar, but at the time of writing, Behave is the most stable, best documented, and most feature-rich of the three. Despite the name, Behave has no relationship with JBehave. There’s also a Thucydides plugin that allows you to generate Thucydides reports from the Behave test results.

6.7. Automating scenarios in .NET with SpecFlow

If you’re in a .NET environment, your best option for BDD is SpecFlow (http://specflow.org). SpecFlow is an open source Visual Studio extension that provides support for Gherkin scenarios in the .NET and Windows development ecosystem. The rest of this section will assume that you’re reasonably familiar with the Visual Studio development environment.

6.8. Automating scenarios in JavaScript with Cucumber-JS

Unit testing is well supported in JavaScript, and low-level BDD unit-testing libraries like Jasmine and Mocha are widely used. We’ll look at these libraries in chapter 10, when we discuss BDD-style unit and integration testing. But in this chapter we’ll focus on options available for higher-level BDD in JavaScript, using Gherkin-style scenario definitions.

There are a number of tools available to JavaScript developers that support Gherkin scenario descriptions, including Cucumber-JS (https://github.com/cucumber/cucumber-js), which is probably the best known of the JavaScript BDD libraries, and Yadda (https://github.com/acuminous/yadda), an alternative to Cucumber-JS that allows more flexibility in the scenario wording.

6.9. Summary

In this chapter, you learned about automating scenarios:

You should now have a good idea of how to automate your acceptance criteria using the language and framework of your choice. But automating scenarios is only the first step—you also need to get these automated scenarios to actually test your application. In the next chapter, you’ll learn how to turn these automated scenarios into effective, expressive, and maintainable automated acceptance tests.