Chapter 2. The specification layer and the automation layer
This chapter covers
- Understanding an executable specification’s layers
- Understanding the structure of those layers
- Writing your first Given-When-Then scenarios
- Exploring how the specification layer interacts with testing code
If you haven’t worked with Gherkin and specification by example (SBE) before, chapter 1 may have shown you your first Gherkin scenario (listing 1.1). We haven’t yet written a full Gherkin specification document together, though, and we’ll need more practical examples to do that.
Chapter 1 also talked about how conversations captured in Gherkin become automated tests and how automation keeps executable specifications up to date. But we didn’t discuss any details. What does automation mean in the context of SBE? How can you automate conversations? Are there any tools you can use? These are all legitimate questions.
In this chapter, you’ll learn to write a simple executable specification from scratch. Throughout the chapter, you’ll work on an example of a calendar application that schedules meetings for its users. (Think Google Calendar.) You’ll write a simple scenario about what it takes to create new meetings. By doing so, we’ll also explore the basics of the relationship between the two layers of every executable specification, in order to understand the technical limitations behind the Given-When-Then template.
Reading Gherkin listings
A quick note on how to read the Gherkin listings placed here and there throughout the book. Because the goal is to get as close to writing the perfect Gherkin scenario as possible, some of the listings are rated on a scale from [BAD] to [BEST]. (Not all scenarios are comparable, so not all are rated.)
Listings marked [BAD] are anti-patterns, should be avoided, and are mentioned only to warn you about their bad effects. The [OK] listings are neither good nor bad; they’re mediocre. The [GOOD] and [BETTER] listings are at the quality level you should strive for after reading this chapter. And the [BEST] listings contain non-obvious solutions that become possible only after you’ve applied the advanced tips and tricks from this chapter. The best scenarios don’t happen every time, but when they do, it’s extremely satisfying.
Every executable specification has two layers: a specification layer and an automation layer (see figure 2.1). Without the specification layer, an executable specification wouldn’t be a specification. Without the automation layer, it wouldn’t be executable. This section provides a brief overview of both.
Figure 2.1. Every executable specification in Gherkin has two layers that complement each other and create automated tests written in natural language.
This book deals mainly with the specification layer and the writing aspect of executable specifications—the art of writing Gherkin. But to do that well, you need to know the basics of the automation layer, too. That’s because automation requires the specification layer to be written in a certain way that allows computers to read it.
Simply put, the specification layer of an executable specification is the document you read when you want to know what you’re going to build. Gherkin specifications are usually short, and even a small project will have multiple documents—all of these documents together form a large specification layer for the entire project.
Definition
Specification layer—Contains text documents written in Gherkin that humans can read
It’s important to understand these three elements of the specification layer (see figure 2.2):
- Acceptance criteria
- Scenarios
- The ubiquitous language
Figure 2.2. The specification layer of an executable specification is the layer where conversations about acceptance criteria are conveyed as Given-When-Then scenarios that create the most accurate reflection of the ubiquitous language between technology and business.
I’ll talk about acceptance criteria, scenarios, and the ubiquitous language in detail later in the chapter, but here’s a brief summary. Chapter 1 said that specifications talk about requirements and acceptance criteria that the delivery team needs to understand if they want to implement new features correctly. Gherkin organizes the acceptance criteria by conveying them as scenarios written using the Given-When-Then template. The scenarios become elaborate expressions of the ubiquitous language, which is a common language among developers, business stakeholders, and end users. As you saw in chapter 1, the scenarios are a way to store conversations about requirements in a written form: they’re a reflection of how the delivery team talks with the business about important domain concepts. You’ll see a similar situation play out later in this chapter.
Feature: Teams
Scenario: Enforcing access rules
For more information on availability, please look at the
invite confirmation scenarios later in the specification.
In short, an unavailable person who already confirmed RSVP
can be invited to another event at the same time, but will
be able to attend only one of them.
Given Mike, Kate, and John were assigned to the same team
And Ada was assigned to another team
When Kate and Mike schedule a 1 hour long meeting at 4 p.m.
Then John should see that Kate and Mike will be unavailable
But Ada shouldn't be able to see Kate and Mike's meeting
As explained in chapter 1, the sequence within each scenario is called the Given-When-Then template. The template and some other repeatable structures visible in the example are what enable automation of every Gherkin document.
The specification layer is created by writing feature files.
Definition
Feature file—A file with a .feature extension written in Gherkin that groups related scenarios within a document that provides a specification for a single functionality
As a matter of principle, both technical and nontechnical stakeholders can have access to the specification layer of the specification suite. They should be at least able to read and understand it if they have to. But as I mentioned, the specification layer also has to be formatted in a way that allows for automation, so some constraints, predefined grammar, and strict syntax are always present. This chapter will show you the basics of Gherkin’s automation syntax later on.
Definition
Specification suite—A collection of all executable specifications and the testing code for a single project. This collection is usually put in the real codebase where it can be automated and executed.
The programmers and testers on your team will use scenarios to write the testing code. The testing code is the executable part of your executable specifications. Every part of the specification suite related to executing automated tests is contained within an automation layer.
Definition
Automation layer—The automation layer executes a simulation of the implemented application to see if the code behaves as defined in the specification.
To make a long story short, every time you want to write a Gherkin specification like the one shown earlier, you have to add a new .feature file to the specification layer of your specification suite, write the scenarios there, and then add the testing code to the suite (see figure 2.3).
Figure 2.3. A specification suite, as well as every executable specification in the specification suite, can be split into two layers—the specification layer and the automation layer—which work together to create a functional testing process.
A specification suite is a kind of test suite. In software development, a test suite is a collection of test cases that are intended to be used to test a software program to show that it implements a specific set of behaviors. Because the tests for executable specifications share some qualities with traditional tests but are different in other aspects, the name should be both familiar and new—thus, a specification suite.
It’s important to understand these three aspects of the automation layer (see figure 2.4):
- The automation layer is a simulation of a working application that behaves as the Given-When-Then scenarios tell it to.
- The simulation performs acceptance tests that check whether the system under test behaves in an acceptable way.
- The automation layer is designed to be a feedback loop that makes accepted specifications pass the testing process and rejects specifications that weren’t accepted, marking them as broken.
Figure 2.4. The automation layer runs a simulation of the system, which uses a series of acceptance tests to check whether the system under test works as you expect it to.
I’ll talk about feedback loops more in chapter 6 when we get to the topic of an executable specification’s life cycle.
As an example of how the automation layer works, suppose you want to test whether a calculator app works well. To do so, you need some samples of addends and the results you expect the calculations to return. The automation layer will then be able to execute the testing code responsible for performing calculations and see whether the result returned by the code matches the expected result defined in the specification (see table 2.1).
Table 2.1. An example of the automation layer’s behavior
| Input given |
Automation layer |
Expected output |
|---|---|---|
| 1 + 0 | The testing code runs ... | 1 |
| 1 + 1 | The testing code runs ... | 2 |
| 1 + 2 | The testing code runs ... | 3 |
| 1 + (-1) | The testing code runs ... | 0 |
Running the automation layer with Cucumber
The Given-When-Then template’s syntax enables automation. That’s why tools like JBehave and Cucumber use Gherkin, which is a more elaborate evolution of the template. But Gherkin isn’t the only available option. Different tools for automated acceptance testing impose their preferred syntax differently. Some tools, like FitNesse, use a wiki-like system as the main way of organizing the specification layer.
JBehave, Cucumber, and FitNesse are all test execution engines.
Definition
Test execution engine—A type of software that’s written to automatically test other software. Throughout the book, I use terms such as test execution engine, test runner, testing framework, and automation framework interchangeably.
This book teaches a Cucumber-flavored version of Gherkin.
Definition
Cucumber—A testing framework that can match Gherkin feature files with testing code. It lets programmers run the code that automates the scenarios.
Cucumber emerged from the Ruby community, but it also supports or has an equivalent for .NET, C++, JS, Lua, PHP, Python, and JVM. It can run Gherkin files written in more than 60 spoken languages—among them Chinese, French, Arabic, and Pirate English.[1]
The division of labor is simple (see the following figure). Gherkin handles everything related to capturing conversations about the business logic, from rewriting acceptance criteria as scenarios to modeling the business domain using the ubiquitous language. Cucumber, on the other hand, allows the delivery team to test the system using the same examples that were captured during the analysis and design phases.
Why did I choose Cucumber and Gherkin for the book? According to the popularity of its open source repository, Cucumber is rising as the most popular testing framework supporting executable specifications, decisively beating both JBehave (10 times as popular) and FitNesse (twice as popular). But I discuss a lot of techniques throughout the book that can come in handy for people who use other test runners, too. (Unfortunately, there are some differences between Cucumber’s Gherkin and JBehave’s Gherkin in terms of syntax and indentation; due to the scope of the book, you’ll have to research the differences between Cucumber and JBehave on the your own.)
You’re probably wondering, “Will this book teach me to automate executable specifications with Cucumber?” Spoiler alert: it won’t. If you want to read more about automating Gherkin tests with Cucumber, read the appendix. It’s a quick tutorial about how to write and run simple automated tests for your feature files. I have to warn you, though: the appendix covers only the basics.
A huge variety of learning resources is already available that teach how to deal with the automation layer, so instead this book focuses on mastering the Gherkin layer—a topic that’s underrated by many developers and testers. Also, because you can use Gherkin with many programming languages such as Ruby, Java, and Python, I wanted to stay as language agnostic as possible. If you’re curious about how to write and execute automated tests, you can easily find a lot of articles on the web. This chapter includes a brief introduction to the topic of automation, explaining only as much as is needed to understand the depths of the specification layer.
At the beginning of this chapter, I mentioned that you’ll work with an example of a calendar application for teams, similar to Google Calendar (see figure 2.5). Every calendar application must allow teams to schedule meetings—to keep things simple, that’s the one and only thing you’re going to specify today.
Figure 2.5. You could imagine your application is a simpler clone of Google Calendar, Microsoft Outlook, or any other popular scheduling software you and your friends use.
It’s time to write some Gherkin. This section presents a step-by-step tour of writing your first scenario. To begin, create an empty text file in any text editor of your choice.
Every Gherkin .feature file starts in a similar manner. You’ve already seen it twice in previous examples of executable specifications:
Feature: Scheduling
Feature is Gherkin’s word for indicating that a new specification has begun—what follows is the name of the feature being documented. Conventionally, every feature file consists of a single Feature in order to encourage writing many small files rather than a few large specifications.
Warning
Don’t get too used to the Feature keyword. In chapter 8, you’ll replace it with more-advanced keywords such as Ability and Business Need that allow for better categorization of requirements.
Below the Feature line, you can add a description of the feature. It’s expected to describe the business value of this feature or to provide additional information that makes the feature easier to understand. In this book, I call this a specification brief.
Definition
Specification brief—Specification line containing important pieces of information such as an answer to the question of why a specification was written in the first place, who the most important stakeholder is, and why they need it.
For the time being, let’s keep the feature file simple. Given the example’s scope, you’ll specify only one or two scenarios. You should inform the reader where other scenarios can be found.
Listing 2.1. Feature line followed by a specification brief
Feature: Scheduling Because scheduling is a huge functionality, this #1 specification file describes only the most important #1 high-level scenario. #1 You can find more detailed scenarios in the rest #1 of the files inside the "meetings" folder in the #1 specification suite. #1
From the point of view of Gherkin’s syntax, the first line is the most important, because the Feature: part can never be changed. (Okay, there are two exceptions, but I’ll talk about them at the end of the book.) The spaces in front of what follows are important, too, because the spaces tell Cucumber what the file’s structure is. There are no strict rules or limitations regarding vertical spacing between the first line and the specification brief. Personally, I like to add a blank line because I think it makes the document clearer. But if you don’t agree, you don’t have to do that.
Exercise 1
Write the rest of the specification brief that explains who will use the scheduling features and why they need them, based on what you know about the application. You can draw some inspiration from the description of the app at the beginning of this section.
In chapter 1, I said that Gherkin captures behavioral requirements. Behavioral requirements are formed as stories about how users behave when they interact with the system. In Gherkin, these stories are called scenarios. In this section, you’ll start writing your first scenario.
Listing 2.2. Your first scenario
Feature: Scheduling Because scheduling is a huge functionality, this specification file describes only the most important high-level scenario. You can find more detailed scenarios in the rest of the files inside the "meetings" folder in the specification suite. Scenario: Creating a meeting #1
In this example, the Scenario line lets both the reader and the automation framework know that a new scenario begins below it. If a scenario needs additional elaboration, you can place any amount of free-flowing text between the Scenario line and the first Given. It’s similar to the specification brief, and that’s why it’s called a scenario brief. There are multiple ways to use the free-flowing space, as we’ll discuss throughout the book: for example, to provide definitions for domain-specific concepts, which you’ll see in action in chapter 7 during the discussion of living documentation.
Every scenario should do the following:
- Define context (the Givens)
- Describe an event that occurs within the system (the Whens)
- Ensure that expected outcomes take place (the Thens)
The sequence is called the Given-When-Then template.
The Givens explain what needs to happen so you can watch the rest of the scenario take place. The Whens neatly organize the template around a single behavior of the system, so a reader doesn’t have to wonder what the purpose of the scenario is. The Thens clarify the consequences of taking that action.
The Given-When-Then template is a simple yet powerful tool. It often works as a harmonious system. A slight change anywhere may influence a new change elsewhere. Givens, Whens, and Thens influence each other, but sometimes they force entire scenarios to change—and when scenarios change, sometimes entire specifications must change as well. I talk more about this in chapter 3, which covers all the details of the template.
Ah, the Givens. Givens answer a single question: what are the prerequisites that allow the scenario to happen?
For example, when a scenario’s main action is, as in this case, creating a meeting, there must be some users who will be able to perform the action—so a user account must have been created first.
Listing 2.3. [OK] Your first Given
Feature: Scheduling
Because scheduling is a huge functionality, this
specification file describes only the most important
high-level scenario.
You can find more detailed scenarios in the rest
of the files inside the "meetings" folder in the
specification suite.
Scenario: Creating a meeting
Given a user #1
Tip
You can see that there are two more spaces before the Given. Scenarios require additional indentation because it helps Cucumber understand which Givens, Whens, and Thens belong to which scenarios.
Given a user—that sounds vague, doesn’t it? All apps have users. And pretty much anyone can be a user. First, this naming doesn’t explain anything. Second, you’re working hard to make your application unique; the specification should reflect that, too.
Listing 2.4. [BETTER] Your first Given, reworked
Feature: Scheduling
Because scheduling is a huge functionality, this
specification file describes only the most important
high-level scenario.
You can find more detailed scenarios in the rest
of the files inside the "meetings" folder in the
specification suite.
Scenario: Creating a meeting
Given a team member #1
Better! Now, you can at least see that the scheduling feature is about collaboration. A team member sounds a bit abstract, though—and software should be made for real people.
Listing 2.5. [BEST] Your first Given, reworked a second time
Feature: Scheduling
Because scheduling is a huge functionality, this
specification file describes only the most important
high-level scenario.
You can find more detailed scenarios in the rest
of the files inside the "meetings" folder in the
specification suite.
Scenario: Creating a meeting
Given Mike, a member of our team #1
Users play key roles in scenarios, and it’s a good practice to use unique, real names for several reasons. I talk about them in the next chapter, when I introduce the topic of outside-in development. Right now, I’ll give you a simple example. Let’s say that two users appear in a single scenario. Calling them team member 1 and team member 2 would sound awkward, wouldn’t it? That’s why you’ll specify Mike—because Mike doesn’t sound awkward, and when things aren’t awkward, more people read and understand them. Hello, Mike. Do you like Gherkin? (I bet he does. It’s the sole reason you brought him into existence).
Givens create the context in which the rest of the scenario takes place. Let’s keep the momentum going and specify what the main action of the scenario should be. And, because Whens describe key actions the user performs, this is the perfect job for a new When. Let’s add one to the scenario.
Listing 2.6. Your first When
Feature: Scheduling
Because scheduling is a huge functionality, this
specification file describes only the most important
high-level scenario.
You can find more detailed scenarios in the rest
of the files inside the "meetings" folder in the
specification suite.
Scenario: Creating a meeting
Given Mike, a member of our team #1
When Mike chooses 2 p.m. as a start time for his meeting #2
As you can see, you’re aiming for an example as concrete as possible: in addition to the user introduced in the previous step, you’ve added specific hours. Also notice that the second step is aligned to the right so the word When ends at the same place Given does. This is a community convention that you’ll see in many feature files, but it isn’t obligatory. You could also write
Given Mike, a member of our team When Mike chooses 2 p.m. as a start time for his meeting
without aligning the When to the right under Given. We’ll continue to use this convention throughout the book, though.
Note that the scenario has already introduced some simple terms into your product’s ubiquitous language: a meeting, a team member, a start time. If programmers or designers read the scenario, they’ll adopt this dictionary without having to invent their own terms. Left to their own devices, technical experts often invent artificial names because they don’t know any better, and introducing artificial terms almost always widens the communication gap between business and delivery, instead of bridging it.
Every scenario should preferably have only one When, because that makes scenarios clearer and easier to read. I’ll talk more about such design rules in chapter 3, where you’ll learn about specifying user tasks and choosing the right abstraction level for your scenarios.
An action without an outcome is wasted. The part of the Given-When-Then template that asserts the outcome is the Then—so why don’t you add it to the scenario?
Listing 2.7. Your first Then
Feature: Scheduling
Because scheduling is a huge functionality, this
specification file describes only the most important
high-level scenario.
You can find more detailed scenarios in the rest
of the files inside the "meetings" folder in the
specification suite.
Scenario: Creating a meeting
Given Mike, a member of our team #1
When Mike chooses 2 p.m. as a start time for his meeting #2
Then he should be able to save his meeting #3
Thens describe consequences. In this case, choosing a valid start time for a meeting results in successfully creating the meeting. A Then is usually a concrete representation of the rule your criteria try to enforce. The representation is usually a change in the system. For example, it may be something new that was created—like Mike created his first meeting—but it may also be something that was removed or rephrased.
The main difference between the scenario in listing 2.7 and the acceptance criteria you usually see is that the scenario seems much more personal. That’s because
- Acceptance criteria are the abstract rules of the system; scenarios tell stories about people who use the system according to these rules.
- Human beings discuss and remember stories much better than they do abstract rules.
Adding a Then means that, in eight simple steps, you’ve arrived at a full Gherkin specification starting from an empty text file. You now have a simple test that specifies what needs to happen if you want to create an event.
Exercise 2
Now’s the perfect time for a quick theoretical break. Let’s slow down to process all the information discussed regarding the specification layer, from the standpoint of the automation layer. Givens, Whens, and Thens are also known as steps.
Definition
Step—The smallest unit of any Gherkin specification. It’s usually a single line of text.
All steps work on two levels:
- A step describes the business logic of the application in natural language.
- Each step is closely related to its underlying testing code, which checks whether the business logic described is implemented properly.
From the technical point of view, every step consists of a restricted keyword and the step’s content written in natural language that follows the keyword:
Definition
Keyword—A special word used to optimize the specification layer for automation. Keywords are imposed and enforced by the test execution engine. They must appear in every executable specification written in Gherkin.
Every keyword has a specific function to serve:
- Givens execute testing code needed to run a scenario: for example, creating accounts or prepopulating the database with example data.
- Whens execute the main action of a scenario that’s expected to change the state of the application by adding new data to the database, editing existing data, or integrating with an external service like email.
- Thens measure consequences of a scenario’s main action to check whether the state of the application changed as you expected it to.
There are also other keywords such as But and And—they’re replacements that help you avoid using the same word multiple times in a row. They make scenarios flow more naturally.
Here’s a full list of the keywords supported by Gherkin. Some of them you already know; others you’ll meet later in this chapter and in the chapters yet to come:
- Feature
- Background
- Scenario
- Given
- When
- Then
- And
- But
- Scenario Outline
- Examples
Scenarios and features use keywords to maintain their structure, too.
You probably remember the Feature keyword. In chapter 8, you’ll learn that Feature can be replaced with more-specific keywords such as Ability and Business Need. They can help you differentiate between your requirements and organize them in a clearer way—but for now, Features are all you need. Similarly, the Scenario keyword lets both the reader and the automation framework know that a new scenario has begun.
Exercise 3
Find all the keywords in the Gherkin file that you’ve written so far in this chapter, separate the keywords from the steps, and write the results in a table. Try to memorize the keywords; they’re the most basic ones, but they’re also the most popular.
Understanding indentation in Gherkin’s syntax
You may have noticed that, like Python and YAML, Gherkin is a line-oriented language that uses indentation to define structure. You can use spaces or tabs, but it’s usually considered a good practice to use spaces, because they maintain the structure more easily on everyone’s computers.
Line-oriented languages require you to keep every command within a single line. You can’t write a two-line Given, for example. The only exceptions to the single-line rule are free-flowing descriptions such as the specification brief and scenario briefs. Free-flowing descriptions aren’t part of the Given-When-Then template, and they don’t make any difference when it comes to automation. That’s why you can format them however you want.
Now is the perfect time for the quick theoretical break to end. You can be proud of yourself. A few pages in, you already know most of Gherkin’s syntax. Be alert, though: we’re not done yet.
Let’s review the scenario again. It’s a good practice to do so once in a while, for two reasons. First, Gherkin is prose—sometimes a bit awkward, but still prose. And prose striving for readability and clearness needs constant reviewing—especially if it’s written by technical people. (Ask my editors.) Second, every scenario is also a test, and delivery teams should continuously work on improving their tests in case they missed anything when they wrote them in the first place.
Listing 2.8. Expected output of the scenario
Feature: Scheduling
Because scheduling is a huge functionality, this
specification file describes only the most important
high-level scenario.
You can find more detailed scenarios in the rest
of the files inside the "meetings" folder in the
specification suite.
Scenario: Creating a meeting
Given Mike, a member of our team
When Mike chooses 2 p.m. as a start time for his meeting
Then he should be able to save his meeting
Have you noticed anything weird?
This scenario looks perfect—too perfect, some might say. Mike accomplished a total success without breaking a sweat. How often does that happen in reality? As a person whose job is to make software for a living, you know the answer all too well. It never happens. When real people interact with software, not only does a total success rarely occur, but sometimes it’s flat-out impossible—and the results look more like carnage.
Developers, designers, testers, and analysts can never assume that customers will use their products in a linear manner. They must always prepare to specify multiple paths a user can choose.
To see why, let’s get back to the example. We’ve taken SBE’s commitment to concrete examples seriously: you can’t get more concrete than the 2 p.m. start time in the When. Have you asked yourself, though, whether there’s any possibility that the start time may fail? For example, what would happen if Mike chose 2 p.m. as the start time, but it was already 3 p.m.? Wouldn’t that break the scenario?
Listing 2.9. Two similar scenarios with different outcomes
Feature: Scheduling
Because scheduling is a huge functionality, this
specification file describes only the most important
high-level scenario.
You can find more detailed scenarios in the rest
of the files inside the "meetings" folder in the
specification suite.
Scenario: Creating a new meeting successfully
Given Mike, a member of our team
And that it isn't 2 p.m. yet #1
When Mike chooses 2 p.m. as a start time for his meeting
Then he should be able to save his meeting #2
Scenario: Failing at creating a new meeting
Given Mike, a member of our team
And that it's already 3 p.m. #3
When Mike chooses 2 p.m. as a start time for his meeting
Then he shouldn't be able to save his meeting #4
By writing two different outcomes for the same action, you’ve written your first comprehensive test in Gherkin. You can either pass or fail a test. In order to make sure your specification suite has acceptable scenario coverage, you must specify both what happens when a user takes a successful approach, as well as an unlucky approach.
Testers usually call scenarios with positive outcomes happy paths. Beginning with the happy-path scenario is a good way to establish a key example as a foundation from which to think about other possibilities. Fortunately, testing heuristics suggest a multitude of other paths, including angry paths, scary paths, embarrassing paths, forgetful paths, and so on. We’ll talk more about all the possibilities in chapters 4 and 5, which are about choosing good examples for your executable specifications.
Exercise 4
What are other circumstances that may result in failures when Mike creates a meeting? Write a third scenario that adds more detail to the specification. You can use a scheduling conflict in Mike’s calendar as a reason for failure, or you can invent your own example.
The two scenarios in listing 2.8 are your first real tests written in Gherkin. But didn’t I say in chapter 1 that SBE’s practitioners automate their tests? How does that happen? I talked a bit about Cucumber, the test execution engine, earlier in this chapter, but I haven’t said anything concrete. How are you going to write good tests in Gherkin without learning to write real testing code in Cucumber? Is that even possible? Let’s decouple these worrisome questions and take them one by one.
Definition
Step definition—An implementation of the testing code for a single step. Step definitions are containers for the testing code that must be executed in order to run the steps in the automation layer.
For example, let’s consider possible step definitions for testing a notification email that’s sent after Mike creates his event. You can find some practical examples of step definitions in table 2.2.
Table 2.2. What needs to happen under the hood to test whether an email was sent?
| Keyword |
Testing code in the step definition |
|---|---|
| Given | Establish connection to the database, populate it with some testing data, or configure the application to a desirable state |
| When | Simulate sending an email based on the data fetched from the database |
| Then | Check if the email was sent by integrating with the mailing service’s internal queue |
In theory, the relationship between features, steps, and step definitions is pretty simple. Figure 2.6 illustrates the high-level flow.
Figure 2.6. Gherkin features test a simulation of the working system with testing code executed through step definitions.
Every feature contains multiple scenarios that contain multiple steps. These steps have corresponding step definitions that are responsible for running the automation code that affects a simulation of a system, as well as checking whether the code yields expected results.
A middle-sized Gherkin project can contain between 200 and 600 scenarios. If you assume that every scenario has at least 3 steps, each of which is one of each kind, that’s between 600 and 1,800 steps in a single specification suite. And remember that, unlike keywords, a step’s content isn’t predefined; you can write whatever you like as long as it fits within a single sentence and makes sense, given the rest of your scenario. All in all, that’s a lot of content and a lot of step definitions to execute. How does that happen?
In general, Gherkin files must be parsed by a testing framework that matches steps and their step definitions (see figure 2.7). That’s precisely what Cucumber does.
Figure 2.7. The testing code contained in step definitions is matched to examples and scenarios through regular expressions executed by the automation engine. Steps and their step definitions are inseparably connected. Steps describe the business logic in natural language; step definitions allow the automation engine to run the testing code needed to check whether the step’s business logic is implemented correctly.
From a technical point of view, every step’s content is matched with a step definition within the automation layer—like in a search-and-replace operation in any text editor you’ve ever used. Here’s the general rule for matching step definitions:
Step's content <- Testing code
You can read this expression as the general rule, “the left-hand side can be replaced with the right-hand side.” When anything that matches the left side—a step—is found, it’s replaced with whatever the right side—testing code—contains. Table 2.3 lists some simple, practical examples with dummy testing code written in Ruby. If you want a more advanced explanation, check out appendix A.
Table 2.3. Specific steps matched with their step definitions
| Step’s content |
Step definition’s code |
Purpose |
|---|---|---|
| Mike, a member of our team | @mike = User.create(name: "Mike") | Create Mike’s account. |
| that it isn't 2 p.m. yet | Time.freeze("1 p.m.") | Freeze the time of the system’s simulation to make sure the event is valid. |
| Mike chooses 2 p.m. as a start time for his meeting | @meeting = Meeting.create(start_time: "2 p.m.", user: @mike) | Attempt to save Mike’s meeting in the database. |
| he should be able to save his meeting | expect(@meeting.saved?).to eq true | Ensure that the system saved Mike’s meeting properly. |
| he shouldn't be able to save his meeting | expect(@meeting.saved?).to eq false | Ensure that the system didn’t save the incorrect meeting. |
You’ll notice, first, that Cucumber rejects keywords when matching a step. Cucumber behaves like that because in some advanced cases, you may want to write interchangeable steps; for example, you may want a When from one scenario to become a Given in another scenario. I’ll talk more about why you might want to do so in chapter 3. Second, Cucumber matches the rest of the step with the testing code in a step definition with the step’s name.
Only together can Gherkin and Cucumber work as an end-to-end tool. The specification layer elaborates requirements and acceptance criteria in natural language using important real-world examples. The examples, expressed in steps, become links that connect acceptance criteria with automated tests—the automation layer—through regular expressions (see figure 2.8). The tests make sure the system behaves as the requirements should require it to.
Figure 2.8. Examples, expressed in steps, act as a link between Gherkin and Cucumber. Steps, scenarios, and feature files constitute the specification layer. Step definitions and the testing code are the fundamentals of the automation layer. Together, the specification layer and the automation layer create an executable specification suite.
You could write scenarios without using Cucumber, but they wouldn’t be executable. And if they weren’t executable, they wouldn’t become acceptance tests. Without acceptance tests, you couldn’t easily know when to update the scenarios when the system breaks or changes. And why would you keep outdated scenarios? You’d probably discard them just as you discard the sticky notes with user stories after you finish the story.
That’s why the system works only if every acceptance criterion becomes a scenario, scenarios become tests, and tests control the design of the system by validating the system frequently with the testing code. These three steps make a cohesive whole: a specification process that covers your development efforts from start to end.
Testing and checking
A question that’s often asked when the topic of automated tests comes up is, “Why don’t we get rid of all of our testers and automate all of our testing?” But you never hear anyone say, “Why don’t we get rid of all of our developers and automate all the development?” Trying to answer why is an interesting thought exercise.
Developers do the smart thing and call their automated work compiling. Testers, too, could differentiate between testing that requires thought and can’t be automated and the checking that’s done with automation. When you design scenarios and think about the possible inputs and expected outputs, you’re doing exploratory testing that could never be done without a conscious mind. Once you design the scenarios, you can automate most of them with testing code—and that’s the checking part. Some tests are more complicated than simple, easy-to-replicate checks, and that’s why they can never be fully automated.
Take our tour and find out more about liveBook's features:
- Search - full text search of all our books
- Discussions - ask questions and interact with other readers in the discussion forum.
- Highlight, annotate, or bookmark.
And—you’re finished. After reading this chapter, you’re ready to write Gherkin specifications for any project.
Let’s review what you’ve done:
- You analyzed your acceptance criteria using concrete examples of how you expect users to use the product in their natural environment.
- You captured these examples as Given-When-Then scenarios to prepare them for future automation.
You ended up with a specification that’s different from traditional requirements -documents:
- It’s much smaller, because Gherkin requires you to break down the scope of your work into singular features. Small specifications fit well into iterative processes that focus on getting small chunks of work done quickly and repeatably.
- It’s concrete and doesn’t leave much to a reader’s imagination. By making the solution easier to comprehend, concrete examples allow engineers to produce working code more quickly.
- It’s ready to be tied up with implementation code through automated tests in order to maintain consistency with the system at all times. The document will stay up to date, because when anything changes, the link between code and scenarios will be broken. If the specification is validated frequently, the delivery team will be able to spot broken scenarios quickly.
- It’s not wasteful. The specification acts as a link between the analysis, design, implementation, and testing phases. The system doesn’t require a separate testing phase that would enable testers to determine whether requirements are satisfied—if they trust the specification, they can focus on exploratory testing and more-difficult edge cases. No other development artifacts are introduced.
What this chapter didn’t cover is Gherkin’s full syntax. Don’t worry; you can find a full specification of the Gherkin language in Cucumber’s repository on GitHub. But in my opinion, you can feel free to skip it. Throughout the book, you’ll see that the key to writing good Gherkin lies somewhere other than getting to know the intricacies of its syntax—and you’ll learn the handiest tricks on the fly.
The real difficulty in using Gherkin is its flexibility. In addition to the keywords, you can type almost anything you want in any order you can imagine. Such flexibility can lead you astray if you’re not careful. In the next chapter, you’ll see how to protect yourself.
EXERCISE 1 Write the rest of the specification brief that explains who will use the scheduling features and why they need them, based on what you know about the application.
Like it or not, meetings are the lifeblood of every organization. Our calendar app is meant for busy teams who work in collaborative environments, have to manage multiple meetings a day with different clients, and want to be as mobile as they can. They need smart reminders, help with different time zones, and a professional meeting management solution. This feature, then, being the simplest meeting management form we could come up with without compromising on powerful capabilities, is the core part of our product. Please be careful and make sure everything is tested properly. Everything else in this application could break and people would still forgive us; if this feature breaks, we'll all be doomed.
EXERCISE 2 Write a simple scenario for canceling a meeting.
Given a meeting by John and Anna at 4 p.m. And that it is 3 p.m. now When Anna cancels the meeting Then the event should be removed from the calendar And John should be notified about the canceled event
EXERCISE 3 Find all the keywords in the Gherkin file that you’ve written so far in this chapter, separate the keywords from the steps, and write the results in a table.
- Feature
- Scenario
- Given
- When
- Then
- And
- But
EXERCISE 4 What are other circumstances that may result in failures when Mike creates a meeting? Write a third scenario that adds more detail to the specification.
Given Mike, a member of our team And another event in Mike's calendar at 2 p.m. When Mike chooses 2 p.m. as a start time for his new meeting Then he should not be able to save his meeting
- A full Gherkin specification is called a feature. A single feature is typically contained in its own file that ends with a .feature extension. A feature typically specifies the behavior of a single functionality.
- The specification layer contains specification files with the .feature extension, written in plain English and filled with domain-specific terms and concepts.
- The automation layer and the specification layer heavily influence each other. You can’t have a good specification layer without understanding the basics of the automation layer.
- Given a set of expected inputs and outputs, the automation layer executes a simulation of the application to see whether the working code behaves as expected.
- Cucumber is a testing framework that matches Gherkin steps with step definitions and lets you run testing code.
- Givens, Whens, and Thens are called steps.
- Steps begin with keywords that help Cucumber parse the feature file.
- Step definitions are containers for all the testing code that must be executed to run a step.