6 Designing a predictable API

In previous chapter we have started our journey to learn how to build usable APIs. We have learned fundamental principles we can use to create straightforward APIs that are easy to understand and easy to use. This is good, we know how to design a quite decent API. But we can do better. What about designing an awesome API? What about designing an API that users will be able to use instinctively without thinking about it even if it is the very first time they use it?

How could we do that? Well, have you ever feel some tremendous pleasure when using an unknown object or application for the first time? You know when everything is so intuitive and so easy that you feel outrageously smart while you discover by yourself all of its possibilities. This is possible not only because you are actually outrageously smart but also because the thing you are using has been designed in order to make it totally predictable. Of course, not every thing is capable of providing such tremendous feeling but everyday you may encounter situations where predictability gives you a hand without you realize it. Why do you know how to open a door? Because it looks like doors you have encounter before. How can you use an ATM in a country using a language you don’t understand? Because it adapts its interface to you. How do you find your way in a huge and labyrinthine subway station? Because there are some indications telling you where to go.

Can we really design such predictable API? Yes, we can! Just like any object, an API can be predicable because it shows similarities with other APIs users have encountered before, because it can adapt to users will or because it provides information to guide them.

6.1  Being consistent

If you encounter a washing machine, such as the one showed on Figure 6.1 , with a button having a triangle icon oriented from left to right, you can guess this button’s purpose easily. Why? Because you already have seen this icon on various media players before.

Since the mid 1960’s all media players use such icon. From obscure DCC (Digital Compact Cassette) audio players to audio cassette players and software media players, each one of these devices uses the same triangle icon for the "start playing" button. Therefore you can guess that this button starts the washing machine. A consistent design is free from variation or contradiction, it helps to make an interface intuitive taking advantage of users previous experiences.

I’m sure you know how the standard pause button looks. What if a media player does not use this standard icon for its pause button? Users will be puzzled and will have to make an effort to understand how to pause the audio or video playing. An inconsistent design introduce variations or contradictions which make an interface harder to understand and use.

Like before, what is true for real world human interface is also true for application programming interfaces. It is essential to keep the design of an API consistent. This can be done with a little bit of discipline, using and meeting standards and shamelessly copying others.

6.1.1  Being consistent within an API

Let’s get back in the 80s. At that time, if you wanted to be cool you had to own a boombox. It is a huge portable radio and cassette player capable of powerful sound. Some of these boomboxes provided two cassette tape players. Imagine such boombox providing totally different controls for the two players. The buttons could be ordered differently and have different icons. Imagine another boombox providing controls that looks exactly the same but have different behaviors from one player to another. On the left player you have to press play once, on the right once you have to press it twice. Such inconsistent designs would look awkward and one would not be really convenient to use. Of course such obviously inconsistent design has not much chance to happen to such a simple a everyday object, but it may definitely happen when designing an API. The look and feel or the behavior of an API can be inconsistent if care is not taken while designing it.

In a poorly designed banking API, as shown in Figure 6.2, an account number could be represented as an accountNumber property for the get accounts goal result, as a number one for the get account one and as a source one for the money transfer input.

The same piece of information in three different contexts is represented with totally different names. Users will not make the connection between them easily. People are used to uniformity in design. Once users have seen an account number represented as accountNumber, they expect to see such piece of information always represented by accountNumber. So, whether this property is part of a fully detailed account or a summary for an account list or used as a path parameter, an account number must be called an accountNumber. When used for a money transfer to identify the source account the original name may be altered in order to provide more information about the nature of the property and call it a sourceAccountNumber for example. Now consumers are able to make a connection between these properties and guess they represent the same concept. So, when choosing names for various representation of a same concept be careful to use similar ones.

But even correctly named, this account number may still be subject to inconsistency. As shown on Figure 6.3, an account number could be represented as a "0001234567" string in get accounts result, as a "01234567" one for get account one and a 1235467 number for money transfer input. Such variations will irremediably cause bugs on the consumer side.

To fix them, consumers will have to standardize these different representations and know when to convert one type or format to another to use it in a given context. People and software don’t like to be surprised with such inconsistency. Once consumers have seen the first account number property as a string with a specific format, they expect all other account number’s representations to be string with the same format. So, and even if they have different names, different representation of the same concept must use the same type and format.

Choosing a data type or format may also have an overall impact on the API. How do you think consumers will react seeing the creationDate of a bank account as a ISO 8601 string such as "2015-02-07" and the date of a transaction represented by a Unix timestamp such as 1530403200. They will not like that, it’s inconsistent. People seek a global uniformity in design. Once consumers have seen a first date and time property represented by a ISO 8601 string, they expect all date and time properties be ISO 8601 strings. So, once a data format has been chosen for a type of data, it must be used for all representations of the same type of data. Consumers seek global uniformity in all aspects of the API, data types and format is only one of them.

What is the problem with the URLs /accounts/{accountNumber} which represents an account and /transfer/{transferId} which represents a money transfer? Its accounts vs transfer, singular vs plural. Once consumers are used to the use of plural for collections, they expect to see all collections with plural names. You may use singular for collection if you want, but whatever you choice: stick to it! And its not only for URLs, from every single name and even value you choose is concerned. Naming conventions can be defined for property names, query parameter names, codes, JSON Schema models in an OpenAPI file… So, once naming conventions have been chosen, follow them strictly.

What is the problem with the /accounts/{accountNumber} and /transfers/delayed/{transferId} URLs shown in Figure 6.4? They don’t have the same organization. The /transfers/delayed/{transferId} URL introduces an unexpected level between the collection name and resource id making the URL harder to understand. We may use a /delayed-transfers/{transferId} instead for example. Each level of a URL should always have the same meaning. Once consumers are used to a data organization pattern they expect to see it used everywhere. It is not only restrained to URLs, data organization in inputs and outputs may present patterns too. Still in Figure 6.4, , the elements of two collections are represented in two different ways. If every collections resource you have designed before is represented by an object containing a property called items which is an array, do not dare to design one as a simple array. Why? Because consumers will be surprised by this variation. So, once data organization conventions have been chosen, follow them strictly.

And finally, consistency concerns not only how the API looks, but also how it behaves. If all previously designed resource creation goals return the created resources in the response, any new one must have the same behavior. If all previously designed sensitive actions (like a money transfer for example) consists in a control <action> and a do <action> goals. The first one doing all possible verifications without executing the action. The second one actually executing the action. Any new sensitive action must be represented with such goals having the same behavior

For an API, the very first step toward consistency is being consistent with itself. Every time you make a design choice you must ensure that it will not introduce a variation or worse a contradiction in the API. When looking at the API as a whole, consumers must see a regular interface. When they jump from one part of the to another they must have the feeling that this new part is familiar. They must be able to guess how this new part works even if they have never used it before.

And if consistency matters within a single it also matters across the many APIs of your organization.

6.1.2  Being consistent across APIs

I own two TVs and a Blu-Ray player manufactured by the same company. Both TVs have exactly the same remote control. Each one as a power button, numbers ones, channel and volume ones, directional buttons to navigate in the menus and even previous/play/pause/next media buttons. The Blu-Ray’s one is slightly different as this device does not offer the same functionalities as the TV but it still have the same look and feel. It also have a power button, number ones, directional buttons to navigate in the menus and of course previous/play/pause/next media buttons. All the buttons which are common to the two types of remote control are exactly the same and roughly organized the same way. I did not got all these devices at the same time. I bought a TV, then another TV and finally the Blu-Ray player. I was immediately comfortable using the remote controls when I got my second TV and then my Blu-Ray player. Simply because they share more or less common features with the first TV’s remote control.

As consistency is important within an API, consistency is also important across the APIs that an organization may provide. An organization may be a team with a single API designer or a whole company with many API designers and everything in between. The consumers of an organization’s APIs do not care if these APIs have been designed by a single or many designers. What they care about is that these APIs share common features so they can understand and use them easily once they have learn to work with one of them. Just like different goals must share common features within an API. Being consistent across APIs "simply" requires to follow the same conventions when designing different APIs. Simply is quoted because it is not always that simple to do that.

A good way to ensure consistency across APIs is to share design conventions in a document usually called "API Design Guidelines" or "API Design Styleguide". Even if you are the only API designer, such guidelines are important because we tend to forget what we have done previously (even in a single API). Defining such guidelines not ony facilitate to standardize the overall organization API surface it also facilitate API designers job. As you have your "API design cheat sheet" you can concentrate on solving real problems and not losing your time reinventing the wheel you have invented a few months ago. Rethinking endlessly about petty details such as what should be an URL structure is not really interesting. We will talk more about all that in Chapter 11 - Growing an API design.

Speaking of loosing time reinventing the wheel, it is more often that not wise to use and follow existing standards instead or inventing your own which will probably be not round enough.

6.1.3  Using and meeting standards

You know the meaning of play and pause symbols shows in Figure 6.5  because you have seen them on various devices. You may have learned their meaning by reading the user manual of the first device using them you have encountered. But after that, every time you see these symbols you can guess their meaning.

On each device using these symbols, their meaning is the same. The look and meaning of play and pause symbols are defined by the ISO 7000 standard.^[1] Once users have encountered one of the ISO 7000 symbol on a device, they are able to guess its purpose on any other device. Users can be able to use a new device without prior experience with this device because they have experience with other devices using the same standards. Like any real world device, an API can take advantage of standards (in a broad sense) to be easier to understand.

A banking API may have to provide some information about amounts in various currencies. Creating our own currency classification and always use it in all of our banking APIs is a good thing. That way we are at least consistent within our organization. But it would be wiser to use ISO 4217. It establishes internationally recognised codes for the representation of currencies. Currencies can be represented in the code in two ways: a three-letter alphabetic code and a three-digit numeric code.^[2] Using such standard, we can be consistent with the entire world. Anyone who have ever used the ISO 4217 elsewhere will understand the meaning of our ISO 4217 currency codes without having to learn a non standard classification. The ISO 8601 we have seen earlier to represent date and time is not only a human friendly format but is also widely adopted in the software industry. You can find standards for data format but also data naming and data organization and even processes. For example, you can find inspiration to design your data model on schema.org which is a collaborative, community activity with a mission to create, maintain, and promote schemas for structured data on the Internet, on web pages, in email messages, and beyond.^[3]

But being standard does not always mean following an ISO or other organization standard. If a banking API represents a delayed transfer with /delayed-transfers/{transferId} URL, I can guess that using the HTTP method DELETE may cancel a delayed transfer. If I get a 410 Gone response, I may guess that the delayed transfer has been executed or canceled before I try to delete How can I guess that? Because I suppose that the banking API which claims to be a REST API follows strictly the HTTP protocol which is defined by RFC7231.^[4] The DELETE HTTP method can be used on a resource to delete, undo or cancel the concept represented by the URL. it. The 410 Gone is quite explicit but it indicates that the resource requested is no longer available and will not be available again. This should be used when a resource has been intentionally removed and the resource should be purged. So, REST APIs can be consistent by simply applying the HTTP protocol rules to the letter. That way anyone can quickly be able to use any REST API because they share the HTTP protocol features.

So, being consistent within and across APIs is a good thing, but we can do more in order to lessen the need of experience practicing our APIs to use them. All we need to do is follow standards in a broad sense used by many others. It can be an ISO or other organization standard or simply apply a protocol rules. Using standards facilitate understanding as consumers may already have used the standards you have chosen. And note that using standards enhance your API interoperability as its data will be easily usable by other APIs using the same standards.

Here’s a scenario to practice this topic. Let’s say you have to design an API which processes images to create a matching color palette for web developers. Free tier users can request up to 10 palettes per month, if they want more they have to pay a subscription. Free or paid users cannot send more than 1 request per second Sent images cannot be larger than 10MB.

But designing APIs is not always about using international standards or strictly applying protocols. It is also about shamelessly copying others.

6.1.4  Following common practices and copying others

Let’s travel again in the 80s, but in the video games industry this time. In 1982, Gunpei Yokoi invented the directional pad or D-pad for the Nintendo’s Donkey Kong Game & Watch handheld LCD game. The D-pad shown in Figure 6.6  consists in a cross that allow to control the movements in four directions.

This D-pad was so useful and praised that everyone copied what was considered as the best thing to do to control directions. Users are quite happy with that as even if there are difference between various gaming controllers, a D-pad always works as expected. More than 35 years later you still can find a D-pad on most video games pad and other devices.

In the API design world, the same sort of things can happen. While there is no standard rules for the URL structure of REST APIs, many of them use the /resources/{resourceId} pattern we have seen in 3.2.3. Representing resources with URLs. As show in Figure 6.7 ”, resources is a collection is identified by a plural noun. It contains elements of type resource.

If not everything is standardized in the API design world, there are common practices that are very close to be standard. It is wise to follow them in order to ensure that your API will be easily understood by consumers based on their experience with other famous APIs.

You can also simply copy what others have done. Why bothering rethinking pagination parameters from the ground when it has been dealt with by so many API designers? You can look at some famous APIs and and reuse the proven design you prefer. It simplifies your life as an API designer. And if your users have used this API, they will feel at home when using yours for the first time. Everybody wins.

Consistency seems to simplify everything that’s great! Well, that’s not always true unfortunately.

6.1.5  Finding balance between consistency and simplicity

The QWERTY keyboard design shown in Figure 6.8  has been created in 1873 for typewriters to avoid the mechanical rods attached to the key to stick to each other when people were typing to fast.

There are other design which are far more simple to learn and use but we will probably be stuck with this 19th century design until the end of days. Why because everybody is used to it. Does it mean that all keyboards use this design (or its declination)? No, sometimes, depending on the context, other designs can be used. Like the one you could found on mobile phone, before they became smart. At that time mobile phones only propose a digit keyboard. On this keyboard the letters are grouped by 3 or 4 on the 2 to 9 buttons, 0 is used for space.

We could follow the ISO 20022 standard to design a banking API. Depending on the type of consumer we target it may be wise or not to use such standard. If we target financial institution who are used to this format it is an interesting option. But if we target consumers who are not financial experts it would be a terrible mistake because this standard is definitely not user friendly.

Being consistent in order to let consumers guess how to use an API is great to be predictable. But you can also cheat and let people choose what they want to get using your API and therefore be even more predictable.

6.2  Being adaptable

I did not own a car anymore but when I had one, I shared it with my wife. Without modifying the standard seat and steering wheel position we both could drive the car. Unfortunately, it was not comfortable for either of us. The distance between pedals and seat was a little bit too short for me and too long for my wife. The steering wheel was too low and too close for me, too high and too far for my wife. Hopefully, each of us could adjust the settings to her or his liking and drive in a very comfortable position. Providing an adaptable design is interesting in order to satisfy different types of users.

APIs can also provide an adaptable design which make the API predictable. Yes, if consumers can tell what they what, they can obviously predict what they will get. We will discuss here three common ways of making an API design adaptable: providing and accepting different formats, internationalizing and localizing and eventually providing filtering, pagination and sorting features. These ways are not exhaustive, you may find other and even create your own when needed.

6.2.1  Providing and accepting different formats

When you buy a book, you can buy it in different versions. It can be sold as a printed book, an e-book or an audio-book. It’s up to you to indicate which one you want when you add this book to your shopping cart. But rest assured that all these versions are different representations of the same book. Managing different representation of the same concept is not reserved to books, we can do that with APIs too.

Nowadays, JSON is the obvious way of representing an account’s transactions list in a banking API. As shown on the left side of Figure 6.9, a list of transactions could be an array of JSON objects. Each one being composed of 3 properties: a date, a label and an amount which is an object composed of a value and currency.

But it is not the only option, right side of Figure 6.9  shows the same data represented as CSV (Comma-Separated Values) data. In that case a list a transaction is represented by text lines. Each line representing a transaction being composed of 4 values separated by commas (,). The first one is the date, the second one the label, the third one the amount and the last one the currency. This transactions list could also be represented by a PDF file. The only limit is your imagination and above all what consumers really need.

But if the get account’s transactions goal can return this list in various formats, how consumers can tell which format they want? As shown in Figure 6.10, we could add some format parameter to this goal in order to let consumers tell if they want the transactions list as JSON, CSV or PDF. To get this transactions list as a CSV document, consumers would have to send a GET /accounts/{accountId}/transactions?format=CSV request. That’s a possibility, but as the banking API is a REST API, we could also take advantage of the HTTP protocol and use content negotiation.

When sending the GET /accounts/{accountId}/transactions request to the API server, consumers can add a Accept: text/csv HTTP header after the HTTP method and URL to indicate that they want this transactions list as CSV data. If everything is OK, the API server will respond with a 200 OK HTTP status code followed by a Content: text/csv header and the transactions as a CSV document. They can also respectively send Accept: application/json or Accept: application/pdf to get JSON data or PDF file. The server respectively returning a response with a Content: application/json or Content-type: application/pdf followed by the document in the adequate format.

This example made us discover two new things about the HTTP protocol: HTTP headers and content negotiation.

HTTP headers are colon separated name value pairs. They can be used in both request and response to provide some additional information. In a request these HTTP headers are located after the request line containing the HTTP method and URL. In a response they are located after the status line containing the HTTP status code and reason. They are around 200 different standard HTTP headers and you can even create your own if needed. They are used for various purposes and one of them is content negotiation.

Content negotiation is a HTTP mechanism which allows to exchange different representations of a single resource. When a HTTP server (hence a REST API server) responds to a request it must indicate the media type of the returned document. This media type is indicated in the Content response header. Most REST APIs use the application/json media type, the documents they return are JSON documents. But consumers may provide an Accept request header containing the media type they wish to get. As shown in Figure 6.11 , the three possible media types for the account’s transactions are application/json, application/pdf and text/csv. If the consumer requests a media type like audio/mp3 which the provider does not handle, the server will respond a 406 Not Acceptable error. Note that a request without Accept header implies that the consumer will accept any media type.

That also works when the consumer have to provide data in the body of the request. In 5.2.1. Requesting straightforward inputs, we have seen that to create a transfer consumers have to send a POST /transfers request which body contains a source account, a destination account and an amount. This body was supposed to be a JSON document, but it could also be of another media type. A consumer may send an XML document containing the information needed to create a transfer. XML (eXtended Markup Language) is a markup language that is supposed to be both human and machine readable. A property such as the amount would be represented as <amount>123.4</amount>. It was the de facto standard for API and web services before JSON. So, if consumers want to send the transfer data as an XML document they have to provide the Content-type: application/xml header. If unfortunately the API server is unable to understand XML, it will return a 415 Unsupported Media Type error. Note that if these consumers wanted to get the result as an XML document instead of a JSON one they may also provide the Accept: application/xml header along with the Content-type one in order to say I send you XML and I would like you to respond using XML too. Note also that we will see in 6.3.3. Taking advantage of the (HTTP) protocol how a consumer can know which are the available media-types.

That’s great, content negotiation (whether provided by the protocol used or handled manually) let consumers choose the format they want to use when communicating with an API as long as it is supported. But we can do more than that.

6.2.2  Internationalizing and Localizing

Books may exists in various formats: printed, e-books and audio-books. But they may also exists in different languages. One day this book will probably be translated in french, hence people who can read french but not read english may buy it. People who understand both english and french will have to choose between this two versions. And of course whichever language is chosen, readers will have to choose the format: printed, e-book or audio-book. Even translated in french, the Le design des APIs du quotidien e-book is only another representation of the same book.

How could we apply this to the banking API example? In 5.2.3. Returning informative error feedback, we have learned to design straightforward error feedbacks. When doing a money transfer, the API may return an error with the Amount exceeds safe to spend message. Such message could be shown to end users, but what if they do not understand english? The developers building the application or web site using the banking API will have to manage translation of this message on their side. From a technical point of view, it is possible as the error is identified with a clearly identifiable AMOUNT_OVER_SAFE type. But maybe we, API designers, can give a hand to developers using our API and propose a way to get error messages in other languages than english.

We could add a language parameter to all Banking API goals. Its value being a ISO 639 language code.^[5] For example, fr stands for french and en for english. We have learned in 6.1.3. Using and meeting standards that using standards is better in order to ensure that a value will be easily understandable and interoperable. But wait, en simply means english. UK english and US english are different languages, just like french and canadian french. So, ISO 639 is not a good idea, it would be better to use a more accurate standard to identify a language. IETF language tags defined by RFC5646 is the standard we are looking for. US english is en-US, UK english is en-UK, french is fr-FR and canadian french is fr-CA. This format uses ISO 639 language codes and ISO 3166 country codes.^[6] As you can see choosing a standard my not be so straightforward, you have to be careful when choosing one and be sure that it really fulfil your needs. However it was complex to find the good standard, when using the money transfer goal with a language parameter set to fr-FR the AMOUNT_OVER_SAFE human readable error message could be Le montant dépasse le seuil autorisé. We can imagine that any text returned by the API, not only error messages, could be translated in the language indicated in the language parameter.

This language parameter could be represented as a query parameter, but the Banking API is a REST API. Hence, we could take advantage of the HTTP protocol to provide this parameter. Content negotiation not only applies to data format but also to languages. Just like we used the Accept and Content-type HTTP headers to tell which media type is used, we can use Accept-Language and Content-Language headers to tell which language we are speaking as shown in Figure 6.12. When using POST /transfers to transfer money, if consumers provide no headers, the API server may return a response with a Content-Language: en-US to tell that any textual content is in US english. But consumers may provide a Accept-Language: fr-FR HTTP header with their request in order to tell that they wish to get any textual content in french. In that case, the API server will respond with a Content-Language: fr-FR header, any textual data being translated in french. If the requested language, italian (it-IT) for example, is not supported, the server returns a 406 Not Acceptable HTTP status code. As this status code can also be returned because the consumer requested a media type which is not supported, it would be nice to provide a straightforward error feedback with a clear error code like UNSUPPORTED_LANGUAGE and message like Requested language not supported.

Adapting data values to developers, their applications and their end-users is not only about translation. In the US, people use the Imperial System, in France, people use the Metric System. US and France do not use the same units, the same number format, the same paper sizes, … Being able to adapt to all these variations is possible if your API supports Internationalization and Localization. So, for our REST Banking API, internationalization is being able to understand that a Accept-Language: fr-FR header means that consumers wants a localized response using french language and conventions. On the server side, it means that if the requested localization is supported, the content will be return localized along with a Content-Language: fr-FR header. If not, the server return a 406 Not Acceptable status code. Still for the Banking API, localization means being actually able to handle the fr-FR locale. The data returned should be in french, using the metric system, or a PDF should be generated using the A4 size and not the US Letter size for example. Internationalization is the mechanisms which allow an application, a software or an API to do Localization. Localization is being able to handle the adaptations to a locale basically composed of a language and a region or a country. This field is not specific to APIs, you have to take care of this in any software development. Note that internationalization is often presented as i18n and localization as l10n (the numbers represent the number of character between the first and last letter of the word).

But, as API designer and API provider, should we really care about internationalization and localization? It is a totally legitimate question that must be answered sooner or later when designing an API. It depends on the nature of your API and the targeted consumers and/or their end users. If you are really lucky, the data exchanged through your API may not be impacted at all by localization concerns so you my bypass it. If you do not target people using different locales you may not need to handle internationalization. Be cautions, sometimes people may use different locale in a country. In the US, people may use en-US or es-US locales for example. However, you can still start without internationalization features and update your API later if needed. But if adding internalization feature on an existing can be done easily and transparently from the consumers point of view. Modifying the implementation which was not build with internationalization in mind might be trickier.

Note that there are other aspects of content negotiation like priorities when requesting multiple variant of a resource and content encoding that will not explore in this book. You can read more about this in the RFC7231 Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content.^[7]

So consumers can tell not only the data format they want to use but also the language and units for example. Is it possible to provide an even more customizable API in order to be even more predictable? Yes, it is!

6.2.3  Filtering, paginating and sorting

When you read a book, you usually do not read it all at once. You read it page after page. When you stop, you mark the last page you read. When you continue reading, you jump directly to the last read page without rereading the book from the beginning. When you read a book, and especially technical one, you may jump directly to a specific chapter or section, hence a specific page. You may also only read the parts concerning a specific topic. You may even not read the chapters in their natural order. Maybe one day some company will even provide a service allowing you to build your own book with chapters from different books just like when you go to a restaurant. At a restaurant, you can order a menu composed of a predefined starter, main course and dessert. But you can also make you own à la carte menu choosing various dishes like two starters and a dessert or only a main course.

If a bank account has been created a long time ago it may have thousands of transactions. If consumers want to get an account’s transactions using the Banking API, they probably do not want to get all transactions at once and prefer to get a subset. Maybe they prefer to get the 10 most recent ones and then possibly go deeper in the list. As shown on Figure 6.13, it could be done by adding some pageSize and page optional parameters to this goal for example.

On the server, the transactions list is virtually split in pages, each page containing pageSize transactions. If pageSize is not provided the server use a default value. If page is not provided, the server returns the first page by default. To get the first page of 10 transactions, consumers would have to provide pageSize=10 and page=1. To get the second page of 10 transactions, they would provide pageSize=10 and page=2. On our REST Banking API, these pagination parameters could be passed as query parameters GET /accounts/1234567/transactions?pageSize=10&page=1.

But we could also take advantage of the HTTP protocol and use the Range HTTP header. To get the "first page of 10 transactions", this header should be Range: items=0-9. And to get the next page, the header is Range: items=10-19. The Range header was created to allow a web browser to retrieve a portion of a binary file. The value of a request Range header is <unit>=<first>-<last>. A standard unit is bytes, bytes=0-500 would return the first 500 bytes of a binary files. Here, we use a custom units items. Sending a Range with the items=10-19 value tells I want collection’s items from index 10 to 19. I could have chose another unit name such as transactions but that would mean that if we want to paginate the /accounts collection resource, the unit would have been accounts. The unit name used to paginated can be guessed from the collection name, but I prefer to favor the items generic name. That way, no need to guess the unit for paginating collections: it is always the same.

But if consumers want a subset of transactions, maybe they would like to have more control on this subset. Maybe they only want transactions which have been categorized as restaurant. To get these specific transactions they may send a GET /accounts/1234567/transactions?category=restaurant request. The category query parameter is used to filter the transactions and only returns the ones categorized as restaurant.

This filter example is a really basic one. If you want to practice, here’s a problem you will have to solve sooner or later as an API designer: filtering a collection on numerical values. Let’s say you design an API dealing with second hand cars. Users should be able to list available cars having a mileage between two value using a GET /cars request and one or more query parameters. Using natural language such query would be something like "list cars with mileage between 15,000 and 30,000 miles".

Consumers may also want to get the transactions sorted by decreasing amount and chronological order instead of the standard reverse chronological order. To get such list they may send a GET /accounts/1234567/transactions?sort=-amount,+date. The sort query parameter defines how the transactions list should be sorted. It contains a list of direction and property couples. The direction + is for ascending and - for descending. The -amount,+date values tells that transactions have to be sorted by descending amount and then ascending date.

These pagination, filtering, and sorting features can be used together. Using the category=restaurant&sort=-amount,+date&page=3 query parameters on GET /accounts/1234567/transactions will return the third pages of the transactions categorized as restaurant order by descending amount and ascending date.

So, besides making our API look like something familiar, a good way of being predictable is basically to let consumers tell what they want and give it to them. A third way of being predictable is giving consumers some clues about what they can do.

6.3  Being discoverable

On most books you know which page you are reading as its number is printed on it. Sometimes you even know where you are as current chapter or section are also indicated on the page’s top or bottom. Earlier we have seen that when you read a book you can directly jump to a specific specific chapter or section. This is possible because a book comes with a handy table of content listing available chapters and sections and on which page they start. So, when you read a book you read its content but you also have access to additional information about the content itself. But you could read the book without using them. We could even remove them all, the book’s content would stay unaffected. But reading the book would be far less convenient. If the book is a novel, that’s not really a problem. A novel is more interesting read page after page without being spoiled by a too explicit table of content like chapter XI - You know what? The character you have been so attached to dies. But, if the book is a practical one, like the one you are reading know, you may like to scan the table of content to have a better idea of what the book is about in order to be sure it is relevant for you. You may want to jump to a specific section on first read or later because you have a specific problem to solve. Without table of content and page numbers, it will not be easy to find what you are looking for.

These additional information make a book discoverable. They are not mandatory but they greatly improve the reading experience. Like books, APIs, can be designed in order to be discoverable mostly by providing additional data in various ways but also by taking advantage of the protocol used. REST APIs have the discoverable feature in their genes has they use URLs and the HTTP protocol.

6.3.1  Providing metadata

Earlier in the chapter in 6.2.3. Filtering, paginating and sorting we have discovered the pagination feature. When accessing an account’s transactions list, consumers of the banking API can indicate which page of transactions they want. But how do they know that there are multiple pages available? For now the server’s response when consumers request an account’s transactions list consists only in an object containing an items property which is an array of transactions. Sounds like a book without page numbers and table of content.

This response could be improved by adding some data about pagination. If a first call to the list account’s transactions goal is made without pagination parameters, the server could return the items array along with the current page’s number page which value is 1 and the total number or pages totalPages which value could be 9. Therefore, consumers know that there are 8 more pages of transactions ahead.

Thanks to these additional data, the transactions list is now discoverable. In computer science such data are called metadata, they are data about data. They can be used to tell consumers where they are and what they can do.

Let’s see another example just to show that metadata are not limited to pagination. Using the Banking API, consumers can transfer money from an account to another immediately or at a predefined date. Therefore, when listing past transfer requests, the server may return executed or postponed transfer requests. An already executed request cannot be canceled but a postponed one which has not yet been executed could be canceled. As shown on Figure 6.14, we could add some metadata describing the possible actions on each transfer request. For an already executed money transfer, the actions list would be empty. For a postponed one it could contain a cancel element. So, consumers know on which one they can use the cancel money transfer goal.

As you can see, by adding metadata we are basically applying what we have learned in 5.1. Designing straightforward representations, we are providing ready to use data. This can be done with any type of APIs. Depending on the type of API, you can rely on other mechanisms to provide such information especially by taking advantage of some of the used protocol features.

6.3.2  Creating hypermedia APIs

Using the REST Banking API, consumers can retrieve accounts list using GET /accounts. Each account comes with a unique id that can be used to build an account’s URL /accounts/{accountId} and retrieve its detailed information using the GET HTTP method on it. This id can also be used to retrieve an account’s transactions with GET /accounts/{accountId}/transactions. Thanks to the pagination metadata we have just added, consumers know that if there are more transactions than the one returned by their first call. In that case they can use GET /accounts/{accountId}/transactions?page=2 to get the next page of transactions. They can even jump directly to the last page of transactions. They just have to take the lastPage value and use it to GET /accounts/{accountId}/transactions?page={lastPage value}.

Sounds like a well designed API with crystal clear URLs and even metadata that helps consumers, right? Now, let’s imagine the same situation when browsing the bank’s web site in which all hypermedia links have been removed. Would you be happy as a user to have to read a user’s manual to learn all available URLs. Would you be happy to have to copy/paste the account number and use it to type a bank’s account web page’s URL yourself? The World Wide Web without its hypermedia links would be quite terrible to use. Hopefully, this is not how it works. Once on a website you can discover its content simply by clicking on links and go from one page to another. REST APIs relying on the World Wide Web principles, why not taking advantage of it?

A hypermedia Banking API would provide a href property on each account returned by GET /accounts. For the 1234567 account its value would be /accounts/1234567. Therefore consumers wishing to access to this account’s detailed information just have to GET this ready to use URL without bothering to build one. And this account detailed information would contain a transactions property which value could be an object containing href which value would be /accounts/1234567/transactions. Again, consumers just have to get this href value to get the account’s transactions. And of course the pagination metadata would also provide URLs like next and last properties which values could be respectively /accounts/1234567/transactions?page=2 and /accounts/1234567/transactions?page=9. Therefore, consumers are able to browse the API without the need to know its available URLs and their structures.

REST APIs provide links just like web pages. It facilitates API discovery and as you will discover later in this book it also facilitates API update. There is no standard way to provide these hypermedia metadata but there are common practices mostly based on how links are represented in HTML pages and the HTTP protocol. Hypermedia metadata usually use names such as href, links or _links for example. But if there is no standard, hypermedia formats have been defined. The most known ones are HAL, Collection+JSON, JSON API, JSON-LD, Hydra or Siren. These formats come with more or less constraints regarding the data structure.

HAL is relatively simple. A basic HAL document contains a links property containing available links. Each link is identified by its _relation_ or _rel_ with the current resource. The self relation is used for the resource’s link. For a bank account resource, the link to its transactions would be located there as transactions.

The concept of link relation is not specific to HAL, it is defined by the RFC 5998 Web Linking. ^[8] But hypermedia APIs do not only provide available URLs, they can also provide available HTTP methods. For example, with the Siren hypermedia format we can describe the cancel action on a postponed money transfer. Siren come with constraints regarding the data structure: the properties are grouped in properties. Links to other resources are located in links and actions are in actions.

Providing hypermedia metadata is the most common way of taking advantage of the web roots of REST APIs to create predictable APIs. The HTTP protocol provides features that can be used to make REST APIs even more predictable.

6.3.3  Taking advantage of the (HTTP) protocol

So far, we have used GET, POST, PUT, DELETE and PATCH HTTP methods. Listing 6.3  shows that a OPTIONS /transfers/000001 request can be used to identify the available HTTP methods on a resource.

If the API server supports this HTTP method and the resource exists, it can return a 200 OK response along with a Allow: GET, DELETE header. The response clearly states that GET and DELETE HTTP methods can be used on /transfers/000001.

Earlier in this chapter we have seen that an account’s transactions list could be returned as a JSON, CSV or PDF. Listing 6.4  shows that when responding to a GET /accounts/1234567/transactions an API server could indicate other available formats with a Link header.

Note that such use of the HTTP protocol by REST APIs may not be widespread. Just like when choosing standard, you should check if such features are really interesting for consumers. If so, you may have to explain them in detail in your documentation for people who are not HTTP protocol experts.

If you want to practice what we have seen in this 6.3. Being discoverable section, you should try update the Shopping API we have been working with in Chapter 3 Designing a programming interface and Chapter 4 Describing an API with an API description format. Using the OpenAPI specification:

6.4  Summary