Chapter 7. Advanced HTTP/2 concepts

This chapter covers the remaining parts of the HTTP/2 protocol, roughly in the order in which they appear in the specification.^[1] Many of these parts aren’t under the direct control of web developers and may even be out of the control of server administrators (unless they’re writing an HTTP/2 server themselves), so these topics are definitely more advanced. Knowledge of them, however, will give you deep understanding of how the protocol works and help with debugging, if you’re looking to implement your own HTTP/2 server. Additionally, in the future, more control may be made available to developers or at least web server administrators. Chapter 8 looks at the HPACK protocol, which is a separate specification from HTTP/2.

¹https://httpwg.org/specs/rfc7540.html

7.1. Stream states

An HTTP/2 stream is created for a single download and then discarded. This is one reason why HTTP/2 streams aren’t exact analogs for HTTP/1.1 connections, even though this is probably the easiest way of explaining them when first teaching HTTP/2. Many diagrams draw parallels between HTTP/2 streams and HTTP/1 connections (like the ones I used in chapter 2 and repeat in figure 7.1), but this convention isn’t strictly true, because streams aren’t reused.

After a stream finishes delivering its resource, the stream is closed. When a new resource is requested, a new stream is started. Streams are a virtual concept and are nothing more than a number each frame is tagged with, known as the stream identifier. The cost of closing a stream or opening a new one, therefore, is considerably lower than the cost of opening an HTTP/1.1 connection (which involves a TCP three-way handshake and an optional HTTPS protocol negotiation before a request is sent). In fact, HTTP/2 connections are even more costly than HTTP/1 connections, as they additionally require the HTTP/2 “magic” preface message and at least one SETTINGS frame to be sent before a single request can be made. HTTP/2 streams are much cheaper.

HTTP/2 streams go through a lifecycle of states. A HEADERS frame sent from a client starts an HTTP request (such as a GET request), the request is answered by the server, and then the stream is closed. This process goes through the following states:

Although this list explains the state transitions for a simple client-initiated HTTP request, the same can happen in a server-initiated request. At present, only HTTP/2 push responses are server-initiated (though there’s nothing to say that some new frame in the future won’t be server-initiated). In this case, a stream starts another stream (the promised stream identifier), and that new promised stream goes through a similar transition of states:

The full HTTP/2 state diagram is shown in figure 7.2, including the two flows indicated in the preceding list and several other possibilities (such as when the RST_STREAM frame is used to end a connection prematurely).

In each of these flows, the client and server have a slightly different view of the stream status, depending on whether it initiated that state or moved to that state based on a message from the other side. Therefore, some of these states have a local or remote indicator (depending on whether you’re the initiator of the stream or the recipient, respectively), and there are send and recv transitions from each state.

Going back to the first example of a GET request, you know that it goes through the following states: idle, open, half-closed, and closed. The half-closed state is ambiguous, however: it’s closed to the client (so it can’t send data and can only receive data), but it’s not half-closed to the server. The client sees the stream as half-closed (local), and the server sees it as half-closed (remote). A request, therefore, doesn’t flow down the state diagram in the same way for the client and the server; it flows down either the left or the right of the diagram at the same time, depending on whether you’re looking at the client or server view.

Also, the state diagram shows only state transitions. Some frames don’t result in a state transition. CONTINUATION frames, for example, are considered to be extensions of the preceding HEADERS frames, so they’re considered to be part of HEADERS in the diagram. Similarly, other frames (such as PRIORITY, SETTINGS, PING, and WINDOW_UPDATE) never result in a state transition, so aren’t captured in this diagram.

To be perfectly honest, the HTTP/2 state diagram isn’t important for most users of HTTP/2 and is more a concern for implementors of low-level HTTP/2 libraries to understand what frames can and can’t be sent at each state. The diagram is in the HTTP/2 specification,^[2] however, and the various states are referenced a lot in this spec, so understanding it helps. Any attempts at state transitions that aren’t allowed by HTTP/2 should result in PROTOCOL_ERROR messages. Again, understanding the state diagram can help you understand why you get such an error (though this error usually is due to a bug in the underlying HTTP/2 implementation and beyond what most web developers can fix themselves).

²https://httpwg.org/specs/rfc7540.html#StreamStates

The HTTP/2 state diagram can be intimidating at first, and unlike some of the concepts I’ve covered so far, it isn’t something you can see in a browser’s developer tools or even by using some of the other tools in this book (such as nghttp and Wireshark). It’s more an internal status that HTTP/2 implementations need to maintain and track. Given that fact, it can be complicated to understand. Going back to the main use case (requesting an HTTP resource), however, as described before the diagram, usually takes some of the mystery out.

7.2. Flow control

Flow control is an important part of networking protocols. It allows a receiver to stop a sender from sending it data if it isn’t yet ready to process, perhaps because it’s too busy to process any more incoming data. Flow control is necessary because different clients can consume data at different speeds. A high-speed server may be able to send data quickly, but if a lower-speed client (such as a mobile phone) isn’t able to keep up, it starts to buffer data in memory, and when that buffer is filled, it starts to drop packets, requiring them to be sent again. As a result, resources are wasted on the server side, the network, and the client side.

Flow control wasn’t required in HTTP/1.1 because there was only one message in flight at any time. Therefore, TCP flow control could be used at a connection level. If the receiver stops consuming TCP packets, it no longer acknowledges those packets, and the sender stops sending them because its TCP congestion window (CWND) would be used up (see chapter 2).

In HTTP/2, you have a multiplexed connection of independent streams, so connection-level flow control is no longer sufficient. Control needs to be at a connection level and at a stream level. It may be that you’re happy to receive more data on one stream but not the other. Chapter 4 provides an example of a website with a video that the user has paused. In this case, you may not want the video to continue downloading while it’s paused, but you want to allow other streams on the HTTP/2 connection to continue to be used.

Flow control is handled in HTTP/2 in a similar manner to TCP. At the beginning of the connection (using the SETTINGS frame), the flow control window size is decided (or the default 65,535 octets is used, if the size isn’t specified). Then each piece of data sent is subtracted from that total, and each bit of data acknowledged (via the WINDOW_UPDATE frame) is added back. There’s a connection-level flow control window, which kind of mirrors the TCP flow control window, and one per stream as well. Senders can send only up to the maximum size of the smallest flow control window (connection-level or for that stream), and when the flow control window reaches zero, the sender must stop sending data until it receives acknowledgments, resulting in a nonzero flow control window. If you implement an HTTP/2 client or server and forget to implement WINDOW_UPDATE frames, you’ll soon notice that the other side stops talking to you!

Flow control is used for DATA frames (though future HTTP/2 frame types may also fall under flow control). Control frames (and in particular the WINDOW_UPDATE frames needed to control flow control) can still be sent when a client has stopped acknowledging frames.

7.2.1. Example of flow control

For an example of flow control, I’ll go back to using the nghttp tool. In this section, we initiate a request to Facebook for the home page and all the required resources and then pipe this into grep to show only the important parts:

Here, you see that the Facebook server has decided to use a flow control window size of 65,536 octets (the SETTINGS_INITAL_WINDOW_SIZE value in the recv SETTINGS frame), and nghttp is using 65,535 octets (the SETTINGS_INITAL_WINDOW_SIZE value in the sender’s SETTINGS frame). Incidentally, 65,535 is also the default size, so nghttp didn’t need to send it at all. The code also shows that the two sides can have different flow control window sizes (even though they’re near enough the same here, differing by only 1 octet).

In the middle of those two SETTINGS frames, you see your first WINDOW_UPDATE frame (highlighted in the code):

This frame states that Facebook is prepared to receive up to 10,420,225 octets, and as the frame was sent on stream 0, this limit is the connection-level limit to be used across all streams, in addition to their stream-level limit. Stream 0 should never be used for DATA frames and doesn’t need its own flow control, which is why it can be used for connection-level flow control. These 10,420,225 allowed octets are on top of the 65,535 octets for the initial window size, so Facebook could also have set the initial size to the sum of both (10,485,761), but it’s also permissible to implement this way.

Next, nghttp acknowledges the server’s settings, followed by a few more frames in which nghttp sets up for prioritizing (incidentally, one of the few instances in which a frame can be created in idle state and stay there until used):

I discuss the PRIORITY frames next, so ignore them for now.

Next, you see that the first request is sent by means of a HEADERS frame on stream 13:

Recall that client-initiated streams must have odd-numbered stream identifiers. Stream 13 is the next free one because 11 was used by the last PRIORITY frame.

Then the server acknowledges the SETTINGS frame, and another WINDOW_UPDATE frame increasing the window size of stream 13 to 10,420,224 octets (oddly, 1 octet smaller than the connection-level size, but nothing says the sizes have to be the same):

Next, nghttp starts to receive the resource’s HEADERS and DATA frames:

After it has received a few DATA frames, nghttp decides to let the server know that it has consumed that much data. Adding up the DATA frames only (1353 + 2571 + 8144 + 5563 + 2572 + 1491+ 2581 + 4072 + 5572) gives 33,919, so that’s what nghttp tells the server that it has consumed at connection level (stream 0) and on stream 13:

It’s important to note that only the length of the DATA frame payload (as given by the length field) is included in the flow control calculation and that the nine-octet frame header is excluded from the flow control.

The connection continues in a similar manner until all the resources are delivered, and the connection is closed by the client sending the ever-so-polite GOAWAY frame:

7.2.2. Setting flow control on the server

Apache allows you to set the flow control window size with the H2WindowSize directive:^[3]

³https://httpd.apache.org/docs/2.4/mod/mod_http2.html#h2windowsize

Other servers may also allow this directive to be set. NodeJS, for example, allows this directive to be set with the initialWindowSize setting,^[4] and the Jetty servlet engine allows it to be set with the jetty.http2.initialStreamRecvWindow setting.^[5] Many other servers (such as nginx and IIS) don’t give you any control of this directive at the time of this writing. The reality, though, is that you’re unlikely to need to change the directive from the default setting unless you want detailed control of your server.

⁴https://nodejs.org/api/http2.html#http2_settings_object

⁵https://github.com/eclipse/jetty.project/blob/master/jetty-documentation/src/main/asciidoc/administration/http2/configuring-http2.adoc

7.3. Stream priorities

Next we look at stream priorities. HTTP/2 introduces the concept of prioritization to allow the client to suggest the relative importance of a request. After a browser downloads a page, it requests the resources needed to view this page. Critical, render-blocking resources (think CSS and any blocking JavaScript) are high-priority, and any images or async JavaScript can be requested with a much lower priority. Priorities can be used by the server to decide the order in which it should send frames; more-important frames can be sent first, so they arrive earlier and aren’t held up due to any flow control or bandwidth issues.

HTTP/2 defines two different methods for setting the priority:

These priorities can be set with requests in a HEADERS frame or can be reprioritized at any time through a separate PRIORITY frame.

7.3.1. Stream dependencies

A stream can be made dependent on another stream, so it should be used to send resources only when the dependent stream doesn’t need to use the connection to send anything. Figure 7.3 shows one such example.

Everything is dependent by default on stream 0 (not shown in figure 7.3), which is the control stream and represents no dependency. In this example, main.css is the first dependency on index.html and should be sent with the highest priority, followed by main.js and finally image.jpg. Usually, index.html is fetched first, followed by the dependencies, so there may be no need to put a dependency on the HTML document file as shown here. But a large index.html may still be downloading as the other requests are made, so it isn’t necessarily wrong to make all the requests dependent on it.

This dependency hierarchy doesn’t mean that dependent streams block on their parents. If main.css isn’t immediately available to the web server and has to be fetched from a backend server, for example, the server can send main.js in the meantime, assuming that it’s available. If neither file is available, image.jpg can be sent while the server waits for those files. The aim of stream prioritization is to make the most efficient use of the connection rather than act as a blocking mechanism.

The server may start to send image.jpg while it fetches main.css and main.js, and when those files are available to send, the server may pause sending image.jpg and send main.css, followed by main.js, before unpausing image.jpg and sending the remainder of that file. Alternatively, the server could have a simpler model and finish sending image.jpg when it has started while the others queue as they become ready. The choice is up to the server.

Streams can also have multiple dependents, as shown in figure 7.4, because each stream can specify the stream it’s dependent on.

In this example, both main.css and main.js are dependent on stream 1, and the image is dependent on main.js stream. If the image file is lower-priority than both these critical resources, ideally it would be dependent on both the CSS and JS streams, as shown in figure 7.5, but the concept of multiple dependencies isn’t supported.

If multiple dependencies were supported, the file would be downloaded only when both main.css and main.js don’t need the connection, but multiple dependencies aren’t supported by the HTTP/2 dependency model (though they can be approximated with the use of weightings).

Stream priorities can be complicated to manage as resources available for the server to send become available. It can be further complicated by adding new requests or finishing in-flight requests. Often, the server must reevaluate dependencies multiple times during the life of a request for optimum performance. Apache discovered early in its HTTP/2 implementation that not doing this reprioritization led to inefficiencies.^[6]

⁶http://icing.github.io/mod_h2/nimble.html

Streams can also be added as exclusive dependencies. A stream should get exclusive access to its dependency, and any existing dependencies should be made dependent on this new exclusive stream. Figure 7.6 shows adding critical.css to the mix and making it dependent on stream 0 without (left) and with (right) the exclusive flag set.

As you see, without the exclusive flag, critical.css is at the same dependency level as main.css and main.js, but when the exclusive flag is set to 1, it takes priority and moves everything to be dependent on it, which may well be what’s needed in this example, based on the name (critical.css).

7.3.2. Stream weighting

The other concept that helps define stream priorities is weighting, which is used to prioritize two requests that are dependent on the same parent resource. Stream weightings allow more complicated scenarios than assuming even weighting for resources at the same dependency level. The critical.css scenario, for example, could have been implemented in an (almost) similar manner with the use of weightings, as shown in figure 7.7.

Here, critical.css (weighting 100) should get 10 times the resource allocations of main.css (weighting 10) and main.js (weighting 10). Using weightings isn’t the same as making them dependent, as with the exclusive flag, but it’s close. When critical.css is delivered, main.css and main.js get 50% of the resources, as they’re evenly weighted.

The 5 weighting for image.jpg isn’t used in the scenario. If main.js finishes sending before main.css (or if main.js can’t be sent yet), image.jpg gets 50% of the resources, as it gets main.js’s share. Therefore, to prevent that situation and give the CSS and JS files much higher weightings than images, a better dependency graph might be the flatter one shown in figure 7.8.

To make prioritization easier, some clients set up dummy streams with the appropriate priorities in advance, using the PRIORITY frame, and hang requests off them. The concept of allowing dummy PRIORITY frames was added late in the ratification of HTTP/2^[7], but provides a lot of flexibility and allows for a lightweight priority model. It allows dependency trees, for example, as shown in figure 7.9.

⁷https://lists.w3.org/Archives/Public/ietf-http-wg/2014OctDec/0467.html

These dummy streams are used only for prioritization and never to send requests directly. You see this situation when nghttp sets up streams 3, 5, 7, 9, and 11 at the beginning of a connection for priority reasons:

This code leads to the dependency tree shown in figure 7.10, which has a high-priority stream 3 (with a dependent stream 11), a low-priority stream 7 (with a dependent stream 9), and a middling priority stream 5.

Any requests are made dependent on one of these streams:

This setup is based on the original Firefox dependency tree. Critical CSS and Java-Script are made dependent on stream 3, noncritical JavaScript is made dependent on stream 5, and everything else is made dependent on stream 11. Note that streams 7 and 9 aren’t used at this writing.^[8] By always being able to hang resources from the same streams, you can create a reasonably efficient dependency model easily.

⁸https://nghttp2.org/documentation/nghttp.1.html#dependency-based-priority

7.3.3. Why does prioritization need to be so complicated?

Why do you need stream dependencies and weighting? This question was debated a fair bit when HTTP/2 was standardized, and SPDY, on which HTTP/2 is based, initially had only weight-based prioritization. The truth is that prioritization is complicated, and allowing both dependencies and weightings, or a mixture of the two, allows the greatest flexibility for prioritization. The added capability to create streams purely for prioritization purposes leads to more implementation options.

There’s no requirement to support stream prioritization, however, and many implementations on both the client and server side choose not to, as I discuss in the next section. As I stated in section 6.2.4, the ability to handle HTTP/2 stream prioritization efficiently could become another key differentiator between browser and server implementations, though the technicalities may be lost on most web users and developers.

Real world usage of HTTP/2 prioritization since HTTP/2 was standardized, and the fact that no implementation has yet found a perfect prioritization scheme, as we will discuss next, have led to more calls to simplify this.^[9] Whether this leads to any changes in HTTP/2 or is considered for future versions (HTTP/3) remains to be seen.

⁹https://lists.w3.org/Archives/Public/ietf-http-wg/2019JanMar/0073.html

7.3.4. Prioritization in web servers and browsers

HTTP/2 prioritization is a potentially powerful option that allows the single HTTP/2 connection to be used efficiently. This option has the advantage over six separate HTTP/1.1 connections when there’s no concept of relative prioritization other than not using one of the connections. Prioritization is complicated, however, and support is limited at this time. Although many implementations on both the server and client sides support prioritization, few give the website owner much control.

Setting priorities for web servers

Server support for prioritization is a mixed bag at this writing. Some servers support it with configuration options, some support it without configuration options, and some don’t support it. Table 7.1 summarizes prioritization support in popular HTTP/2 web servers.

^ahttps://httpd.apache.org/docs/2.4/mod/mod_http2.html#h2pushpriority

^bhttps://forums.iis.net/t/1233780.aspx

^chttps://www.nginx.com/blog/http2-module-nginx/#prioritization

^dhttp://nginx.org/en/docs/http/ngx_http_v2_module.html

^ehttps://nodejs.org/api/http2.html#http2_http2stream_priority_options

^fhttps://nghttp2.org/blog/2014/04/27/how-dependency-based-prioritization-works/

Most of the other web servers make little reference to HTTP/2 prioritization, suggesting that it’s perhaps not supported and certainly isn’t configurable if it is. Among servers that do support it, the prevailing thought seems to be to allow the client to specify the priority in requests rather than to allow server-side prioritization configuration.

Shimmercat is a fairly new web server that takes an interesting approach, allowing image requests to be sent with an initial high prioritization and then dialed back to lower priority. This approach allows the first few bytes to be sent, which allows the browser to know the size of the image and other metadata needed to lay out the page as early as possible and dial down the priority for the remainder of the image file.

Perhaps more web servers will allow this type of innovation or more control. But for now, most servers use the client-suggested priorities or don’t support them.

Setting priorities for web browsers

Web browser support is also a bit hit-and-miss. Finding documentation on this topic is tricky, but it’s possible to see stream prioritization and infer what it’s doing. To do so, you can set up Wireshark as discussed in chapter 4, but this technique allows you to intercept only browsers that export the HTTPS key settings (such as Chrome, Opera, and Firefox on the desktop). A better way is to run the nghttpd server in verbose mode and look at the incoming messages. You can also pipe the output into grep to filter only the important messages. Windows users without a Linux terminal can do the equivalent with findstr or select-string if they’re using PowerShell:

Then create a dummy index.html file in the same folder, with a load of references to various media types, to get a flavor of how each media type is sent by each browser:

It isn’t important for the referenced stylesheets, JavaScript, or image files to exist for this simple test. The test is slightly easier if these items don’t exist, in fact, as you see only 404 HEADERS frame responses rather than HEADERS frame and DATA frame responses, which only add noise.

Next, you connect to the server (such as https://localhost) and look at the frames sent. Firefox (v62) sends the frames similarly to the nghttp client, which is unsurprising, because nghttp is based on the Firefox implementation:

The output shows that Firefox has added an extra stream 13 with a weight of 241 (a super-urgent stream?) used for the original request, making it higher-priority than any CSS request.

Chrome (v69) uses no up-front PRIORITY frames, like nghttp or Firefox, but it sets a priority on requests when they’re sent and adds dependencies on previous streams. It also likes exclusive dependencies, creating a tall dependency graph:

The benefit of such use of the exclusive bit is still in debate.^[10] The Chromium team’s main argument seems to be that most requests are unusable until the full resource is received (HTML and progressive JPEGs being the primary exceptions), so it often doesn’t make sense to dilute the connection by sending multiple resources at the same time.

¹⁰https://bugs.chromium.org/p/chromium/issues/detail?id=651538

Opera (v59) does the same thing as Chrome (being another Chromium-based browser), but Safari (v12.0) seems to do weighting based on prioritization and doesn’t use stream dependencies (the opposite of Chrome!):

Edge (v41) has the poorest implementation, choosing not to use stream priorities at this writing, so every resource gets the default priority weighting of 16:

As you can see, a large variance exists among the browsers, which leads to different performance at the same site. Some researchers have performed much more extensive testing of the differences among browsers.^[11] There are likely to be more research studies and improvements in this area to come. HTTP/2 provides the tools for specific prioritization, but I’ve yet to find the best ways to use them.

¹¹https://speakerdeck.com/summerwind/2-prioritization and https://www.researchgate.net/publication/324514529_HTTP2_Prioritization_and_its_Impact_on_Web_Performance

7.4. HTTP/2 conformance testing

Now that you understand all the finer details of HTTP/2, you can compare the various implementations on both the client and server sides.

7.4.1. Server conformance testing

H2spec^[12] is an HTTP/2 conformance tester that sends various messages to an HTTP/2 server and checks whether it follows the specification accurately. Download the version for your computer type,^[13] and point it at an HTTP/2 server:

¹²https://github.com/summerwind/h2spec

¹³https://github.com/summerwind/h2spec/releases

This code should run several tests against your server and show you whether each test passes or fails:

I’ve run the tool against some popular web servers, and the results are shown in table 7.2.

I made similar tests on the home pages of some of common content delivery networks, under the assumption that the home pages run on the CDN infrastructure, which admittedly may not be a valid assumption. Table 7.3 shows the results.

Kudos to Apache for achieving the only perfect score. But does it matter that some implementations don’t match the specification as they should? Arguably not, because they often fail when trying to process incorrect messages that shouldn’t be sent in the first place. Many popular servers/CDNs handle HTTP/2 traffic successfully and without problems despite not getting 100% conformance.

If you look at nginx’s results as one example, you see the following as one of the tests the server is failing:

nginx doesn’t handle a large DATA frame as it should, but at the same time, no client should be sending such a frame. Moving on to the next errors, you see some state errors:

Again, nginx isn’t correctly handling frames sent incorrectly when the stream is in an idle state, but again, these frames shouldn’t be sent by the client. Most of the other errors follow suit.

If you’re writing an HTTP/2 server, the h2 spec tool is useful for checking whether your server is implementing the specification correctly, but the reality is that many major web servers get away with less-than-perfect implementations. The web has always been a forgiving place on the technology side, and (unlike many programming languages) slight errors are often overlooked. Still, these errors can lead to more unexpected errors later, so it’s interesting to know how your server behaves. When I published the preceding statistics on Twitter,^[14] several server implementations took note and sought to improve their compliance.

¹⁴https://twitter.com/tunetheweb/status/988196156697169920

7.4.2. Client conformance testing

A client equivalent of the tool (such as for testing browsers) does exist,^[15] though built versions aren’t supplied, so this tool must be compiled from source. I leave this task to the reader as an exercise.

¹⁵https://github.com/summerwind/h2spec/pull/74

Summary