Adam Prescott

SPDY’s colon headers

When Chrome is talking to a web server using SPDY, if you check the HTTP request headers in Developer Tools, you’ll find some non-standard-looking headers that all start with colons.

:host: www.google.com
:method: GET
:path: /search?q=spdy+colon+headers&qscrl=1
:scheme: https
:version: HTTP/1.1

These headers don’t exist for other requests, so what are they?

spdy-colon-headers.png

This information already exists in the first line of the HTTP request, right?

GET https://www.google.com/search?q=spdy+colon+headers&qscrl=1 HTTP/1.1

If you had no idea you were looking at headers for a SPDY connection, it might look like the browser is adding request headers where it doesn’t need to. After all, why not just parse the info out of the first line of the request? Well, because it’s SPDY, and there’s nowhere to put a single line like GET /foo HTTP/1.1 in the SPDY protocol. These required colon headers take the place of the first HTTP request line, to provide a consistent metadata system.

By “unfolding” the values in the first line of a HTTP request — which is sort of an arbitrary string — to explicit headers in the client to be sent in the request, the whole thing is much more uniform. A simpler protocol.

Reading these explicit values on the server requires only parsing headers instead of also an arbitrary string corresponding to the first line of a HTTP request. This tiny gain is of course multiplied by every HTTP router that might be involved.

There’s an interesting little aside here, with the way that SPDY actually carries HTTP, compared to something like TLS/SSL.

Encapsulation and framing

Encapsulation is like wrapping one thing in something else. For example, consider TCP and IP.

                     IP packet

   +-----------+---------------------------------+
   |           |/                               \|
   |           |                                 |
   |    IP     |           TCP packet            |
   |           |                                 |
   |  headers  |         payload / data          |
   |           |                                 |
   |           |\                               /|
   +-----------+---------------------------------+

The TCP packet is wrapped up, as-is, as the payload (data) segment of an IP packet. When the IP packet is received, the TCP packet is unwrapped and then used. The data encapsulated by an IP packet does not need to be TCP, it could instead be UDP, or something else. That IP packets are ignorant of this allows flexibility higher up the stack.

In the same way, TLS/SSL encapsulates (most often) HTTP. You set up a secure tunnel and then send the HTTP payload, unmodified, through that secure tunnel.

SPDY doesn’t really encapsulate HTTP in the same way IP encapsulates TCP. (The version 3 draft spec doesn’t mention the word “encapsulate”.) But it does take the HTTP request, both header and data, and chop it up into different frames that get sent over to the server.

HTTP headers are lines separated by CRLF newline characters, but SPDY defines a way to send this data in a different form, as bytes over the wire. A spdy-dev mailing list post explains this (as well as backing up, further down, the simpler parsing point above):

SPDY is a way of converting the concept of an “HTTP message” into bytes to be sent over the wire. This is confusing because the method of doing this currently in widespread use is also called HTTP. As an example, an HTTP message contains a list of headers, each of which is a name/value pair. In SPDY, this is serialized as a series of length-delimited strings, the block of which is then gzipped. In HTTP, this is serialized as a series of CRLF-delimited strings, where names and values are separated by a colon (and isn’t gzipped).

In section 4.1 of the SPDYv3 draft the blending of the framing layer and application layer is touched upon:

Readers may note that this specification sometimes blends the framing layer (Section 2) with requirements of a specific application - HTTP (Section 3). This is reflected in the request/response nature of the streams, the definition of the HEADERS and compression contexts which are very similar to HTTP, and other areas as well.

This blending is intentional - the primary goal of this protocol is to create a low-latency protocol for use with HTTP. Isolating the two layers is convenient for description of the protocol and how it relates to existing HTTP implementations. However, the ability to reuse the SPDY framing layer is a non goal.


(Thanks to Ilya Grigorik for giving some input on this post.)