Hands-On HTTP/3 with .NET: Creating a Conformance Testing Tool from Scratch

Photo by Aperture Vintage on Unsplash

Introduction

I believe the best way to understand technology is to create a simple version myself. This process helps me learn about the challenges faced by the original developers, the compromises they made, and the best ways to use its features.

In this post, I will explain how to create a basic HTTP/3 client that behaves like an internet browser. We’ll establish an HTTP/3 connection to a server and start communicating using HTTP/3 semantics. Additionally, I’ll show you how to observe and capture different server behaviours when we intentionally violate HTTP/3 specifications, essentially creating an HTTP/3 specification checker similar to existing tools for HTTP/2.

The code presented in this blog post is part of a lightweight, open-source HTTP/3 conformance testing tool, which you can access in this GitHub repository. Feel free to fork it or propose any changes.

Disclaimer: In order to implement the HTTP/3 client, I was highly inspired by the .NET implementation and used some .NET components, such as QPACK, for building blocks of HTTP/3.

HTTP3: Shift from TCP to QUIC

HTTP relies on a network connection that ensures data is delivered reliably and in order. Until recently, this was achieved using TCP. However, TCP doesn’t account for the HTTP/2 Multiplexer and streams, as it only guarantees reliability at the transport layer. This leads to the Head-of-line blocking problem. Additionally, applying TLS as an extension to HTTP/2 adds extra round trips for establishing a new connection, resulting in increased latency.

Head-of-line blocking in HTTP/2 occurs when a delay in one stream blocks the progress of other streams. Even though HTTP/2 allows multiple streams over a single TCP connection, if a packet is lost or delayed, it stalls the entire connection until the issue is resolved. This results in a slowdown for all active streams, not just the one with the problem.

Comparison of HTTP/2 and HTTP/3 Stacks

QUIC is a new approach that overcomes the problems posed by TCP-based HTTP connections. Unlike TCP, QUIC is a lighter protocol based on UDP. Since UDP doesn’t handle reliability, ordering, or congestion control, QUIC manages these functions at the application layer in a way that is more compatible with the HTTP protocol.

Improving HTTP/2 on top of TCP is quite cumbersome. TCP is a core protocol managed by the OS at the kernel level, which means any changes are inherently slow. Additionally, protocol ossification, caused by middleboxes enforcing strict rules and expecting specific behaviours, further slows down updates and improvements. On the other hand, most QUIC implementations are in the user space, which makes them flexible and allows for rapid updates.

Beyond addressing TCP’s problems, QUIC also supports connection migration. This feature allows you to start your connection over Wi-Fi at home and move to a mobile network without needing to establish a new connection. Unlike TCP, a QUIC connection is not tied to specific IP addresses and ports on either side.

QUIC replaces most of what TCP traditionally provides (handshake, reliability, and congestion control), improves HTTPS handshake delays, and even enhances parts of HTTP/2 (flow control and multiplexing).

QUIC Implementation

To implement QUIC, we also need to implement its corresponding TLS layer, which has been a major challenge for developers. OpenSSL, a well-known security library that is installed by default on most operating systems, does not currently support the TLS requirements for QUIC. The OpenSSL Management Committee decided to implement a complete QUIC stack on their own. For now, OpenSSL 3.2 and later versions only support QUIC connectivity in the client mode.

Given this situation, developers have three options for obtaining QUIC TLS support for their HTTP/3 needs:

1. OpenSSL-QUIC: This option supports only the client side and does not allow control over the QUIC implementation.
2. OpenSSL Fork: This involves adding QUIC APIs to OpenSSL until it officially supports QUIC. It requires building the SSL library from the source and installing it on every server that needs QUIC and HTTP/3 support, which might not be feasible for everyone.
3. OpenSSL Compatibility Layer: This allows running QUIC and HTTP/3 on top of OpenSSL without needing to patch or rebuild it, as used by Nginx.

In this blog post, we will use Microsoft’s implementation of QUIC, known as MsQuic. MsQuic is written in C and designed to be a general-purpose QUIC library, exposing interoperability layers for both Rust and C#. MsQuic is available on Windows and Linux. On Windows, MsQuic relies on built-in support from Schannel for TLS functionality, while on Linux, it relies on a fork of OpenSSL for QUIC/TLS support. In the .NET runtime, System.Net.Quic serves as an interoperability layer to call MsQuic APIs.

HTTP/3 Implementation in .NET

ٰIn this part of the blog, I will explain the essential building blocks required in HTTP/3 to make an HTTP request from a client to a server. I did not reinvent the wheel and have utilised as much of the .NET implementation for various components of HTTP/3 as possible. Note that some implementations might change or be removed over time. You can find all the source code in this repository.

The following figure provides an overview of the different concepts needed to make an HTTP request based on the HTTP/3 specification over QUIC. I will explain each of these concepts in the rest of this section.

Building Blocks for HTTP/3 Requests

Establish a QUIC Connection

In this section, I will show you how to establish a QUIC connection to an HTTP/3 server using System.Net.Quic which will be used to communicate with the server according to HTTP/3 specifications. I will not delve into the implementation details of QUIC.

A QUIC connection is a binary transport communication protocol, while HTTP/3 is a specification that explains how to implement HTTP semantics over QUIC connections and their streams. The Kestrel implementation heavily inspires this project, and most of the HTTP/3-related code is derived from the .NET HttpClient and ASP.NET Core Kestrel implementation.

To establish a new QUIC connection, you need to instantiate and populate a QuicClientConnectionOptions object and pass it to the following static method QuicConnection.ConnectAsync, which returns a QUIC connection instance.

var clientConnectionOptions = new QuicClientConnectionOptions
{
    RemoteEndPoint = new DnsEndPoint(server.Host, server.Port),
    DefaultStreamErrorCode = (long)Http3ErrorCode.RequestCancelled,
    DefaultCloseErrorCode = (long)Http3ErrorCode.NoError,
    MaxInboundUnidirectionalStreams = 5, // Minimum is 3 (1x control stream + 2x QPACK).
    MaxInboundBidirectionalStreams = 0, // Client doesn't support inbound streams
    ClientAuthenticationOptions = new SslClientAuthenticationOptions
    {
        ApplicationProtocols = new List<SslApplicationProtocol>() { SslApplicationProtocol.Http3 }
    },
};
QuicConnection quicConnection = await QuicConnection.ConnectAsync(clientConnectionOptions);

QUIC supports two types of streams: unidirectional and bidirectional. Unidirectional streams are mainly used to create control streams in HTTP/3 to exchange settings between the client and server. Bidirectional streams are used to make requests and send responses in HTTP/3.

To initialise a stream, you can use the following method on the QUIC instance, specifying the type of stream you want to create. According to the HTTP/3 specification, the server can only initialise unidirectional streams, while the client is allowed to initialise both types of streams.

var streamType = QuicStreamType.Unidirectional; // or QuicStreamType.Bidirectional
QuicStream quicStream = await quicConnection.OpenOutboundStreamAsync(streamType);

Frames

HTTP/3 frames are the basic units of communication in the HTTP/3 protocol. Each frame carries a specific type of information, such as HEADERS, DATA, or SETTINGS, and is transmitted over QUIC streams. All frames have the following format:

HTTP/3 Frame Format {
  Type (i),
  Length (i),
  Frame Payload (..),
}

In the repository, the Http3FrameWriter class is responsible for writing different frames based on their underlying format. For example, to write the request headers, there is the WriteRequestHeaders method, which accepts a list of key-value pairs and writes their binary HEADERS frame representation into the request stream. This ensures the headers are correctly formatted and transmitted according to the HTTP/3 specification.

internal class Http3FrameWriter 
{
  internal Task WriteSettingsAsync(List<Http3PeerSetting> settings){}
  internal Task WriteGoAwayAsync(long id){}
  internal Task WriteRequestHeaders(IDictionary<string, string> headers){}
  internal Task WriteDataAsync(in ReadOnlySequence<byte> data){}
}

QPACK

QPACK is a compression format designed for efficiently representing HTTP fields, particularly headers that are often repetitive. Before writing the HEADERS frame, the header key-value pairs must be compressed according to the QPACK specification. While explaining the inner workings and implementation of QPACK is beyond the scope of this post, I will use the .NET internal implementation of QPACK from System.Net.Http.QPack for handling the compression and decompression of request and response headers in HTTP/3.

Makes an HTTP/3 Request

To make an HTTP request following the HTTP/3 specification, several steps need to be completed between the client and server once a QUIC connection is established. The sequence diagram below illustrates how to send an HTTP request with the HTTP/3 specification to a server over a QUIC connection.

Sending HTTP/3 Request Over QUIC: Sequence Diagram

The first step is to establish a QUIC connection to an HTTP/3 server. This can be done using the following code:

private async Task<Http3Connection> CreateConnection(Http3ServerOptions server)
{
    var clientConnectionOptions = new QuicClientConnectionOptions
    {
        ...
    };
    var quicConnection = await QuicConnection.ConnectAsync(clientConnectionOptions);
    return new Http3Connection(quicConnection);
}

I’ve encapsulated the QUIC connection within an Http3Connection class, which simplifies the creation of streams within this connection.

Unidirectional streams, in either direction, are used for a range of purposes. The purpose is indicated by a stream type, which is sent as a variable-length integer at the start of the stream. The format and structure of data that follows this integer is determined by the stream type.

Unidirectional Stream Header {
  Stream Type (i),
}

To initiate a unidirectional stream to send client settings, you can use the following code:

var outboundControlStream = await connection.OpenStreamAsync(QuicStreamType.Unidirectional);
await outboundControlStream.WriteStreamTypeId((long)Http3StreamType.Control); // 0x00

Each side must initiate a single control stream, which is a unidirectional stream, at the beginning of the connection. The first frame sent on this stream should be the SETTINGS frame, which exchanges configuration parameters that affect how endpoints communicate, such as preferences and constraints on peer behaviour. You can send settings to the server with the following line of code:

await outboundControlStream.WriteSettingsFrameAsync([
    new(Http3SettingType.QPackMaxTableCapacity, 1)
]);

After exchanging settings, we can make an HTTP request by initialising a bidirectional request stream. This stream will be used to send the HTTP request and receive the response from the server. Once the request is complete, the stream will be closed and never used again.

var requestStream = await connection.OpenStreamAsync(QuicStreamType.Bidirectional);

The first frame in the request stream must be the HEADERS frame. In addition to HTTP headers, we need to specify some mandatory pseudo-header fields that convey message control data. Pseudo-header fields are only valid in the context in which they are defined. For example, we can specify the HTTP method with the :method header. The following code sends the HEADERS frame with request pseudo-header fields:

var headers = new Dictionary<string, string>()
{
    {":authority", connection.QuicConnection.TargetHostName },
    {":method", "POST" },
    {":path", "/" },
    {":scheme", "https" },
    {"Content-Type", "application/x-www-form-urlencoded" }
};
await requestStream.WriteRequestHeader(headers);

DATA frames are not mandatory, but in this case, we want to POST something to the server. We can write a byte array as the payload of the DATA frame.

byte[] body = ...;
await requestStream.WriteData(body);

The request stream is bidirectional, allowing us to wait for the server to write its response to the client. In response, a single :status pseudo-header field is defined to carry the HTTP status code along with other response headers.

The server behaves just like the client and sends data in frames. In our code, we also implement the necessary logic to interpret incoming bytes on the stream as the appropriate HEADERS and DATA frames.

Conformance Testing

Now that we’ve comprehensively understood and implemented all the complex building blocks of HTTP/3 over a QUIC connection to enable HTTP/3 requests, we can proceed to construct a conformance testing tool for HTTP/3 implementations. I will demonstrate how we can send various messages to an HTTP/3 server and verify whether it adheres to the specifications accurately. Here are some test cases that I have implemented and executed against different HTTP/3 servers such as Kestrel, Cloudflare, Facebook, and others.

Local Kestrel HTTP3 Server

To test the HTTP/3 implementation in .NET Kestrel, we are going to run a lightweight ASP.NET Core application on a local machine, which only supports the HTTP/3 protocol on localhost:6001.

To run this server, you need to ensure that you have the .NET HTTP/3 requirements on your local machine. For Windows 11, simply install the latest preview version of .NET SDK 9. While it is possible to run with .NET 8, I prefer to use the latest implementation for conformance testing. On Linux, you need to install MsQuic. Unfortunately, macOS is not supported yet.

In conformance testing, we skip certificate validation for simplicity. However, to test it in the browser, you need to generate a test certificate that is trusted on your local machine using the following command:

dotnet dev-certs https -ep ./certs/certificate.pfx -p 1234567 --trust
dotnet dev-certs https --check --trust

You can now easily run your server locally with the following command:

cd src\h3server
dotnet run

To test it in a browser that supports HTTP/3, like Chrome, you can start your browser with the following arguments(make sure all Chrome instances are closed on your local machine):

chrome.exe --origin-to-force-quic-on=localhost:6001 https://localhost:6001

In Chrome, you can enable QUIC by going to the address bar and typing chrome://flags#enable-quic. Set the experimental QUIC protocol flag to enabled. Close Chrome and relaunch it.

Specification test cases scenario

What happens if the client violates part of the specification and makes a malformed request to the server? For example, consider the following sentence from section section-6.2.1–2 of the HTTP/3 specification:

Each side MUST initiate a single control stream at the beginning of the connection and send its SETTINGS frame as the first frame on this stream. If the first frame of the control stream is any other frame type, this MUST be treated as a connection error of type H3_MISSING_SETTINGS.

To evaluate this section, we need a test scenario that sends a GOAWAY frame at the beginning of a control stream instead of a SETTINGS frame. As a result, we expect an exception to be raised by the server and captured by the stream. We then need to verify the exception to ensure the server returns an error of type H3_MISSING_SETTINGS.

// Arrange
var outboundControlStream = await connection.OpenStreamAsync(QuicStreamType.Unidirectional);

// Act
await outboundControlStream.WriteStreamTypeId((long)Http3StreamType.Control);
await outboundControlStream.WriteGoAway(0);

var inboundControlStream = await connection.AcceptStreamAsync();
var inboundTask = inboundControlStream.ProcessRequestAsync();
await WaitForInboundStreamTask(inboundTask);

// Assert
var exceptoin = inboundControlStream.Exception;
exceptoin.Should().BeOfType<QuicException>();
exceptiHi Mon.ApplicationErrorCode.Should().Be((int)Http3ErrorCode.MissingSettings);

I implemented some simple test scenarios in this part of the repository. We can run these tests against different servers to evaluate their adherence to the specified behaviours.

To run the tools, first, amend the appsettings.json file to specify the HTTP/3 servers. I’ve already included some server configurations from Cloudflare, Facebook, and other well-known HTTP/3 implementations. The following command will run the conformance testing tool:

cd src\h3spec
dotnet run

The tool will pick up each test scenario one by one and write its results to the console, similar to the following output, which I obtained from running the tool against Kestrel:

Console Output of Test Scenarios Run Against Kestrel

Wrap Up

In this blog post, I’ve explained HTTP/3 and QUIC, highlighting the advantages and differences compared to HTTP/2. I also mentioned an interesting implementation issue related to the lack of TLS functionality in OpenSSL, which developers have to address differently.

We then had a brief overview of the important building blocks of HTTP/3 over QUIC, aimed at making an HTTP request to a server following the HTTP/3 specifications. I demonstrated how you can accomplish this.

In the final section, I discussed the possibility of using these building blocks to develop a conformance tool to examine different server behaviours based on various parts of the HTTP/3 specification. While the tool is still under development and may contain bugs or mistakes, feel free to raise issues or contribute improvements to its repository.

I hope this blog post has helped you gain a solid understanding of the HTTP/3 protocol, just as it has for me.