Golang Http2 Fundamentals and Implementation

Author

Reads 1.2K

Programming Code on Screen
Credit: pexels.com, Programming Code on Screen

Http2 is a multiplexing protocol that allows multiple requests to be sent over a single connection, reducing latency and improving performance.

Golang has a built-in http2 package that makes it easy to implement http2 in your applications.

Http2 uses a binary framing layer to multiplex multiple requests over a single connection, which is more efficient than the text-based framing layer used in http1.1.

This efficiency is achieved by using a header compression algorithm that reduces the size of the headers sent over the wire.

In Golang, the http2 package uses the hpacce header compression algorithm, which is a variant of the HPACK algorithm used in http2.

You might like: Install Golang Package

Configuring Server and Transport

To add HTTP/2 support to a net/http Server, you must call the ConfigureServer function before the server begins serving. This is a crucial step to enable HTTP/2 functionality.

ConfigureServer returns an error if the server has already been HTTP/2-enabled, so it's essential to check the return value. The ConfigureServer function adds HTTP/2 support to a net/http Server.

When configuring a net/http HTTP/1 Transport to use HTTP/2, you can use the ConfigureTransports function. This function returns a new HTTP/2 Transport for further configuration.

Configure Server

Credit: youtube.com, Configuring the Server

Configure Server is a crucial step in setting up your server. You must call ConfigureServer before your server starts serving.

To add HTTP/2 support to your net/http Server, use the ConfigureServer function. This function must be called before your server begins serving.

The options for the Server.ServeConn method are defined by the ServeConnOpts type. If you pass nil for the opts parameter, default values will be used.

Take a look at this: Golang Func Type

Transport

Transport is an HTTP/2 Transport that internally caches connections to servers. It's safe for concurrent use by multiple goroutines.

If you want to use HTTP/2 with your net/http Server, you'll need to configure it properly. The `ConfigureTransport` function configures a net/http HTTP/1 Transport to use HTTP/2, but it's recommended to use `ConfigureTransports` instead.

`ConfigureTransports` returns a new HTTP/2 Transport for further configuration, but it will return an error if the Transport has already been HTTP/2-enabled. This is because HTTP/2 is enforced and requires additional configuration.

Here's a brief summary of the key differences between `ConfigureTransport` and `ConfigureTransports`:

ClientConn

Credit: youtube.com, HTTP/2 and http2 in Go 1.6

ClientConn is the state of a single HTTP/2 client connection to an HTTP/2 server. It's a crucial component for making HTTP/2 requests in Go.

To manage a pool of HTTP/2 client connections, you can use ClientConnPool. This helps you keep track of multiple connections and reuse them when needed.

The http.Client in Go is used for HTTP/2 requests, but you need to specify http2.Transport instead of http.Transport in the client's Transport field. This is a subtle difference that's easy to overlook.

On a similar theme: Golang Go

Client Conn Pool

Client Conn Pool is a pool of HTTP/2 client connections managed by ClientConnPool.

This pool is used to efficiently handle multiple requests over a single connection, making it a great way to improve performance.

The ClientConnPool is particularly useful for applications that need to send and receive data in a full duplex manner, like Brad Fitzpatrick's example code in the Go Client Implementation.

In this code, data is sent through the pipe to the request, and data is returned from the server in the response body as it becomes available, allowing for simultaneous communication in both directions.

Curious to learn more? Check out: Golang Comments

Credit: youtube.com, What is database pooling and why it is important? #shorts #youtubeshorts

However, it's worth noting that connections in the pool are not closed until they're no longer needed, which can lead to idle connections if not properly managed.

To address this, you can use the CloseIdleConnections method, which closes any connections that are no longer in use, freeing up resources for other requests.

This method doesn't interrupt any connections currently in use, so you can safely close idle connections without affecting ongoing requests.

On a similar theme: Golang Use Cases

(*ClientConn) Ping

The (*ClientConn) Ping function is a vital part of maintaining a healthy connection with an HTTP/2 server.

It sends a PING frame to the server and waits for the ack, essentially keeping the connection alive and checking for any issues.

This process is crucial for ensuring that the connection remains active and functional, even in the face of network issues or other problems.

By sending a PING frame and waiting for the ack, the Ping function helps to detect any potential issues with the connection before they become major problems.

In practice, this means that the Ping function can help prevent connection timeouts and other issues that might otherwise arise from a dormant or faulty connection.

HTTP/2 Client

Credit: youtube.com, Hacking with Andrew and Brad: an HTTP/2 client

The http.Client in Go is used for HTTP/2 requests as well, but with a slight difference. You need to use http2.Transport instead of http.Transport in the client's Transport field. This is a subtle but important distinction.

The Go net/http package already supports HTTP/2 "out of the box", but it's not always enabled by default. If your service works over HTTPS, chances are it's already using HTTP/2 automatically. However, if it works over plain HTTP, it's likely not.

If you want to enable HTTP/2 without TLS, you can use the protocols.SetUnencryptedHTTP2 function. This allows the client and server to interact over HTTP/2 even without HTTPS.

A unique perspective: Golang vs Go

Go Client Implementation

The Go Client Implementation is pretty interesting. We create a request with a dynamic body, not something straightforward for the average Go programmer. Then, we receive a dynamic response body, which is more normal, yet not too common.

Brad Fitzpatrick suggests using a code similar to the following, which is pretty low level, so I added explanations in comments. The example is pretty interesting because we create a request with a dynamic body and receive a dynamic response body.

Credit: youtube.com, Go's backwards way of testing HTTP clients

Here's an example of how to create a request with a dynamic body:

```

func main() {

req := &http.Request{

Method: "POST",

URL: "http://example.com",

Body: http.MaxBytesReader(os.Stdout, []byte("Hello, world!"), 1024),

}

// ...

}

```

This code creates a POST request to `http://example.com` with a dynamic body that can be read from `os.Stdout`. The `http.MaxBytesReader` function is used to limit the size of the body to 1024 bytes.

A FrameType is a registered frame type as defined in https://httpwg.org/specs/rfc7540.html#rfc.section.11.2.

WriteSettings

The WriteSettings function is a crucial part of the HTTP/2 protocol, and it's used to write SETTINGS frames with zero or more settings.

The ACK bit is not set when using the WriteSettings function, which means that the sender is not acknowledging any previous frames.

SETTINGS frames are used to negotiate parameters and settings for the HTTP/2 connection, such as the maximum frame size and the initial window size.

HTTP/2 Server

The HTTP/2 server in Go is a low-level package intended for direct use by few people. Most users will use it indirectly through the net/http package, which automatically enables HTTP/2 support for HTTPS connections from Go 1.6 and later.

Credit: youtube.com, Exploring HTTP/2 with Go

To use the HTTP/2 server in earlier Go versions, you need to call the ConfigureServer function before the server starts serving. This function adds HTTP/2 support to a net/http Server.

You can also use the x/net/http2/h2c package to enable H2C (HTTP/2 Cleartext) protocol, which is supported by the standard library from Go 1.12.

Http/2 Server

To set up an HTTP/2 server, you need to call the ConfigureServer function before your server starts serving. This function adds HTTP/2 support to a net/http Server.

The net/http package in Go supports HTTP/2 automatically from version 1.6 and later, but for earlier versions, you need to use ConfigureServer.

To generate a private key and certificate, you can run a command on Linux, which will create two files: server.key and server.crt. The server.key file contains your server's private key and should remain private and secret in production systems.

You can use Go's standard library HTTP server and enable TLS with the generated SSL files. If you want to use H2C (HTTP/2 Cleartext) protocol, which is HTTP/2 with no TLS, you can use the x/net/http2/h2c package.

Credit: youtube.com, HTTP/2 Push

Here are some common scenarios where HTTP/2 might not be used:

  • Your service works over the HTTP protocol using a simple ListenAndServe.
  • You are behind a proxy server like Cloudflare.
  • You are behind Nginx with HTTP/2 support.

To use HTTP/2 directly, you need to set up your service with SSL/TLS. Technically, you can use HTTP/2 without TLS, but it's not recommended for external traffic.

If you want to use HTTP/2 without TLS, you can set the protocols.SetUnencryptedHTTP2(true) flag, but this only works for simple HTTP and not for HTTPS.

Here are some examples of how to set up HTTP/2:

Public Certificate

A public certificate is crucial for securing your HTTP/2 server. If you don't have one, you're leaving your server vulnerable to attacks.

The Common Name field in your certificate is very important. It needs to match what your client uses as part of the request host/URL or tls.Config.ServerName, or you'll get an error.

This can be frustrating if you're not expecting it, but it's an easy mistake to make.

HTTP/2 Basics

HTTP/2 is a protocol that builds on top of TLS connections. The process starts with a ClientHello message that includes an ALPN extension, listing the protocols supported by the client, such as "h2" for HTTP/2.

Credit: youtube.com, Golang basic HTTP2 demo in 5 min (Hello world)

The TLS server stack checks this list against the protocols it supports, and if both sides agree on "h2", the server confirms the choice in its ServerHello response.

The scheme (HTTP or HTTPS) implies the type of connection. If it's TLS on port 443, you know it's HTTPS.

The Host header, added in HTTP/1.1 for virtual hosting, is just a regular header and not a formal part of the request structure.

In summary, here are the key points about HTTP/2 basics:

  • Scheme (HTTP or HTTPS) implies the type of connection.
  • Host header is a regular header, not a formal part of the request structure.

HTTP/2 in Go

HTTP/2 in Go is a powerful feature that allows for efficient and concurrent communication between clients and servers. It's supported by the standard library, and most users will use it indirectly through the net/http package.

To use HTTP/2 with Go, you'll need to generate a private key and a certificate. This can be done using a command like "openssl req -x509 -newkey rsa:4096 -nodes -keyout server.key -out server.crt -days 365" on Linux. The resulting files, server.key and server.crt, contain the server's private key and public key, respectively.

Credit: youtube.com, London Go Gathering 2015 | HTTP/2 with Brad Fitzpatrick

You can enable TLS for your server by using the generated SSL files and the net/http package. Alternatively, you can use the H2C (HTTP/2 Cleartext) protocol, which is supported by the x/net/http2/h2c package. However, keep in mind that this protocol is not recommended for production use.

Here are some common scenarios where HTTP/2 may not be used:

  • Your service works over HTTP, using simple ListenAndServe.
  • You're behind a Cloudflare proxy. In this case, requests from users to Cloudflare may use HTTP/2, but the connection from Cloudflare to your service (the origin) usually uses HTTP/1.1.
  • You're behind an Nginx server with HTTP/2 support. Nginx acts as a TLS terminator, decrypting the request and re-encrypting the response, while forwarding all requests to your service over HTTP/1.1.

To enable HTTP/2 on your service, you'll need to configure it with SSL/TLS. This can be done using the standard library or the golang.org/x/net/http2 package.

Http/2 Advanced Features

HTTP/2 provides more features beyond a secured and efficient connection, including server push, priority scheduling, and frame writing.

Server push, for example, allows a server to send multiple responses to a client in a single request, but unfortunately, it's currently not possible to consume server push with a Go client.

However, server push does work with certain clients, such as Google Chrome, which can handle server push and receive multiple responses from the server.

Credit: youtube.com, HTTP tunneling in Go using HTTP/2 streams

The Go client transport sets an HTTP/2 setting flag that indicates push is disabled, which is why server push doesn't work with Go clients.

A WriteScheduler can be used to schedule frames by following HTTP/2 priorities as described in RFC 7540 Section 5.3, which is useful for optimizing performance.

You can construct a WriteScheduler using the NewPriorityWriteScheduler function, which takes an optional configuration object as an argument.

Frames can be written to the underlying writer using the WriteHeaders, WriteContinuation, and WritePriority methods of the Framer struct.

WriteHeaders writes a single HEADERS frame, which is a low-level header writing method that encodes headers and splits them into any necessary CONTINUATION frames elsewhere.

WriteContinuation writes a CONTINUATION frame, which will perform exactly one Write to the underlying Writer.

It's the caller's responsibility to not call other Write methods concurrently when using WriteContinuation.

A PriorityFrame specifies the sender-advised priority of a stream, which is useful for optimizing performance by prioritizing certain streams over others.

Check this out: Golang Writefile

Http/2 In Go

Credit: youtube.com, GopherCon 2015: HTTP/2: What It Is and How We Implemented It In Go - Brad Fitzpatrick

HTTP/2 in Go is a powerful feature that allows for faster and more efficient communication between clients and servers. The standard library in Go already supports HTTP/2, but it's not enabled by default for plain HTTP connections.

If your service works over plain HTTP, it's likely that HTTP/2 is not used. However, if you're behind a proxy like Cloudflare, the client may use HTTP/2 to connect to Cloudflare, but the connection from Cloudflare to your service is usually HTTP/1.1.

To use HTTP/2, you need to set up SSL/TLS on your server. Technically, you can use HTTP/2 without TLS, but it's not recommended for external traffic. However, it can be used in internal environments like microservices or private networks.

Here are some common scenarios where HTTP/2 may not be used:

  • Your service works over plain HTTP using the `ListenAndServe` function.
  • You're behind a proxy like Cloudflare, which may use HTTP/2 to connect to the client, but the connection from Cloudflare to your service is HTTP/1.1.
  • You're behind Nginx with HTTP/2 support, which acts as a TLS termination point, decrypting the request and re-encrypting the response, and then forwarding the request to your service over HTTP/1.1.

If you want to use HTTP/2 directly, you need to configure it manually. You can do this by using the `SetUnencryptedHTTP2` function to enable unencrypted HTTP/2. This allows the client and server to communicate over HTTP/2 even without HTTPS.

Credit: youtube.com, How HTTP/2 Works, Performance, Pros & Cons and More

Here's an example of how to do this:

```

import (

"net/http"

"golang.org/x/net/http2"

)

func main() {

h := http2.NewServerTransport()

h.SetUnencryptedHTTP2(true)

http.DefaultTransport = h

}

```

This code enables unencrypted HTTP/2 on the default transport. Note that this is not recommended for external traffic, but it can be useful for internal environments.

Variables

HTTP/2 in Go involves working with variables that define the behavior of the protocol.

The ErrFrameTooLarge error is returned by Framer.ReadFrame when a peer sends a frame that is larger than declared with SetMaxReadFrameSize.

Frames in HTTP/2 can be quite large, but the SetMaxReadFrameSize function helps prevent issues by limiting their size.

String

In HTTP/2, a string is a sequence of characters that can be used to identify a specific protocol or header field.

Strings are composed of UTF-8 encoded characters, which allows for the representation of a wide range of languages and characters.

The maximum length of a string in HTTP/2 is 16,384 bytes, which is a significant increase from the 255-byte limit in HTTP/1.1.

Strings can be used to represent header field names and values, as well as protocol names and versions.

Oscar Hettinger

Writer

Oscar Hettinger is a skilled writer with a passion for crafting informative and engaging content. With a keen eye for detail, he has established himself as a go-to expert in the tech industry, covering topics such as cloud storage and productivity tools. His work has been featured in various online publications, where he has shared his insights on Google Drive subtitle management and other related topics.

Love What You Read? Stay Updated!

Join our community for insights, tips, and more.