I Write HTTP Servers

[Main] [Articles] [Projects] [Now] [Resume] [Talks] [Training]

October 31 2025

I am surprised I haven't written this sooner, especially since I have written about How I Write HTTP Clients. It was inevitable this post was going to be written. With that said, I want this post to follow the same format so a lot of this will read very similar since I copied a lot of the language from it and just adapted it for HTTP servers.

I have written a lot of HTTP server that serve APIs with different types of content over the years and there have been some patterns that have emerged and so I thought I would share them in hopes they help others. The major concepts I want to highlight are:

• Composing Server Dependencies and Options
• Composing Middleware
• Data Encoding/Decoding/Validation
• Testing Handlers

To illustrate these concepts I am going to write a REST API with the following key features:

• Support JSON / XML
• Authorization Tokens
• Stores data in a database

Write Handlers First

Most of you might think to start writing the main HTTP server first, which makes a lot of sense, but I like to focus on the micro and write my individual handlers first. This allows me to discover dependencies as they come and keeps me focused on small bits of single-purpose and testable logic first.

I also like to start with the routes that create data since we need data to exist to read it and delete it.

func CreateArticle(logger Logger, ds DataStore) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // ... }) }

The CreateArticle() is a simple function that takes in its dependencies as arguments and returns a http.Handler which we can mount to a route. This particular handler requires a Logger and a DataStore to do its work. By keeping these as interfaces we can pass in any implementation of them we want.

For now lets complete the rest of the handlers for working with articles.

func CreateArticle(logger Logger, ds DataStore) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // ... }) } func DeleteArticle(logger Logger, ds DataStore) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // ... }) } func ListArticles(logger Logger, ds DataStore) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // ... }) } func GetArticle(logger Logger, ds DataStore) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // ... }) } func GetArticleImage(logger Logger, bs BlobStore) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // ... }) }

Looking over the dependencies for all of these handlers we can can see we need a Logger, DataStore, and a BlobStore.

Interface Implementations

Since we discovered the dependenices we need for our handlers we can rough out what those would be. The important concept here is that it doesn't matter how each of them are implemented since we are only concerned with the handlers themselves. For illustrative purposes we could have the following interfaces defined as:

type Logger interface { Log(lvl Level, msg string, kvpairs ...string) } type DataStore interface { SaveArticle(ctx context.Context, article *Article) error QueryArticles(ctx context.Context, filterby string) ([]Articles, error) FindArticle(ctx context.Context, id int) (*Article, error) DeleteArticle(ctx context.Context, id int) error } type BlobStore interface { ReadBlob(ctx context.Context, path string) (*Blob, error) WriteBlob(ctx context.Context, path string, r io.Reader) error }

Each of these could have a few implementations and in most scenarios they would have only one, especially when the server is running in production. However, there are times where they might need to change and/or its impossible to test these handlers with a concrete implementation of something that requires network access to a 3rd party system.

• Logger could be implemented with log/slog, zap.Logger, etc.
• DataStore could be PostgreSQL with pgx or as simple as database/sql.
• BlobStore could be the local filesystem, AWS S3, or even scp if you're feeling spicy

JSON / XML Support

APIs should leverage the Content-Type and Accept HTTP headers to negotiate what encoding format should be used during the lifecycle of a request and response. Let's just focus on the CreateArticle() handler since the rest will follow the same idea.

func CreateArticle(logger Logger, ds DataStore) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { var req ReqPayload var res ResPayload switch r.Header.Get("Content-Type") { case "application/json": // ... case "application/xml" // ... } article := Article{ Title: req.Title, Contents: req.Contents, } err := ds.CreateArticle(r.Context(), &article) res = ResPayload{ ID: article.ID, CreatedAt: time.Now(), } switch r.Header.Get("Accept") { case "application/json": // ... case "application/xml" // ... } }) }

This would be the ideal setup for each.

1. Set up the request and response structs
2. Decode the request depending on the Content-Type header
3. Do the work of the handler and build the response
4. Encode the response depending on the Accept header

This can get very repetative for every route so, fortuately, we can leverage some generics to make this a bit easier on ourselves.

func decode[T any](r *http.Request) (T, error) { var req T switch r.Header.Get("Content-Type") { case "application/json": if err := json.NewDecoder(r.Body).Decode(&req); err != nil { return nil, err } case "application/xml" if err := xml.NewDecoder(r.Body).Decode(&req); err != nil { return nil, err } } return req, nil } func encode[T any](w http.ResponseWriter, r *http.Request, status int, res T) error { mimeType := r.Header.Get("Accept") w.Header().Set("Content-Type", mimeType) w.WriteHeader(status) switch mimeType { case "application/json": if err := json.NewEncoder(w).Encode(v); err != nil { return err } case "application/xml": if err := xml.NewEncoder(w).Encode(v); err != nil { return err } } return nil }

Now that we have some nice encoding/decoding functions we can reuse these in our handlers and the proper encoding format will be used based on the headers in the http.Request.

func CreateArticle(logger Logger, ds DataStore) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { var res ResPayload req, err := decode[ReqPayload](r) article := Article{ Title: req.Title, Contents: req.Contents, } err := ds.CreateArticle(r.Context(), &article) res = ResPayload{ ID: article.ID, CreatedAt: time.Now(), } err := encode[ResPayload](w, r, http.StatusOK, res) }) }

We can even take this a step further and introduce some nice request validation too. We can accomplish this by expanding the decode() to not just use any type T, but a certain Validator and if the request is invalid decoding fails.

type Validator interface { Valid(ctx context.Context) error } func decode[T Validator](r *http.Request) (T, error) { var req T switch r.Header.Get("Content-Type") { case "application/json": if err := json.NewDecoder(r.Body).Decode(&req); err != nil { return nil, err } case "application/xml" if err := xml.NewDecoder(r.Body).Decode(&req); err != nil { return nil, err } } if err := req.Valid(r.Context()); err != nil { return nil, err } return req, nil }

We define a new Validator interface and change our decode[T Validator]() to work with that interface. Once decoding is completed we can call Valid() on the struct and return any validation errors that occur back to our handler. We didn't even need to change out handler at all!

Testing Handlers

Now that we have a fully built handler we can begin to test it. I do want to stress that what we are testing should be the HTTP logic only. We should not be testing the behavior of DataStore.CreateArticle() since that should be handled by its own tests of its own implementations. We are merely testing the request / response lifecycle to ensure out payloads uphold the contracts with HTTP clients.

type mockLogger struct {} type mockDataStore struct {} func TestCreateArticle(t *testing.T) { handler := handlers.CreateArticle(&mockLogger{}, &mockDataStore{}) // Create the reqPayload := bytes.NewStringBuffer(`{"title":"Article Title","contents":"This is the contents of the article"}`) // The "/" path does not matter since we're calling the handler directly r := httptest.NewRequest(http.MethodPost, "/", reqPayload) r.Header.Set("Content-Type": "application/json") r.Header.Set("Accept": "application/json") w := httptest.NewRecorder() handler.ServeHTTP(w, r) if w.StatusCode != http.StatusOK { t.Fail("expected 200 response code") } if !strings.Contains(w.Body.String(), "id") { t.Fail("expected response to contain generated article id") } }

Notice we didn't even need to start a real HTTP server and bind to a port to test this. We only needed to create an instance of the handler with its dependencies. We were able to provide our own in-memory implementation of them since they are just interfaces. Next, we created a request with httptest.NewRequest() with the body of the request as some JSON. bytes.NewStringBuffer() lets us make some JSON that implements the required io.ReadCloser. Then we make a httptest.NewRecorder() so our handler can write results to an in-memory buffer that simulates the client end of the connection. We can use parts of the record to test the status code and the contents of the response body to ensure our handler is doing what it is meant to do.

Layers of Middleware

At this point we have all of our handlers written and tested and everything is working as expected. All tests run in-memory and can also run in parallel.

Next we need to layer some middleware on top of them to protect them with authorization tokens and get insight into them with some basic logging. We just keep following the rule of writing functions that return http.Handler, but now they also accept a http.Handler as their first argument.

func LoggerMiddleware(next http.Handler, logger Logger) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { logger.Log(logger.INFO, "request", "method", r.Method, "path", r.RequestURI) next(w, r) }) } func AuthorizationMiddleware(next http.Handler, as AuthStore) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { authheader := r.Header.Get("Authorization") method, token, found := strings.Cut(authheader, " ") if !found { http.Error(w, http.StatusUnauthorized, "not authorized") return } if method != "Bearer" { http.Error(w, http.StatusBadRequest, "invalid authorization method") return } err := as.ValidToken(r.Context(), token) if err != nil { http.Error(w, http.StatusUnauthorized, "not authorized") return } next(w, r) }) }

Both middlewares accept and return http.Handler which will allows us to chain them together. It also allows us to test them in a similar fashion as our other handlers.

Pulling it All Together

We have handlers and middleware all ready to go now. Each are small discrete functions that can be reasoned about and tested by themselves. Now it is time to pull them all togther to make our HTTP server. I like to make a NewServer() function that does all of this for me and, you guessed it, it will return a http.Handler. Shocking, I know.

func NewServer( logger Logger, ds DataStore, bs BlobStore, as AuthStore, ) http.Handler { mux := http.NewServeMux() mux.Handle("GET /articles", handlers.ListArticles(logger, ds)) mux.Handle("GET /articles/{id}", handlers.GetArticles(logger, ds)) mux.Handle("POST /articles", handlers.CreateArticle(logger, ds)) mux.Handle("DELETE /articles/{id}", handlers.DeleteArticle(logger, ds)) var handler http.Handler = mux handler = LoggerMiddleware(handler, logger) handler = AuthorizationMiddleware(handler, as) return handler }

Now we have one http.Handler to rule them all. We composed a http.ServeMux which is also a http.Handler so we can attach our handlers to routes. Then we layer on the middleware passing one into the next to build it up and finally returning the completed http.Handler.

The Real HTTP Server

The last piece is attaching all of our work to a real HTTP server and binding it to a port with some additional options. This is where http.Server gets used in main.go.

func main() { var logger logger.Logger = logger.NewNoOp() { switch config.logger { case "slog": logger = logger.NewSlog() case "zap": logger = logger.NewZap() } } var ds datastore.DataStore = datastore.NewSQLite() { switch config.datastore { case "postgresql": ds = datastore.NewPostgreSQL() case "in-memory": ds = datastore.NewInMemory() } } var bs bs.BlobStore = bs.NewLocalFileSystem() { switch config.bs { case "s3": bs = bs.NewS3() case "scp": bs = bs.NewSCP() } } var as as.AuthStore = as.NewSQLite() { switch config.as { case "redis": as = as.NewRedis() case "in-memory": as = as.NewInMemory() case "bypass": as = as.NewByPass() } } handler := handlers.NewServer(logger, ds, bs, as) srv := http.Server{ Addr: ":8080" Handler: handler, ReadTimeout: 5*time.Seconds, WriteTimeout: 20*time.Seconds, IdleTimeout: time.Minute, } idleConnsClosed := make(chan struct{}) go func() { sigint := make(chan os.Signal, 1) signal.Notify(sigint, os.Interrupt) <-sigint if err := srv.Shutdown(context.Background()); err != nil { logger.Log(logger.ERROR, "server shutdown", "error", err) } close(idleConnsClosed) }() if err := srv.ListenAndServe(); err != http.ErrServerClosed { logger.Log(logger.ERROR, "server stopped listening", "error", err) } <-idleConnsClosed }

Based on some configurations we set up each of the Logger, DataStore, BlobStore, and AuthStore. Since NewServer() only cares about interfaces we can compose many different permutations on dependencies our API can use and decouples the HTTP request / response lifecycle from the core business logic in each of the inferface implementations.

Each layer of this API can also be tested thouroughly as independent units or as one service.

func TestServer(t *testing.T) { tests := []struct { name string req *http.Request resCode int }{ { name: "sample request", req: httptest.NewRequest(http.MethodGet, "/articles", nil), resCode: http.StatusOk, }, // Many more requests to test } for _, test := range tests { t.Run(test.name, func(t *testing.T) { t.Parellel() logger := logger.NewNoOp() ds := logger.NewInMemory() bs := logger.NewLocalFileSystem() as := logger.NewBypass() handler := handlers.NewServer(logger, ds, bs, as) w := httptest.NewRecorder() handler.ServeHTTP(w, test.req) if w.StatusCode != test.resCode { t.Fail("failed to get expected status code") } }) } }

We set up our test harness to test the entire server with no logging, in-memory data store, a local filesystem blob store, and we bypass auth all together. We also run each test within its own t.Run() so we can then call t.Parallel() indicating all subtests should run concurrently. This will drastically speed up our tests. Take notice that we set up everything inside each subtests so we do not share resources among them. By doing this we ensure test isolation and each subtests gets its own instance. This can be expanded to set up different states for each subtests as well if some requests require existing data in a database or file on disk too. Since this is all composable the options are pretty endless.

server.Shutdown(ctx)

This concludes this post on how I write my HTTP servers. They are not always the same every time, but these are the goals I try to strive for. You can even use these ideas when refactoring, adding features, or bug fixing. If you focus on trying to always return http.Handler and pass in your dependencies to those handlers your life and you're team's lives will be much easier.

Feedback is always welcome so feel free to send me a message or even a patch to my public inbox!