Backend Web Framework Design Exploration - Part 1

This is the first in a series of articles I am writing in which I do a Design Exploration of a Backend Web Framework with the following constraints.

• only use functions & data - no classes w/ methods
• use strong typing (via TypeScript) - no use of any
• trivially testable via basic unit style state based tests

Note: Within this article I will include simplified code examples. If however you are interested in the real code & tests I used to do this design exploration you can see them via the commits at my be-wf-design-exploration Git repository.

Why the constraints?

It has been my experience that applying constraints like this pushes you to think about problem solving in different ways. My hope is that by forcing myself to think within these constraints that I will have some valuable take aways in terms of deeper understandings of design principles and best practices.

Why these constraints?

Throughout my career I have used a lot of different backend web frameworks in a lot of different languages. Sadly I can't think of one that actually facilitates end-to-end request -> response testing in a simple unit testing style. Generally you have to do a bunch of horrible magic to facilitate dealing with dependencies (database, etc.) or use a bunch of unique magic provided by the framework.

I believe this magic is unnecessary nonsense and that it is possible to design a framework in such a way that would naturally facilitate simple unit style testing. Hence, the constraint of requiring everything be trivially testable via basic unit style state based tests.

Additionally, I am constantly faced with developers having this strange impulsive fear of functions. These developers generally build classes that have no real state and no need to actually be a class. In fact they seem to simply use the class to implicitly share dependencies.

I think this conciously or unconciously boils down to people thinking that explicit dependency declarations via function paramaters are bad. I believe this is due to people worrying about "threading" dependencies through layers and layers of functions and not understanding how modularity works.

My instinct is that explicit dependency declaration is extremely valuable. Not only does it facilitate testing but it can also act as an indicator of poor design. For example seeing heavy parameter "threading" should tell you something is likely off with the design. Therefore I have included the constraint of only using functions & data to try and prove not only to myself but hopefully to others that using explicit dependency declarations is extremely valuable.

Beyond that, over the years I have moved further and further away from dynamic languages and closer to static strongly typed languages. The amount of aid the the compiler provides these days is amazingly valuable and I believe it outways the limitations. So I thought it would be interesting to see what sort of implications there are from having that requirement in this context.

Guiding Architectural Concept

For the sake of discussion lets say we want a web framework that would help us build a REST API. REST APIs are built on top of the HTTP protocol and the HTTP protocol is a Request-Response protocol. That means that a client makes a request with some data and the server responds with some data.

When I think about this Request-Response concept it feels very similar to the concept of a function.

function add(x: number, y: number): number {
  return x + y;
}

When you call the function add(5, 2) it is equivalent in my eyes to a client making a request to a server with the data { "x": 5, "y": 2 } and getting back a response, in this case the return value of the function.

Given such similarity to a concept that exists naturally in all these languages I am going to think of each request-response pair as a function in the hope that it can help guide my design.

What a Backend Web Framework does?

I am going to start by focusing on the core functionality of the framework and see if I can evolve it from there. At the most basic level I think a Backend Web Framework needs to do the following.

1. Receive a request
2. Identify the request & it's associated handler
3. Execute the matched handler
4. Receive the response from the handler
5. Transport the response back to the client

Design 001

Naively this might look something like the following.

However this design has one major problem. The router doesn't meet single responsibility principle. Infact in this design it is responsible for doing the following.

• mapping a request to a handler
• coordinating execution of the handler & tunneling the response

This problem would prevent the router from being tested in isolation from the execution of the mapped handler which goes against defined constraints.

Design 002

Taking the learnings from the first design. I decided to separate the concerns of the mapping a request to a handler and the coordination of the execution of a handler.

This shift now makes it so the process request & router abide by the single responsibility principle. In turn enabling testing of the router configuration in isolation.

Initial Framework

The following is the initial framework implementation based on Design 002. To see the full implementation at this stage checkout commit 9bd3e3.

export interface Request {
  path: string;
  // ...
}

export interface Response {
  status: number;
  // ...
}

export type RouteHandler = (request: Request) => Response;

export interface Route {
  pattern: string;
  handler: RouteHandler;
}

export type Routes = Array<Route>;

export type Router = (request: Request) => Route;

function routeMatches(request: Request, route: Route): Bool {
  return true;
}

export function router(routes: Routes, notFoundRoute: Route): Router {
  return function (request: Request): Route {
    const matchedRoute = routes.find((route) => routeMatches(request, route));
    if (matchedRoute) {
      return matchedRoute;
    } else {
      return notFoundRoute;
    }
  };
}

export function process(request: Request, router: Router): Response {
  return router(request).handler(request);
}

The above seems pretty reasonable at least at a high level. Probably the most interesting technique I have used thus far is in the implementation of the router function. It is simply a builder function that builds the actual instance of type Router and returns it. It accomplishes this by returning an anonymous function that matches the Router type signature.

That by itself isn't really that interesting. However the fact that the builder function can take in parameters which are then available to the scope of actual router implementation (anonymous function) is extremely valuable. It is what enables us to feed dependencies into a function while having it's signature map to typed signature that knows nothing about those dependencies.

This technique is commonly used in functional programming.

To make sure it supports all the levels of testing I want before adding support for anything else. I whip up a couple tests.

test("routing configuration", () => {
  const request = ...;
  const route = router(routes, notFoundRoute)(request)
  expect(route).toStrictlyEqual({ pattern: "some/path", handler: somePathHandler });
});

test("end-to-end request -> response", () => {
  const request: wf.Request = { path: "some/path" };
  const response = process(request, router(routes, notFoundRoute));
  expect(response.status).toBe(200)
});

The above allows us to test the following things.

• end-to-end test with a request
• test routing configuration in isolation
• test the route handler in isolation

Usage

To get a better feel for what this framework might actually look like to use in this state. I have broken down a rough example.

import * as wf from "./wf";

function fooHandler(request: wf.Request): wf.Response {
	return { status: 200 };
}

function barHandler(request: wf.Request): wf.Response {
	return { status: 200 };
}

function notFoundHandler(request: wf.Request): wf.Response {
	return { status: 404 };
}

const routes = [
    { pattern: "/foo", handler: fooHandler },
    { pattern: "/bar", handler: barHandler }];

const router = wf.router(routes, notFoundRoute);

// Normally this request would come in via an HTTP Server and get translated
// into a wf.Request object and in turn handed to the wf.process() function.
// However, this works as a means of simulating this for the development of
// the framework without needing to build the HTTP Server portion.
const request: wf.Request = { path: "/foo" };
const response = wf.process(request, router);

I threw the above into a single file to facilitate seeing it all at once. In reality this would obviously be broken up into different files.

Review & Next Up

Looking at what I have thus far I am feeling pretty good about the design. It feels simple and clean yet still extremely robust and flexible while meeting all of the outlined constraints.

Next up is going to be interesting as I need to figure out how to ideally facilitate dependency injection within the framework.