Exploring domain problems cheaply and fast with Rust: a web router

Domain modeling is the art and science of figuring out how to map some messy, fuzzy, real-life ideas and things, into a computer program that we can execute.

It is usually easier to say than it is to do, so I figured I'd give you an example of how I've done domain modeling in the past and how I like to explore domain problems with the Rust type-system.

Shapes of Things

There are only so many kinds of data we can have in our programs. You have single things, you have collections of the same things. You have collections of different things that also happen to be a thing, and they can either be one of many things that are the same thing, or many things together making a thing!

Where I'm going with this is that the shapes of data that you normally work with come in a few packages.

We have many things that belong together but are distinct on their own. In Rust, we call these enums when we know all of them already.

We have many things that belong together and form a single unit, where every piece has its own place, and the place doesn't have a name. In Rust, we call these tuples. But when these pieces don't have a given place and instead have a name, in Rust we call these structs.

We also have things that share behaviors, and in some way form belong together. In Rust, we call those behaviors traits.

We have things that are collections of other things. Lists, Vectors, HashMaps, Sets, etc.

In fact, we can create most of these different shapes of data with some of the simpler ones. Structs can be made with tuples. Lists can be made with enums. And so on.

Usually, when I don't yet know what the kind of data is, I use an empty struct as a placeholder.

Okay, enough of this. Let's start using these shapes for some specific domain problems.

Modeling a Web Router

We'll start with one that most of you will likely be familiar with: a web router. That is a router that helps a web framework figure out where to send each request. There are many out there, and I'm not pitching you to write your own (but you should, because it's a good way to learn!).

But what really is a web router? It's a collection of patterns and handler functions. Not unlike a match expression really. It matches the pattern against a web request object, and if it succeeds it will execute the function that corresponds to it.

So we can start by defining what we know:

• there are patterns, and
• there are handler functions,
• a route is a pair of a pattern and a handler
• a router has routes

struct Pattern;
struct Handler(dyn Fn(Request) -> Response);
struct Route(Pattern, Handler);
struct Router(Vec<Route>);

Brilliant! Now we have our types in place, we can start exploring how they interact with each other.

A router typically will receive some form of Request and turn it into a Response. After all, we expect to reply to our users with something. We'll call this behavior WebRouter and use a trait to bring it into life in our model:

trait WebRouter {
  fn route(&self, req: Request) -> Response
}

To make a response we need to find a handler. We can find a handler by matching our request against every pattern in our router. When that happens, calling the handler will produce a response. This matching behavior, where one pattern matches one request, we'll call RequestMatch:

trait RequestMatch {
  fn is_match(&self, req: Request) -> bool;
}

Now slowly we are building up the right vocabulary not just around the problem, but also in the code that deals with it.

And that's it. We have our first model for a router. We have a clearer understanding of what the moving pieces are, and how they connect together.

From here we can take it in many directions but what I like to do is to do challenge the model.

Challenging the model

In the process of challenging, we want to grab individual pieces and ask what's important about them, and how are they different than other things, and why are they really needed.

For example, why is a pattern a separate entity and not just a behavior of a handler? A handler could very well inspect and ignore a request and just let the next handler handle it.

This would lead to a slightly different model, where a handler either tells us it has handled or ignored something, or the handler itself calls the next thing.

In the first case, we can model it by making a new type that can be either an ignored handler result or a handled handler result. Since know this has a limited number of options to choose from (2: ignored and handled), we can use an enum:

enum HandlerResult {
  /// This represents a handler that _skipped_ handling this request.
  Ignored,
  
  /// this is the result of a handler actively handling a request.
  Handled(Response)
}

And so our handler type:

struct Handler(dyn Fn(Request) -> HandlerResult);

This leads naturally to some implementations, such as folding over the list of routes, and bailing as soon as we find a route that returned Handled(res). This is super flexible when it comes to letting the route itself decide how or if it will process a request.

In pseudorust:

let request
for route in routes {
    if let Handled(response) = route.handler(request) {
      return response;
    }
}

But in the second case, we can see we have an even more powerful model. In this one, we are making sure every handler receives the next handler, which it can call at any point. This second model looks like this:

// We use a type-alias to make the Handler struct easier to read.
type NextFn = dyn Fn(Request) -> Response;

pub struct Handler(dyn Fn(Request, NextFn) -> Response);

This model leads to an implementation that looks a little more like this:

let request
let route_fn = fold_right(
    starting_with: |_req| { respond_with_404_not_found },
    collection: routes.reversed(), 
    fold: |last, current| {
      |req| => {
         current(req, last)
      } 
    }
}
route_fn(request)

Which is rather complex, as it has to thread together all the routes, passing along the request object, and letting every route receive a function to call the next route if they need to.

This can be much trickier than the last models we saw. So we'll stick to our first model for now.

Refining the model

Once we find a model that we like, and in this case, we'd like to stick with the first one, we can start doing some refinements.

Refining is the process of adding detail to the model, and it helps us see how it can materialize as a working application.

For example, we can take our pattern struct and start looking into what shapes it can actually take. Usually, a domain expert here is the best person to ask: "what really is a pattern?"

In our case, we want to be able to match on the route URL or path; the kind of HTTP method they are using, or verb; and we'd like to know what is in the body.

To do this we expand our pattern struct once to include some data, and in the process we define a new type for the HTTP method, since we knew we needed that and we roughly understand upfront the shape it has: it can be one of some options. The new pattern now looks like this:

pub enum HttpMethod {
    Get,
    Post,
}

pub struct Pattern {
    path: String,
    verb: HttpMethod,
    body: Option<String>,
}

Excellent, but now what exactly happens in the body?

As it is above, it can either be present and be a string or be missing. If it is Some(string) then either an empty string "" and the entire works of Shakespeare in Korean would be valid bodies. Is this really what we mean to say?

Here's where our refining doubles down, and asks if there's anything special about the body in this specific pattern, or in this specific route. So we're relating the current refining learnings with our other explorations.

Turns out the body should actually be something that the handler can in fact handle, so we need somehow to make it fit into what the handler expects.

So does the handler really expect a request? Or does it expect a specific kind of request? Let's see if we can be more specific in a few steps:

1. Let our pattern be more specific about a request payload
2. Make our handler be specific about the request payload
3. Make our router work with our new handler and pattern types.

We will start by making our pattern take type parameter. During modeling a type parameter is usually a good way of handling large groups of data that, even though they look homogeneous from afar, can be subdivided into smaller groups that shouldn't be mixed.

pub struct Pattern<Body> {
  path: String,
  verb: HttpMethod,
  body: Option<Body>
}

But wait, how are we going to create Patterns if we need to know the Body already? Seems like what we really want is a way of checking if the body corresponds to some expected type. Let's use a function instead and let the creator of the pattern figure out how to do this:

pub struct Pattern<Body> {
  path: String,
  verb: HttpMethod,
  body: Box<dyn FnOnce(String) -> Option<Body>>
}

Excellent.

Next up, we have our handler, which now should receive a type parameter for our payload and look a bit more like this:

pub struct Handler<Body>(Box<dyn Fn(Request, Body) -> Response>);

And finally, it seems that our route and router type doesn't need much amending. Because we really just need a list of patterns and handlers, and that's exactly what they are, right?

Right?! 🙈

Oh no. If we follow this current refinement and thread in a Body parameter to route struct, we will end up with a single type of payload in the vector of routes. This is because the vector type can only hold one type of elements, and every Route<T> is essentially a new type!

• Route<()> is a type of routes that have no payloads
• Route<User> is a type of routes that have payloads of a type user
• and every one of these is not mixable with the rest :(

So we can either backtrack, and move this body parsing function inside the handler, to let the handlers figure out how to work with it. Or we can find another way of putting all these handlers together in a vector, while maintaining the model as close to the domain.  

The simplest thing we can reach for is rethinking our Route struct:

pub struct Route(Box<dyn FnOnce(Request) -> Option<Response>>);

So that the information about the type is hidden. But this now means we need to create this new function to stitch together a pattern and a handler, that we are guaranteed will have a matching type.  

impl Route {
    pub fn new<Body: 'static>(pattern: Pattern<Body>, handler: Handler<Body>) -> Self {
        Route(Box::new(move |req| {
            if pattern.is_match(&req) {
                if let Some(body) = (pattern.body)(req.body()) {
                    return Some(handler.0(req, body));
                }
            }
            None
        }))
    }
}

This now lets us collect many routes that will be guaranteed to fit together:

fn main() {
    let router = vec![
    
    	// This route handles POST / where the body should be an integer
        Route::new(
            Pattern {
                path: "/".to_string(),
                verb: HttpMethod::Post,
                body: Box::new(|str| Some(2112)),
            },
            Handler(Box::new(|req, body| {
                Response::ok()
            })),
        ),
        
        // This route handles POST /bool where the body should be a bool
        Route::new(
            Pattern {
                path: "/bool".to_string(),
                verb: HttpMethod::Post,
                body: Box::new(|str| Some(false)),
            },
            Handler(Box::new(|req, body| {
                Response::ok()
            })),
        ),
    ];
}

Now we can go to our last step.

Rewrite the model

What we wrote above works, but doesn't look like idiomatic Rust to me.

In fact, when I was first writing this post I saw that Router::new required a Body: 'static because there's this closure and  lifetimes wouldn't necessarily match up...oh boy. Sometimes a lifetime is a great indicator that the domain is being mapped truthfully. In this case, it was a good indicator that the model was leaking into the domain. Which means the model needs rethinking.

AND THAT IS OKAY.

It's okay because this iteration with types and traits helped me get to a bottleneck so much faster than committing to an implementation before doing any experimentation.

In other words, if I hadn't experimented before, then my whole thing would have been an experiment.

Now that I've learned more about the problem I'm solving, I can actually try to solve it with some chance of success. For example:

1. A Handler and a Pattern are too close together to meaningfully be separate Things
2. I immediately noticed that I can't seem to capture anything in the paths of my routes
3. There are established ways of parsing external data into typed data safely, such as expecting types that implement serde::de::DeserializeOwned
4. Some of our traits weren't really reused, so they could be part of specific type implementations

Solving for these, here's another iteration:

pub enum PathPattern {
    Prefix(String),
    Variable { name: String, value: String },
}

impl PathPattern {
    pub fn new(str: &str) -> Self {
        Self::Prefix(str.to_string())
    }
}

pub type HandlerFn<Input> = Box<dyn Fn(&Request, Input) -> Response>;
pub struct StaticHandler<Input: serde::de::DeserializeOwned> {
    verb: HttpMethod,
    path: PathPattern,
    handler: HandlerFn<Input>,
}

impl<Input: serde::de::DeserializeOwned> StaticHandler<Input> {
    pub fn new(verb: HttpMethod, path: PathPattern, handler: HandlerFn<Input>) -> Self {
        Self {
            verb,
            path,
            handler,
        }
    }
}

trait Handler {
    fn matches(&self, req: &Request) -> bool;
    fn handle_request(self, req: Request) -> Response;
}

impl<Input: serde::de::DeserializeOwned> Handler for StaticHandler<Input> {
    fn matches(&self, _req: &Request) -> bool {
        todo!()
    }

    fn handle_request(self, req: Request) -> Response {
        (self.handler)(&req, serde_json::from_str(&req.body()).unwrap())
    }
}

#[derive(serde::Deserialize)]
struct CreateUserReq {
    name: String,
}

pub struct Router(Vec<Box<dyn Handler>>);

fn main() {
    let _router = Router(vec![
        Box::new(StaticHandler::new(
            HttpMethod::Post,
            PathPattern::new("/user"),
            Box::new(move |_req, _body: CreateUserReq| Response::ok()),
        )),
        Box::new(StaticHandler::new(
            HttpMethod::Post,
            PathPattern::new("/bool"),
            Box::new(move |_req, _body: bool| Response::ok()),
        )),
    ]);
}

Notice we are using a new trait for a Handler, that helps us abstract over all of the handlers (so we hide information about the specific types of requests they support). This lets us put them all together within a single vector in our router.

The nice thing is that the final router implementation for this can be done with the same easy algorithm we used before:

impl Router {
    pub fn route(&self, req: Request) -> Response {
        for handler in &self.0 {
             if handler.matches(&req) {
                return (*handler).handle_request(req)
             }
        }
        Response::not_found()
    }
}

Conclusions from Modeling a Router

Like this, we've quickly gone through several iterations of our model, tried to understand better what problem we are trying to solve, what are some of the constraints it has, and how our model leads to different implementations.

We've even rewritten our last model once we understood that we could represent some concepts better.

It is very important to understand that this first implementation is meant to be correct, and not necessarily optimal. But it can make a great first implementation to test things against, and eventually, help you optimize making sure you are not breaking good behavior!

I'd love to go on with some more examples, like:

1. modeling regulatory compliance for betting companies
2. modeling the publishability window of content in the music industry following geographic restrictions
3. modeling the optimal publishing of photography content to a social network
4. modeling an offline-first graph database for the edge
5. and more!

But we're already over 2,500 words and I'd like you to get a glass of water and maybe stretch your legs. So let me know which modeling example you'd like to see in a next post.

If you liked this, please subscribe so you get the next issue of Practical Rust right in your inbox, and share it with your rustacean friends on lobste.rs, hackernews, x.com, and so on.

And I would love to hear if this is the kind of content you're expecting from a Practical Rust blog, so let me know! I'm on x.com: @leostera

Happy Rusting! 🦀