Often, a backend application is a black box, even if we look at its code, we still can’t simply reason about it locally without some assosiated context. It is hard to figure out what is going on at particular layer of the application, even though it can be a mainly stateless application. One of the main problem here is that we can’t reason about an application code, because of too weak method or function type signatures.
Functional programming is great paradigm to bring clarity into any codebase. What would it mean for a backend developer to implement a REST service in functional style? The answer to this is referential transparency1, usage of functions as first-class citizens, and explicit side-effect control.
The Scala ecosystem provides lots of libraries to build backend applications. One popular choice to build REST APIs is Play and Akka-HTTP. However, if we want to build a pure-functional application with full control of side-effects and work with programs like with values, then HTTP4s library is a perfect choice for that. Often, we need to manage some state in some SQL database. Slick library is a good and mature option to work with an SQL database via the JDBC. However, to have a pure-functional persistence layer, one could use the Doobie library instead of Slick.
What we want to achieve:
- Get a pure-functional program/service in Scala.
- Look at transition to Typelevel libraries.
- Bonus goal: have an opportunity to choose an effect type by specifying it at a single place.
Before we start achieving these goals, let’s define what we mean at the goal #1 based on this talk. It is important to get an idea what the pure functional program is.
Everything starts with the referential transparency:
e is referentially transparent if for all programs
p every occurrence of
p can be
replaced with the result of evaluating
e without changing the meaning of
A pure function
- A function
fis pure if
f(x)is RT when
- A pure function does not depend on anything other than its argument.
A pure program
A pure functional program consists of the referentially transparent (RT) expressions.
We start an implementation of a service with Akka + Slick and migrate to HTTP4s + Doobie to get pure-functional program using monads.
We want to register user trips by taxi, bike, car within some distance and price. This is to be implemented by a REST API.
- Add: POST /api/v1/trips, body = JSON
- Update: PUT /api/v1/trips/<id>, body = JSON
- Delete: DELETE /api/v1/trips/<id>
- Select all: GET /api/v1/trips?sort=id&page=1&pageSize=100
- Select one: GET /api/v1/trips/<id>
Our service logic and data layer representation must be 3rd-party libraries-free. That means, our core abstractions should not depend or import some modules from the infrastructure libraries, which we are going to use in our application. Let’s take care of that, before we implement and combine all the layers.
F[_] stands for some generic high-order type, which represents an effect type. This is used for almost
every return type of the service and repository layers. For now, this effect type remains abstract. It will be defined
when we are ready to launch the application or a unit test for this application.
Build with Akka-Http and Slick
Akka and Slick are both asynchronous libraries and they work with the standard Scala
That means, we have to go with Scala
Future when we fix
F[_] at the very beginning of the program execution path.
We could use another effect type, like Cats
IO, but this would require transformation of
IO to Scala
behind the scene, which would change the
IO behaviour from lazy to eager. Basically, this would break the original
idea to get pure-functional service. Why? Because Scala
Future is running immediately upon its construction and it is caching
its computed value. This leads to violation of referential transparency. Here is very good and short explanation
how Future brakes that.
Macwire is a lightweight Scala dependency injection library. Method
wire is a Scala macro, which generates
object creation code at compile time. See Macwire documentation for more details.
AkkaModule combines all the parts together. In the result, we can use
routes to start the Akka-HTTP server.
This is what Routing DSL of Akka-HTTP looks like:
In a similar way, we define the rest endpoints (see source code for the full code example).
Slick is a functional relational mapping library, which works with the SQL Databases over the JDBC. We map each case class to its own Slick Table entity:
Next, we write our Repository implementation. It uses a Slick table instance:
As we can see,
Future is used in implementation of Akka-HTTP routes (implicitly)
and in Slick database actions. That is an actual problem to achieve our goal, since Scala
Future is not
referentially transparent as we concluded above.
Build with HTTP4s and Doobie
HTTP4s and Doobie are both Typelevel projects and very well integrated
with the Cats library. Therefore, our effectful2 code can be conveniently wrapped with the Cats-Effect
This module structure is very similar to previous Akka module. However, using HTTP4s and Doobie we are able to
abstract effect type. Implicit
ContextShift[F] are required by the Doobie
What is an IO Monad?
An IO stands for input and output. An IO Monad is a special monad used for effectful programs. These are programs that eventually produce some side-effects in the external world. Before we run the IO programs, they stay referentially transparent. When we use IO Monad, we basically lift side-effects to the type signature. The type signature shows that the program does interact with the external world from its inside, and thus must be used carefully.
For quick comprehension, a short implementation version of an IO Monad can be:
A chunk of code inside the
body is lazy. Method
run is here to execute the I/O action,
which is captured by the IO constructor.
When to run an IO program?
In functional programming, we tend to avoid side-effects by delaying them. Usually, we launch such a side-effectful program inside the entry point of an application, for example from the main method of a Scala program. IO monad is a wrapper around some code and this captured code is talking to the external world. Some examples of I/O actions can be read/write to console, database, file or network.
Other HTTP endpoints are to be defined in a similar way.
To stress one important point: HTTP4s and Doobie take a type of the effect on the API level. That allows us to keep
abstraction even further by saying that our effect is some
F[_]. So it is very easy to parametrise the HTTP and data
layers of our application when using HTTP4s and Doobie with some concrete effect type like Cats IO, Monix Task,
Scalaz Task, Identity or even Scala Future (like IO -> Future via
Doobie is a pure functional layer on top the JDBC API. This means, we have to write our SQL queries manually, like we would
do with the JDBC API using e.g.
PreparedStatement, etc. There are no ORM3 or FRM4 capabilities, but this is perfectly
fine for most projects. SQL queries can be written using SQL fragments, which helps to reduce code duplication.
There above two examples use the Doobie API. One is done via string interpolation, the other creates SQL query via SQL fragments.
Similar to HTTP routes, we use the implicit
cats.effect.Sync monad to let the Doobie to wrap the result into
The Doobie transactor instance is also parameterised with the target effect type
F. Again, we will define the concrete
type to be used for
F in the main method of our application. Stay tuned.
Composition with IO monad
We are going to use the Cats IO monad when it comes to define our abstract
Our Scala program can be represented using below pseudo-code:
Note that HTTP4s is based on the fs2 library, that is why, at some point, we construct a
instance to serve the HTTP requests. However, not much knowledge regarding the fs2 API is needed to implement HTTP
servers that are similar to the example given here.
The Cats-Effect library provides an
IOApp trait, which leaves
run method to be implemented. Its signature very similar
to the standard Scala program
main method, however we need to return a
cats.effect.IO instance instead of a
The main composition is done inside the
stream method and based on for-comprehension of
There are three IO instances and one block of non-effectful code, which creates an instance of HTTP4s app.
The first IO instance creates all the objects needed. At the moment, it does not do any IO action, but this
may change when it starts to depend on the external world checks, for example checking database connection.
The second IO calls the
init method on the module to initialize the database schema. The third IO creates an
instance of HTTP server and binds it to a particular user port and hostname.
AkkaMain program consist of similar blocks, but relies on Scala
at some point. See Akka-HTTP version
Pure-functional libraries as building blocks help to achieve the functional approach within the entire application. This brings the following benefits:
- As a developer, one can see that a particular part of the program is doing IO action based on the type signature.
- IO actions become composable when the effect type is a monad.
- Referential transparency in the whole application helps us to reason locally and test program pieces in isolation.
- Separation of IO code from the business logic code.
- To represent the program effect type, an abstract generic type is used. This can greatly help when it comes to unit testing or while experimenting with different effect types. We might want to switch from Cats IO to something else. A change of a couple of lines makes this happen. However this benefit comes from generic programming, rather than functional paradigm itself.
- Cats-Effect IO Data Type
- HTTP4s site and documentation
- Doobie microsite
- Deep talk on Purely Functional I/O from Runar Bjarnson
- Source code of the sample project
for an expression to be referentially transparent—in any program, the expression can be replaced by its result without changing the meaning of the program. And we say that a function is pure if calling it with RT arguments is also RT. (Functional Programming in Scala. Paul Chiusano and Runar Bjarnason) ↩
Effectful code is a code which is doing some side–effect to the external world. ↩
Object–Relational Mapping ↩
Functional–Relational Mapping ↩