Recently, 47 Degrees released a new Scala library called Fetch that allows you to write your data-fetching code in a concise way, where the library takes care of batching requests to the same data source, caching results that have been requested before, and running requests in parallel that do not depend on another. A similar library is Clump, which was initially inspired by Twitter’s closed-source Stitch project.clu
Using free monads, applicatives and so on to achieve this, and shield developers from having to take care of batching, caching and parallelization themselves, is certainly an exciting and challenging feat of engineering, and I have the utmost respect for every single person who has worked on the mentioned libraries.
However, looking at these libraries, I keep asking myself if we’re trying to solve the wrong problems. According to the Clump website,
The microservices architecture introduces many new challenges when dealing with complex systems. One of them is the high number of remote procedure calls and the cost associated to them.
Maybe, when we are facing such problems, we should not immediately rush to an engineering solution. Maybe, we would be better off if we took the time to lie back and ask ourselves a few simple questions: Is this problem I am trying to solve cast in stone? Or is it just the result of a different, underlying problem?
In this instance, I would like to challenge the unquestioned assumption that you have to deal with a high number of remote procedure calls in order to serve a single request to the end user. Maybe, instead of writing fancy code that makes it easier and more efficient to deal with all these service calls, we should re-think our architecture. If tons of requests to various different services are necessary, maybe the responsibilities of our services are not divided in an optimal way.
The next time you are about to jump to a solution for a problem, try to find out if it wouldn’t be more beneficial to tackle the root cause of that problem. More often than not, that root cause is an architectural problem.