Getting rid of Monoliths

I am working for Jumbo Supermarkten, which is a large grocery chain in the Netherlands. This family owned business started to venture into e-commerce at about 10 years ago with a very small team of developers. Currently, we've grown into an IT department that consists of over 450 developers working on all digital solutions. We've made some changes over time on how we manage our software. I gave a talk together with my colleague on this topic, so let's share the write up here.

So, if we rewind the clock the the founding of the Jumbo Tech Campus (JTC), where in the beginning the main purpose was to get started with e-commerce. What happened was that the team at the time did the thing that is most obvious with limited resources and a generic goal: they grabbed an off the shelf solution and started to implement it. This meant connecting to the existing brick and mortar stores, the delivery and storage APIs and whatnot.

As you can imagine, this ended up in a more custom implementation that realised at first. This meant that, over time, the application grew in complexity, while the department grew as well. Having multiple teams working in the same code base, often with the same goals: you can imagine that it causes friction.

We decided that this situation has reached the limit if scalability and doesn't suit our needs.

So we came up with a plan

During the late monolithic era (as we call it) we already started to offload some of the responsibility of the monolith to an isolated environment, by using a front end library (VueJS) and executing certain actions on the client side. This resulted in a component library which we could apply to the templates that the e-commerce engine was happily serving.

Also, we started to introduce a separate Content Management System (CMS) to help content editors take control of their domain (content) without the need for developers to do deployments for a content change.

What we value at the JTC is taking ownership of your own domain and you can only take full ownership when you can own the code. So. We needed to split up the code some more.

Teams started to collectively try out new technologies and we saw the emergence of micro applications that handled a specific step in the online process. The first micro application was the checkout. You can imagine the importance of a checkout for an e-commerce platform. The checkout was a NuxJS application being served from a routing service in front of the whole e-commerce engine. And it worked like a charm! 🙌

In fact, it worked so well, that more teams started to follow suit. Each and every team chipping off small blocks of the monolith to adopt for themselves. That was exactly what we wanted, because now the teams weren't working in the same codebase anymore. The were not dependant on other teams' release cycles. We reduced the friction!

The flipside

We did start to see other detractions though. Something we highly value is the omni channel experience, which to us basically means that as a customer you have one entity to deal with. It's not a customer problem that the entity behind the curtain is actually multiple micro applications. Also, we like to do A/B tests. In order for an A/B test to be validated, you want to be able to isolate the test in a stable user interface.

With our component library, we have the tool to recreate applications with identical components. But with all teams contributing to the component library, versions could bump several times over the course of a day. That's detrimental of both a omni channel experience as well as your A/B tests. A new problem arises!

The solution

So again, what we did, is come up with a solution to address the challenges we're facing. We realised that while we do value the individuality of teams, we need to have a point where all of the code converges to a single source of truth. Enter the Frontend Platform.

The platform allows a certain level of freedom for teams, and enables the retention of ownership of a teams' domain, while facilitating the part where the code gets served to the customer.

Frontend Platform

The system we've designed consists of four sub systems which all work together. Let's take a look:

Content Management System

We still need a place to store our content. And since we already have a CMS our content editors are happy with, we investigated what we needed to do to incorporate it. Turns out: not so much. We did need to improve the headless capabilities somewhat, but basically we are running this out of the box. We did write an adapter for the output of the content, but we needed to do this anyway (we'll get back to that later).

Component Library

Again, something we already had. Our component library (named: Kompas) is built in VueJS, is battle tested, has full coverage, quality assurance and visual regression tests. Good to go! We did add something, not in code, but as a convention among developers. We agreed that every component should land in the component library. We don't care if it's big, small, smart or dumb, it needs to be stored here. The reason is: we're using the component library as the tool box with which our content editors can construct pages with. They need to have access to all of the possible components in order to create simple pages as well as entire application flows.

Content Service

This is the heart and brain of the platform. It connects Kompas to the CMS, exposing all of the components. It also ingests the output of the CMS via the adapter to send it to the next system. Under the hood it also validated that the content adheres to the models we've drafted and designed.

Render service

In order to present all the UI goodness to the customer, we need a means of serving pages. An application that can use the output from the Content service and makes sense of it, while server side rendering the page that gets requested on this service. Currently, since we're using VueJS it makes sense for us to deploy a single NuxtJS instance as the render service. Based on the route, it queries the Content service to receive a structured representation of the components it should render back to the user.

That's it!

What we've done here, is to offload responsibilities to specific systems. It's now fairly easy to swap out a CMS with the next one, just by making sure we have an adapter that can transform the output to something the Content Service can make sense of.

We even have parts of the monolith running, since it still has its value within the landscape. That's okay. We can get rid of it when we want, because we have an architecture that facilitates breaking up complex problems into manageable parts.

With this transformation, we have developers that are not bogged down with tedious dependency management: they're contributing to the component library to work on the features that are required. Any component that is added, will automatically be provisioned to the CMS so that our content editors can spin up pages at a rapid pace.