When Digital Products Get Big: Monoliths, Megaliths and Microservices
Large digital services are entirely normal in the modern world. There’s lots of them around and we use them every day, and because we use them every day they need to be resilient, to withstand the scale of demand they experience.
I’ve written previously about the importance of testing the resilience of your large digital service, so you can sleep well at night. In this post, I’m going to discuss an approach to designing your digital system to create the necessary resilience and aid your restful sleep.
Pushing a big rock uphill
There is a classic pattern when building a successful digital service. Initially it’s a relatively tightly focussed tool, designed to fit a specific and well-defined need or interconnected set of needs. When it’s released, it’s great, gets widely used and everyone’s happy. Part of its success is its tight focus. But success breeds opportunity, so new tools, features and functionality get added. In the classic pattern, these additions extend the same codebase and the application grows.
Each addition to the codebase, each new feature, adds to the complexity of the system and increases its fragility. A simple addition — to patch a security issue, for example — can cause a major headache for fear of cascading knock-on effects and trickle-down impacts. Making upgrades to a part of the system just amplifies the problem. There are delays adding them and pushing them out to a production environment where, especially in the need of security patches, they’re much needed.
Success breeds opportunity, so new tools, features and functionality get added
Large systems on singular codebases like this are often called monoliths; extremely large ones are megaliths. I’ve worked on quite a few, some of which felt like Sisyphean projects.
The clear fragility of monolithic and megalithic digital systems in an age of increasing reliance on them has prompted a re-evaluation of the way these systems are designed and the way they work. The most common approach is to break things into smaller parts, to break up the monoliths.
Goodbye mega; hello micro
Instead of trying to do lots of things in one codebase, as with mono- and megalithic digital systems, approaches of recent years have tried instead to break out discrete functions into distinct codebases, to put things that can run complete and self-contained into a complete and self-contained environment. The more you are able to do this the more resilient your system will likely become (with important caveats that I’ll discuss below). This known as a microservices approach.
The rationale behind breaking up a mono- or megalithic system into microservices might be well served by a simple analogy (hat tip to Donald Merand).
Monolithic applications are like having one best friend. If that friend is busy or gets poorly you have no one with whom you can do fun things. Instead, by having a wide group of friends, if your cinema pal is unavailable or unwell then you can go for a bike ride with your cycling buddy instead.
However, these are negative reasons, as it were, to turn to microservices (i.e. they’re not monoliths). Beyond overcoming the problems of monolithic services, what are the advantages of microservices?
Learning from bees: the microservices honeycomb
The idea with adopting a microservices architecture is to emulate bees building the honeycomb of their hive.
The honeycomb is built from individual cells. Each cell is relatively mundane, independent and unitary. In a technical vocabulary we call this decoupled. It is does not have to be built of the same material of the cells around it, and the internals of the cell is abstracted, not visible to the outside world.
But the cells are integrated: they connect together to form a large structure that as a whole is strong, robust and resilient.
Damage to one cell does not necessarily affect other cells, and a cell can be rebuilt without affecting other cells or the honeycomb as a whole.
So, in less analogous terms, why should you consider adopting a microservice architecture?
Why should I use microservices?
There are a number of key benefits to adopting a microservices architecture, but before setting them out I’ll outline two specific use-cases, one when building from scratch and one when you have an existing system.
In product development
Building with a microservices architecture separates out features into distinct elements, each doing one specific task.
When you are developing your product this gives you a huge advantage where things can be worked on in relative isolation. The user interface can be developed without waiting for authentication to be built, or user identity, or order management, or payment processing, or product listings, or product recommendations, or surveys, or comments, or stock management or … whatever. Each can be developed without dependency.
Obviously, ultimately your microservices will need to fit into an organised whole in the larger system. How you do that is a large subject worthy of further posts, but for the time being I will indicate some rules-of-thumb below.
In live service
If you have an existing monolithic system in live service microservices give you the several advantages. Ultimately it will increase the resilience of your live service. But also, you can follow a progressive migration.
There is no need or expectation to refactor from fresh and then throw the big switch. Rather, you can progressively add new features as microservices. Or, you can progressively abstract and then move distinctive elements of functionality into self-contained microservices. By way of example, authentication and user management are two obvious self-contained units to abstract initially. The user interface may be abstracted in a fairly straightforward way also, if can be driven by a set of data APIs, for example. After that, you are well on your way to migrating to a microservices architecture.
How microservices can make things better
The greatest benefit of microservices is the independence of each element.
Independent, easy and frequent deployment
If you make an update to your authentication microservice or your recommendation system, you only need to push out the improvement to that microservice, not the whole codebase. You don’t have to worry about issues in your payment processing because no new updates are being rolled out to that system. It also means you can run deployments more frequently that with megalithic systems because you have taken away the worry of downtime across the whole system.
Independent in the event of failure
If your comments system falls over, you don’t have to worry about your order processing microservice. In addition, a failure in the functioning or code of one microservice does not cause a cascading failure in other codebases (though, of course, code should’ve been rigorously tested before ever being released, naturally).
Independent in scalability
If your order management microservice becomes a bottleneck and needs more resources, you only need to increase the capacity of that one system, not the whole thing. Less critical microservices can remain untouched. And when, demand on a pressurised service subsides it can be scaled down independently also.
Independent from any one technology
The way that a particular microservice does what it does is totally independent. As long as the interface with the world outside itself is the same, you can make any internal changes you want. If you want to try out a new programming language or new architecture, you’re completely free to do so.
Independent knowledge
If you bring a new member into your team to work on your payment system, there is no requirement to understand the way the user interface works as well. That new developer only needs to get to grips with the working of the one microservice.
Sounds like microservices are just swell, eh? They’ll fix everything and make you coffee while you put your feet up. Well, there are some gotchas that’ll bite you if you’re not careful.
Swings and roundabouts?
Microservices are not a silver bullet that will solve all your digital service problems.
The approach will bring many benefits, but also some difficulties you will need to understand and handle appropriately. And if you’re not aware of them or don’t take them into account you may well make life harder for yourself.
Developing distributed systems can be complex
A system designed around microservices increases complexity, there’s no doubt about it. Keeping an overview of that complexity can be difficult. If you’re not careful, your system architecture can make that complexity worse.
It’s easy, for example, to end up with a web of interactions between microservices that ends up stupidly hard to understand, track or manage.
Complexity arises when any individual microservice can command or query any other microservice directly. That creates a dependency between those two that must be accounted for whenever there is a change made in either one, meaning any change in either is potentially a breaking change. Bad, bad, not good.
It doesn’t have to be that way, though. Complexity can be kept to a minimum with a good system design.
The centrepiece of a good microservice system architecture is a central conduit that facilitates communication between microservices without them talking directly to each other. This is usually a specific tool called an event stream, but it could be a simple database.
In an architecture like this, microservices consume events from the event stream (also known as ‘event sourcing’) that are triggers to action (e.g. here’s a reply to a comment — save it). Once they’ve taken action, they can then produce events that summarise their actions and/or trigger other actions (e.g. here’s the current view of the comment conversation thread).
Increased effort of operations, deployment and monitoring
If there are more distinct systems, albeit smaller ones, then there’s more moving parts to watch. Each is a distinct unit of deployment. That means more to operate through the release cycle of testing, deploying and monitoring.
Multiple databases can be painful
A basic design principle of microservice architecture is that there is no single database microservice. Data is stored close to the thing that uses or manipulates that data, with in each microservice. Each database is smaller, but there’s more of them to manage.
Testing an application with a microservice architecture can be cumbersome
Each microservice needs a distinct set of tests to ensure it works as expected. That’s important, and necessary, and a basic principle of good development. But designing adequate tests for the system as a whole, or sub-sets of it, is also very important, and may take a fair bit of effort to design and implement.
With an event stream system in place, this is easier than it might be otherwise.
Transaction safety
If a complete transaction from a client or user perspective requires transactions across several microservices, ensuring the transaction is completed safely is more challenging. This is especially so if the transaction in one service needs to be rolled back — how downstream or dependent microservices handle that needs to be co-ordinated as well.
With an event stream system in place, this is easier than it might be otherwise.
Challenges in refactoring
If the architecture is poorly structured, then any updates to one microservice will need to be co-ordinated with dependent services. (This is also a substantial challenge when migrating from a monolithic system to one with a microservice architecture, as elements are moved out into new microservices.)
This problem is taken away, however, by using an event stream. Then, all each microservice is keep its integration with the event stream consistent — the events it consumes and produces — to ensure overall coherence.
Summary
So, whether you are preparing to build a large-scale digital service, or you are currently running one that is getting too big to handle, adopting or moving to a microservices architecture may be a good and healthy approach for your project.
The advantages of microservices are huge, and are adopted by most of the largest digital service providers. For instance, eBay have talked about their shift to microservices and about what their service looks like under the hood. Of course, eBay has an eBay-sized budget to put this decision into action.
While the benefits are substantial, there are some consequences and without good planning, structured design and deliberative implementation those risks may turn and bite you. Beware of them, if you choose the microservices path.
In following blogs I shall look at some more detailed aspects of implementing microservices and the approaches we’ve taken as we implement them for ourselves and our clients. I’ll look at aspects of designing a microservices system, at event sourcing in particular, at what is called ‘command/query responsibility segregation’, and things to do with working with microservices in PHP, our programming language of choice.
Convivio helps organisations to work better for people.
We do this mainly by helping them transform their services using digital tools, but also by spreading new ways of working.
Read our blog: blog.weareconvivio.com
Follow us on twitter: @weareconvivio
Get in touch: hello@weareconvivio.com
Visit our website at weareconvivio.com
