Container Tidbits: When Should I Break My Application into Multiple Containers?

There is a lot of confusion around which pieces of your application you should break into multiple containers and why. I recently responded to this thread on the Docker user mailing list which led me to writing today's post. In this post I plan to examine an imaginary Java application that historically ran on a single Tomcat server and to explain why I would break it apart into separate containers. In an attempt to make things interesting - I will also aim to

justify this action (i.e. breaking the application into separate containers) with data and (engineering) logic... as opposed to simply stating that "there is a principle" and that one must adhere to it all of the time.

Let's take an example Java application made up of the following two components:

1. A front-end application built on the Struts Web Framework
2. A back-end REST API server built on Java EE

As mentioned, this application historically ran in a single Tomcat server and the two components were communicating over a REST-based API... so the question becomes:

Should I break this application into multiple containers?

Yes. I believe this application should be decomposed into two different Docker containers... but only after careful consideration.

Instead of breaking applications up into multiple containers "just because" or in an attempt to adhere to some newfangled principle (e.g. "run only one process per container") - I suggest we think through the engineering requirements to make an informed and intelligent decision. Whether or not all applications should be broken into multiple containers - containerizing them should, at the very least, make your life easier by providing you with a simpler deployment strategy.

Let's pause (briefly) on the analysis of our example application and do some design thinking:

1. The JVM is multi-threaded, so you are not necessarily running multiple Unix/Linux processes. In fact, I think this is rightfully confusing to engineers coming from the Java world. Historically, Java developers actually liked having multiple applications running in the same JVM; doing so can, in practice, save quite a bit of memory at scale. Furthermore, web application servers like Tomcat were built from the ground up to support running multiple applications in a single JVM. That's actually one of the main differences between running a simple Java program versus a Java EE application (...which could be comprised of multiple threads and multiple different programs).
2. In reality, many applications use multiple processes per container. The Apache Web Server prefork and MPM modules both use multiple processes within a container. Modern web applications featuring event driven programming or the reactor pattern (take, for example, Nginx) actually fire off many sub processes. I would argue, the whole idea of FastCGI, AIO, and the reactor pattern is to offload work to other processes (or threads) and let the kernel handle the I/O. The Linux kernel is quite good at scheduling sub processes. Kubernetes/Swarm and individual Docker containers are not as good at this. Processes (and threads) are about kernel resource allocation; containers are about cluster resource allocation.
3. The run one process per container "best practice" is widely cited as a "principle" but sounds a lot (more) like philosophy. As an engineer, I want to understand the technical components and make logical decisions. I would argue that this "best practice" is not even universally agreed upon and its over-application stems from a wide-spread lack of understanding with respect to how Unix works.
4. Linux Containers have historically come in many forms and many actually recommend running multiple processes per container. What makes Docker containers any different? Linux containers are essentially the clone() system call, SELinux, and Cgroups; whether they are of a LXC or Docker (through libcontainer) type is mostly irrelevant as the Linux kernel itself "does" the process isolation.
5. There are valid times when processes communicate over sockets, through files, over the network, etc. - and - each approach has its own pros and cons. A given application's approach to communication will (most certainly) have an impact on whether or not you want to break your application into multiple containers.
6.  Separation of code, configuration and data will also play into your ability to break your application into multiple containers (or not). If your application has good separation of code, configuration and data, it should be easy to decompose the application. If your application is very old and not well understood - it may make (unwanted) changes to your file system and could be very difficult to decompose. Note that this is OK (!), you don't have to re-write your application to containerize it. You can still get the benefits of the Docker container format by putting your application into a single container. You will then be able to easily move it around (using a registry server) and deploy it (using docker run).

OK, with some Unix / application 101 out of the way, let's get back to analyzing our example Java application:

1. The two Java components as described above seem to be doing very different things. One component is a web front-end, the other is an API server. Since these components are doing different things (i.e they are indeed different services), there is little chance that there would be a performance benefit from being in the same JVM (...though, of course, I can't be 100% certain without actually testing performance).
2. These two applications communicate using a REST API (...instead of using sockets, shared memory or files, etc.).
3. Generally, if an application contains an API layer and a front-end layer, it can be useful to scale these independently. For example, if the API is also consumed by a mobile application it might be useful to scale it up and down with user load, where the web front-end may not need to be scaled. Conversely, if I scale the web front-end, I may also need to scale the API server portion, but I may only need one more API server for every five web front-ends. Long story short, scaling logic can be complicated and being able to scale these independently with something like Kubernetes could be very useful.

Based on these three observations alone - I would recommend breaking these two components into separate containers. I would also recommend using container orchestration such as Kubernetes or OpenShift to wire the services together. I would not base this decision on "principles" or supposed "best practices", I would base it on the application architecture and some form of informed reasoning.

Here's where things get wild... I submit (to you) a new "best practice", drum roll please:

...yes, if your application / service has good separation of code, configuration, and data, installs cleanly (as installer scripts can make this whole process difficult), and features a clean communication paradigm - it does make sense to break the application up / allocate one service per container.

Fundamentally, I suggest that we all begin to think more rationally about how to put applications into containers. Let's also realize that containerization is more than just philosophy, it's about solving technical pain(s). I love containers and I want to use them (all of the time) - but I also want to do so in an intelligent and informed manner. I'm positive that people have opinions and ideas about containerizing applications - I encourage you to share your thoughts in the comments section below.