Speed Up Docker Compose with Parallel Builds

Date Published: 07 April 2021

Speed Up Docker Compose with Parallel Builds

I've been using docker-compose quite a bit lately for a distributed app that includes 3 front end apps, 2 databases, RabbitMQ, and PaperCut (test email server). For the most part, this works great as a way to encapsulate all of these processes and run them in a containerized manner so that everything works together. But building this whole solution is pretty slow, by default.

When you work with docker-compose, you generally have two commands you run: build and up. Build is used to build all of the containers using their individual DOCKERFILEs (or just to download the image if it's a prebuilt and published image). Up is used to launch everything and run the app(s).

To measure how long it takes to run docker-compose build, you can use the Measure-Command PowerShell commandlet which I described in my previous article. Here's the command I used to generate the images below:

Measure-Command { docker-compose build | Out-Default}

First run:

docker build time 1

Change something in a file (to force rebuild) and run again:

docker build time 1

You can see the times are both just under 4 minutes: 3:43 and 3:56.

Parallel Docker Build

Now let's try running them again using the parallel switch on the docker-compose build command, making it docker-compose build --parallel.

First run in parallel:

docker build time 3

Change something in a file (to force rebuild) and run again (in parallel):

docker build time 4

Looking at the times, now we get 2:11 and 2:05.

Looking at total seconds, the average build without parallel took 230 seconds. The average build using parallel took 128 seconds. That's a 44% drop in overall time for the build, which is pretty significant, especially when you're running this frequently while trying to troubleshoot a problem.

Of course, this won't help for every docker-compose or on every machine. But if you have a large number of containers being built and a bunch of CPU cores, it's definitely worth a try.

Summary

Most computers these days, and especially most developer workstations, have a lot of CPU cores that can perform work in parallel. The problem is, a lot of software doesn't work in parallel by default. Fortunately, docker-compose can be made to take advantage of those extra CPU cores by adding the --parallel switch, which can yield pretty dramatic improvements in overall build time.

Steve Smith

About Ardalis

Software Architect

Steve is an experienced software architect and trainer, focusing on code quality and Domain-Driven Design with .NET.


Ardalis

Copyright © 2021