Speed up bundle install with this one trick

Did you know bundler can download and install gems in parallel?

That's right, using the --jobs option for bundle install means you can set the number of gems that can download and install at the same time.

As a bonus, you don't even have to remember to use the --jobs option every time. You can set this in your global bundler config with this command:

$ bundle config --global jobs 4

Does it really help?

I found all of this out recently, even though this has been available since bundler version 1.4.0, and I wanted to test how useful it actually was. Downloading and installing gems in parallel sounds like a delight, but I needed to test it with something to be sure.

I went looking for a large Rails project to try this out on. I happened upon the GitLab Community Edition app and cloned the project. I didn't need to actually get the project running, I was just testing the installation time of the gems.

I was looking for a large application. rake stats told me that it contains 172 controllers, 208 models and 86,019 lines of code. In the Gemfile there are 197 gem dependencies which shakes out to 369 gems in total. I checked out Discourse and Diaspora too, but GitLab certainly had the largest number of gems, so it was perfect to test my theory.

I ran time bundle install --path=./gems --quiet --force --jobs=n 5 times each for n equal to 1 and 4. The median results for each were:

time bundle install --path=./gems --quiet --force --jobs=1
real  4m39.836s
user  1m59.692s
sys 0m50.291s

time bundle install --path=./gems --quiet --force --jobs=4
real  2m55.857s
user  2m0.005s
sys 0m47.897s

These tests were run on a MacBook Pro with 2.5 GHz Intel Core i7 and 16GB RAM.

With these results we can see that installing the gems for GitLab in parallel using 4 workers was ~1.6 times faster.

You should run your own tests with your own setup to see if this genuinely will save you time with installing gems. There has been some research into the best number of jobs. I've certainly set the default number of jobs to 4 for the future.

Why isn't this default?

You might be wondering why Bundler doesn't just turn on parallel installation by default. A quick glance at the comments in the source code gives this one away.

# the order that the resolver provides is significant, since
# dependencies might affect the installation of a gem.
# that said, it's a rare situation (other than rake), and parallel
# installation is SO MUCH FASTER. so we let people opt in.

Reading the docs

As a final point, I'd like to point out that reading the documentation, even for a project you may use every day, can turn up some interesting options you didn't know you could use. Bundler itself is full of other useful commands and options, check out bundle pristine, bundle gem and bundle outdated for some ideas of what is available.

At least if you'd not known about the --jobs option for bundle install you can now go out there and install your gems faster.

Speed up bundle install with this one trick was originally published at philna.sh on June 12, 2017.