Greenkeeper and Lockfiles: A match made in heaven

Photo by NeONBRAND on Unsplash

If you’ve used Node.js in the last years, it’s very likely that you’ve used lockfiles. You know, that package-lock.json file that gets auto-generated every time you run npm install? That's a lockfile. Lockfiles are also commonly known as Manifest files, and their purpose is to take a snapshot of your application's dependencies. Think of them as taking a cross-section of your entire dependency tree, and putting it into a machine-readable format. That way, when someone else runs npm install on their machine, they'll get the exact same dependency versions as you do.

It’s important to know that you should never touch your package-lock.json (or yarn.lock) unless you know what you're doing, since it'll likely get overwritten the next time you use npm anyways.

So what do you gain by keeping a lockfile in version control? Well, there are numerous benefits:

• As said before, it makes sure that you and your coworkers/collaborators always install the exact same dependency versions, and honestly, for the time investment? It’s worth it.
• It mirrors your node_modules directory, which means that via version control, you can jump back to previous states of the directory. Time travel!
• The coolest thing: It saves a bunch of time because all of the dependency metadata is already written down. This means npm doesn’t have to read through every directory, but can just directly pull from the lockfile, making installation faster.

Do you even need Lockfiles?

A question to ask yourself before starting a larger project is whether lockfiles are necessary at all for your use. The answer to this is not always clear. Most times, a lockfile is useful for ensuring synchronization of dependency versions, but there are a few common scenarios where there’s less use in keeping a lockfile:

• When publishing a library to npm. npm does not publish package-lock.json under any circumstance. This is specifically what shrinkwrap is made for, so use that instead.
• When targeting older versions of Node/npm. npm 5 is the first version that supported lockfiles, and Node.js 8 was the first version that bundled npm 5. If you want to target Node.js versions below that, consider ditching the lockfile. It’s not incompatible, but potential users might get confused.

Most times, though, keeping a lockfile is recommended, as it provides more stability and less “why is my app crashing? I didn’t even do anything!” moments.

How Greenkeeper uses Lockfiles

An example workflow using greenkeeper-lockfile

You may think that Greenkeeper automatically updates your lockfile when it updates a dependency, without any additional setup. Well, it doesn’t! Let me explain. Since lockfiles are an optional feature of npm, we wanted Greenkeeper to also have lockfiles as an optional feature. Instead of integrating lockfile functionality into the core Greenkeeper software, we decided to open-source a library called greenkeeper-lockfile that does this. It plugs right into your CI workflow and updates your lockfile for every Greenkeeper pull request. That means you don't have to spend any time manually updating it after Greenkeeper updates your versions.

greenkeeper-lockfile has support for multiple CI providers, like:

• Travis CI
• Circle CI
• Jenkins
• Codeship
• TeamCity

And many more!

Next up, let’s investigate how to set up greenkeeper-lockfile for our project.

Setting up greenkeeper-lockfile

Setting it all up is simpler than you might think. All you need is to modify your existing CI configuration and add an environment variable to your CI environment. We’re going to use Travis CI in this example, but it should work similarly with other services.

First off, you need to generate a Personal Access Token. This is essentially a key that allows arbitrary access to your GitHub account. Don’t worry, greenkeeper-lockfile won't use it for anything sketchy. (it is open-source after all) To generate a token, head over to this GitHub settings page. Generate a new token with the public_repo scope:

Creating a new personal access token on GitHub

This will generate a random string of letters and numbers. Do not close this tab, as this is the only time the token will be displayed to you. Open your Travis CI project page, go to settings and add the token as a GH_TOKEN environment variable:

Adding the Personal Access Token on Travis CI

Now you can close the tab with the token in it. That’s step one! Now, let’s configure Travis CI to actually use greenkeeper-lockfile.

Adjust your .travis.yml like so:

before_install:
- '[[ $(node -v) =~ ^v9.*$ ]] || npm install -g npm@latest'
- npm install -g greenkeeper-lockfile
install: npm install
before_script: greenkeeper-lockfile-update
after_script: greenkeeper-lockfile-upload

That’s a lot to take in, so let’s go through it one by one.

before_install:
- '[[ $(node -v) =~ ^v9.*$ ]] || npm install -g npm@latest'
- npm install -g greenkeeper-lockfile

If we ignore the shell magic in the second line, all it does is check whether the Node version came bundled with a version of npm that supports lockfiles. If not, it installs the latest version of npm manually. After that, we install greenkeeper-lockfile globally, since it works through shell commands.

install: npm install

You might not know about npm ci, which is a faster version of npm install made specifically for running in a CI environment. (hence the name) Travis CI recently started using it by default, and that's good! However, npm ci does not update the lockfile when detecting different versions in package.json and package-lock.json, instead it just quits with an error. This isn't what we want, so we manually override it to use npm install.

before_script: greenkeeper-lockfile-update after_script: greenkeeper-lockfile-upload

Before running your tests, greenkeeper-lockfile updates your package-lock.json. After running your tests, it pushes a Git commit to the Greenkeeper branch where the build was initiated.

Don’t worry, greenkeeper-lockfile only runs in situations where it's supposed to. It doesn't run on your normal builds, just on Greenkeeper-initiated ones. What's better, now that you've completed the setup, you don't need to do anything else! Next time Greenkeeper opens a pull request, your lockfile should automatically be updated along with your package.json.

Further Reading

We just shipped support for monorepos! This also works with greenkeeper-lockfile, which will update all of your lockfiles in one go. You don't need to change any of your settings, everything will still work as before. If you manually put a version constraint in the installation (e.g. npm install -g greenkeeeper-lockfile@1), you're going to have to update that to greenkeeper-lockfile@2.

If you have any suggestions to how we can make interactions with lockfiles easier, we’d love for you to share your thoughts! Just open an issue on the greenkeeper-lockfile repository: https://github.com/greenkeeperio/greenkeeper-lockfile/issues/new