30 November 2021

8 min read

Managing dependencies

Engineering

Nowadays, most codebases (not to say every) depend on third-party code to avoid reinventing the wheel for every project.

Dependencies exist in production-ready code as well as in tools that are used to prepare the final application. But let’s be honest, most dependencies are of the second kind, and managing nested dependencies (meaning the dependencies of a dependency) can quickly become a nightmare.

Node Package Manager (npm)

Most modern programming languages have dependency managers. The latter has strongly facilitated sharing pieces of code, created for specific purposes, and has enabled other developers to integrate them in their own applications. To illustrate, NPM is the world’s largest software registry and is the one we also are currently using at PayFit.

An NPM package (which could itself rely on other packages) has an average dependency tree depth of 4.39.This means that when an NPM package is downloaded, an average count of 4.39 other packages are also downloaded to make it work. It is also interesting to note that the average dependency tree size of a project is 86.55 NPM packages (sources here). And, as this article suggests, dependencies grow fast and sometimes unexpectedly.

The following image captures this spirit nicely:

Moreover, since the beginning of the npm registry, there have been more than 1.7 million packages published (source, when logged in). This stresses the importance of the number of packages published.

Choosing the right dependency

Finding your package candidates can be done in several ways: through the use of the search engines on the internet or simply by searching directly on the NPM website (or by using a third-party npm browsing website).

NPM and GitHub expose statistics about packages which provide indications and ultimately help you choose the dependency that fits your needs. See for instance the npm page for the Lodash library.

It’s important to follow an adequate set of criteria when choosing a dependency, because by including it in your project, you become dependent on it, for a long time (if not forever).
You want to refrain from choosing pieces of code which are not supported, not updated, overly dependent from other pieces of code you don’t know about -- or worse, code that could be replaced by malevolent code, either through hacking or through an evil company buying the whole repository.

Trying out young/new/unpopular dependencies is okay for one-shot small projects, but as soon as you are in a professional setting, you need to really know what you’re doing in terms of dependencies, not just blindly trust whatever the internet has to offer.

For instance, you can rely on the following information such as:

The number of downloads
The packages it depends on
The last time a version has been published
…

Which criteria you base your dependency choice on is entirely up to you. Even within PayFit, we do not have strict, company-wide rules for choosing dependencies (yet!), as projects vary wildly in size and scope.

Let’s explore some of the most telling stats given by the npm website.

Popularity

The popularity indicates how many times a package has been downloaded over time, and therefore how many other projects depend on it. It is a strong indicator as it informs you about how many people rely on it.

GitHub gives you the number of repositories and packages which depend on a potential dependency (here, Lodash).

Contributors

It also is a good thing to pay attention to the number of contributors on a project. The more contributors a project has, the higher the code quality and organization is supposed to be.

Maintenance

Frequent updates to packages usually mean that the dependency has fewer bugs and is more stable. It is an indicator for solved issues and released new features. Therefore, a package with many releases indicates that it is active and still maintained obviously.

It works both ways, though: a package that is very much downloaded but has few changes can mean that it is very reliable and bug-free -- even though it doesn evolve much. For bigger projects, you might want to focus on truly stable dependencies.

Size

The size of a package depends on the size of its sources and assets, but also on the size of its dependencies. As a consequence, it can directly affect your application and its performances.

A good way to avoid relying on a dependency that will bloat your project is to check the size of a package. NPM provides some basic information about it, but I would recommend using a tool like BundlePhobia.

TypeScript support

TypeScript is (becoming) our first-class language at PayFit. In order to maximise the type safety of our code, our external dependencies should also be type-safe. There are different scenarios regarding types in libraries, ordered by preference:

Library is written in TypeScript.
Library is not written in TypeScript, but maintainers publish type definitions.
Library is not written in TypeScript, but a @types/... package is available (usually not published and maintained by library maintainers).
Library is not written in TypeScript, we need to add type definitions in our codebases.

Quality

And last but not least, you can check the presence of up-to-date dependencies, documentation, stability, tests, the existence of a dedicated website, of a strict Pull Request policy, and so on.

Updating a dependency

What for?

As we just saw in the Quality section above, having up-to-date dependencies is considered a good practice, but you may ask “why?”.
Let’s take a basic example. Often, you create your project and you use the most recent dependencies at this moment. You will add new functionalities to this project later on but you will mostly forget to update the versions of your dependencies. This usually happens for one simple reason: you just don’t know that a new version of a dependency was released

And that’s when you may find yourself in some difficult positions.

If you need a new version

Let’s say you need a new feature that one of your dependencies now has. Then you need to update this dependency. But since you are several major versions late on this dependency, updating it is not only difficult, it is time-consuming (e.g. because of the risk of breaking changes). Therefore you won’t feel very confident about it, and you will get anxious about delivering it in production.

If a critical vulnerability was found

Now let’s say that a critical vulnerability was found in one of your dependencies and you need to update to a newer version quickly: you find yourself in the same situation as above.

In all cases you will need to update to a new version of the dependency, but as you did not maintain it regularly, the update is painful, time-consuming, and risky.

How to do it in a safe way?

Be reassured because nowadays, tools have been developed to help you in the process of updating your dependencies.

NPM

NPM comes with several tools to keep your dependencies up to date, starting with the update command, available through the CLI. This command will update every package your project depends on.

Let’s say your application depends on packages called dep1 and dep2. The published versions of dep1 are:

{
  "dist-tags": { "latest": "1.1.0" },
  "versions": [
    "1.1.0",
    "1.0.0"
  ]
}

And the ones of dep2 are:

{
  "dist-tags": { "latest": "1.1.0" },
  "versions": [
    “1.1.0”,
    "1.0.1",
    "1.0.0"
  ]
}

Suppose your application has a package.json that looks like this:

{
  "name": "my-app",
  "dependencies": {
    "dep1": "^1.0.0",
    "dep2": "~1.0.0"
  }
}

Then after running the npm update command, you will end up with:

{
  "name": "my-app",
  "dependencies": {
    "dep1": "^1.1.0",
    "dep2": "~1.0.1"
  }
}

NPM will update all your dependencies and will respect the caret (meaning that you only care about the major version being fixed) and the tilde (meaning that you only care about the major and the minor version being fixed).

npm audit

NPM also comes with a command named npm audit. This command will analyse your entire dependency tree and get you a report of known vulnerabilities in your project. If any vulnerability is found, then the impact and a suggested mitigation will be displayed like this:

Sometimes the command --fix allows you to fix them, but sometimes you may need to do it manually.

Dependency update automatization

Even if NPM provides some tools to update your dependencies, these are mainly manual tools that you will need to run from time to time to ensure the quality of your application.

But, lucky for us, there also exists tools that improve automation in this area.
GitHub provides dependabot to scan dependencies and provide alerts if any of the dependencies have known security issues. So it is basically the automation of the npm audit part.

I also recommend the use of a tool like Renovate: It scans your dependencies files and creates Pull Requests for you to update them. If you configured any continuous integration (CI) tool to run some tests and build on your code in order to have confidence on the updates, you can even let these tools merge the PRs automatically for you as well. Your project fixes itself while you sleep!

Thanks to these two tools, we at PayFit gained a lot in productivity and confidence regarding dependency updates.

Should you pin your dependencies?

The Renovate team did a great job explaining dependency pinning with their article here https://docs.renovatebot.com/dependency-pinning/
I will try to sum up their ideas. You can always read the article if you need more context.

Librairies

When developing a npm library you should always use SemVer ranges for the dependencies part but use pinned dependencies for the devDependencies part.
This is to avoid duplicating dependencies when your library will be installed in the project of someone else.

Apps (web or node)

When developing a web or node application, you should always pin all of your dependencies. Combined with the package-lock.json file, this will ensure the consistency of your dependencies.

These to me are the two main key learnings from that article from RenovateBot, but I do encourage you to read it all.

Conclusion

Regularly updating dependencies of projects helps in reducing bugs and vulnerability issues. It also helps in bringing value to these projects (faster build, faster application, improved maintainability, etc.) and in integrating new up-to-date features.

Even if we still have to pay attention to which packages or versions we choose as dependencies, there are now solutions to ease/automatize their management afterward.

Header photo by Erol Ahmed on Unsplash.