Nodejs Dependency Hell & Semver

Here’s a short story about nodeJs/JavaScript dependency hell.

The incident happened between 30/07 and 03/08.

We are about to ship a new feature to our existing app, everything works fine ‘as usual’, we push the code to our development branch then to the staging branch. Ci/CD is fine and our app was ready to be tested. We were on friday, we don’t push to production at the end of week.

The next week, after tests, we were ready to merge the staging code to pre-prod and then to prod but the CI/CD went red displaying the following message.

error yargs@17.0.1: The engine "node" is incompatible with this module. Expected version ">=12". Got "10.24.1"

Ok, it seems that yargs caused the issue.

So first step we will check for yargs then do a quick fix using the latest version of yargs that support node 10.

We have about 12 node js apps that we manage using lerna. We did a dummy search for yargs with hope to find it in some package.json files but oops, no yargs found .

Of course, we need to update the node version but we should check all the impact and side effects before upgrading to a new nodeJs version. But for now, let’s ship the new feature.

Inspection with yarn

Since we are using yarn to handle node packages. Let’s see what we have :

yarn --verbose

First we tried yarn --verbose but we just got an endless list of lines in the terminal with no significant help.

yarn why

Since we weren’t able to identify which node module uses yargs, hopefully the why yarn command can help us do the job.

So the first attempt was yarn why yargs@17.0.1, but nope, no help. Yarn didn’t find anything.

Then we tried without specifying the exact version and yarn why yargs and boom, we have all the list of npm packages that uses yargs

=> Found "@..../package#yargs@17.0.1"
info Reasons this module exists
 - "_project_#@package#package#typeorm" depends on it
 - Hoisted from "_project_#@....#typeorm#yargs"

So typeorm which we use as our ORM depends on yargs@17.0.1.

The issue

We use docker with nodejs 10. We use the caret prefix ^version in our package.js for typeorm and we don’t expect such breaking changes that drops the support of nodejs 10.

Since the 17.0.0, yargs did some breaking changes including dropping node 10 (17.0.0). The typeorm team upgraded to yargs 17.0.1 and released a new version 0.2.36. So, as a result typeorm no longer supports node js 10.

What happened between friday and monday

Friday afternoon, the typeorm team pushed the new version to npm. Monday, the new version (0.2.36) was already in the npm registry. This happened after our test on develop. We reported the issue to the typeorm team to update Nodejs minimum version (which was 4). Hopefully, the team were very kind and they replied (in the same night) by fixing the documentation.

Thoughts

• The changes made by the typeorm team should be in a MAJOR version release because it contains incompatible API changes https://semver.org/.
• Yargs is used for command line tools but the majority of features can work without cmd. Why typeorm should drop the support of older Node Js versions just because the cmd tool doesn’t support them.

Moral of the story

• Make sure to use semver in a consistent way when you provide a public api. This helps your users to better integrate your library.
• CI/CD is a must. There are many advantages of using CI/CD such as making tests easy, consistent deployment, better packaging, better versioning, blue-green deployment, increased velocity …. In our case, it helped us detect the issue using the gitlab notifications.
• Set up your prevention tools so you can be notified when something went wrong before it harms your users or your application.
• Nodejs dependency hell is real. Despite being one of the most used languages in the web, node js packaging managers lack some features. Be careful about dependencies and hidden node_modules.
• Try using some tools such as (update-notifier) https://www.npmjs.com/package/update-notifier to better communicate with your library users.