Speed up your npm install in Docker build on the Jenkins pipeline

Got tired of npm build time? Why don’t you cache it?

Caching is the technique of storing the state of data, at some particular point in time.
Not every time we need some to be processed so we can store that data for further use.

How does caching work in node.js?

While building the npm packages it dynamically installs the dependencies from the package.json file into the “node_modules” directory in the project folder.

node_modules is the local directory where all the installed packages are stored and executed inside the project directory

~/.npm is the root directory where all the cache is stored. While installing the node packages at another location it checks the packages at the ~/.npm folder first.

How to cache in npm?

Now, the question arises what to cache in the npm build “node_modules” or “.npm” folder

While using the “npm ci” or “npm i” for installing the node dependencies
before installation it will delete node_modules and installs again from the package file. So, it is recommended to cache the “.npm” folder instead of“node_modules”.

Caching the .npm folder in Jenkins CI/CD using a declarative pipeline

For caching in Jenkins, we use the jenkins-pipeline-cache-plugin we can store the cache either in a local directory or an S3 bucket.But here I am showing the demo for the Jenkins on the k8s cluster, where for every build it will launch the new pod. So, the local cache won’t be worked here. So, we prefer to store the cache in the S3 bucket.

Installing the plugin

In Jenkins, go to Manage Jenkins > Manage Plugins > Advanced > Upload Plugin or Deploy Plugin then “Enter the URL”

Upload Plugin or Deploy Plugin then “Enter the URL”

github.com/j3t/jenkins-pipeline-cache-plugi..

Restart Jenkins after the installation is complete.

Configuring the plugin

Under “Manage Jenkins”, go to “Configure System”

In the Cache Plugin block,

Enter the Access Key, Secret Key, Bucket name, Region, and Endpoint.

The endpoint will be https://s3-<region_name>.amazonaws.com

NOTE: Make sure you have already created the bucket in your AWS account as it won’t create the new bucket.

Implement Caching in Jenkins Pipeline:

For the jenkins-pipeline-cache-plugin plugin, the syntax for caching block would be

node {
git(url: 'github.com/spring-projects/spring-petclinic', branch: 'main')
cache(path: "~/.npm", key: "$BUILD-${hashFiles('package-lock.json')}" , restoreKeys: ["$BUILD-${hashFiles('package-lock.json')}"]) {
}
}

Here, it simply creates one node and pulls the git repo from the URL. The cache( ) is blocked to implement the caching step.
In the cache block, “path”: is the parameter for the directory path to be cached.
Here “key”: It is an identifier that gets attached to the cache stored.
For validating the cache we use the hashing mechanism for checking the checksum of the file. If any changes are updated in package-lock.json then the hash value changes.

“restoreKeys”: It is an identifier to restore the cache.

First when the cache() block starts executing it looks for the cache with the restoreKeys on the s3 bucket if the cache is found, it restores it.

If the cache is not found, then before the end of the cache block the directory gets cached in the s3 bucket with the key.

Caching the “.npm” folder in the docker build.

If you want to use the npm caching to reduce the docker build, you need to pass the restored .npm folder to the docker build.

For that while building the Dockerfile we need to pass the cache to the docker container and copy it to the ~/.npm path

During the “npm ci” while building the Dockerfile this cache will be used.

Here is the sample Dockerfile we used for copying the cache to the docker

FROM node:latest
WORKDIR /app
COPY package* /app/
COPY . /app
COPY .npm ~/.npm
RUN npm ci
CMD ["pm2-runtime", "start", "index.js"]

Here we stored the cache in the current directory and while building the Dockerfile it copies the cache to the ~ (root directory)
So, when the “npm ci” tries to install it uses the cache from the ~/.npm folder. During the first build, it would take time to cache and upload to the s3 bucket. While running the pipeline it first checks the cache with restoreKeys, if found it will restore the cache.

Cache restored successfully
1461248 bytes in 0.79 secs (1841711 bytes/sec)

Before the cache block ends, it checks if any change in package-lock.json it uploads the cache to the s3 bucket.

Cache saved successfully (cache-d41d8cd98f00b204e9800998ecf8427e)
1461248 bytes in 1.68 secs (869186 bytes/sec)

Else, it will skip restoring the cache.

Cache not saved (cache-d41d8cd98f00b204e9800998ecf8427e already exists)

But after the next build, you can see around 20–30% speed up in your build time.

Results:

Before implementing the caching it took around 11 min to complete the build.

After implementing the npm caching in the docker build, the pipeline speed was optimized to

Hurray! We have saved the 2 minutes 42 seconds.
But the optimized time depends on the build, if it is a smaller build it may give less result.