- SAP Managed Tags:
Here’s a quick, and clean, trick to speed up the build step in SAP CAP Node.js projects, and improve deploy times to SAP BTP as a side-effect.
I’ve been exploring ways to improve CAP Node.js build and deploy times. I’ve tried crazy shell scripts, NPM caching + offline, replacing NPM with PNPM, using the parameter --no-clean in the command $ cds build --production --no-clean, etc. Everything gave me better results than the default build, and I’d love to see more PNPM in CAP projects, but they required too many changes or were not intuitive. Then, I landed on this which gave me incredible results and seemed foolproof…
The “secret” starts with: do NOT build and deploy node_modules folder! Here’s the reason and a little tweak to take it to the next level…
Why SAP CAP build is slow
When building a CAP project with Cloud MTA Build Tool (MBT), it uses the mta.yaml instructions to generate an archive file (*.mtar) that is used to deploy to SAP BTP. By default, the archive contains the artefacts from the project (frontend + backend + database), including all their dependencies. As you may know, Node.js dependencies are stored in the node_modules folder and it can be MASSIVE.
Downloading all dependencies and including them into the archive file may take a lot of time.
Here’s an example of how node_modules can get ugly, it’s from a famous package used for test automation: Jest.
Yes, those are the dependencies from just ONE and ONLY package: 155 MB. Node.js projects may have dozens of dependencies…
How to leverage the tooling
I remember when I started working as a developer, before ABAP, building web applications with vanilla HTML/JavaScript as the frontend and PHP for the backend. Those were the days of ADSL modem and its impressive 256 kbit/s (not dial up, I’m not that old) and the golden rule for web performance was: the best performance improvement you get, is from the bytes you don’t send. Or, as Julius Rock would say:
Cloud Foundry has a built-in feature to execute $ npm install when applications are deployed missing node_modules. Taking advantage of this feature, we can completely ignore node_modules in our built/deployed projects and leave Cloud Foundry to install the dependencies for us. Thanks nodejs-buildpack!
MBT can exclude files and folders from the built *.mtar file using the parameter ignore in build-parameters.
Build the default way
This is what you see out there in most of the CAP Node.js projects. A simple MTA module declaration using heaps of defaults, not explicit setting the configuration. This results in a non-optimal build which may be OK if you want to. It also results in the node_modules folder inflating the *.mtar file.
Build the project: $ mbt build -t gen --mtar archive
modules:
- name: btp-cap-rag-ai-srv-001
type: nodejs
path: gen/srv
build-parameters:
builder: npm
provides:
- name: srv-api
properties:
srv-url: ${default-url}
requires:
- name: btp-cap-rag-ai-db-001
- name: btp-cap-rag-ai-db-deployer-001
type: hdb
path: gen/db
requires:
- name: btp-cap-rag-ai-db-001
I’m testing an application which has 2 Fiori apps + CAP backend + HANA database. Building this mta.yaml takes over 6 minutes to complete and creates a *.mtar file with a size of 58 MB. Add another couple minutes to send it over to SAP BTP. If the internet isn’t the fastest, you may spend 10 minutes to build/deploy the application.
Build 6x faster
Now, let MBT know we don’t want the node_modules folder included in the archive. Make sure to do it for all Node.js applications in the project which also includes the one in the db folder. I know you don’t even have a package.json in the db folder, but you should…
Build the project: $ mbt build -t gen --mtar archive
modules:
- name: btp-cap-rag-ai-srv-001
type: nodejs
path: gen/srv
build-parameters:
builder: npm
ignore:
- node_modules/
provides:
- name: srv-api
properties:
srv-url: ${default-url}
requires:
- name: btp-cap-rag-ai-db-001
- name: btp-cap-rag-ai-db-deployer-001
type: hdb
path: gen/db
build-parameters:
builder: npm
ignore:
- node_modules/
requires:
- name: btp-cap-rag-ai-db-001
The build took 1 minute! And the generated *.mtar file size is 136 KB (Kilobytes, not Megabytes)! That’s 6x faster than the default behaviour. We reduced build time by 84% with a single parameter.
That’s impressive. However, we want more!
Build 25x faster
Usually, the builder parameter is set to npm or npm-ci to install dependencies in build time. This $ npm install happens in the gen folder (not in the project’s root), this is what gets added to the archive. Now, think. If we’re excluding node_modules from the *.mtar file, why do we need to install the dependencies during build? We don’t! We can skip installing the dependencies altogether!!! Let’s change our mta.yaml one more time to customize the builder.
In the modules build-parameters section, set builder to custom and pass any valid NPM command (or nothing at all). I like to use the command $ npm root because it’s super light and it runs locally, it doesn’t send any request out to the internet.
Build the project: $ mbt build -t gen --mtar archive
modules:
- name: btp-cap-rag-ai-srv-001
type: nodejs
path: gen/srv
build-parameters:
builder: custom
commands:
- npm root
ignore:
- node_modules/
provides:
- name: srv-api
properties:
srv-url: ${default-url}
requires:
- name: btp-cap-rag-ai-db-001
- name: btp-cap-rag-ai-db-deployer-001
type: hdb
path: gen/db
build-parameters:
builder: custom
commands:
- npm root
ignore:
- node_modules/
requires:
- name: btp-cap-rag-ai-db-001
Build took 15 SECONDS!!! That’s 25x faster than the default behaviour. We reduced build time by 96% with almost no effort, just a simple configuration.
Now, deploying 136 KB of *.mtar file to SAP BTP should be much faster than the initial 60 MB. The application still requires the dependencies to run, but they’ll be installed by Cloud Foundry on the server side. I’m pretty sure SAP BTP in AWS/Azure/GCP data centers have a faster internet connection than your home office 😜
I’d love to see MBT delivering a new builder parameter value called “none”, “skip” or “too-lazy-to-build” to mimic the behaviour I’d described here… Blazing fast build time without a cost!
Deployment
A lot of people asked me about the impact on deployment times, so, here I'm adding these numbers for reference.
A lean *.mtar file, ignoring node_modules, was 3x faster! Remember, this is not just the time to send the file over. This is the whole deployment time, the process is only completed once the newly deployed application is up and running. It means the whole app has become available in less than 1 minute rather than waiting for over 3 minutes...
PS: added the full build command for reference + custom command [ ] as suggested in the comments (thanks Marcel)!
9 Comments
