Question Re: How to avoid changes in package-lock.json when using local and BAS development in parallel? in Technology Q&A

How to avoid changes in package-lock.json when using local and BAS development in parallel?

gregorw — Tue, 07 Dec 2021 09:34:24 GMT

Hello BAS Team,

we've developed a CAP application in our local VS Code environment. Now we're handing over the applicaiton to the customer and they must use BAS as no local IDE installation is allowed. During the setup of the project in BAS I've noticed the following issue:

When running "npm install" in BAS the original content of package-lock.json e.g.:

    "@sap/cds-odata-v2-adapter-proxy": {
      "version": "1.8.2",
      "resolved": "https://registry.npmjs.org/@sap/cds-odata-v2-adapter-proxy/-/cds-odata-v2-adapter-proxy-1.8.2.tgz",
      "integrity": "sha512-fN36iWJSPIOvxnPgbKmbD4FHvXV2QxUR3cVzWLNHmVY9/C1ky1oODfsYN4fkIuxj4nuIfjYdKIsaLjnv50fRmw==",

is replaced with:

    "@sap/cds-odata-v2-adapter-proxy": {
      "version": "1.8.2",
      "resolved": "http://nginx-redirector.repo-cache.svc.cluster.local/repository/appstudio-npm-group/@sap/cds-odata-v2-adapter-proxy/-/cds-odata-v2-adapter-proxy-1.8.2.tgz",
      "integrity": "sha512-fN36iWJSPIOvxnPgbKmbD4FHvXV2QxUR3cVzWLNHmVY9/C1ky1oODfsYN4fkIuxj4nuIfjYdKIsaLjnv50fRmw==",

as you can see the version and integrity fields are exactly the same. But the resolved field does change and this results in a change that either needs to be commited to the Git repository or discarded. My temporary solution for the moment is to use:

npm ci

I understand that the registry is configured to a different location:

registry = "http://nginx-redirector.repo-cache.svc.cluster.local/repository/appstudio-npm-group/"<br>

to improve performance and avoid not necessary traffic. But maybe there is a way to keep the original resolved value?

Best Regards
Gregor

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

SebastianEsch — Tue, 07 Dec 2021 11:16:51 GMT

I noticed this too. This is quite annoying if you have mixed setups with BAS and VS Code developers.

The BAS internal resolved URL also breaks in CI/CD pipelines. From my point of view, a NPM cache should be transparent and not change your development artefacts.

Kind regards,

Sebastian

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

thomas_jung — Wed, 08 Dec 2021 14:39:15 GMT

I asked the BAS development team internally about this and their response was: Package-lock.json should be excluded from git. BAS resolves against NPM mirror, so these URLs will be in the package-lock.json. In any environment that does not have access to Nexus it breaks builds.

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

gregorw — Wed, 08 Dec 2021 15:57:58 GMT

Hi Thomas,

thank you for jumping in hear. According to the documenation for the package-lock-json which states:

"This file is intended to be committed into source repositories"

I have to disagree with the answer from the BAS development team to not add the file to version control. Please ask them to join the discussion here directly.

Best Regards
Gregor

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

thomas_jung — Wed, 08 Dec 2021 17:00:45 GMT

I have shared the link to this thread with them. I also had similar questions back as I also believe package-lock.json committed to git is a best practice particularly in CI/CD setup.

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

gregorw — Fri, 10 Dec 2021 12:50:26 GMT

Hi Thomas,

today I faced another issue that is realated to the package-lock.json. As I try to keep the MTAR resulting from the build as small as possible I've added:

    build-parameters:
      ignore: ["node_modules/"]

to my srv and db module. But when I now build this in BAS the deploy fails as the CF build environment seems to have no access to http://nginx-redirector.repo-cache.svc.cluster.local. I've now posted: Deployment of MTAR fails when build-parameters: ignore: ["node_modules/"] is set on this topic.

Best Regards
Gregor

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

thomas_jung — Fri, 10 Dec 2021 13:46:55 GMT

I can see where that would be an issue. You could also exclude the package-lock.json with the same build-parameters ignore configuration; but in doing so you are of course running the risk of deploying into production with a different version of modules than you developed and tested with. I'll share this additional item with development as well.

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

WRoeckelein — Fri, 10 Dec 2021 17:54:33 GMT

I agree with Gregor and Thomas that package-lock.json should be commited to git to get reproducible builds. And for reproducible builds the package-lock.json needs to contain public acessible URLs.

If you fear direct outside access to arbitrary npm repos, you should leave the package-lock.json as it is and put some other checks in between.

And please do not start the need to commit node-modules or put it into mtars. We already have enough problems with the node-modules file system space usage (I remember at least one incident by us caused by this), we should not carry this over to git and mtar (i would wish to have non-inclusion of node-modules as the default)! I would even think if its time for pnpm usage...

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

SebastianEsch — Mon, 13 Dec 2021 08:02:46 GMT

Why should package-lock.json be excluded from Git? Node.js documentation states the opposite: https://nodejs.dev/learn/the-package-lock-json-file

Kind regards,

Sebastian

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

SebastianEsch — Mon, 13 Dec 2021 08:03:52 GMT

Same here: https://docs.npmjs.com/cli/v8/configuring-npm/package-lock-json

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

MarcusSchlzel — Mon, 13 Dec 2021 08:53:38 GMT

I agree with Gregor and the others that the best practice is to commit the package-lock and use "npm ci" to get a reproducable installation (btw not consistently used by SAP generators and buildpacks). I think it is a good option for beginners or BAS only users, but it should be possible to override this artifact repository somehow (with .npmrc it is currently not possible).

But it looks like they are currently doing it only for npm. Yarn is accessing the public repo directly. And i think that yarn is also installed in the default CF buildpack.

So my solution, for the time being, would be to switch to yarn.

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

michelle-van-lieshout — Tue, 14 Dec 2021 12:00:49 GMT

I developed in VS before too and then wanted to change to the BAS environment to add some Fiori elements -elements. From BAS I need to deploy to a cloud foundry environment.

This change, that BAS makes in comparison to Visual Studio Code, also results in errors trying to deploy to a CF environment. Deploying to CF now is not possible from the BAS environment. I tried removing the "ignore: "*node_modules*"" from the build parameters in the mta.yaml but this didn't solve the problem. I keep getting the error: Unable to build dependencies: exit status 1

This error is due to the network requests to the local cache failing:

network request to http://nginx-redirector.repo-cache.svc.cluster.local/repository/appstudio-npm-group/yallist/-/yallist-4.0.0.tgz failed, reason: getaddrinfo ENOTFOUND nginx-redirector.repo-cache.svc.cluster.local (STDOUT, STG)#

A solution to this problem would therefore be very helpful to me too! Keep me updated 🙂

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

gregorw — Tue, 14 Dec 2021 21:59:48 GMT

michelle-van-lieshout in which step does your error occur? During mbt build in BAS or during the MTA deployment?

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

michelle-van-lieshout — Wed, 15 Dec 2021 08:10:36 GMT

gregorw it occurs during the MTA deployment. I solved it for now by pulling everything from git in VS, then running all npm installs and copying everything within the package-lock.json to BAS (so you get the registry.xxxx link instead of the nginx-redirectorxxxxx links within the package-lock.json files).

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

gregorw — Wed, 15 Dec 2021 08:20:50 GMT

Can you please file an incident via SAP Support so we get pressure also via an official support channel.

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

michelle-van-lieshout — Wed, 15 Dec 2021 09:48:30 GMT

I have filed an incident as well 😉

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

michelle-van-lieshout — Wed, 15 Dec 2021 09:50:16 GMT

gregorw: The incident: 896313 / 2021

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

thomas_jung — Mon, 20 Dec 2021 13:42:06 GMT

The official answer back from development is that you should use an .npmrc (in the project or user level). Point the registry configuration in the .npmrc to the normal, public NPM registry. This will override the default BAS configuration and force it to use the normal registry instead of the cache one.

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

gregorw — Tue, 21 Dec 2021 16:47:19 GMT

Hi Thomas,

thank you for this follow up. I've added now a .npmrc with the following content:

registry=https://registry.npmjs.com/

To all folders that contain a package.json and in which an npm install is executed. That avoids changes on the package-lock.json.

Best Regards
Gregor

Re: How to avoid changes in package-lock.json when using local and BAS development in parallel?

former_member6792 — Fri, 01 Apr 2022 03:56:12 GMT

Hi Marcus, I agree with you (with .npmrc ) it is not overriding. Even with manual overriding in package-lock.json it is having the same error on cf push. Still there is no solution from SAP End