Lock transitive Node dependencies#413
Conversation
Improve docs. 359390
| * The `pnpm-install` script in the `package.json` files. | ||
| * In Eclipse: The run configuration shell scripts: `js build.sh` or `js build.cmd`. | ||
| . Replace the version specifier of the `@eclipse-scout/releng` dependency in all `package.json` files with `~26.2.0`. | ||
| . Add `pnpm-lock.yaml` to your `.gitignore` file to ensure it is not commited as it will not be used anyway. |
There was a problem hiding this comment.
Delete the pnpm-lock.yaml and add ...
| * In Eclipse: The run configuration shell scripts: `js build.sh` or `js build.cmd`. | ||
| . Replace the version specifier of the `@eclipse-scout/releng` dependency in all `package.json` files with `~26.2.0`. | ||
| . Add `pnpm-lock.yaml` to your `.gitignore` file to ensure it is not commited as it will not be used anyway. | ||
| . Ensure all `package.json` files include values for the `name` and `version` attributes. |
There was a problem hiding this comment.
Are the last points really necessary? Or will it create an error and you can fix it later?
|
|
||
| == Locking of Transitive Node Dependencies | ||
|
|
||
| Scout includes support to lock the Node dependencies. |
There was a problem hiding this comment.
Please add a little bit more text, e.g.
To mitigate supply chain attacks and ensure reproducible builds, Scout now provides support to lock the Node.js dependencies.
|
|
||
| The dependency version to use is part of the dependency declaration in the `package.json` and follows the https://semver.org/[SemVer] specification. | ||
| This specification allows to declare range dependencies which might match several versions (even versions in the future which have not yet been published). | ||
| When a dependency is installed, pnpm decides which version to use based on the SemVer constraint declared. |
There was a problem hiding this comment.
based on the declared SemVer constraint.
| This setup gives you the possibility to easier integrate and update 3rd party JavaScript frameworks available in the huge https://www.npmjs.com/[npm registry]. | ||
| Using ranged versions has the benefit that you automatically get newer versions e.g. including bugfixes without changing anything in your dependency declarations. | ||
| On the other hand you also get regressions automatically, and it may prevent reproducible build outputs. | ||
| Furthermore, there have been several malicious dependency updates of packages whose developer accounts have been hacked. |
There was a problem hiding this comment.
Maybe rewrite the last sentence include the wording "supply chain attack".
E.g: It also increases the risk of supply chain attacks, where an attacker compromises an npm account and publishes a malicious package that will be downloaded automatically.
| Furthermore, there have been several malicious dependency updates of packages whose developer accounts have been hacked. | ||
|
|
||
| Scout itself is also published to that registry and will therefore be downloaded automatically once you execute `npm install`, as long as your `package.json` contains a Scout dependency. You will recognize a Scout module based on its name: all official Scout modules are published using the scope `@eclipse-scout`. The most important one is https://www.npmjs.com/package/@eclipse-scout/core[@eclipse-scout/core] which contains the core runtime functionality. Other modules are https://www.npmjs.com/package/@eclipse-scout/cli[@eclipse-scout/cli] for the building support, https://www.npmjs.com/package/@eclipse-scout/eslint-config[@eclipse-scout/eslint-config] for our ESLint rules, or https://www.npmjs.com/package/@eclipse-scout/karma-jasmine-scout[@eclipse-scout/karma-jasmine-scout] for enhanced testing support. | ||
| To address these issues Scout 26.2 or newer by default locks most of the ranged dependencies to a fixed version. |
There was a problem hiding this comment.
| To address these issues Scout 26.2 or newer by default locks most of the ranged dependencies to a fixed version. | |
| To address these issues, Scout 26.2 or newer locks most of the ranged dependencies to a fixed version by default. |
| Excluded are `snapshot` dependencies which should always point to the newest available snapshot version. | ||
| Because Scout only needs partial locking, the pnpm default locking mechanism using a `pnpm-lock.yaml` file cannot be used (because this always locks the full dependency tree). |
There was a problem hiding this comment.
| Excluded are `snapshot` dependencies which should always point to the newest available snapshot version. | |
| Because Scout only needs partial locking, the pnpm default locking mechanism using a `pnpm-lock.yaml` file cannot be used (because this always locks the full dependency tree). | |
| Excluded are `snapshot` dependencies which should always point to the newest available snapshot version. | |
| These `snapshot` dependencies are the reason Scout cannot use pnpm's default locking mechanism based on a `pnpm-lock.yaml` file. The lock file always pins the entire dependency tree, whereas snapshot dependencies must not be locked. |
|
|
||
| Instead, the locked versions are written to the https://pnpm.io/settings#overrides[pnpm overrides] in the `pnpm-workspace.yaml` file. | ||
| The new `install` command of the `@eclipse-scout/releng` module generates these overrides when dependencies are installed locally during development. | ||
| The Maven build scripts only use the available overrides and therefore exactly use the versions specified. |
There was a problem hiding this comment.
| The Maven build scripts only use the available overrides and therefore exactly use the versions specified. | |
| The Maven build scripts only use the available overrides and therefore exactly use the specified versions. |
| ** `snapshots`: Only `snapshot` dependencies are updated. All other dependencies use the version declared in the overrides. As only `snapshots` can be updated, and they are never part of the overrides, the overrides will not be touched in this mode. | ||
| * `convergenceLogLevel`: Specifies which non-convergent dependencies should be logged. Default is `own`. Must be one of the following: | ||
| ** `all`: Log all dependencies for which several versions are in use (non-unique versions). | ||
| ** `single-external`: Log dependencies for which a single pnpm-workspace-external dependency exists. This may be handy to eliminate duplicate dependencies by adapting the workspace to the single external one. |
There was a problem hiding this comment.
I don't really understand the single-external log-level, maybe you can provide an example?
Also, it would be useful to know when I need to use these log levels and how, maybe add an extra chapter
| . Remove the overrides from the `pnpm-workspace.yaml` if existing. | ||
| . Replace the `pnpx @eclipse-scout/releng install` command in the `pnpm-install` script of your `package.json` files with the following: `pnpm install --ignore-scripts`. | ||
| . This will generate the `pnpm-lock.yaml` file again when executed. | ||
| .. If you want to lock the whole tree: commit it to your VCS. |
There was a problem hiding this comment.
| .. If you want to lock the whole tree: commit it to your VCS. | |
| .. If you want to lock the whole tree (including snapshot): commit it to your VCS. |
| The `install` command of the `@eclipse-scout/releng` module has the following parameters: | ||
|
|
||
| * `updateMode`: Specifies which dependencies should be updated during install. Default is `snapshots`. Must be one of the following: | ||
| ** `required`: Update dependencies that do not fulfill the version declaration in the `package.json`. This should be used during development so that the overrides are adapted as required in case you modify the dependencies in the `package.json`. |
There was a problem hiding this comment.
Maybe add some examples for each mode to make it more clear what happens
| The `install` command of the `@eclipse-scout/releng` module has the following parameters: | ||
|
|
||
| * `updateMode`: Specifies which dependencies should be updated during install. Default is `snapshots`. Must be one of the following: | ||
| ** `required`: Update dependencies that do not fulfill the version declaration in the `package.json`. This should be used during development so that the overrides are adapted as required in case you modify the dependencies in the `package.json`. |
There was a problem hiding this comment.
I was wondering, why certain dependencies were not included in scout-overrides. I changed the jQuery version and nothing happened even if I used updateMode required. Then I noticed jQuery is not included in the yaml. Maybe describe how the scout-overrides are created?
| * `updateMode`: Specifies which dependencies should be updated during install. Default is `snapshots`. Must be one of the following: | ||
| ** `required`: Update dependencies that do not fulfill the version declaration in the `package.json`. This should be used during development so that the overrides are adapted as required in case you modify the dependencies in the `package.json`. | ||
| ** `possible`: Update all dependencies to the newest possible version that currently exists. This can be used if you want to perform a manual full update of everything. | ||
| ** `snapshots`: Only `snapshot` dependencies are updated. All other dependencies use the version declared in the overrides. As only `snapshots` can be updated, and they are never part of the overrides, the overrides will not be touched in this mode. |
There was a problem hiding this comment.
Maybe add that snapshots mode is typically used by CI builds
359390