Skip to content

Commit dd25d8f

Browse files
committed
fixup! test(@angular/build): add library builder scaling benchmark suite
Implemented the style-flavor matrix (`--style-flavor`, see "Style-flavor matrix" above): `inline`, `inline-css`, `inline-scss`, `external`, `external-css`, `external-scss`, as a sweep axis independent of `--layout`/`--count`/`--depth`. Defaults to `inline` only (matching the prior behavior) since the full matrix multiplies build count by up to 6x; opt in with `--style-flavor=<list>`.
1 parent e1a1171 commit dd25d8f

6 files changed

Lines changed: 286 additions & 68 deletions

File tree

‎scripts/benchmark.mts‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
import fs from 'node:fs';
1010
import path from 'node:path';
1111
import { type BenchmarkCliOptions, runI18nBenchmarks } from './benchmarks/i18n/index.mts';
12+
import { STYLE_FLAVORS, type StyleFlavor } from './benchmarks/library-builder/fixtures.mts';
1213
import {
1314
type LibraryBuilderBenchmarkOptions,
1415
runLibraryBuilderBenchmarks,
@@ -44,16 +45,26 @@ function parseIntList(raw: string | number | undefined): number[] | undefined {
4445
if (raw === undefined) {
4546
return undefined;
4647
}
48+
4749
return String(raw)
4850
.split(',')
4951
.map((s) => Number(s.trim()));
5052
}
5153

54+
function parseStringList(raw: string | undefined): string[] | undefined {
55+
if (raw === undefined) {
56+
return undefined;
57+
}
58+
59+
return raw.split(',').map((s) => s.trim());
60+
}
61+
5262
async function runLibraryBuilderSubsystem(
5363
options: {
5464
layout?: string;
5565
count?: string;
5666
depth?: string;
67+
'style-flavor'?: string;
5768
iterations?: string | number;
5869
json?: boolean;
5970
csv?: string | boolean;
@@ -108,10 +119,26 @@ async function runLibraryBuilderSubsystem(
108119
return 1;
109120
}
110121

122+
const styleFlavor = parseStringList(options['style-flavor']);
123+
const validStyleFlavors: readonly string[] = STYLE_FLAVORS;
124+
if (styleFlavor !== undefined && styleFlavor.some((f) => !validStyleFlavors.includes(f))) {
125+
// eslint-disable-next-line no-console
126+
console.error(
127+
'Error: --style-flavor must be a comma-separated list of: ' +
128+
STYLE_FLAVORS.join(', ') +
129+
'. Got "' +
130+
options['style-flavor'] +
131+
'".',
132+
);
133+
134+
return 1;
135+
}
136+
111137
const cliOptions: LibraryBuilderBenchmarkOptions = {
112138
layout: options.layout as LibraryBuilderBenchmarkOptions['layout'],
113139
count,
114140
depth,
141+
styleFlavor: styleFlavor as StyleFlavor[] | undefined,
115142
iterations,
116143
json: Boolean(options.json),
117144
csv: options.csv !== undefined ? path.resolve(cwd, options.csv as string) : undefined,
@@ -140,6 +167,7 @@ export default async function (
140167
layout?: string;
141168
count?: string;
142169
depth?: string;
170+
'style-flavor'?: string;
143171
help?: boolean;
144172
[key: string]: unknown;
145173
},
@@ -178,6 +206,9 @@ Options (library-builder):
178206
--depth=<n,n,...> Explicit tree depths, layout=deep only. Overrides the depths
179207
that would otherwise be derived from --count (the smallest
180208
depth with 2^depth >= each target count).
209+
--style-flavor=<f,f,...> Style-flavor targets, one or more of: inline, inline-css,
210+
inline-scss, external, external-css, external-scss
211+
(default: inline)
181212
--iterations=<n> Measured iterations (default: 3)
182213
--json Output raw measurements as JSON to stdout
183214
--csv=<path> Write raw per-iteration measurements as CSV to <path>

‎scripts/benchmarks/library-builder/README.md‎

Lines changed: 70 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -56,9 +56,31 @@ This suite supports two primary layouts to test different filesystem and entry-p
5656
- Design: Creates a nested directory structure (`sub-a`/`sub-b` at each level).
5757
- Depth $N$ is not an independent sweep parameter -- see "Sweep rounds" below.
5858

59-
Each entry point consists of a `public-api.ts` and a component file with an inline template
60-
containing a bit of per-entry content derived from the entry's id (e.g. `<h2>Component no.
61-
0042</h2>`), so entries aren't byte-identical copies of each other.
59+
Each entry point consists of a `public-api.ts` and a component file with a bit of per-entry
60+
content derived from the entry's id (e.g. `<h2>Component no. 0042</h2>`), so entries aren't
61+
byte-identical copies of each other.
62+
63+
### Style-flavor matrix
64+
65+
`--style-flavor` is a second, independent sweep axis (crossed with `--layout` and the
66+
count/depth sweep) controlling each entry point's template location and stylesheet
67+
presence/language:
68+
69+
| Flavor | Template | Stylesheet |
70+
| :-------------- | :--------------------- | :------------------------------------------------ |
71+
| `inline` | inline `template` | none |
72+
| `inline-css` | inline `template` | inline `styles`, plain CSS |
73+
| `inline-scss` | inline `template` | inline `styles`, SCSS (via `inlineStyleLanguage`) |
74+
| `external` | external `templateUrl` | none |
75+
| `external-css` | external `templateUrl` | external `styleUrl`, `.css` file |
76+
| `external-scss` | external `templateUrl` | external `styleUrl`, `.scss` file |
77+
78+
`external` isolates the cost of the extra per-entry template file read from the cost of
79+
stylesheet processing (it has no stylesheet, like `inline`, but reads its template from disk
80+
like the other `external-*` flavors). SCSS content uses a per-entry variable
81+
(`$ref-color-NNNN`) so the preprocessor does real work rather than passing through literal CSS.
82+
Defaults to `inline` only -- the full 6-flavor matrix is opt-in via `--style-flavor=<list>`
83+
since it multiplies the number of builds run by up to 6x.
6284

6385
### Sweep rounds
6486

@@ -88,14 +110,15 @@ pnpm admin benchmark library-builder [options]
88110

89111
### Options
90112

91-
| Option | Description | Default |
92-
| :---------------------- | :--------------------------------------------------------------------------------------------------- | :--------------------------------------- |
93-
| `--layout=<flat\|deep>` | Fixture layout | `flat` |
94-
| `--count=<n,n,...>` | Entry-point count targets, drives both layouts | `10,20,50,100,200,300,400,500,1000,2000` |
95-
| `--depth=<n,n,...>` | **`layout=deep` only.** Explicit tree depths, overriding the depths otherwise derived from `--count` | derived from `--count` |
96-
| `--iterations=<n>` | Measured iterations per size | `3` |
97-
| `--json` | Output raw per-iteration measurements as JSON to stdout | N/A |
98-
| `--csv=<path>` | Write raw per-iteration measurements as CSV to `<path>` | N/A (opt-in only) |
113+
| Option | Description | Default |
114+
| :------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------- |
115+
| `--layout=<flat\|deep>` | Fixture layout | `flat` |
116+
| `--count=<n,n,...>` | Entry-point count targets, drives both layouts | `10,20,50,100,200,300,400,500,1000,2000` |
117+
| `--depth=<n,n,...>` | **`layout=deep` only.** Explicit tree depths, overriding the depths otherwise derived from `--count` | derived from `--count` |
118+
| `--style-flavor=<f,f,...>` | Style-flavor targets (see "Style-flavor matrix" above), one or more of `inline`, `inline-css`, `inline-scss`, `external`, `external-css`, `external-scss` | `inline` |
119+
| `--iterations=<n>` | Measured iterations per size | `3` |
120+
| `--json` | Output raw per-iteration measurements as JSON to stdout | N/A |
121+
| `--csv=<path>` | Write raw per-iteration measurements as CSV to `<path>` | N/A (opt-in only) |
99122

100123
> `--count` and `--depth` are not independent sweep axes for `layout=deep`: `--count` is translated
101124
> into depths (smallest `depth` with `2^depth >= n` per target, de-duplicated); `--depth` only
@@ -110,6 +133,15 @@ Sweep the flat layout at a single size:
110133
pnpm admin benchmark library-builder --layout=flat --count=500
111134
```
112135

136+
Sweep the full style-flavor matrix at a single size, both layouts:
137+
138+
```sh
139+
pnpm admin benchmark library-builder --layout=flat --count=500 \
140+
--style-flavor=inline,inline-css,inline-scss,external,external-css,external-scss
141+
pnpm admin benchmark library-builder --layout=deep --count=500 \
142+
--style-flavor=inline,inline-css,inline-scss,external,external-css,external-scss
143+
```
144+
113145
## Benchmark Results
114146

115147
Each entry below is one benchmark run, with its own summary and conclusions.
@@ -161,10 +193,36 @@ but not a regression, and not yet investigated further.
161193

162194
All 60 builds (10 sizes x 3 iterations x 2 layouts) completed with `status: "ok"`; zero failures.
163195

196+
### 2026-10-16: Style-flavor matrix implemented, fixture-correctness verification
197+
198+
Implemented the style-flavor matrix (`--style-flavor`, see "Style-flavor matrix" above):
199+
`inline`, `inline-css`, `inline-scss`, `external`, `external-css`, `external-scss`, as a sweep
200+
axis independent of `--layout`/`--count`/`--depth`. Defaults to `inline` only (matching the
201+
prior behavior) since the full matrix multiplies build count by up to 6x; opt in with
202+
`--style-flavor=<list>`.
203+
204+
This entry only verifies fixture correctness at small scale, not full-sweep performance
205+
numbers -- a full style-flavor sweep is still Future Work.
206+
207+
Verification performed:
208+
209+
- `--layout=flat --count=5` and `--layout=deep --count=5` (8 entries, depth 3), all 6 flavors,
210+
`--iterations=1`: all 12 builds completed with `status: "ok"`.
211+
- Manually inspected a generated `inline-scss` fixture's `angular.json` to confirm
212+
`inlineStyleLanguage: "scss"` is set only for that flavor, and compiled it standalone to
213+
confirm the per-entry SCSS variable (`$ref-color-NNNN`) resolves to a literal color in the
214+
built `.mjs` output (not left as unresolved `$...` syntax) -- i.e. SCSS is actually being
215+
preprocessed, not passed through as plain CSS.
216+
- Repeated the same compiled-output check for `external-scss` with the `deep` layout --
217+
confirms external `.scss` stylesheets referenced via `styleUrl` also compile correctly.
218+
- Verified `--csv` and `--json` output both include the new `styleFlavor` field.
219+
- Verified `--style-flavor` rejects an unrecognized flavor name with a clear error listing the
220+
valid set, consistent with existing `--layout`/`--count`/`--depth` validation.
221+
164222
## Future Work
165223

166-
- Style-flavor matrix: inline / inline-css / inline-scss / external / external-css / external-scss, swept across both layouts.
167224
- A/B comparison against `ng-packagr` (both are already registered Architect builders — `@angular/build:ng-packagr` and `@angular/build:library` — so this is a same-builder-package comparison, not a separate tool invocation).
225+
- Full style-flavor matrix sweep (default `--count` sweep x 6 flavors x 2 layouts) to quantify the actual performance impact of stylesheet processing and external file reads, now that the matrix is implemented.
168226
- Dependencies between entry points.
169227
- Shared stylesheet tokens imported by multiple components.
170228
- Watch mode.

0 commit comments

Comments
 (0)