From afdc1a23a68464264f4837cdd6d93948f594ed91 Mon Sep 17 00:00:00 2001 From: unadlib Date: Sat, 10 Oct 2026 09:53:45 +0800 Subject: [PATCH 1/6] perf: record the benchmark of main at ce0c25e in the performance summary --- perf-testing/reports/README.md | 2 +- perf-testing/reports/SUMMARY.md | 409 +++++++++++++++----------------- 2 files changed, 194 insertions(+), 217 deletions(-) diff --git a/perf-testing/reports/README.md b/perf-testing/reports/README.md index 26bf9407..39c24ea5 100644 --- a/perf-testing/reports/README.md +++ b/perf-testing/reports/README.md @@ -1,6 +1,6 @@ # Performance evidence and archives -This directory contains two maintained documents: this archive index and the [current performance summary](./SUMMARY.md). The summary records the October 6 and 7 cross-library measurements of `main`, with the Set scenarios measured again on October 8 for [PR #195](https://github.com/unadlib/mutative/pull/195), and its [history](./SUMMARY.md#history) summarizes earlier batches, including the October 5 base/head matrix of PR #184, whose paired CI measurements are uploaded by the [PR checks](https://github.com/unadlib/mutative/pull/184/checks). Benchmark source, tests, pinned versions, [`budgets.json`](../budgets.json), and the [build-size baseline](../../scripts/build-size-baseline.json) remain versioned. Historical measurements are not inputs to CI: the performance jobs measure their actual base/head checkouts and upload their own results. +This directory contains two maintained documents: this archive index and the [current performance summary](./SUMMARY.md). The summary records the October 9 and 10 cross-library measurements of `main` at `ce0c25e`, and its [history](./SUMMARY.md#history) summarizes earlier batches, including the October 6 and 7 batch with the cells that PRs #195, #196 and #198 measured again or added, and the October 5 base/head matrix of PR #184, whose paired CI measurements are uploaded by the [PR checks](https://github.com/unadlib/mutative/pull/184/checks). Benchmark source, tests, pinned versions, [`budgets.json`](../budgets.json), and the [build-size baseline](../../scripts/build-size-baseline.json) remain versioned. Historical measurements are not inputs to CI: the performance jobs measure their actual base/head checkouts and upload their own results. ## Retention policy diff --git a/perf-testing/reports/SUMMARY.md b/perf-testing/reports/SUMMARY.md index 8f1796e6..d0329149 100644 --- a/perf-testing/reports/SUMMARY.md +++ b/perf-testing/reports/SUMMARY.md @@ -1,22 +1,16 @@ # Performance summary -Measurements from 2026-10-06 and 2026-10-07 UTC of the source of `main` at [`2ab58fc`](https://github.com/unadlib/mutative/commit/2ab58fce1210b2b4724fcecce10ff0e1a864ddce). Since the source of the previous batch, `e6b6563`, `main` has added the cheap `current()` of changed arrays of [PR #180](https://github.com/unadlib/mutative/pull/180), the returned-value, frozen-array, Set and moved-element improvements of [PR #183](https://github.com/unadlib/mutative/pull/183), and the draft lifecycle, patch, Map and Set fixes of [PR #184](https://github.com/unadlib/mutative/pull/184); PRs #181, #182, #186 and #189 changed development builds, packaging and rarely used paths. The candidate and the pinned npm baseline both report Mutative 1.3.0. Immer is pinned to 11.1.18 and Mitata to 1.0.34. The main comparison runs Immer without its array-method plugin; a separate dataset below enables `enableArrayMethods`. The raw datasets of this batch were not archived; the identities and commands below reproduce them, and the [archive index](./README.md) describes the retention policy for batches that are. +Measurements from 2026-10-09 and 2026-10-10 UTC of the source of `main` at [`ce0c25e`](https://github.com/unadlib/mutative/commit/ce0c25e9f1efb43e735ad4d2abe0e03ce29b1161). Since the source of the previous batch, `2ab58fc`, `main` has added the array draft fixes of [PR #190](https://github.com/unadlib/mutative/pull/190), the `apply()`, mark, `current()` and async recipe fixes of [PR #192](https://github.com/unadlib/mutative/pull/192), the Set drafts of [PR #195](https://github.com/unadlib/mutative/pull/195) and the patch application change of [PR #196](https://github.com/unadlib/mutative/pull/196); PR #191 changed packaging, and [PR #198](https://github.com/unadlib/mutative/pull/198) added four scenarios to the harness. The candidate and the pinned npm baseline both report Mutative 1.3.0. Immer is pinned to 11.1.18 and Mitata to 1.0.34. The main comparison runs Immer without its array-method plugin; a separate dataset below enables `enableArrayMethods`. The raw datasets of this batch were not archived; the identities and commands below reproduce them, and the [archive index](./README.md) describes the retention policy for batches that are. -The Set scenarios were measured again on 2026-10-09 UTC, after [PR #195](https://github.com/unadlib/mutative/pull/195) made Set drafts change their copy directly instead of mapping all their items: `set-add`, `set-delete`, `set-update` and `set-update-10pct` at 100 rows and `set-add` and `set-update` at 1,000 and 10,000 rows, every library in processes of its own, three runs each (312 trials from 96 processes), and the memory of `set-add` at every size. The results below use those cells in place of the earlier ones, except where [shared and isolated processes](#shared-and-isolated-processes) compares the cells of October 6 and 7; [Set drafts](#set-drafts) compares the change with `main`. - -The patch application scenarios were measured again on 2026-10-09 UTC as well, after [PR #196](https://github.com/unadlib/mutative/pull/196) made `apply()` read the type of each draft from its state instead of asking its proxy: `apply-update-10pct`, `apply-reverse` and `apply-array-ops` at 100 rows and `apply-update-10pct` and `apply-reverse` at 1,000 and 10,000 rows, every library in processes of its own, three runs each (126 trials from 63 processes), and the memory of `apply-update-10pct` at 100 rows. The results below use those cells too, with the same exception; [Patch application](#patch-application) compares the change with `main`. - -Four scenarios were added on 2026-10-09 UTC by [PR #198](https://github.com/unadlib/mutative/pull/198) for improvements of PRs #183 and #184 that only micro-benchmarks had measured: `set-read`, `map-forEach`, `shift-and-update` and `map-set-beside`. They ran with every library in processes of its own, three runs each, at 100, 1,000 and 10,000 rows (468 trials from 144 processes), and `shift-and-update` also with Immer's array-method plugin (72 trials from 18 processes). The results below include their cells, with the same exception; [Set reads, Map iteration, moved rows and unchanged collections](#set-reads-map-iteration-moved-rows-and-unchanged-collections) reports them. - -This batch remeasures every scenario and replaces the cross-library comparison of 2026-10-03 and 2026-10-04, which the [history](#history) summarizes. Unlike earlier batches, every timing below comes from processes that each run one scenario with one library. With all libraries in one process, as earlier batches measured every scenario but the wide-object ones, each library ran slower by a different amount, which favored the candidate against Immer and hid much of the hand-written reducer's lead; [shared and isolated processes](#shared-and-isolated-processes) compares the two. The harness is unchanged since the previous batch except for the three `search-*` scenarios of PR #180, which now also run at 1,000 and 10,000 rows and with Immer's array-method plugin. +This batch remeasures every scenario with the method of the previous one: every timing below comes from processes that each run one scenario with one library, and the shared-process matrices are kept as a control, which [shared and isolated processes](#shared-and-isolated-processes) compares. It replaces the batch of 2026-10-06 and 2026-10-07 together with the cells that PRs #195 and #196 measured again and PR #198 added, which the [history](#history) summarizes; [changes since the previous batch](#changes-since-the-previous-batch) compares them with this batch. ## Scope and method Apple M1 Max, 64 GiB RAM, darwin/arm64 (kernel 25.6.0), Node 24.16.0, V8 13.6.233.17-node.49. Jobs ran sequentially with normal desktop activity. -- **Workloads.** 93 scenarios at 100 rows, of which 26 also ran at 1,000 and 10,000 rows: the 23 of the previous batch and the three searches. With the four scenarios added on October 9, which ran at every size, the results cover 97 scenarios, 30 of them at 1,000 and 10,000 rows. The [benchmark guide](../README.md#workloads-and-units) describes each one. +- **Workloads.** 97 scenarios at 100 rows, of which 30 also ran at 1,000 and 10,000 rows. The [benchmark guide](../README.md#workloads-and-units) describes each one. - **Libraries.** The current source, pinned npm Mutative 1.3.0, Immer 11.1.18, and the hand-written reducer that is every scenario's reference result. Each library runs both freeze modes and both patch modes, except that the hand-written reducer runs only without either, and patch application only without patch generation. -- **Timing.** Every scenario ran with every library in processes of its own (`--isolate`), three runs each, covering both freeze and patch modes in each process. Each benchmark first runs its workload for 30 ms on a throwaway fixture. These datasets hold 5,508 trials from 1,719 processes. As a control, the shared-process matrices of earlier batches were measured too, with Map and Set scenarios in processes of their own: 5,508 trials from 18 processes. +- **Timing.** Every scenario ran with every library in processes of its own (`--isolate`), three runs each, covering both freeze and patch modes in each process. Each benchmark first runs its workload for 30 ms on a throwaway fixture. These datasets hold 5,976 trials from 1,863 processes. As a control, the shared-process matrices of earlier batches were measured too, with Map and Set scenarios in processes of their own: 5,976 trials from 18 processes. - **Isolation.** Code that ran earlier in a process changes how fast V8 runs a library. Copying objects with hundreds of properties took up to about 70 times as long after other code, and every library ran slower beside the others than alone. Isolated processes remove the influence of the other libraries and scenarios, not that of a real application's code. - **Memory.** 1,110 isolated workers for 13 scenarios at 100 rows and 8 at 1,000 and 10,000 rows. @@ -30,32 +24,32 @@ Cells faster / within 5% / slower, and the geometric mean of comparator time ove | Comparator | 100 rows | 1,000 and 10,000 rows | All | | --- | ---: | ---: | ---: | -| Immer 11.1.18 | 374 / 7 / 1 of 382, 3.06 | 221 / 9 / 2 of 232, 4.68 | 595 / 16 / 3 of 614, 3.59 | -| Immer 11.1.18 with its defaults | 95 / 1 / 1 of 97, 6.28 | 58 / 0 / 2 of 60, 7.31 | 153 / 1 / 3 of 157, 6.66 | -| Pinned npm Mutative 1.3.0 | 348 / 33 / 1 of 382, 3.48 | 195 / 37 / 0 of 232, 6.02 | 543 / 70 / 1 of 614, 4.28 | +| Immer 11.1.18 | 374 / 8 / 0 of 382, 3.04 | 222 / 8 / 2 of 232, 4.67 | 596 / 16 / 2 of 614, 3.58 | +| Immer 11.1.18 with its defaults | 96 / 0 / 1 of 97, 6.23 | 58 / 0 / 2 of 60, 7.31 | 154 / 0 / 3 of 157, 6.62 | +| Pinned npm Mutative 1.3.0 | 346 / 33 / 3 of 382, 3.46 | 192 / 40 / 0 of 232, 6.03 | 538 / 73 / 3 of 614, 4.27 | | Hand-written reducer | 7 / 0 / 87 of 94, 0.11 | 11 / 3 / 42 of 56, 0.13 | 18 / 3 / 129 of 150, 0.12 | Against Immer by workload group, faster / within 5% / slower and geometric mean: | Group | 100 rows | 1,000 rows | 10,000 rows | | --- | ---: | ---: | ---: | -| Upstream workloads | 82 / 1 / 1, 2.03 | — | — | -| Reads | 20 / 0 / 0, 1.59 | 12 / 0 / 0, 1.60 | 12 / 0 / 0, 1.87 | -| No-op producers | 7 / 1 / 0, 1.69 | — | — | -| Small state | 12 / 0 / 0, 1.66 | — | — | -| Mutation density | 12 / 0 / 0, 2.10 | 4 / 0 / 0, 1.90 | 4 / 0 / 0, 1.92 | -| Array methods | 132 / 0 / 0, 5.75 | 20 / 0 / 0, 23.77 | 20 / 0 / 0, 31.12 | -| Moved elements | 4 / 0 / 0, 12.93 | 4 / 0 / 0, 21.84 | 4 / 0 / 0, 30.39 | -| Map and Set | 46 / 2 / 0, 2.26 | 25 / 3 / 0, 2.52 | 24 / 4 / 0, 2.51 | -| Object records | 11 / 1 / 0, 1.26 | 8 / 0 / 0, 2.34 | 8 / 0 / 0, 1.61 | -| Class instances | 6 / 2 / 0, 1.74 | 6 / 0 / 2, 1.61 | 6 / 2 / 0, 1.64 | -| Deep path | 4 / 0 / 0, 1.54 | — | — | -| Push and insert | 8 / 0 / 0, 2.01 | 4 / 0 / 0, 2.41 | 4 / 0 / 0, 3.34 | -| Patch application | 6 / 0 / 0, 4.08 | 4 / 0 / 0, 1.65 | 4 / 0 / 0, 1.58 | -| Returned values | 12 / 0 / 0, 3.83 | 12 / 0 / 0, 5.60 | 12 / 0 / 0, 8.85 | -| Searches | 12 / 0 / 0, 5.14 | 12 / 0 / 0, 8.33 | 12 / 0 / 0, 9.47 | - -The 3 cells slower than Immer are `class-wide-update` with freeze on at 1,000 rows (411 µs and 415 µs against 239 µs and 238 µs) and `concat` with freeze and patches on at 100 rows (22.5 µs against 20.8 µs, a cell whose three candidate processes measured 19.2–22.8 µs); see the limits below. With each library's defaults, the candidate is slower only in `return-replace`, at every size: without auto-freeze it searches the returned state for drafts, which Immer's auto-freeze skips because the state is frozen (27.0 µs, 256 µs and 2.59 ms against 0.78–0.79 µs). The one cell slower than Mutative 1.3.0 is the same `concat` cell, against 18.3 µs. +| Upstream workloads | 84 / 0 / 0, 2.02 | — | — | +| Reads | 20 / 0 / 0, 1.59 | 12 / 0 / 0, 1.60 | 12 / 0 / 0, 2.20 | +| No-op producers | 6 / 2 / 0, 1.66 | — | — | +| Small state | 12 / 0 / 0, 1.63 | — | — | +| Mutation density | 12 / 0 / 0, 2.08 | 4 / 0 / 0, 1.89 | 4 / 0 / 0, 1.88 | +| Array methods | 132 / 0 / 0, 5.71 | 20 / 0 / 0, 23.71 | 20 / 0 / 0, 30.26 | +| Moved elements | 4 / 0 / 0, 12.93 | 4 / 0 / 0, 22.19 | 4 / 0 / 0, 30.64 | +| Map and Set | 45 / 3 / 0, 2.26 | 25 / 3 / 0, 2.52 | 25 / 3 / 0, 2.37 | +| Object records | 11 / 1 / 0, 1.25 | 8 / 0 / 0, 2.38 | 8 / 0 / 0, 1.60 | +| Class instances | 6 / 2 / 0, 1.73 | 6 / 0 / 2, 1.61 | 6 / 2 / 0, 1.64 | +| Deep path | 4 / 0 / 0, 1.50 | — | — | +| Push and insert | 8 / 0 / 0, 2.04 | 4 / 0 / 0, 2.40 | 4 / 0 / 0, 3.35 | +| Patch application | 6 / 0 / 0, 4.12 | 4 / 0 / 0, 1.64 | 4 / 0 / 0, 1.59 | +| Returned values | 12 / 0 / 0, 3.83 | 12 / 0 / 0, 5.60 | 12 / 0 / 0, 8.84 | +| Searches | 12 / 0 / 0, 5.11 | 12 / 0 / 0, 8.26 | 12 / 0 / 0, 9.42 | + +The 2 cells slower than Immer are `class-wide-update` with freeze on at 1,000 rows (415 µs and 410 µs against 237 µs and 241 µs); see the limits below. With each library's defaults, the candidate is slower only in `return-replace`, at every size: without auto-freeze it searches the returned state for drafts, which Immer's auto-freeze skips because the state is frozen (26.2 µs, 257 µs and 2.61 ms against 0.78–0.79 µs). The 3 cells slower than Mutative 1.3.0, by 7–16%, are `add` and `update-largeObject1` with freeze on at 100 rows (19.5 µs against 18.3 µs, and 141 µs and 148 µs against 129 µs and 128 µs), cells in which the three processes of each library overlapped: 1.3.0 measured 17.4–21.8 µs and 117–152 µs. ### Large arrays @@ -63,81 +57,81 @@ Against Immer without its array-method plugin, by size and mode, over the 30 sca | Rows | Freeze | Patches | All scenarios | Geometric mean | Moves, geometric mean | Moves, range | | ---: | --- | --- | ---: | ---: | ---: | ---: | -| 1,000 | off | off | 29 / 1 / 0 | 8.59 | 211 | 72.0–492 | +| 1,000 | off | off | 29 / 1 / 0 | 8.58 | 209 | 72.1–485 | | 1,000 | off | on | 27 / 1 / 0 | 4.11 | 7.1 | 6.1–7.5 | -| 1,000 | on | off | 28 / 1 / 1 | 3.84 | 32.4 | 18.5–46.9 | -| 1,000 | on | on | 27 / 0 / 1 | 2.68 | 6.6 | 6.1–6.9 | -| 10,000 | off | off | 28 / 2 / 0 | 10.58 | 442 | 176–1,213 | -| 10,000 | off | on | 27 / 1 / 0 | 4.61 | 7.5 | 5.0–9.7 | -| 10,000 | on | off | 28 / 2 / 0 | 4.18 | 39.7 | 22.2–53.0 | -| 10,000 | on | on | 27 / 1 / 0 | 2.86 | 7.1 | 5.1–9.0 | +| 1,000 | on | off | 28 / 1 / 1 | 3.84 | 32.4 | 18.6–47.2 | +| 1,000 | on | on | 27 / 0 / 1 | 2.68 | 6.6 | 6.2–6.8 | +| 10,000 | off | off | 29 / 1 / 0 | 10.61 | 431 | 177–1,051 | +| 10,000 | off | on | 27 / 1 / 0 | 4.59 | 7.3 | 5.1–8.9 | +| 10,000 | on | off | 28 / 2 / 0 | 4.17 | 38.8 | 22.5–54.3 | +| 10,000 | on | on | 27 / 1 / 0 | 2.84 | 6.8 | 5.2–8.0 | Immer time over candidate time for the moves at 10,000 rows, by freeze and patch mode: | Scenario | Off, off | Off, on | On, off | On, on | | --- | ---: | ---: | ---: | ---: | -| array-shift-nested | 1,213 | 9.7 | 49.1 | 9.0 | -| array-unshift-nested | 474 | 8.9 | 41.8 | 8.2 | -| array-splice-insert-nested | 176 | 6.4 | 22.2 | 6.0 | -| array-reverse-nested | 551 | 8.8 | 40.5 | 8.1 | -| array-reverse-primitive | 302 | 5.0 | 53.0 | 5.1 | +| array-shift-nested | 1,051 | 8.4 | 43.2 | 7.8 | +| array-unshift-nested | 476 | 8.5 | 42.4 | 7.7 | +| array-splice-insert-nested | 177 | 6.5 | 22.5 | 5.9 | +| array-reverse-nested | 550 | 8.9 | 39.3 | 8.0 | +| array-reverse-primitive | 306 | 5.1 | 54.3 | 5.2 | -The gap widens with size. With freeze and patches off, the moves were 211 times faster at 1,000 rows and 442 times at 10,000 on geometric mean: the candidate moves elements natively on its copy, while Immer moves each element through its draft proxy. Removing the first of 10,000 rows took 7.10 µs against 8,605 µs. With patches, both libraries emit one patch per moved index, which bounds the gain to 5–10 times. At these sizes Immer was faster only in `class-wide-update` with freeze on at 1,000 rows, and within 5% in Map updates without freezing and `class-wide-update` with freeze on at 10,000 rows, which the limits below discuss, and in three cells of `set-read`, where both libraries only read the Set. +The gap widens with size. With freeze and patches off, the moves were 209 times faster at 1,000 rows and 431 times at 10,000 on geometric mean: the candidate moves elements natively on its copy, while Immer moves each element through its draft proxy. Removing the first of 10,000 rows took 7.07 µs against 7,426 µs. With patches, both libraries emit one patch per moved index, which bounds the gain to 5–9 times. At these sizes Immer was faster only in `class-wide-update` with freeze on at 1,000 rows, and within 5% in Map updates without freezing and `class-wide-update` with freeze on at 10,000 rows, which the limits below discuss, and in two cells of `set-read`, where both libraries only read the Set. ### Freeze off, patches on -With patches on and freeze off, the candidate was faster than Immer in 148 of 150 cells and within 5% in the other 2, the Map updates at 1,000 and 10,000 rows (geometric mean 3.30). It was faster than Mutative 1.3.0 in 132 and within 5% in 18 (4.59). In the 70 array-related cells, array methods, the update of a moved row, searches and the upstream array workloads at every size, it was faster than both in every cell: 4.44 times Immer and 13.23 times Mutative 1.3.0 on geometric mean. +With patches on and freeze off, the candidate was faster than Immer in 147 of 150 cells and within 5% in the other 3, the Map updates at 1,000 and 10,000 rows and `map-delete` at 100 rows (geometric mean 3.29). It was faster than Mutative 1.3.0 in 132 and within 5% in 18 (4.58). In the 70 array-related cells, array methods, the update of a moved row, searches and the upstream array workloads at every size, it was faster than both in every cell: 4.41 times Immer and 13.14 times Mutative 1.3.0 on geometric mean. -Patches cost little when an update changes a few paths. They added a median 1% to the candidate's time at 1,000 and 10,000 rows and 20% at 100 rows. The former `benchmark:base` workload, `push-and-insert` at 10,000 rows, took 62.8 µs without patches and 63.3 µs with them; Mutative 1.3.0 went from 67.0 µs to 208 µs, and Immer from 162 µs to 364 µs. Moving elements is the exception: both libraries emit one patch per moved index, so `shift` at 10,000 rows produces 10,000 forward and 10,000 inverse patches in each. The candidate's time grows from 7.10 µs to 1,071 µs, against 10,376 µs for Immer producing the same patches. +Patches cost little when an update changes a few paths. They added a median 1% to the candidate's time at 1,000 and 10,000 rows and 20% at 100 rows. The former `benchmark:base` workload, `push-and-insert` at 10,000 rows, took 62.9 µs without patches and 63.0 µs with them; Mutative 1.3.0 went from 64.8 µs to 210 µs, and Immer from 165 µs to 364 µs. Moving elements is the exception: both libraries emit one patch per moved index, so `shift` at 10,000 rows produces 10,000 forward and 10,000 inverse patches in each. The candidate's time grows from 7.07 µs to 1,066 µs, against 8,967 µs for Immer producing the same patches. -Array methods on nested rows at 100 rows, in µs per call. The primitive and shallow shapes behave alike; over all 33 cells the candidate is 4.05 times faster than Immer (1.3–7.4) and 8.51 times faster than Mutative 1.3.0 (1.4–217). +Array methods on nested rows at 100 rows, in µs per call. The primitive and shallow shapes behave alike; over all 33 cells the candidate is 4.04 times faster than Immer (1.2–7.3) and 8.48 times faster than Mutative 1.3.0 (1.4–209). | Operation | Candidate | Immer | Mutative 1.3.0 | Immer/candidate | 1.3.0/candidate | | --- | ---: | ---: | ---: | ---: | ---: | -| push | 3.84 | 7.01 | 5.54 | 1.83 | 1.44 | -| pop | 1.50 | 6.14 | 3.71 | 4.08 | 2.46 | -| shift | 11.4 | 82.2 | 2,391 | 7.24 | 210.55 | -| unshift | 14.3 | 87.8 | 2,333 | 6.13 | 162.84 | -| splice, insert | 8.89 | 48.8 | 1,201 | 5.49 | 135.15 | -| splice, remove | 6.24 | 45.1 | 1,204 | 7.23 | 192.97 | -| splice, replace | 3.50 | 9.51 | 6.22 | 2.72 | 1.78 | -| fill | 4.27 | 8.11 | 6.97 | 1.90 | 1.63 | -| copyWithin | 2.45 | 5.70 | 74.8 | 2.33 | 30.54 | -| sort | 103 | 129 | 2,395 | 1.25 | 23.26 | -| reverse | 11.3 | 83.4 | 2,465 | 7.35 | 217.15 | +| push | 3.80 | 6.97 | 5.49 | 1.83 | 1.45 | +| pop | 1.50 | 6.01 | 3.63 | 4.00 | 2.42 | +| shift | 11.3 | 81.7 | 2,367 | 7.20 | 208.65 | +| unshift | 14.1 | 86.4 | 2,288 | 6.11 | 162.01 | +| splice, insert | 8.83 | 48.0 | 1,192 | 5.44 | 135.02 | +| splice, remove | 6.14 | 44.4 | 1,191 | 7.23 | 194.11 | +| splice, replace | 3.45 | 9.49 | 6.21 | 2.75 | 1.80 | +| fill | 4.24 | 7.99 | 6.81 | 1.88 | 1.60 | +| copyWithin | 2.41 | 5.56 | 79.2 | 2.31 | 32.88 | +| sort | 103 | 130 | 2,363 | 1.25 | 22.86 | +| reverse | 11.3 | 82.5 | 2,356 | 7.33 | 209.39 | The array workloads of the upstream suite at 100 rows: | Scenario | Candidate | Immer | Mutative 1.3.0 | Immer/candidate | 1.3.0/candidate | | --- | ---: | ---: | ---: | ---: | ---: | -| add | 1.79 | 4.19 | 3.57 | 2.33 | 1.99 | -| remove | 11.6 | 84.9 | 2,983 | 7.33 | 257.69 | -| filter | 22.3 | 31.5 | 36.0 | 1.41 | 1.61 | -| concat | 0.79 | 1.00 | 1.05 | 1.27 | 1.33 | -| mapNested | 40.1 | 59.6 | 78.2 | 1.49 | 1.95 | -| sortById-reverse | 66.5 | 83.5 | 2,950 | 1.26 | 44.38 | -| reverse-array | 11.4 | 82.2 | 2,975 | 7.22 | 261.15 | -| update-multiple | 7.41 | 16.8 | 16.3 | 2.26 | 2.20 | -| remove-high | 19.8 | 49.6 | 602 | 2.51 | 30.43 | -| remove-reuse | 134 | 806 | 27,939 | 6.01 | 208.24 | -| mixed-sequence | 51.0 | 92.3 | 142 | 1.81 | 2.77 | +| add | 1.81 | 4.21 | 3.50 | 2.33 | 1.94 | +| remove | 11.4 | 83.2 | 2,949 | 7.28 | 257.87 | +| filter | 22.5 | 31.3 | 35.9 | 1.39 | 1.59 | +| concat | 0.78 | 0.98 | 1.04 | 1.26 | 1.34 | +| mapNested | 40.4 | 59.5 | 77.3 | 1.47 | 1.91 | +| sortById-reverse | 65.1 | 83.4 | 2,904 | 1.28 | 44.60 | +| reverse-array | 11.3 | 82.3 | 2,923 | 7.28 | 258.53 | +| update-multiple | 7.33 | 16.6 | 16.0 | 2.26 | 2.19 | +| remove-high | 19.9 | 49.0 | 610 | 2.46 | 30.62 | +| remove-reuse | 133 | 792 | 26,802 | 5.97 | 201.91 | +| mixed-sequence | 50.9 | 90.1 | 140 | 1.77 | 2.76 | Moving elements at 1,000 and 10,000 rows, with Immer's array-method plugin from the plugin datasets for reference: | Scenario | Rows | Candidate | Immer | Mutative 1.3.0 | Immer + plugin | | --- | ---: | ---: | ---: | ---: | ---: | -| array-shift-nested | 1,000 | 109 | 798 | 25,196 | 125 | -| array-splice-insert-nested | 1,000 | 58.0 | 436 | 11,963 | 82.9 | -| array-reverse-nested | 1,000 | 110 | 809 | 26,537 | 124 | -| array-shift-nested | 10,000 | 1,071 | 10,376 | 273,162 | 1,380 | -| array-unshift-nested | 10,000 | 1,079 | 9,567 | 281,936 | 1,393 | -| array-splice-insert-nested | 10,000 | 703 | 4,478 | 132,028 | 934 | -| array-reverse-nested | 10,000 | 1,074 | 9,416 | 277,427 | 1,375 | -| array-reverse-primitive | 10,000 | 1,196 | 6,036 | 5,295 | 1,979 | +| array-shift-nested | 1,000 | 110 | 801 | 25,431 | 124 | +| array-splice-insert-nested | 1,000 | 57.9 | 436 | 12,181 | 82.5 | +| array-reverse-nested | 1,000 | 110 | 811 | 25,045 | 125 | +| array-shift-nested | 10,000 | 1,066 | 8,967 | 273,171 | 1,390 | +| array-unshift-nested | 10,000 | 1,080 | 9,144 | 263,088 | 1,395 | +| array-splice-insert-nested | 10,000 | 710 | 4,615 | 134,943 | 939 | +| array-reverse-nested | 10,000 | 1,073 | 9,525 | 276,034 | 1,381 | +| array-reverse-primitive | 10,000 | 1,181 | 6,056 | 5,251 | 2,014 | #### Why Immer's array-method plugin is not the main comparator -Immer's optional `enableArrayMethods()` also runs array methods on the draft's copy, and with patches it comes within 1.13–1.71 times of the candidate on the moves above. The comparisons nevertheless run Immer without it, because in Immer 11.1.18 the plugin breaks guarantees that its other configurations and Mutative keep. [`test/immer-array-methods.md`](../../test/immer-array-methods.md) reproduces four root causes: +Immer's optional `enableArrayMethods()` also runs array methods on the draft's copy, and with patches it comes within 1.14–1.71 times of the candidate on the moves above. The comparisons nevertheless run Immer without it, because in Immer 11.1.18 the plugin breaks guarantees that its other configurations and Mutative keep. [`test/immer-array-methods.md`](../../test/immer-array-methods.md) reproduces four root causes: - `shift`, `pop` and `splice` return raw base objects. Editing a removed object changes the previous state, or throws when that state is frozen. - `reverse` and `sort` can expose original objects at inserted indices as raw values, with the same effect; inverse patches then restore the modified values instead of the original ones. @@ -152,26 +146,26 @@ The `search-*` scenarios find the last row of an array of nested rows and update | Scenario | Rows | Freeze | Mutative | Mutative 1.3.0 | Immer | Hand-written | Immer/Mutative | | --- | ---: | --- | ---: | ---: | ---: | ---: | ---: | -| search-draft | 100 | off | 20.4 | 37.3 | 30.1 | 0.15 | 1.48 | -| search-draft | 100 | on | 22.9 | 40.2 | 38.0 | — | 1.66 | -| search-current | 100 | off | 12.3 | 43.6 | 238 | 0.14 | 19.36 | -| search-current | 100 | on | 14.9 | 46.0 | 34.9 | — | 2.35 | -| search-current-shifted | 100 | off | 3.58 | 147 | 97.4 | 0.13 | 27.23 | -| search-current-shifted | 100 | on | 5.86 | 151 | 105 | — | 17.97 | -| search-draft | 1,000 | off | 192 | 342 | 274 | 0.89 | 1.43 | -| search-draft | 1,000 | on | 211 | 361 | 345 | — | 1.63 | -| search-current | 1,000 | off | 24.3 | 333 | 2,287 | 0.88 | 94.16 | -| search-current | 1,000 | on | 42.4 | 357 | 259 | — | 6.11 | -| search-current-shifted | 1,000 | off | 22.7 | 1,406 | 928 | 0.86 | 40.85 | -| search-current-shifted | 1,000 | on | 41.4 | 1,435 | 999 | — | 24.14 | -| search-draft | 10,000 | off | 2,126 | 3,729 | 3,465 | 65.4 | 1.63 | -| search-draft | 10,000 | on | 2,258 | 3,881 | 4,057 | — | 1.80 | -| search-current | 10,000 | off | 176 | 3,441 | 22,888 | 21.9 | 129.80 | -| search-current | 10,000 | on | 354 | 3,618 | 2,449 | — | 6.92 | -| search-current-shifted | 10,000 | off | 257 | 15,874 | 11,230 | 64.7 | 43.74 | -| search-current-shifted | 10,000 | on | 432 | 16,115 | 11,913 | — | 27.58 | - -A snapshot search was faster than the draft search at every size: 1.7 times at 100 rows, 7.9 times at 1,000 and 12 times at 10,000 with freeze off. The draft search drafts every row it visits, while `current()` copies the changed array from its current elements. After `shift()`, the search at 10,000 rows took 257 µs, against 772 µs when PR #180 was measured: updating the moved last row now finds its original index by searching the original array instead of building a map of all 10,000 original indices (PR #183). Immer's `current()` walks every unfrozen object, so with freeze off its snapshot searches are slower than its draft search. +| search-draft | 100 | off | 20.6 | 36.8 | 29.8 | 0.14 | 1.44 | +| search-draft | 100 | on | 23.0 | 39.7 | 37.6 | — | 1.63 | +| search-current | 100 | off | 12.2 | 43.3 | 232 | 0.14 | 19.08 | +| search-current | 100 | on | 14.6 | 45.9 | 34.6 | — | 2.37 | +| search-current-shifted | 100 | off | 3.60 | 145 | 96.1 | 0.13 | 26.70 | +| search-current-shifted | 100 | on | 5.86 | 150 | 105 | — | 17.86 | +| search-draft | 1,000 | off | 195 | 340 | 274 | 0.90 | 1.40 | +| search-draft | 1,000 | on | 214 | 360 | 345 | — | 1.61 | +| search-current | 1,000 | off | 24.2 | 332 | 2,287 | 0.88 | 94.67 | +| search-current | 1,000 | on | 42.8 | 358 | 259 | — | 6.05 | +| search-current-shifted | 1,000 | off | 22.9 | 1,405 | 929 | 0.86 | 40.57 | +| search-current-shifted | 1,000 | on | 41.6 | 1,442 | 1,005 | — | 24.18 | +| search-draft | 10,000 | off | 2,175 | 3,729 | 3,452 | 64.8 | 1.59 | +| search-draft | 10,000 | on | 2,308 | 3,880 | 4,077 | — | 1.77 | +| search-current | 10,000 | off | 177 | 3,443 | 22,711 | 65.2 | 128.53 | +| search-current | 10,000 | on | 353 | 3,636 | 2,444 | — | 6.92 | +| search-current-shifted | 10,000 | off | 257 | 15,961 | 11,379 | 23.4 | 44.25 | +| search-current-shifted | 10,000 | on | 435 | 16,186 | 11,981 | — | 27.56 | + +A snapshot search was faster than the draft search at every size: 1.7 times at 100 rows, 8.1 times at 1,000 and 12 times at 10,000 with freeze off. The draft search drafts every row it visits, while `current()` copies the changed array from its current elements. After `shift()`, the search at 10,000 rows took 257 µs, against 772 µs when PR #180 was measured: updating the moved last row now finds its original index by searching the original array instead of building a map of all 10,000 original indices (PR #183). Immer's `current()` walks every unfrozen object, so with freeze off its snapshot searches are slower than its draft search. ### Set drafts @@ -191,7 +185,7 @@ A snapshot search was faster than the draft search at every size: 1.7 times at 1 | set-update | 10,000 | off | off | 1,053 | 542 | 0.52 | | set-update-10pct | 100 | off | off | 10.90 | 6.98 | 0.64 | -Over all 32 cells, four Set scenarios at 100 rows and two at 1,000 and 10,000 rows in every mode, the PR took 0.28 of the time of `main` on geometric mean and was faster in each. Additions and deletions no longer copy the items into a Map and back, so without patches they now cost little more than copying the Set: `set-add` at 10,000 items allocated 325 KiB per update, against 2,878 KiB for Mutative 1.3.0. With patches, a Set that added or deleted items still compares all items to emit its own patches, which bounds that gain to about 3 times, and a changed item still rebuilds the Set in order. Against the other libraries the candidate was faster than Immer in every Set cell, 2.9–51.1 times, and faster than the hand-written reducer in `set-add` at 1,000 and 10,000 rows, where `Set.prototype.difference` copies the Set faster than the reducer's `new Set()`. +Over all 32 cells, four Set scenarios at 100 rows and two at 1,000 and 10,000 rows in every mode, the PR took 0.28 of the time of `main` on geometric mean and was faster in each. Additions and deletions no longer copy the items into a Map and back, so without patches they now cost little more than copying the Set: in this batch, `set-add` at 10,000 items allocated 330 KiB per update, against 2,883 KiB for Mutative 1.3.0. With patches, a Set that added or deleted items still compares all items to emit its own patches, which bounds that gain to about 3 times, and a changed item still rebuilds the Set in order. In this batch the candidate was faster than Immer in every cell of these four Set scenarios, 3.0–50 times, and faster than the hand-written reducer in `set-add` at 1,000 and 10,000 rows, where `Set.prototype.difference` copies the Set faster than the reducer's `new Set()`. ### Patch application @@ -216,35 +210,35 @@ Over all 14 cells, three patch application scenarios at 100 rows and two at 1,00 ### Set reads, Map iteration, moved rows and unchanged collections -PRs #183 and #184 measured four of their improvements with micro-benchmarks only, so [PR #198](https://github.com/unadlib/mutative/pull/198) added a scenario for each. Microseconds per scenario, medians of three runs, patches off: +PRs #183 and #184 measured four of their improvements with micro-benchmarks only, so [PR #198](https://github.com/unadlib/mutative/pull/198) added a scenario for each. Microseconds per scenario in this batch, medians of three runs, patches off: | Scenario | Rows | Freeze | Mutative | Mutative 1.3.0 | Immer | Hand-written | Immer/Mutative | 1.3.0/Mutative | | --- | ---: | --- | ---: | ---: | ---: | ---: | ---: | ---: | -| set-read | 100 | off | 0.63 | 4.83 | 0.67 | 0.03 | 1.07 | 7.70 | -| set-read | 1,000 | off | 0.62 | 39.8 | 0.66 | 0.04 | 1.06 | 64.39 | -| set-read | 10,000 | off | 0.62 | 558 | 0.65 | 0.03 | 1.04 | 894 | -| map-forEach | 100 | off | 2.55 | 9.25 | 3.23 | 0.59 | 1.27 | 3.63 | -| map-forEach | 1,000 | off | 26.4 | 87.4 | 29.9 | 5.46 | 1.13 | 3.31 | -| map-forEach | 10,000 | off | 263 | 866 | 290 | 54.3 | 1.10 | 3.29 | -| shift-and-update | 100 | off | 1.60 | 84.2 | 71.8 | 0.07 | 44.98 | 52.71 | -| shift-and-update | 1,000 | off | 2.96 | 801 | 675 | 0.32 | 228 | 270 | -| shift-and-update | 10,000 | off | 15.9 | 9,610 | 8,768 | 1.40 | 553 | 606 | -| shift-and-update | 100 | on | 4.93 | 86.5 | 78.7 | — | 15.96 | 17.54 | -| shift-and-update | 1,000 | on | 32.3 | 831 | 748 | — | 23.18 | 25.77 | -| shift-and-update | 10,000 | on | 301 | 9,693 | 9,304 | — | 30.94 | 32.23 | -| map-set-beside | 100 | off | 8.89 | 17.0 | 23.0 | 4.34 | 2.58 | 1.91 | -| map-set-beside | 1,000 | off | 44.7 | 114 | 167 | 42.5 | 3.74 | 2.54 | -| map-set-beside | 10,000 | off | 576 | 1,537 | 2,013 | 594 | 3.50 | 2.67 | -| map-set-beside | 100 | on | 12.1 | 38.5 | 29.1 | — | 2.41 | 3.19 | -| map-set-beside | 1,000 | on | 66.4 | 309 | 201 | — | 3.03 | 4.65 | -| map-set-beside | 10,000 | on | 775 | 3,471 | 2,308 | — | 2.98 | 4.48 | - -- `set-read` checks whether a Set of numbers holds its middle number and a missing one, and reads its size. Mutative 1.3.0 mapped every item of a Set draft as soon as the recipe read the Set, and copied the Set when a lookup missed; the candidate does neither until the recipe changes the Set (PR #183), as Immer does. Both took 0.6–0.7 µs at every size, and 4 of the 12 cells against Immer are within 5%. +| set-read | 100 | off | 0.62 | 4.78 | 0.66 | 0.03 | 1.07 | 7.71 | +| set-read | 1,000 | off | 0.62 | 40.2 | 0.66 | 0.04 | 1.07 | 65.04 | +| set-read | 10,000 | off | 0.62 | 550 | 0.66 | 0.03 | 1.07 | 890 | +| map-forEach | 100 | off | 2.51 | 9.16 | 3.20 | 0.58 | 1.27 | 3.64 | +| map-forEach | 1,000 | off | 26.5 | 86.9 | 30.0 | 5.53 | 1.13 | 3.28 | +| map-forEach | 10,000 | off | 264 | 869 | 296 | 54.4 | 1.12 | 3.29 | +| shift-and-update | 100 | off | 1.56 | 83.2 | 70.2 | 0.07 | 44.90 | 53.22 | +| shift-and-update | 1,000 | off | 2.90 | 805 | 680 | 0.32 | 235 | 278 | +| shift-and-update | 10,000 | off | 15.8 | 9,570 | 8,780 | 1.40 | 557 | 607 | +| shift-and-update | 100 | on | 4.83 | 85.5 | 78.8 | — | 16.29 | 17.68 | +| shift-and-update | 1,000 | on | 32.3 | 823 | 750 | — | 23.20 | 25.47 | +| shift-and-update | 10,000 | on | 300 | 9,676 | 9,320 | — | 31.08 | 32.27 | +| map-set-beside | 100 | off | 8.75 | 16.8 | 22.4 | 4.29 | 2.56 | 1.92 | +| map-set-beside | 1,000 | off | 44.7 | 113 | 167 | 42.4 | 3.74 | 2.54 | +| map-set-beside | 10,000 | off | 571 | 1,502 | 1,970 | 589 | 3.45 | 2.63 | +| map-set-beside | 100 | on | 11.9 | 37.8 | 29.1 | — | 2.43 | 3.16 | +| map-set-beside | 1,000 | on | 65.8 | 309 | 202 | — | 3.08 | 4.71 | +| map-set-beside | 10,000 | on | 770 | 3,474 | 2,312 | — | 3.00 | 4.51 | + +- `set-read` checks whether a Set of numbers holds its middle number and a missing one, and reads its size. Mutative 1.3.0 mapped every item of a Set draft as soon as the recipe read the Set, and copied the Set when a lookup missed; the candidate does neither until the recipe changes the Set (PR #183), as Immer does. Both took 0.6–0.7 µs at every size, and 3 of the 12 cells against Immer are within 5%. - `map-forEach` sums a Map of numbers with `forEach()`. Mutative 1.3.0 called `get()` through the draft's proxy for each entry; the candidate reads each entry from the draft's state (PR #184), which takes 0.3 of the time, and 0.8–0.9 of Immer's. The hand-written reducer's native `forEach()` took a fifth of the candidate's time. - `shift-and-update` removes the first row with `shift()` and updates the middle row. The candidate moves the rows natively on its copy and finds the moved row's original index by searching the original array from its end (PR #183), while Immer and Mutative 1.3.0 move every row through the draft's proxy. With Immer's array-method plugin, the candidate was faster in every cell, 1.1–24 times. With auto-freeze, finding the moved row takes longer; see the limits below. - `map-set-beside` inserts a row into a Map of rows and a number into a Set of numbers, which copies both, and then updates a number beside them in nine more producers. Without auto-freeze the copies dominate in every library: at 10,000 rows the candidate was within 5% of the hand-written reducer. With auto-freeze, Mutative 1.3.0 walked both copies again in each later producer, since it froze Map and Set instances only by replacing their mutators; the candidate freezes the instances themselves (PR #184), so the later producers skip them, as Immer's do. -Each scenario detects the loss of the improvement it covers. Built from the commit that made each improvement and from the commit before it, with Mutative alone in three alternating processes at 1,000 rows, the earlier build took 62–65 times as long in `set-read` in every mode (39.8 µs against 0.62 µs without freezing or patches), 3.5 times in `map-forEach` (91.9 µs against 25.9 µs), 11 times in `shift-and-update` without freezing or patches (32.7 µs against 2.94 µs) and 1.5 times with freezing alone, and 2.0–2.3 times in `map-set-beside` with auto-freeze (311 µs against 135 µs without patches). The CI budgets fail a cell at 1.30 times. +Each scenario detects the loss of the improvement it covers. Built from the commit that made each improvement and from the commit before it, with Mutative alone in three alternating processes at 1,000 rows, the earlier build took 62–65 times as long in `set-read` in every mode (39.7 µs against 0.62 µs without freezing or patches), 3.5 times in `map-forEach` (91.9 µs against 25.9 µs), 11 times in `shift-and-update` without freezing or patches (32.7 µs against 2.94 µs) and 1.5 times with freezing alone, and 2.0–2.3 times in `map-set-beside` with auto-freeze (311 µs against 135 µs without patches). The CI budgets fail a cell at 1.30 times. ### Selected times @@ -252,67 +246,67 @@ Microseconds per complete scenario, medians of three runs. | Scenario | Rows | Freeze | Patches | Mutative | Mutative 1.3.0 | Immer | Hand-written | Immer/Mutative | | --- | ---: | --- | --- | ---: | ---: | ---: | ---: | ---: | -| small-object-update | 100 | off | off | 0.51 | 0.81 | 0.70 | 0.02 | 1.37 | -| read-index | 100 | off | off | 45.8 | 61.4 | 86.6 | 0.14 | 1.89 | -| rtkq-sequence | 100 | off | off | 1,018 | 1,830 | 1,159 | 355 | 1.14 | -| sortById-reverse | 100 | off | off | 53.0 | 93.7 | 70.4 | 1.22 | 1.33 | -| array-shift-nested | 100 | off | off | 0.72 | 80.0 | 69.3 | 0.05 | 96.76 | -| update-largeObject1 | 100 | off | off | 51.0 | 51.0 | 146 | 129 | 2.87 | -| update-largeObject1 | 100 | on | off | 132 | 138 | 231 | — | 1.75 | -| object-update | 100 | off | off | 3.16 | 6.12 | 3.45 | 2.14 | 1.09 | -| map-update | 100 | off | off | 4.55 | 5.19 | 4.95 | 3.66 | 1.09 | -| set-update | 100 | off | off | 3.39 | 8.98 | 42.0 | 2.14 | 12.37 | -| class-update | 100 | on | off | 3.86 | 5.22 | 9.24 | — | 2.40 | -| apply-array-ops | 100 | off | off | 2.99 | 115 | 100 | — | 33.57 | -| return-filter | 100 | off | off | 23.8 | 26.2 | 32.6 | 0.19 | 1.37 | -| search-current | 100 | off | off | 12.3 | 43.6 | 238 | 0.14 | 19.36 | -| array-splice-insert-nested | 1,000 | off | off | 4.90 | 394 | 352 | 2.03 | 71.98 | -| object-update | 1,000 | off | off | 51.9 | 53.2 | 127 | 129 | 2.45 | -| class-wide-update | 1,000 | on | off | 411 | 409 | 239 | — | 0.58 | -| map-update | 1,000 | off | off | 40.4 | 41.2 | 41.1 | 39.3 | 1.02 | -| array-shift-nested | 10,000 | off | off | 7.10 | 8,548 | 8,605 | 1.40 | 1212.71 | -| array-unshift-nested | 10,000 | off | off | 15.9 | 8,220 | 7,547 | 28.3 | 474.21 | -| array-reverse-nested | 10,000 | off | off | 13.7 | 8,627 | 7,555 | 9.68 | 551.48 | -| array-reverse-nested | 10,000 | off | on | 1,074 | 277,427 | 9,416 | — | 8.76 | -| read-index | 10,000 | off | off | 5,094 | 6,293 | 11,773 | 55.5 | 2.31 | -| set-update | 10,000 | off | off | 544 | 1,057 | 6,119 | 411 | 11.24 | -| object-update | 10,000 | off | off | 1,175 | 1,189 | 2,109 | 2,113 | 1.79 | -| apply-reverse | 10,000 | off | off | 9,918 | 13,772 | 13,889 | — | 1.40 | -| push-and-insert | 10,000 | off | on | 63.3 | 208 | 364 | — | 5.76 | -| return-replace | 10,000 | on | off | 0.53 | 2,630 | 0.79 | — | 1.48 | -| search-current-shifted | 10,000 | off | off | 257 | 15,874 | 11,230 | 64.7 | 43.74 | +| small-object-update | 100 | off | off | 0.51 | 0.81 | 0.69 | 0.02 | 1.35 | +| read-index | 100 | off | off | 45.6 | 61.6 | 86.7 | 0.14 | 1.90 | +| rtkq-sequence | 100 | off | off | 1,012 | 1,846 | 1,173 | 352 | 1.16 | +| sortById-reverse | 100 | off | off | 52.5 | 92.7 | 69.3 | 1.20 | 1.32 | +| array-shift-nested | 100 | off | off | 0.73 | 79.7 | 69.2 | 0.05 | 94.23 | +| update-largeObject1 | 100 | off | off | 51.5 | 51.4 | 149 | 127 | 2.90 | +| update-largeObject1 | 100 | on | off | 141 | 129 | 231 | — | 1.64 | +| object-update | 100 | off | off | 3.14 | 6.04 | 3.45 | 2.14 | 1.10 | +| map-update | 100 | off | off | 4.55 | 5.19 | 4.90 | 3.64 | 1.07 | +| set-update | 100 | off | off | 3.33 | 8.94 | 41.6 | 2.14 | 12.50 | +| class-update | 100 | on | off | 3.90 | 5.25 | 9.30 | — | 2.39 | +| apply-array-ops | 100 | off | off | 3.00 | 115 | 99.8 | — | 33.27 | +| return-filter | 100 | off | off | 24.0 | 26.3 | 32.8 | 0.19 | 1.37 | +| search-current | 100 | off | off | 12.2 | 43.3 | 232 | 0.14 | 19.08 | +| array-splice-insert-nested | 1,000 | off | off | 4.93 | 393 | 355 | 2.04 | 72.10 | +| object-update | 1,000 | off | off | 50.9 | 51.2 | 129 | 125 | 2.53 | +| class-wide-update | 1,000 | on | off | 415 | 410 | 237 | — | 0.57 | +| map-update | 1,000 | off | off | 39.9 | 40.3 | 40.2 | 38.4 | 1.01 | +| array-shift-nested | 10,000 | off | off | 7.07 | 8,184 | 7,426 | 1.41 | 1050.79 | +| array-unshift-nested | 10,000 | off | off | 15.9 | 8,573 | 7,572 | 28.4 | 476.22 | +| array-reverse-nested | 10,000 | off | off | 13.8 | 8,504 | 7,577 | 9.70 | 550.42 | +| array-reverse-nested | 10,000 | off | on | 1,073 | 276,034 | 9,525 | — | 8.88 | +| read-index | 10,000 | off | off | 4,766 | 6,465 | 11,933 | 54.8 | 2.50 | +| set-update | 10,000 | off | off | 533 | 1,036 | 6,014 | 406 | 11.28 | +| object-update | 10,000 | off | off | 1,199 | 1,195 | 2,102 | 2,091 | 1.75 | +| apply-reverse | 10,000 | off | off | 9,928 | 13,892 | 13,865 | — | 1.40 | +| push-and-insert | 10,000 | off | on | 63.0 | 210 | 364 | — | 5.78 | +| return-replace | 10,000 | on | off | 0.53 | 2,626 | 0.79 | — | 1.48 | +| search-current-shifted | 10,000 | off | off | 257 | 15,961 | 11,379 | 23.4 | 44.25 | ### Against a hand-written reducer -The hand-written reducer is faster than the candidate in 129 of 150 cells, and across all 150 the candidate takes 8.3 times as long on geometric mean: the reducer copies only what each update changes and has no draft to create, track, or finalize. The candidate is faster in 18 cells. Fifteen copy wide objects: objects with 1,000 and 3,000 properties, records with 1,000 or 10,000 keys, and class instances with 100 or more fields, by 1.5–2.7 times. One inserts at the front of an array of 10,000 rows, where the reducer's spread copies every element and `unshift` runs natively on the draft's copy: 15.9 µs against 28.3 µs; at 100 and 1,000 rows the reducer was faster. Two add a number to a Set of 1,000 or 10,000 numbers, which the candidate copies with `Set.prototype.difference`, faster than the reducer's `new Set()`: 1.28 µs against 4.10 µs and 34.1 µs against 66.0 µs; at 100 numbers the reducer was faster, 0.44 µs against 0.70 µs. Copying a Map of 1,000 or 10,000 entries dominates `map-update` in both, which are within 5%, and so does copying a Map and a Set of 10,000 entries in `map-set-beside`. The hand-written reducer is the same spread-based code that validates every scenario; faster hand-written code exists for many of them. +The hand-written reducer is faster than the candidate in 129 of 150 cells, and across all 150 the candidate takes 8.3 times as long on geometric mean: the reducer copies only what each update changes and has no draft to create, track, or finalize. The candidate is faster in 18 cells. Fifteen copy wide objects: objects with 1,000 and 3,000 properties, records with 1,000 or 10,000 keys, and class instances with 100 or more fields, by 1.5–2.8 times. One inserts at the front of an array of 10,000 rows, where the reducer's spread copies every element and `unshift` runs natively on the draft's copy: 15.9 µs against 28.4 µs; at 100 and 1,000 rows the reducer was faster. Two add a number to a Set of 1,000 or 10,000 numbers, which the candidate copies with `Set.prototype.difference`, faster than the reducer's `new Set()`: 1.29 µs against 4.06 µs and 30.9 µs against 63.2 µs; at 100 numbers the reducer was faster, 0.44 µs against 0.69 µs. Copying a Map of 1,000 or 10,000 entries dominates `map-update` in both, which are within 5%, and so does copying a Map and a Set of 10,000 entries in `map-set-beside`. The hand-written reducer is the same spread-based code that validates every scenario; faster hand-written code exists for many of them. ### Shared and isolated processes -Earlier batches ran every scenario except the wide-object ones with all four libraries together, in one process per run and a second one for the Map and Set scenarios. This batch measured every cell both ways, and in a process of its own every library was faster, by different amounts. This comparison uses the cells of October 6 and 7, before the Set scenarios were measured again. Over all 566 cells, the shared-process times were longer on geometric mean by 1.08 times for the candidate, 1.15 times for Mutative 1.3.0 and 1.12 times for Immer, and over the 138 cells of the hand-written reducer by 1.67 times; at the 95th percentile by 1.26, 1.42, 1.45 and 5.67 times. In a shared process the recipes, the hand-written reducer and the libraries handle the objects of every scenario, so V8 optimizes them for many object shapes at once; in a process of its own each handles the shapes of one scenario. Small code gains the most: the hand-written reducer summed 100 rows in 0.14 µs alone and in 0.71 µs beside the others. Immer gained more than the candidate, most of all in Set updates: adding a number to a 100-item Set took Immer 5.9 µs alone and 14.0 µs beside the others, and the candidate 6.4 µs and 6.5 µs. +Batches before the previous one ran every scenario except the wide-object ones with all four libraries together, in one process per run and a second one for the Map and Set scenarios. This batch measured every cell both ways, and in a process of its own every library was faster, by different amounts. Over all 614 cells, the shared-process times were longer on geometric mean by 1.09 times for the candidate, 1.14 times for Mutative 1.3.0 and 1.15 times for Immer, and over the 150 cells of the hand-written reducer by 1.64 times; at the 95th percentile by 1.31, 1.39, 1.77 and 5.53 times. In a shared process the recipes, the hand-written reducer and the libraries handle the objects of every scenario, so V8 optimizes them for many object shapes at once; in a process of its own each handles the shapes of one scenario. Small code gains the most: the hand-written reducer summed 100 rows in 0.14 µs alone and in 0.71 µs beside the others. Immer gained more than the candidate, most of all in Set updates: adding a number to a 100-item Set took Immer 5.85 µs alone and 13.9 µs beside the others, and the candidate 0.69 µs and 0.79 µs. | Comparator | All in shared processes | Isolated processes | | --- | ---: | ---: | -| Immer 11.1.18 | 551 / 12 / 3 of 566, 3.50 | 536 / 21 / 9 of 566, 3.37 | -| Immer 11.1.18 with its defaults | 142 / 0 / 3 of 145, 6.10 | 141 / 1 / 3 of 145, 5.98 | -| Pinned npm Mutative 1.3.0 | 497 / 61 / 8 of 566, 3.79 | 470 / 95 / 1 of 566, 3.57 | -| Hand-written reducer | 17 / 1 / 120 of 138, 0.16 | 16 / 2 / 120 of 138, 0.11 | +| Immer 11.1.18 | 603 / 11 / 0 of 614, 3.79 | 596 / 16 / 2 of 614, 3.58 | +| Immer 11.1.18 with its defaults | 154 / 0 / 3 of 157, 6.82 | 154 / 0 / 3 of 157, 6.62 | +| Pinned npm Mutative 1.3.0 | 570 / 40 / 4 of 614, 4.48 | 538 / 73 / 3 of 614, 4.27 | +| Hand-written reducer | 19 / 2 / 129 of 150, 0.18 | 18 / 3 / 129 of 150, 0.12 | -Most of the difference against Immer lies at 100 rows (3.08 shared, 2.92 isolated); at 1,000 and 10,000 rows the geometric means are 4.43 and 4.39. The isolated processes give the more even comparison between libraries, and every result above uses them. Each one measures a workload alone, as a microbenchmark does; in an application, the application's own code shares V8's state as the other libraries did here. +Most of the difference against Immer lies at 100 rows (3.25 shared, 3.04 isolated); at 1,000 and 10,000 rows the geometric means are 4.87 and 4.67. The isolated processes give the more even comparison between libraries, and every result above uses them. Each one measures a workload alone, as a microbenchmark does; in an application, the application's own code shares V8's state as the other libraries did here. -Inserting a property into the 1,000-property object also depended on the order of the code that ran before it in a shared process. With libraries in alternating orders in six processes, the hand-written reducer took 1.70–1.73 µs when it ran after the libraries and 130–133 µs when it ran first; Immer 2.69–2.70 µs after the hand-written reducer and 152–155 µs after Mutative; the candidate 53.9–57.6 µs first and 97.9–103 µs after the others. In processes of their own the candidate and Mutative 1.3.0 took 51.0 µs, Immer 146 µs, and the hand-written reducer 129 µs. A separate effect made the first timed run of a scenario in a process slower: updating a 100-key record took 12–15 µs in every library there and 3–4 µs afterwards. Priming each benchmark removed it. +Inserting a property into the 1,000-property object also depended on the order of the code that ran before it in a shared process. With libraries in alternating orders in six processes, the hand-written reducer took 1.66–1.69 µs when it ran after the libraries and 124–130 µs when it ran first; Immer 2.68–2.77 µs after the hand-written reducer and 146–148 µs after Mutative; the candidate 50.9–54.7 µs first and 94.2–98.1 µs after the others. In processes of their own the candidate took 51.5 µs, Mutative 1.3.0 51.4 µs, Immer 149 µs, and the hand-written reducer 127 µs. A separate effect made the first timed run of a scenario in a process slower: updating a 100-key record took 12–15 µs in every library there and 3–4 µs afterwards. Priming each benchmark removed it. ### Memory -Sampled allocation of the candidate was below Mutative 1.3.0 in 107 of 114 memory cells. The exceptions are `noop-empty` in three modes and `small-object-update` with patches, by at most 0.2 KiB; and `map-update` at 10,000 entries, by 0.4% with patches and without freezing, and by 20–29% with auto-freeze (3.7 MiB against 2.9–3.1 MiB). That last difference depends on what a worker runs before it measures: in workers that validate only the measured patch mode, the current source and that of the previous batch both allocated 2.9 MiB, and the cell's time equals that of 1.3.0 (692 µs against 691 µs). It was below Immer in 77 of 114 cells: lower for reads, array moves, Set additions, returned filters, and patch application; higher for Map drafts (up to 2.3 times), the push-and-insert fixture (1.1–2.6 times), the 1,000-key record (113 against 16 KiB), and by a few KiB for no-op, small-object, record, and class-instance producers. Objects with more than 128 keys are copied into dictionary-mode objects, which also retain 48 KiB per output at 1,000 keys, against 8 KiB for the other libraries. +Sampled allocation of the candidate was below Mutative 1.3.0 in 109 of 114 memory cells. The exceptions are `map-update` at 100 and 1,000 entries in three modes, by 0.7–1.6%, `object-update` at 10,000 keys with patches, by 0.1%, and `map-update` at 10,000 entries with auto-freeze and patches, by 28% (3.7 MiB against 2.9 MiB); with auto-freeze and without patches, the candidate allocated 2.9 MiB against 3.1 MiB, and the time of both cells equals that of 1.3.0 (704 µs against 705 µs and 712 µs). It was below Immer in 80 of 114 cells: lower for reads, array moves, Set additions, returned filters, and patch application; higher for Map drafts (up to 2.4 times), the push-and-insert fixture (1.1–2.6 times), the 1,000-key record (113 against 16 KiB), and by a few KiB for no-op, small-object, record, and class-instance producers. Objects with more than 128 keys are copied into dictionary-mode objects, which also retain 48 KiB per output at 1,000 keys, against 8 KiB for the other libraries. | Scenario | Rows | Patches | Candidate | Mutative 1.3.0 | Immer | Hand-written | | --- | ---: | --- | ---: | ---: | ---: | ---: | -| read-index | 100 | off | 98.6 | 150.1 | 192.3 | 1.7 | -| push-and-insert | 100 | off | 115.2 | 1,365.6 | 66.0 | 19.3 | -| object-update | 1,000 | off | 112.9 | 1,311.7 | 15.9 | 10.9 | -| set-add | 10,000 | off | 325.4 | 2,878.1 | 1,606.8 | 321.7 | -| array-reverse-nested | 10,000 | off | 89.0 | 11,077.6 | 12,840.4 | 83.4 | -| array-reverse-nested | 10,000 | on | 3,028.5 | 268,460.6 | 15,102.0 | — | +| read-index | 100 | off | 98.4 | 150.5 | 194.9 | 1.8 | +| push-and-insert | 100 | off | 114.8 | 1,365.0 | 65.6 | 19.2 | +| object-update | 1,000 | off | 113.2 | 1,312.2 | 16.3 | 11.0 | +| set-add | 10,000 | off | 330.2 | 2,882.6 | 1,613.6 | 325.5 | +| array-reverse-nested | 10,000 | off | 87.9 | 11,090.0 | 12,807.4 | 83.8 | +| array-reverse-nested | 10,000 | on | 3,020.3 | 268,357.3 | 15,068.1 | — | Sampled KiB allocated per iteration with freeze off, medians of three isolated workers. With patches, nested reverse at 10,000 rows retains about 1,797 KiB per output, against 9,891 KiB for Mutative 1.3.0 and 1,797 KiB for Immer. @@ -322,54 +316,47 @@ The plugin datasets rerun the array-related scenarios with Immer's array-method | Comparator | Faster / within 5% / slower | Geometric mean Immer/candidate | | --- | ---: | ---: | -| Immer 11.1.18 with `enableArrayMethods`, all plugin cells | 264 / 6 / 58 of 328 | 1.86 | -| Array-method scenarios only | 158 / 1 / 13 of 172 | 2.37 | +| Immer 11.1.18 with `enableArrayMethods`, all plugin cells | 265 / 6 / 57 of 328 | 1.87 | +| Array-method scenarios only | 158 / 1 / 13 of 172 | 2.35 | | Scenario | Rows | Freeze | Patches | Candidate | Immer + plugin | Immer/candidate | | --- | ---: | --- | --- | ---: | ---: | ---: | -| array-shift-nested | 10,000 | off | off | 7.06 | 396 | 56.10 | -| array-shift-nested | 10,000 | off | on | 1,069 | 1,380 | 1.29 | -| array-shift-nested | 10,000 | on | off | 182 | 1,133 | 6.23 | -| array-reverse-nested | 10,000 | off | off | 13.8 | 385 | 27.97 | -| array-reverse-nested | 10,000 | off | on | 1,073 | 1,375 | 1.28 | -| array-splice-insert-nested | 10,000 | off | off | 20.9 | 414 | 19.79 | -| array-reverse-primitive | 10,000 | off | off | 13.7 | 369 | 26.93 | -| array-shift-nested | 1,000 | off | on | 110 | 125 | 1.14 | -| array-shift-nested | 100 | off | off | 0.73 | 2.58 | 3.55 | -| array-push-primitive | 100 | off | off | 1.08 | 0.79 | 0.73 | -| array-sort-nested | 100 | off | off | 90.3 | 8.46 | 0.09 | -| search-draft | 10,000 | off | off | 2,135 | 91.1 | 0.04 | -| read-missing | 10,000 | off | off | 2,006 | 69.4 | 0.03 | - -Immer with the plugin is faster in 58 cells. Most are scenarios where the plugin hands raw base elements to a callback or comparator, which Mutative never does: `find` in `read-missing` (25–29 times at 1,000 and 10,000 rows), `findIndex` in `remove-high`, `remove-high-reuse` and `search-draft` (up to 23 times at 10,000 rows), `filter` in `filter` and `return-filter`, and sorts of objects in `sortById-reverse`, `array-sort-shallow` and `array-sort-nested`. The others are `push` and `pop` on 100 rows with freeze and patches off, which Mutative leaves on the proxy path: 1.1–1.3 µs against 0.7–1.0 µs, and `add`, which pushes a row. Every `shift`, `unshift`, `splice` and `reverse` cell is faster than Immer with the plugin; at 10,000 rows by 20–56 times with freeze and patches off, 5.5–11 times with freeze on, and 1.3–2.1 times with patches on. So is every cell of `shift-and-update`, by 1.1–24 times. +| array-shift-nested | 10,000 | off | off | 7.10 | 385 | 54.24 | +| array-shift-nested | 10,000 | off | on | 1,067 | 1,390 | 1.30 | +| array-shift-nested | 10,000 | on | off | 185 | 1,129 | 6.11 | +| array-reverse-nested | 10,000 | off | off | 13.8 | 396 | 28.77 | +| array-reverse-nested | 10,000 | off | on | 1,077 | 1,381 | 1.28 | +| array-splice-insert-nested | 10,000 | off | off | 20.9 | 409 | 19.57 | +| array-reverse-primitive | 10,000 | off | off | 13.7 | 373 | 27.21 | +| array-shift-nested | 1,000 | off | on | 109 | 124 | 1.14 | +| array-shift-nested | 100 | off | off | 0.74 | 2.51 | 3.38 | +| array-push-primitive | 100 | off | off | 1.07 | 0.78 | 0.73 | +| array-sort-nested | 100 | off | off | 90.4 | 8.54 | 0.09 | +| search-draft | 10,000 | off | off | 2,190 | 91.1 | 0.04 | +| read-missing | 10,000 | off | off | 2,026 | 69.2 | 0.03 | + +Immer with the plugin is faster in 57 cells. Most are scenarios where the plugin hands raw base elements to a callback or comparator, which Mutative never does: `find` in `read-missing` (25–29 times at 1,000 and 10,000 rows), `findIndex` in `remove-high`, `remove-high-reuse` and `search-draft` (up to 24 times at 10,000 rows), `filter` in `filter` and `return-filter`, and sorts of objects in `sortById-reverse`, `array-sort-shallow` and `array-sort-nested`. The others are `push` and `pop` on 100 rows with freeze and patches off, which Mutative leaves on the proxy path: 1.1–1.3 µs against 0.7–1.0 µs, and `add`, which pushes a row. Every `shift`, `unshift`, `splice` and `reverse` cell is faster than Immer with the plugin; at 10,000 rows by 20–54 times with freeze and patches off, 5.7–11 times with freeze on, and 1.3–2.1 times with patches on. So is every cell of `shift-and-update`, by 1.1–24 times. ### Paired performance budget -The local budget gate compared the current build with itself at 1,000 rows, since this batch changes no Mutative source file. All 48 decisions passed: five latency pairs and three memory pairs per cell, both freeze and patch modes. Paired candidate/base median ratios span 0.970–1.023 for sampled allocation and 1.000 for retained heap. For latency they span 0.995–1.036 in 18 of the 32 cells; the 14 others are array moves with patches (1.17–1.26) or with freeze on (1.07–1.09), which four of the five candidate processes and one of the five base processes ran more slowly. The two builds were identical, so this is variance between processes, and it came close to the 1.30 budget. The timing matrices did not show it: the three process means of each of those cells at 1,000 rows stayed within 4% of each other. +The local budget gate compared the current build with itself at 1,000 rows, since this batch changes no Mutative source file. All 222 decisions of the five budget groups passed: five latency pairs and three memory pairs per cell, both freeze and patch modes. The latency ratios of the fastest processes spanned 0.875–1.189; above 1.10 were only `search-current-shifted` with patches (1.19 with and without freezing) and `push-and-insert` with freeze on (1.18 with patches, 1.10 without), below the 1.30 budget. The medians of the paired ratios spanned 0.977–1.061 for sampled allocation and 1.000–1.039 for retained heap. ### Changes since the previous batch -This comparison uses the shared-process matrices that the previous batch published, with the wide-object scenarios in processes of their own in both batches, so that each cell compares the same setup. Over the 530 cells that both batches measured, the candidate's time fell by 11% on geometric mean (previous/current time 1.12: 1.07 at 100 rows, 1.17 at 1,000 and 1.26 at 10,000). Immer and Mutative 1.3.0, whose artifacts did not change, measured 1.02 times faster than in the previous session, which bounds the drift between the two sessions. In 122 cells every process of this batch was faster than every process of the previous one, among them: +This comparison uses the cells that the previous summary published: the batch of October 6 and 7, with the Set and patch application cells that PRs #195 and #196 measured again on October 9 and the cells that PR #198 added the same day. Over all 614 cells, the candidate's time changed by less than 1% on geometric mean (previous/current time 1.004), as did those of Immer (1.008), Mutative 1.3.0 (1.007) and the hand-written reducer (1.003), whose code did not change; the drift between the sessions is of that size. The process means of most cells are steady enough that such a drift separates them: in 98 cells every process of this batch was faster than every process of the previous measurement and in 37 slower, 128 of the 135 by less than 3%, and the cells of Immer and Mutative 1.3.0 separated as often, 123 and 17 times and 121 and 13 times. -- Returned values with auto-freeze: `return-replace` took 0.57 µs at every size, against 30.0 µs, 291 µs and 2.9 ms. Production builds with auto-freeze skip a frozen returned value instead of searching it (PR #183). -- Patch generation: with patches and without freezing, `deep-update` took 3.98 µs instead of 7.77 µs, `object-update-10pct` 10.5 µs instead of 16.8 µs, `update` 1.72 µs instead of 2.17 µs, and `mutation-density-100pct` 81.8 µs instead of 110 µs at 100 rows and 9.29 ms instead of 11.6 ms at 10,000. A changed draft's patch path is checked once per level against its parent's copy instead of being resolved again from the root of the state (PR #184). -- Map reads: `map-read` took 31.6 µs instead of 39.4 µs at 100 rows and 3.56 ms instead of 4.33 ms at 10,000. Map drafts iterate without calling `get()` through the proxy (PR #184). -- Arrays with auto-freeze: most array operations on 100 rows with freeze on took about 10% less. Frozen arrays are copied with a spread (PR #183). -- Patch application: `apply-array-ops` took 4.36 µs instead of 6.87 µs. `apply()` copies object patch values with a spread (PR #184). -- `search-current-shifted` at 10,000 rows took 331 µs, against 772 µs when PR #180 was measured. The moved row's original index is found by searching the original array (PR #183). - -In 10 cells every process was slower than every process of the previous batch: `add` and `reverse-array` with freeze and patches on at 100 rows (21.9 µs against 18.1 µs, and 27.6 µs against 24.4 µs), and at 1,000 rows `map-update` in all four modes, `set-add` in three and `map-read` in one, by 3–8%. They did not reproduce with Mutative alone in each process. Same-session runs of benchmark bundles built from `d3cb6e6`, whose production artifact is byte-identical to that of `e6b6563`, and from `2607b22`, `dd5118e`, `ba8d0cd`, `29ad35b` and `2ab58fc`, with Mutative alone in each process and rotating order, measured the previous and current sources within 2% of each other in the Map and Set cells at 1,000 rows, for example `map-update` 39.6 µs against 39.8 µs and `set-add` 66.7 µs against 66.5 µs, medians of three rounds; and `add` with freeze and patches on at 19.4 µs against 19.3 µs over five more rounds. In processes of their own, the candidate's Map and Set updates at 1,000 rows were as fast as those of Mutative 1.3.0 or up to 3.4% faster. These cells depend on the code that runs beside them in a process. +The source changes since `2ab58fc` outside the cells that PRs #195 and #196 measured again are the fixes of PRs #190 and #192. Among the candidate's cells that moved by more than 10%, only `add` with freeze on at 100 rows was slower in every process: 19.5 µs against 15.9 µs without patches. Ten alternating rounds of benchmark bundles built from `2ab58fc` and `ce0c25e`, with Mutative alone in each process, measured 18.9 µs against 19.4 µs with overlapping ranges, and 1.52 µs for both without freezing, so those fixes did not slow it; with freeze on, the processes of this cell varied by up to 1.3 times in both batches. `concat`, `remove` and `update` with freeze on at 100 rows measured 10–19% faster, with processes that overlapped those of the previous batch, and `set-add` at 10,000 items without freezing or patches took 30.9 µs instead of 34.1 µs in every process, with the same Set code that PR #195 measured. ## Tradeoffs and limits -- Returned values: without auto-freeze, the candidate searches every object of a returned value for drafts, as Immer does: `return-replace` took 2.59 ms at 10,000 rows, against 5.63 ms for Immer without auto-freeze. Immer's default auto-freeze skips frozen values (0.79 µs), and so do production builds of the candidate with auto-freeze (0.53 µs). Wrapping the value in `rawReturn()` skips the search (0.26 µs). -- A Set draft copies the Set on its first change and rebuilds it in order once an item that the recipe read through an iterator changed: `set-update` took 544 µs at 10,000 items, against 411 µs for the hand-written reducer, which copies the Set once. With patches, a Set that added or deleted items compares every original item with its copy and every item of the copy with the original: `set-add` took 448 µs with patches at 10,000 items, against 34.1 µs without. Copying the Map dominates `map-update` at 10,000 entries in every library, about 520 µs. -- With auto-freeze, updating a row that `shift()` moved costs more than the move: `shift-and-update` took 301 µs at 10,000 rows, against 189 µs for `array-shift-nested`, which only removes the row, while without auto-freeze they took 15.9 µs and 7.10 µs. The candidate finds the moved row's original index with `lastIndexOf` on the original array, which is frozen then, and V8 runs `lastIndexOf` on a frozen array about 15 times as slowly: 116 µs against 7.9 µs to find an element in the middle of 10,000, while `indexOf` takes 1.1 µs on both. -- With auto-freeze, Immer updated a class instance with 1,000 fields faster: 239 µs against 411 µs. Without freezing the candidate is faster, 98 µs against 206 µs, so the difference lies in freezing the candidate's copy, which it makes property by property with descriptors where Immer 11 uses `Object.assign`. -- Even in processes of their own, some cells varied between processes of the same build by up to about 1.5 times, for example `remove-reuse` with freeze on (127–191 µs) and Immer's `map-read` at 10,000 rows; `class-wide-update` without freezing at 1,000 rows took either about 95 µs or about 200 µs in both Mutative builds. Medians of three processes absorb one outlier, not two; compare such cells through geometric means rather than one by one. +- Returned values: without auto-freeze, the candidate searches every object of a returned value for drafts, as Immer does: `return-replace` took 2.61 ms at 10,000 rows, against 5.73 ms for Immer without auto-freeze. Immer's default auto-freeze skips frozen values (0.79 µs), and so do production builds of the candidate with auto-freeze (0.53 µs). Wrapping the value in `rawReturn()` skips the search (0.26 µs). +- A Set draft copies the Set on its first change and rebuilds it in order once an item that the recipe read through an iterator changed: `set-update` took 533 µs at 10,000 items, against 406 µs for the hand-written reducer, which copies the Set once. With patches, a Set that added or deleted items compares every original item with its copy and every item of the copy with the original: `set-add` took 444 µs with patches at 10,000 items, against 30.9 µs without. Copying the Map dominates `map-update` at 10,000 entries in every library, about 525 µs. +- With auto-freeze, updating a row that `shift()` moved costs more than the move: `shift-and-update` took 300 µs at 10,000 rows, against 189 µs for `array-shift-nested`, which only removes the row, while without auto-freeze they took 15.8 µs and 7.07 µs. The candidate finds the moved row's original index with `lastIndexOf` on the original array, which is frozen then, and V8 runs `lastIndexOf` on a frozen array about 15 times as slowly: 116 µs against 7.9 µs to find an element in the middle of 10,000, while `indexOf` takes 1.1 µs on both. +- With auto-freeze, Immer updated a class instance with 1,000 fields faster: 237 µs against 415 µs. Without freezing the candidate is faster, 99.7 µs against 205 µs, so the difference lies in freezing the candidate's copy, which it makes property by property with descriptors where Immer 11 uses `Object.assign`. +- Even in processes of their own, some cells varied between processes of the same build by up to about 1.6 times, for example `update-reuse` with freeze and patches on (170–240 µs) and Immer's `map-read` at 10,000 rows (5.1–8.0 ms); `class-wide-update` without freezing at 1,000 rows took either about 95 µs or about 200 µs in both Mutative builds. The freeze-on upstream workloads at 100 rows vary most among the candidate's cells, which is why `add` and `update-largeObject1` with freeze on are slower than Mutative 1.3.0 in this batch. Medians of three processes absorb one outlier, not two; compare such cells through geometric means rather than one by one. - Isolated processes measure each library without the other libraries and scenarios, not inside a real application, whose own code also changes V8's state; wide-object timings in particular depend on it, by up to about 70 times. - Freeze-on inputs are already frozen, so freezing cold external data is excluded. Allocation is sampled, and small retained deltas are noisy. All measurements come from one machine and Node.js; browser engines may differ. -- Bundle size: as the README measures it with esbuild 0.24.0, importing only `create` bundles to 7.7 kB with Brotli against 3.6 kB for Immer's `produce`, and with patches, Map and Set support and array methods to 8.2 kB against 6.4 kB for Immer with its plugins. `size-limit` measures the production CJS artifact at 8.3 kB against its 8.4 kB cap (8,114 Brotli bytes), and the ESM consumers of `create` and of all exports at 7.27 kB and 8.16 kB against 7.4 kB and 8.3 kB. +- Bundle size: as the README measures it with esbuild 0.24.0, importing only `create` bundles to 7.8 kB with Brotli against 3.6 kB for Immer's `produce`, and with patches, Map and Set support and array methods to 8.4 kB against 6.4 kB for Immer with its plugins. `size-limit` measures the production CJS artifact at 8.56 kB against its 8.6 kB cap (8,367 Brotli bytes), and the ESM consumers of `create` and of all exports at 7.5 kB and 8.41 kB against 7.5 kB and 8.5 kB. ## Native array method contract @@ -387,13 +374,13 @@ The fast paths are made for data arrays. On them, values, own-property presence ## Source and artifact identity -The measured source is that of [`2ab58fc`](https://github.com/unadlib/mutative/commit/2ab58fce1210b2b4724fcecce10ff0e1a864ddce), with the harness of the same commit. Every dataset reports the same source and production hashes. +The measured source is that of [`ce0c25e`](https://github.com/unadlib/mutative/commit/ce0c25e9f1efb43e735ad4d2abe0e03ce29b1161), with the harness of the same commit. Every dataset reports the same source and production hashes. The production artifact is the one that PR #196 measured, since PRs #197 and #198 changed no source file. SHA-256 identities (`source` uses the sorted path/content algorithm in [`build.mjs`](../build.mjs); `production` is `dist/mutative.cjs.production.min.js`): ```text -Candidate source: 4f4ce89ca64753973262e952f3ade4bde8632d20941bee0b23f7cf6452a7cc94 -Candidate production: f210318adca2f60d6d0c5bd807dcb76ff34d345120e779092facc264f453666d +Candidate source: 763c08619d0a1892c3dfd9e4cbbc165763fcd71eb263481d2ca793deae09c5ff +Candidate production: bd8ff3a63a6007bb800f8782985e6156c57aac1bbd0c8f7251708e863c2c19c9 Pinned v1 production: 15ad9df11178c64f80e66797ffe784a4eeedfb03759ca59ef61a64f4142fde41 Immer production: 30fec64eb16c235f5fe771ed7e4be08eb0f67cb46c44917424a6e643f3d677b9 ``` @@ -412,14 +399,7 @@ PR #196 production: bd8ff3a63a6007bb800f8782985e6156c57aac1bbd0c8f7251708e863c main production: 74adeacf19aaa8df15d94751824c3d8f12dd821f854eb8a9f137ee9daff41d05 ``` -The four scenarios of PR #198 measured the source of `main` at `d96aa98` with the harness of that PR. Its production artifact is that of PR #196, since PR #197 changed only documentation: - -```text -Candidate source: 763c08619d0a1892c3dfd9e4cbbc165763fcd71eb263481d2ca793deae09c5ff -Candidate production: bd8ff3a63a6007bb800f8782985e6156c57aac1bbd0c8f7251708e863c2c19c9 -``` - -The comparison of each scenario with the commit before the improvement it covers built `538b7ae` and `715a41f` for `set-read`, `bad9902` and `13460fb` for `shift-and-update`, `00d4829` and `2168780` for `map-set-beside`, and `0bf2740` and `ce20399` for `map-forEach`. +The comparison of `add` in [changes since the previous batch](#changes-since-the-previous-batch) built `2ab58fc`, whose production artifact (`f210318a…`) is the one the previous batch measured, and `ce0c25e`. The comparison of each scenario of PR #198 with the commit before the improvement it covers built `538b7ae` and `715a41f` for `set-read`, `bad9902` and `13460fb` for `shift-and-update`, `00d4829` and `2168780` for `map-set-beside`, and `0bf2740` and `ce20399` for `map-forEach`. ## Reproduce @@ -449,6 +429,8 @@ for rows in 1000 10000; do done ``` +This batch ran the isolated command at 100 rows in five groups of scenarios, one after another, with the same options. + The shared-process control, the process-order control, and the Immer plugin datasets in shared processes: ```sh @@ -461,18 +443,12 @@ for rows in 1000 10000; do done ``` -The comparison of sources in [changes since the previous batch](#changes-since-the-previous-batch) builds each source in a checkout of its own, as the PR #184 matrix in the history did, and runs the resulting bundles with Mutative alone, rotating their order: +The comparison of `add` in [changes since the previous batch](#changes-since-the-previous-batch) and that of each scenario of PR #198 with the commit before the improvement it covers build each source in a checkout of its own and run the resulting bundles with Mutative alone, in alternating order: ten rounds for `add`, three for each scenario of PR #198 at 1,000 rows. ```sh # For each source, from a built checkout of it at $CHECKOUT: MUTATIVE_PERF_CANDIDATE_DIR="$CHECKOUT" MUTATIVE_PERF_BUILD_DIR="perf-testing/dist/$NAME" node perf-testing/build.mjs -MUTATIVE_PERF_BUNDLE="perf-testing/dist/$NAME/immutability-benchmarks.mjs" node perf-testing/run-benchmarks.mjs --library mutative --runs 1 --array-size 1000 --freeze both --patches both --filter '^(map-update|set-add|set-update)$' -MUTATIVE_PERF_BUNDLE="perf-testing/dist/$NAME/immutability-benchmarks.mjs" node perf-testing/run-benchmarks.mjs --library mutative --runs 1 --freeze on --patches both --filter '^(add|reverse-array)$' -``` - -The comparison of each scenario of PR #198 with the commit before the improvement it covers builds both commits the same way, with the harness of PR #198, and runs that scenario alone in three rounds of alternating order: - -```sh +MUTATIVE_PERF_BUNDLE="perf-testing/dist/$NAME/immutability-benchmarks.mjs" node perf-testing/run-benchmarks.mjs --library mutative --runs 1 --freeze both --patches both --filter '^add$' MUTATIVE_PERF_BUNDLE="perf-testing/dist/$NAME/immutability-benchmarks.mjs" node perf-testing/run-benchmarks.mjs --library mutative --runs 1 --array-size 1000 --freeze both --patches both --filter '^set-read$' ``` @@ -483,6 +459,7 @@ Earlier versions of this summary, in Git history, describe each round in detail. - 2026-10-09, PR #198: the scenarios `set-read`, `map-forEach`, `shift-and-update` and `map-set-beside` cover improvements of PRs #183 and #184 that only micro-benchmarks had measured, and CI budgets them. Measured with every library in processes of its own at 100, 1,000 and 10,000 rows, their cells join the results: 595 / 16 / 3 of 614 against Immer (geometric mean 3.59), 153 / 1 / 3 of 157 with each library's defaults (6.66), 543 / 70 / 1 against 1.3.0 (4.28), and 18 / 3 / 129 against the hand-written reducer. Built from the commit before the improvement it covers, each scenario took at least 2.3 times as long in one or more modes, and `set-read` 62–65 times as long in every mode. - 2026-10-09, PR #196: `apply()` reads the type of each draft on a patch path from its state instead of asking its proxy twice per step. Against `main` at `4ca6345`, Mutative alone, the PR took 0.68 of the time over 14 patch application cells, and `apply-update-10pct` 0.49–0.61 of it. The patch application scenarios, measured again with every library in processes of its own, replace the earlier cells: 551 / 12 / 3 of 566 against Immer (geometric mean 3.66), 142 / 0 / 3 of 145 with each library's defaults (6.76), and 495 / 70 / 1 against 1.3.0 (3.88). The production CJS artifact grew from 8,334 to 8,367 Brotli bytes. - 2026-10-09, PR #195: Set drafts add and delete items in their copy, map only the items that drafts stand for and are rebuilt once, by their own producer, drafts of Set subclasses keep their items in order, and a Set draft that left its key keeps its changes, which 1.3.0 also lost. Against `main` at `e6af4b0`, Mutative alone, the PR took 0.28 of the time over 32 Set cells; `set-add` at 10,000 items took 33.2 µs instead of 987 µs. The Set scenarios, measured again with every library in processes of its own, replace the earlier cells: 548 / 12 / 6 of 566 against Immer (geometric mean 3.62), 142 / 0 / 3 of 145 with each library's defaults (6.62), 495 / 70 / 1 against 1.3.0 (3.84), and 18 / 2 / 118 against the hand-written reducer. The production CJS artifact grew from 8,201 to 8,334 Brotli bytes. +- 2026-10-06 and 2026-10-07, the source of `main` at `2ab58fc`, 93 scenarios with every library in processes of its own for the first time: 536 / 21 / 9 of 566 cells against Immer (geometric mean 3.37), 141 / 1 / 3 of 145 with each library's defaults (5.98), 470 / 95 / 1 against 1.3.0 (3.57), and 16 / 2 / 120 of 138 against the hand-written reducer; in shared processes the same cells gave 551 / 12 / 3 against Immer (3.50). Against the shared-process matrices of 2026-10-03 and 2026-10-04, the candidate's time fell by 11% on geometric mean over 530 cells. PRs #195, #196 and #198 later measured cells again or added them, as their entries describe. - 2026-10-05, PR #184 review fixes: the fixed source at `6b08f57` against `main` at `ba8d0cd`, Mutative only, all 93 scenarios at 100 rows in both freeze and patch modes, three runs each: fixed/main geometric mean 0.982 over 366 cells, 42 cells more than 5% faster and 6 more than 5% slower; the slower wide-object cells measured 0.77–1.02 in isolated processes. Map iteration with the iterator lifetime checks took 1–2.4% longer than without them and about 70% less than `main` for primitive entries. An update beside an unchanged frozen 10,000-entry Map or Set took 0.44 µs instead of 66.7 µs and 26.1 µs. The production CJS artifact grew from 8,079 to 8,106 Brotli bytes. - 2026-10-04, PR #180 at `e70aeed`: `current()` copies changed array drafts from their current array and leaves moved base elements as they are, and strict mode warns once in development builds when a recipe leaves 1,000 or more drafts unchanged. The three `search-*` scenarios were added: a `current()` snapshot search at 10,000 rows took 209 µs instead of 3,262 µs, and 772 µs instead of 7,490 µs after `shift()`. Gate 48 of 48. The production CJS artifact grew from 8,031 to 8,104 Brotli bytes against `main`. - 2026-10-03 and 2026-10-04, the source of `main` at `e6b6563` with the harness of PR #177, 90 scenarios at 100 rows and 23 at 1,000 and 10,000, in shared processes except for the wide-object scenarios: 508 / 11 / 11 of 530 cells against Immer (geometric mean 3.07), 133 / 0 / 3 of 136 with each library's defaults (6.01), 417 / 111 / 2 against pinned 1.3.0 (3.16), and 18 / 2 / 109 of 129 against the hand-written reducer (0.17). Immer was faster in `return-replace` with freeze on (2.9 ms against 0.9 µs at 10,000 rows), `apply-update-10pct` and `class-wide-update` with freeze on at 1,000 rows. With Immer's `enableArrayMethods`, 229 / 5 / 46 of 280 cells (1.63). Allocation below 1.3.0 in 104 of 114 memory cells; gate 48 of 48. From 8cf07a88a7d2ddb7a0b2b510358c6be7bebd0d3c Mon Sep 17 00:00:00 2001 From: unadlib Date: Sat, 10 Oct 2026 09:53:45 +0800 Subject: [PATCH 2/6] docs: refresh the README's benchmark figures from the benchmark of ce0c25e --- README.md | 84 +++++++++++++++++++++++++++---------------------------- 1 file changed, 42 insertions(+), 42 deletions(-) diff --git a/README.md b/README.md index ddcb98de..febeba02 100644 --- a/README.md +++ b/README.md @@ -8,9 +8,9 @@ [![NPM Downloads](https://img.shields.io/npm/dm/mutative)](https://npmtrends.com/mutative) ![license](https://img.shields.io/npm/l/mutative) -**Mutative** - A JavaScript library for efficient immutable updates. In the [benchmark](./perf-testing/reports/SUMMARY.md), it was about 3.6x faster than Immer with the same settings and 6.7x faster with each library's defaults across 97 workloads. +**Mutative** - A JavaScript library for efficient immutable updates. In the [benchmark](./perf-testing/reports/SUMMARY.md), it was about 3.6x faster than Immer with the same settings and 6.6x faster with each library's defaults across 97 workloads. -In that benchmark the gap widened on large arrays, where Mutative moved elements up to 1,213x faster than Immer. When copying dominates, such as updating objects with thousands of keys, inserting at the front of a large array or adding an item to a large Set, Mutative is even faster than hand-written reducers. +In that benchmark the gap widened on large arrays, where Mutative moved elements up to 1,051x faster than Immer. When copying dominates, such as updating objects with thousands of keys, inserting at the front of a large array or adding an item to a large Set, Mutative is even faster than hand-written reducers. **How does Mutative compare with the spread operation (hand-written reducers)?** @@ -35,67 +35,67 @@ This is why Mutative was created. The [benchmark suite](./perf-testing/README.md) times 97 workloads: Immer's own performance tests, array methods, reads, Map and Set values, object records, class instances, a deep path, patch application, returned values, and searches. It compares Mutative with Immer 11.1.18 and with reducers written by hand, after checking every result against those reducers. The [performance summary](./perf-testing/reports/SUMMARY.md) has the complete results, the method, and their limits. -With matched settings, both freezing or both not and both generating patches or both not, Mutative was faster than Immer in 595 of 614 measured cases, 3.6x on geometric mean. With each library's defaults, Mutative without auto-freeze and Immer with it, Mutative was faster in 153 of 157 cases, 6.7x on geometric mean. +With matched settings, both freezing or both not and both generating patches or both not, Mutative was faster than Immer in 596 of 614 measured cases, 3.6x on geometric mean. With each library's defaults, Mutative without auto-freeze and Immer with it, Mutative was faster in 154 of 157 cases, 6.6x on geometric mean. Times are microseconds per update, medians of three runs on an Apple M1 Max with Node.js 24.16.0, each library in a process of its own; lower is better. Mutative, the first Immer column, and the hand-written reducers run without auto-freeze; the second Immer column shows Immer's default. | Workload | Rows | Mutative | Immer | Immer, auto-freeze | Hand-written | | ----------------------------------- | -----: | -------: | -----: | -----------------: | -----------: | -| Update one field of a small object | — | 0.51 | 0.70 | 0.92 | 0.02 | -| Update an array item found by ID | 100 | 1.11 | 1.73 | 25.2 | 0.38 | -| 200 RTK Query-style updates | — | 1,018 | 1,159 | 4,934 | 355 | -| Read every row by index | 10,000 | 5,094 | 11,773 | 12,138 | 55.5 | -| Remove the first row | 10,000 | 7.10 | 8,605 | 9,265 | 1.40 | -| Update every row | 10,000 | 6,629 | 9,783 | 13,591 | 211 | -| Update one of 10,000 records | 10,000 | 1,175 | 2,109 | 3,135 | 2,113 | -| Insert into a 1,000-property object | — | 51.0 | 146 | 231 | 129 | -| Update a Map value | 10,000 | 517 | 521 | 730 | 518 | -| Add a number to a Set | 10,000 | 34.1 | 929 | 1,004 | 66.0 | -| Update a class instance | 10,000 | 2.92 | 3.43 | 668 | 1.43 | -| Update a value ten levels deep | — | 3.47 | 4.46 | 7.83 | 0.75 | -| Apply patches to 10% of rows | 10,000 | 736 | 1,169 | 2,102 | — | -| Search `current()` after `shift()` | 10,000 | 257 | 11,230 | 11,913 | 64.7 | -| Return draft.filter() | 10,000 | 2,647 | 4,306 | 4,735 | 78.8 | -| Return a new state | 10,000 | 2,594 | 5,628 | 0.79 | 0.02 | -| Return it with rawReturn() | 10,000 | 0.26 | 5,699 | 0.79 | 0.02 | - -Each library ran each workload in a process of its own. With all of them in one process every library ran slower, and not by the same amount: Mutative and Immer took 1.1 times as long on geometric mean, and the hand-written reducers 1.7 times. The search removes the first row with `shift()`, then finds the last row in a snapshot of the array and updates it. Immer has no `rawReturn()` and returns the same plain value in the last two rows; with auto-freeze, Mutative skips a returned state that is already frozen, as Immer does, taking 0.53 µs. Immer was faster in 3 of the 614 matched cases: with auto-freeze, when updating a class instance with 1,000 fields; and in one concatenation with auto-freeze and patches, by 8%. The Set workloads were measured again after Set drafts began to change their copy directly; adding or deleting a Set item became 3-50x faster. The patch application workloads were measured again after `apply()` began to read the type of each draft from its state; applying patches that replace nested values became 1.6-2x faster. Four workloads were added later and measured the same way: reading a Set, iterating a Map, updating a row that `shift()` moved, and updating a value beside a Map and a Set that stay unchanged. +| Update one field of a small object | — | 0.51 | 0.69 | 0.92 | 0.02 | +| Update an array item found by ID | 100 | 1.14 | 1.75 | 23.8 | 0.38 | +| 200 RTK Query-style updates | — | 1,012 | 1,173 | 4,985 | 352 | +| Read every row by index | 10,000 | 4,766 | 11,933 | 11,980 | 54.8 | +| Remove the first row | 10,000 | 7.07 | 7,426 | 8,186 | 1.41 | +| Update every row | 10,000 | 6,534 | 9,644 | 13,501 | 212 | +| Update one of 10,000 records | 10,000 | 1,199 | 2,102 | 3,164 | 2,091 | +| Insert into a 1,000-property object | — | 51.5 | 149 | 231 | 127 | +| Update a Map value | 10,000 | 526 | 530 | 741 | 523 | +| Add a number to a Set | 10,000 | 30.9 | 913 | 987 | 63.2 | +| Update a class instance | 10,000 | 2.90 | 3.42 | 668 | 1.43 | +| Update a value ten levels deep | — | 3.56 | 4.41 | 7.78 | 0.76 | +| Apply patches to 10% of rows | 10,000 | 727 | 1,179 | 2,101 | — | +| Search `current()` after `shift()` | 10,000 | 257 | 11,379 | 11,981 | 23.4 | +| Return draft.filter() | 10,000 | 2,639 | 4,307 | 4,752 | 79.7 | +| Return a new state | 10,000 | 2,606 | 5,728 | 0.79 | 0.02 | +| Return it with rawReturn() | 10,000 | 0.26 | 5,747 | 0.79 | 0.02 | + +Each library ran each workload in a process of its own. With all of them in one process every library ran slower, and not by the same amount: Mutative took 1.1 times as long on geometric mean, Immer 1.2 times, and the hand-written reducers 1.6 times. The search removes the first row with `shift()`, then finds the last row in a snapshot of the array and updates it. Immer has no `rawReturn()` and returns the same plain value in the last two rows; with auto-freeze, Mutative skips a returned state that is already frozen, as Immer does, taking 0.53 µs. Immer was faster in 2 of the 614 matched cases, both with auto-freeze when updating a class instance with 1,000 fields. Run `pnpm benchmark:immer` to measure the suite; the [benchmark guide](./perf-testing/README.md) describes its options. ### Large arrays -At 1,000 and 10,000 rows the gap grows. Against Immer without its array-method plugin, Mutative was faster in 221 of 232 cases at these sizes, 4.7x on geometric mean, and faster in every case that moves elements: +At 1,000 and 10,000 rows the gap grows. Against Immer without its array-method plugin, Mutative was faster in 222 of 232 cases at these sizes, 4.7x on geometric mean, and faster in every case that moves elements: | Auto-freeze | Patches | All workloads, 1,000 rows | All workloads, 10,000 rows | Moves, 1,000 rows | Moves, 10,000 rows | | ----------- | ------- | ------------------------: | -------------------------: | ----------------: | -----------------: | -| off | off | 8.6x | 10.6x | 211x | 442x | -| off | on | 4.1x | 4.6x | 7.1x | 7.5x | -| on | off | 3.8x | 4.2x | 32x | 40x | -| on | on | 2.7x | 2.9x | 6.6x | 7.1x | +| off | off | 8.6x | 10.6x | 209x | 431x | +| off | on | 4.1x | 4.6x | 7.1x | 7.3x | +| on | off | 3.8x | 4.2x | 32x | 39x | +| on | on | 2.7x | 2.8x | 6.6x | 6.8x | -Each value is the geometric mean of Immer's time over Mutative's; moves are `shift`, `unshift`, `splice` insertion, and `reverse`. Mutative moves elements natively on its copy, while Immer moves each one through its draft proxy: removing the first of 10,000 rows took 7.10 µs against 8,605 µs, 1,213x. With patches, both libraries emit one patch per moved index, which bounds the gain to 5-10x. The [performance summary](./perf-testing/reports/SUMMARY.md) breaks these results down by scenario. +Each value is the geometric mean of Immer's time over Mutative's; moves are `shift`, `unshift`, `splice` insertion, and `reverse`. Mutative moves elements natively on its copy, while Immer moves each one through its draft proxy: removing the first of 10,000 rows took 7.07 µs against 7,426 µs, 1,051x. With patches, both libraries emit one patch per moved index, which bounds the gain to 5-9x. The [performance summary](./perf-testing/reports/SUMMARY.md) breaks these results down by scenario. ### With patches -With patches on and auto-freeze off, Mutative was faster than Immer in 148 of 150 cases and within 5% in the other 2, 3.3x on geometric mean, and faster than Mutative 1.3.0 in 132 and within 5% in the rest, 4.6x. In every array case measured it was faster than both: 4.4x Immer and 13x Mutative 1.3.0 on geometric mean. Patches cost little when an update changes a few paths: pushing a row and inserting a property at 10,000 rows took 63.3 µs with patches against 62.8 µs without. Moving elements emits one patch per moved index in every library, so removing the first of 10,000 rows produces 10,000 forward and 10,000 inverse patches. +With patches on and auto-freeze off, Mutative was faster than Immer in 147 of 150 cases and within 5% in the other 3, 3.3x on geometric mean, and faster than Mutative 1.3.0 in 132 and within 5% in the rest, 4.6x. In every array case measured it was faster than both: 4.4x Immer and 13x Mutative 1.3.0 on geometric mean. Patches cost little when an update changes a few paths: pushing a row and inserting a property at 10,000 rows took 63.0 µs with patches against 62.9 µs without. Moving elements emits one patch per moved index in every library, so removing the first of 10,000 rows produces 10,000 forward and 10,000 inverse patches. Times are microseconds per update with patches on and auto-freeze off: | Workload | Rows | Mutative | Immer | Mutative 1.3.0 | | ---------------------------------- | -----: | -------: | -----: | -------------: | -| Push a row and insert a property | 10,000 | 63.3 | 364 | 208 | -| Update an array item found by ID | 100 | 1.47 | 3.20 | 3.36 | -| 200 RTK Query-style updates | — | 1,150 | 1,500 | 2,077 | -| Remove the first row with `splice` | 100 | 11.6 | 84.9 | 2,983 | -| Insert a row in the middle | 100 | 8.89 | 48.8 | 1,201 | -| Sort rows | 100 | 103 | 129 | 2,395 | -| Remove the first row with `shift` | 10,000 | 1,071 | 10,376 | 273,162 | -| Reverse the rows | 10,000 | 1,074 | 9,416 | 277,427 | -| Update every row | 10,000 | 8,195 | 20,871 | 21,391 | -| Update one of 10,000 records | 10,000 | 1,192 | 2,097 | 1,178 | - -Immer's optional `enableArrayMethods()` plugin also runs array methods on the draft's copy, and with patches it comes within 1.13-1.71x of Mutative when moving elements of 1,000 or 10,000 rows. These comparisons leave it off because in Immer 11.1.18 it breaks guarantees that Immer otherwise keeps: `shift`, `pop` and `splice` return raw base objects, so editing a removed object changes the previous state; reordering can expose original objects the same way and leave revoked drafts in the result; and its forward patches can fail to replay. In [the audit](./test/immer-array-methods.md), 123 of 4,913 three-step operation sequences break with the plugin and none without it. The [performance summary](./perf-testing/reports/SUMMARY.md) reports Immer with the plugin separately. +| Push a row and insert a property | 10,000 | 63.0 | 364 | 210 | +| Update an array item found by ID | 100 | 1.51 | 3.18 | 3.35 | +| 200 RTK Query-style updates | — | 1,155 | 1,511 | 2,097 | +| Remove the first row with `splice` | 100 | 11.4 | 83.2 | 2,949 | +| Insert a row in the middle | 100 | 8.83 | 48.0 | 1,192 | +| Sort rows | 100 | 103 | 130 | 2,363 | +| Remove the first row with `shift` | 10,000 | 1,066 | 8,967 | 273,171 | +| Reverse the rows | 10,000 | 1,073 | 9,525 | 276,034 | +| Update every row | 10,000 | 8,132 | 20,060 | 22,073 | +| Update one of 10,000 records | 10,000 | 1,200 | 2,114 | 1,183 | + +Immer's optional `enableArrayMethods()` plugin also runs array methods on the draft's copy, and with patches it comes within 1.14-1.71x of Mutative when moving elements of 1,000 or 10,000 rows. These comparisons leave it off because in Immer 11.1.18 it breaks guarantees that Immer otherwise keeps: `shift`, `pop` and `splice` return raw base objects, so editing a removed object changes the previous state; reordering can expose original objects the same way and leave revoked drafts in the result; and its forward patches can fail to replay. In [the audit](./test/immer-array-methods.md), 123 of 4,913 three-step operation sequences break with the plugin and none without it. The [performance summary](./perf-testing/reports/SUMMARY.md) reports Immer with the plugin separately. ### Bundle size @@ -111,7 +111,7 @@ Mutative's `create` includes patches, `Map`/`Set` support and the native array m ## Features and Benefits - **Mutation makes immutable updates** - Immutable data structures supporting objects, arrays, Sets and Maps. -- **High performance** - About 6x faster than Immer with each library's defaults, and faster than hand-written spreads in measured wide-object and large-array insertion workloads. +- **High performance** - About 6.6x faster than Immer with each library's defaults, and faster than hand-written spreads in measured wide-object and large-array insertion workloads. - **Optional freezing state** - No freezing of immutable data by default. - **Support for JSON Patch** - Full compliance with JSON Patch specification. - **Custom shallow copy** - Support for more types of immutable data. From d107fa3d6b430427605fc106df1ffbc4122a87df Mon Sep 17 00:00:00 2001 From: unadlib Date: Sat, 10 Oct 2026 09:53:45 +0800 Subject: [PATCH 3/6] docs: refresh the website's benchmark figures from the benchmark of ce0c25e --- website/app/layout.tsx | 2 +- website/components/home/benchmarks.tsx | 6 +- website/components/home/features.tsx | 2 +- .../extra-topics/comparison-with-immer.mdx | 4 +- .../docs/getting-started/performance.mdx | 82 +++++++++---------- website/content/docs/intro.mdx | 4 +- 6 files changed, 50 insertions(+), 50 deletions(-) diff --git a/website/app/layout.tsx b/website/app/layout.tsx index 7b98793d..53a40f86 100644 --- a/website/app/layout.tsx +++ b/website/app/layout.tsx @@ -15,7 +15,7 @@ export const metadata: Metadata = { template: '%s | Mutative', }, description: - "Efficient immutable updates, about 3.6x faster than Immer with matched settings and 6.7x faster with each library's defaults.", + "Efficient immutable updates, about 3.6x faster than Immer with matched settings and 6.6x faster with each library's defaults.", }), }; diff --git a/website/components/home/benchmarks.tsx b/website/components/home/benchmarks.tsx index 62e01b73..7a1f1266 100644 --- a/website/components/home/benchmarks.tsx +++ b/website/components/home/benchmarks.tsx @@ -7,12 +7,12 @@ const stats = [ { value: '3.6x', label: 'Faster than Immer with matched settings', - detail: 'Faster in 595 of 614 measured cases', + detail: 'Faster in 596 of 614 measured cases', }, { - value: '6.7x', + value: '6.6x', label: "Faster than Immer with each library's defaults", - detail: 'Faster in 153 of 157 measured cases', + detail: 'Faster in 154 of 157 measured cases', }, { value: '97', diff --git a/website/components/home/features.tsx b/website/components/home/features.tsx index fe032aaa..8e02f3c1 100644 --- a/website/components/home/features.tsx +++ b/website/components/home/features.tsx @@ -30,7 +30,7 @@ const features: { icon: Zap, title: 'High performance', description: - 'About 6x faster than Immer out of the box, and faster than hand-written spreads for objects with thousands of keys.', + 'About 6.6x faster than Immer out of the box, and faster than hand-written spreads for objects with thousands of keys.', href: '/docs/getting-started/performance', }, { diff --git a/website/content/docs/extra-topics/comparison-with-immer.mdx b/website/content/docs/extra-topics/comparison-with-immer.mdx index 279995b2..b115a5dd 100644 --- a/website/content/docs/extra-topics/comparison-with-immer.mdx +++ b/website/content/docs/extra-topics/comparison-with-immer.mdx @@ -74,9 +74,9 @@ Mutative passes the test suite of Immer, adapted to its API. The [non-support te ## Mutative vs Immer Performance -With matched settings, both freezing or both not and both generating patches or both not, Mutative was faster than Immer 11.1.18 in 595 of 614 measured cases, 3.6x on geometric mean. With each library's defaults, Mutative without auto-freeze and Immer with it, Mutative was faster in 153 of 157 cases, 6.7x on geometric mean. Immer was faster with auto-freeze when updating class instances with 1,000 fields, and in one concatenation with auto-freeze and patches. +With matched settings, both freezing or both not and both generating patches or both not, Mutative was faster than Immer 11.1.18 in 596 of 614 measured cases, 3.6x on geometric mean. With each library's defaults, Mutative without auto-freeze and Immer with it, Mutative was faster in 154 of 157 cases, 6.6x on geometric mean. Immer was faster only with auto-freeze when updating class instances with 1,000 fields. -With patches on and auto-freeze off, Mutative was faster in 148 of 150 cases and never slower, and faster in every array case: 4.4x Immer and 13x Mutative 1.3.0 on geometric mean. The comparisons run Immer without its `enableArrayMethods()` plugin. In Immer 11.1.18 the plugin returns raw base objects from `shift`, `pop` and `splice`, so editing them changes the previous state, and its patches can fail to replay; the audit above reproduces these failures. +With patches on and auto-freeze off, Mutative was faster in 147 of 150 cases and never slower, and faster in every array case: 4.4x Immer and 13x Mutative 1.3.0 on geometric mean. The comparisons run Immer without its `enableArrayMethods()` plugin. In Immer 11.1.18 the plugin returns raw base objects from `shift`, `pop` and `splice`, so editing them changes the previous state, and its patches can fail to replay; the audit above reproduces these failures. Immer 11.0.0 ported the finalization callbacks of Mutative, which replace a recursive traversal of the draft tree, in [immerjs/immer#1183](https://github.com/immerjs/immer/pull/1183). Immer 11.1.18, which these measurements use, includes them. diff --git a/website/content/docs/getting-started/performance.mdx b/website/content/docs/getting-started/performance.mdx index 1564acfa..bdeac9cc 100644 --- a/website/content/docs/getting-started/performance.mdx +++ b/website/content/docs/getting-started/performance.mdx @@ -2,7 +2,7 @@ title: Performance --- -Mutative is about 3.6x faster than Immer with matched settings and 6.7x faster with each library's defaults, across 97 benchmarked workloads. A hand-written reducer that copies only what it changes is faster still in most of them. +Mutative is about 3.6x faster than Immer with matched settings and 6.6x faster with each library's defaults, across 97 benchmarked workloads. A hand-written reducer that copies only what it changes is faster still in most of them. ## Benchmark suite @@ -12,69 +12,69 @@ The [benchmark suite](https://github.com/unadlib/mutative/tree/main/perf-testing > Mutative passed all of Immer's test cases. -With matched settings, both freezing or both not and both generating patches or both not, Mutative was faster than Immer in 595 of 614 measured cases, 3.6x on geometric mean. With each library's defaults, Mutative without auto-freeze and Immer with it, Mutative was faster in 153 of 157 cases, 6.7x on geometric mean. +With matched settings, both freezing or both not and both generating patches or both not, Mutative was faster than Immer in 596 of 614 measured cases, 3.6x on geometric mean. With each library's defaults, Mutative without auto-freeze and Immer with it, Mutative was faster in 154 of 157 cases, 6.6x on geometric mean. Times are microseconds per update, medians of three runs on an Apple M1 Max with Node.js 24.16.0, each library in a process of its own; lower is better. Mutative, the first Immer column, and the hand-written reducers run without auto-freeze; the second Immer column shows Immer's default. | Workload | Rows | Mutative | Immer | Immer, auto-freeze | Hand-written | | --- | ---: | ---: | ---: | ---: | ---: | -| Update one field of a small object | — | 0.51 | 0.70 | 0.92 | 0.02 | -| Update an array item found by ID | 100 | 1.11 | 1.73 | 25.2 | 0.38 | -| 200 RTK Query-style updates | — | 1,018 | 1,159 | 4,934 | 355 | -| Read every row by index | 10,000 | 5,094 | 11,773 | 12,138 | 55.5 | -| Remove the first row | 10,000 | 7.10 | 8,605 | 9,265 | 1.40 | -| Update every row | 10,000 | 6,629 | 9,783 | 13,591 | 211 | -| Update one of 10,000 records | 10,000 | 1,175 | 2,109 | 3,135 | 2,113 | -| Insert into a 1,000-property object | — | 51.0 | 146 | 231 | 129 | -| Update a Map value | 10,000 | 517 | 521 | 730 | 518 | -| Add a number to a Set | 10,000 | 34.1 | 929 | 1,004 | 66.0 | -| Update a class instance | 10,000 | 2.92 | 3.43 | 668 | 1.43 | -| Update a value ten levels deep | — | 3.47 | 4.46 | 7.83 | 0.75 | -| Apply patches to 10% of rows | 10,000 | 736 | 1,169 | 2,102 | — | -| Search `current()` after `shift()` | 10,000 | 257 | 11,230 | 11,913 | 64.7 | -| Return draft.filter() | 10,000 | 2,647 | 4,306 | 4,735 | 78.8 | -| Return a new state | 10,000 | 2,594 | 5,628 | 0.79 | 0.02 | -| Return it with rawReturn() | 10,000 | 0.26 | 5,699 | 0.79 | 0.02 | - -Each library ran each workload in a process of its own. With all of them in one process every library ran slower, and not by the same amount: Mutative and Immer took 1.1 times as long on geometric mean, and the hand-written reducers 1.7 times. The search removes the first row with `shift()`, then finds the last row in a snapshot of the array and updates it. Immer has no `rawReturn()` and returns the same plain value in the last two rows; with auto-freeze, Mutative skips a returned state that is already frozen, as Immer does, taking 0.53 µs. Immer was faster in 3 of the 614 matched cases: with auto-freeze, when updating a class instance with 1,000 fields; and in one concatenation with auto-freeze and patches, by 8%. The Set workloads were measured again after Set drafts began to change their copy directly; adding or deleting a Set item became 3-50x faster. The patch application workloads were measured again after `apply()` began to read the type of each draft from its state; applying patches that replace nested values became 1.6-2x faster. Four workloads were added later and measured the same way: reading a Set, iterating a Map, updating a row that `shift()` moved, and updating a value beside a Map and a Set that stay unchanged. +| Update one field of a small object | — | 0.51 | 0.69 | 0.92 | 0.02 | +| Update an array item found by ID | 100 | 1.14 | 1.75 | 23.8 | 0.38 | +| 200 RTK Query-style updates | — | 1,012 | 1,173 | 4,985 | 352 | +| Read every row by index | 10,000 | 4,766 | 11,933 | 11,980 | 54.8 | +| Remove the first row | 10,000 | 7.07 | 7,426 | 8,186 | 1.41 | +| Update every row | 10,000 | 6,534 | 9,644 | 13,501 | 212 | +| Update one of 10,000 records | 10,000 | 1,199 | 2,102 | 3,164 | 2,091 | +| Insert into a 1,000-property object | — | 51.5 | 149 | 231 | 127 | +| Update a Map value | 10,000 | 526 | 530 | 741 | 523 | +| Add a number to a Set | 10,000 | 30.9 | 913 | 987 | 63.2 | +| Update a class instance | 10,000 | 2.90 | 3.42 | 668 | 1.43 | +| Update a value ten levels deep | — | 3.56 | 4.41 | 7.78 | 0.76 | +| Apply patches to 10% of rows | 10,000 | 727 | 1,179 | 2,101 | — | +| Search `current()` after `shift()` | 10,000 | 257 | 11,379 | 11,981 | 23.4 | +| Return draft.filter() | 10,000 | 2,639 | 4,307 | 4,752 | 79.7 | +| Return a new state | 10,000 | 2,606 | 5,728 | 0.79 | 0.02 | +| Return it with rawReturn() | 10,000 | 0.26 | 5,747 | 0.79 | 0.02 | + +Each library ran each workload in a process of its own. With all of them in one process every library ran slower, and not by the same amount: Mutative took 1.1 times as long on geometric mean, Immer 1.2 times, and the hand-written reducers 1.6 times. The search removes the first row with `shift()`, then finds the last row in a snapshot of the array and updates it. Immer has no `rawReturn()` and returns the same plain value in the last two rows; with auto-freeze, Mutative skips a returned state that is already frozen, as Immer does, taking 0.53 µs. Immer was faster in 2 of the 614 matched cases, both with auto-freeze when updating a class instance with 1,000 fields. ## Large arrays -At 1,000 and 10,000 rows the gap grows. Against Immer without its array-method plugin, Mutative was faster in 221 of 232 cases at these sizes, 4.7x on geometric mean, and faster in every case that moves elements: +At 1,000 and 10,000 rows the gap grows. Against Immer without its array-method plugin, Mutative was faster in 222 of 232 cases at these sizes, 4.7x on geometric mean, and faster in every case that moves elements: | Auto-freeze | Patches | All workloads, 1,000 rows | All workloads, 10,000 rows | Moves, 1,000 rows | Moves, 10,000 rows | | --- | --- | ---: | ---: | ---: | ---: | -| off | off | 8.6x | 10.6x | 211x | 442x | -| off | on | 4.1x | 4.6x | 7.1x | 7.5x | -| on | off | 3.8x | 4.2x | 32x | 40x | -| on | on | 2.7x | 2.9x | 6.6x | 7.1x | +| off | off | 8.6x | 10.6x | 209x | 431x | +| off | on | 4.1x | 4.6x | 7.1x | 7.3x | +| on | off | 3.8x | 4.2x | 32x | 39x | +| on | on | 2.7x | 2.8x | 6.6x | 6.8x | -Each value is the geometric mean of Immer's time over Mutative's; moves are `shift`, `unshift`, `splice` insertion, and `reverse`. Mutative moves elements natively on its copy, while Immer moves each one through its draft proxy: removing the first of 10,000 rows took 7.10 µs against 8,605 µs, 1,213x. With patches, both libraries emit one patch per moved index, which bounds the gain to 5-10x. The [performance summary](https://github.com/unadlib/mutative/blob/main/perf-testing/reports/SUMMARY.md) breaks these results down by scenario. +Each value is the geometric mean of Immer's time over Mutative's; moves are `shift`, `unshift`, `splice` insertion, and `reverse`. Mutative moves elements natively on its copy, while Immer moves each one through its draft proxy: removing the first of 10,000 rows took 7.07 µs against 7,426 µs, 1,051x. With patches, both libraries emit one patch per moved index, which bounds the gain to 5-9x. The [performance summary](https://github.com/unadlib/mutative/blob/main/perf-testing/reports/SUMMARY.md) breaks these results down by scenario. ## With patches -With patches on and auto-freeze off, Mutative was faster than Immer in 148 of 150 cases and within 5% in the other 2, 3.3x on geometric mean, and faster than Mutative 1.3.0 in 132 and within 5% in the rest, 4.6x. In every array case measured it was faster than both: 4.4x Immer and 13x Mutative 1.3.0 on geometric mean. Patches cost little when an update changes a few paths: pushing a row and inserting a property at 10,000 rows took 63.3 µs with patches against 62.8 µs without. Moving elements emits one patch per moved index in every library, so removing the first of 10,000 rows produces 10,000 forward and 10,000 inverse patches. +With patches on and auto-freeze off, Mutative was faster than Immer in 147 of 150 cases and within 5% in the other 3, 3.3x on geometric mean, and faster than Mutative 1.3.0 in 132 and within 5% in the rest, 4.6x. In every array case measured it was faster than both: 4.4x Immer and 13x Mutative 1.3.0 on geometric mean. Patches cost little when an update changes a few paths: pushing a row and inserting a property at 10,000 rows took 63.0 µs with patches against 62.9 µs without. Moving elements emits one patch per moved index in every library, so removing the first of 10,000 rows produces 10,000 forward and 10,000 inverse patches. Times are microseconds per update with patches on and auto-freeze off: | Workload | Rows | Mutative | Immer | Mutative 1.3.0 | | --- | ---: | ---: | ---: | ---: | -| Push a row and insert a property | 10,000 | 63.3 | 364 | 208 | -| Update an array item found by ID | 100 | 1.47 | 3.20 | 3.36 | -| 200 RTK Query-style updates | — | 1,150 | 1,500 | 2,077 | -| Remove the first row with `splice` | 100 | 11.6 | 84.9 | 2,983 | -| Insert a row in the middle | 100 | 8.89 | 48.8 | 1,201 | -| Sort rows | 100 | 103 | 129 | 2,395 | -| Remove the first row with `shift` | 10,000 | 1,071 | 10,376 | 273,162 | -| Reverse the rows | 10,000 | 1,074 | 9,416 | 277,427 | -| Update every row | 10,000 | 8,195 | 20,871 | 21,391 | -| Update one of 10,000 records | 10,000 | 1,192 | 2,097 | 1,178 | - -Immer's optional `enableArrayMethods()` plugin also runs array methods on the draft's copy, and with patches it comes within 1.13-1.71x of Mutative when moving elements of 1,000 or 10,000 rows. These comparisons leave it off because in Immer 11.1.18 it breaks guarantees that Immer otherwise keeps: `shift`, `pop` and `splice` return raw base objects, so editing a removed object changes the previous state; reordering can expose original objects the same way and leave revoked drafts in the result; and its forward patches can fail to replay. In [the audit](https://github.com/unadlib/mutative/blob/main/test/immer-array-methods.md), 123 of 4,913 three-step operation sequences break with the plugin and none without it. The [performance summary](https://github.com/unadlib/mutative/blob/main/perf-testing/reports/SUMMARY.md) reports Immer with the plugin separately. +| Push a row and insert a property | 10,000 | 63.0 | 364 | 210 | +| Update an array item found by ID | 100 | 1.51 | 3.18 | 3.35 | +| 200 RTK Query-style updates | — | 1,155 | 1,511 | 2,097 | +| Remove the first row with `splice` | 100 | 11.4 | 83.2 | 2,949 | +| Insert a row in the middle | 100 | 8.83 | 48.0 | 1,192 | +| Sort rows | 100 | 103 | 130 | 2,363 | +| Remove the first row with `shift` | 10,000 | 1,066 | 8,967 | 273,171 | +| Reverse the rows | 10,000 | 1,073 | 9,525 | 276,034 | +| Update every row | 10,000 | 8,132 | 20,060 | 22,073 | +| Update one of 10,000 records | 10,000 | 1,200 | 2,114 | 1,183 | + +Immer's optional `enableArrayMethods()` plugin also runs array methods on the draft's copy, and with patches it comes within 1.14-1.71x of Mutative when moving elements of 1,000 or 10,000 rows. These comparisons leave it off because in Immer 11.1.18 it breaks guarantees that Immer otherwise keeps: `shift`, `pop` and `splice` return raw base objects, so editing a removed object changes the previous state; reordering can expose original objects the same way and leave revoked drafts in the result; and its forward patches can fail to replay. In [the audit](https://github.com/unadlib/mutative/blob/main/test/immer-array-methods.md), 123 of 4,913 three-step operation sequences break with the plugin and none without it. The [performance summary](https://github.com/unadlib/mutative/blob/main/perf-testing/reports/SUMMARY.md) reports Immer with the plugin separately. ## Mutative vs hand-written reducers -A hand-written reducer that copies only what it changes is faster than any library that drafts state, and it was faster than Mutative in 129 of the suite's 150 cases. Mutative was faster when copying wide objects, such as records with 1,000 keys or class instances with 100 fields, by 1.5-2.7x, when inserting at the front of 10,000 rows: 16 µs instead of 28 µs, and when adding a number to a Set of 1,000 or 10,000 numbers, which it copies faster than `new Set()` does: 33 µs instead of 66 µs at 10,000. +A hand-written reducer that copies only what it changes is faster than any library that drafts state, and it was faster than Mutative in 129 of the suite's 150 cases. Mutative was faster when copying wide objects, such as records with 1,000 keys or class instances with 100 fields, by 1.5-2.8x, when inserting at the front of 10,000 rows: 16 µs instead of 28 µs, and when adding a number to a Set of 1,000 or 10,000 numbers, which it copies faster than `new Set()` does: 31 µs instead of 63 µs at 10,000. Spreads also become slow when code copies the same structure again and again. Mutative copies each object at most once per update, however many changes a recipe makes. diff --git a/website/content/docs/intro.mdx b/website/content/docs/intro.mdx index 8a617fa6..49efba8c 100644 --- a/website/content/docs/intro.mdx +++ b/website/content/docs/intro.mdx @@ -2,7 +2,7 @@ title: Introduction --- -**Mutative** - A JavaScript library for efficient immutable updates, about 3.6x faster than Immer with matched settings and 6.7x faster with each library's defaults, across [97 benchmarked workloads](/docs/getting-started/performance). +**Mutative** - A JavaScript library for efficient immutable updates, about 3.6x faster than Immer with matched settings and 6.6x faster with each library's defaults, across [97 benchmarked workloads](/docs/getting-started/performance). ## What is Mutative? @@ -68,7 +68,7 @@ This is why Mutative was created. ## Features and Benefits - **Mutation makes immutable updates** - Immutable data structures supporting objects, arrays, Sets and Maps. -- **High performance** - About 6x faster than Immer out of the box, and faster than hand-written spreads when updating objects with thousands of keys or inserting at the front of large arrays. +- **High performance** - About 6.6x faster than Immer out of the box, and faster than hand-written spreads when updating objects with thousands of keys or inserting at the front of large arrays. - **Optional freezing state** - No freezing of immutable data by default. - **Support for JSON Patch** - Full compliance with JSON Patch specification. - **Custom shallow copy** - Support for more types of immutable data. From 86be79eda86dd1a3cce67a83154e09f9216274f2 Mon Sep 17 00:00:00 2001 From: unadlib Date: Sat, 10 Oct 2026 09:53:45 +0800 Subject: [PATCH 4/6] docs: refresh the process order example in the benchmark guide --- perf-testing/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/perf-testing/README.md b/perf-testing/README.md index 973ee8c6..252b9ede 100644 --- a/perf-testing/README.md +++ b/perf-testing/README.md @@ -113,7 +113,7 @@ The `search-*` scenarios find the last row of an array of nested rows and update - The hand-written reducer is the reference implementation of every scenario: spreads, `slice`, `map`, and new `Map` and `Set` copies, with plain loops for the values that recipes read through drafts. It runs with freeze and patches off only, since it neither freezes nor produces patches. It is an ordinary reducer, not the fastest possible code. - Immer needs `enableMapSet()` for Map and Set values. Once loaded, Immer calls the plugin while finalizing every draft: `small-object-update` measured about 4% slower with it (0.67 µs instead of 0.64 µs), 100-row reads and updates unchanged. Map and Set scenarios therefore run in a process of their own within each run, and memory workers load the plugin only for them, so other scenarios measure Immer without it. Reports record `immerMapSetEnabled`. - Immer drafts the class instances through a static `immerable` property; Mutative through its `mark` option, which returns `immutable` for those classes and is passed to `apply` as well. Mutative copies such an instance property by property with its descriptors; Immer 11 copies it with `Object.assign` onto its prototype, since its `useStrictShallowCopy` option is off by default. -- Objects with hundreds of properties are sensitive to the object shapes V8 keeps from the code that ran earlier in the same process. Inserting a property into the 1,000-property object took the hand-written reducer 1.7 µs when it ran after the libraries and 130–133 µs when it ran first; Immer took 2.7 µs after the hand-written reducer and 152–155 µs after Mutative. Alternating the library order between runs does not cancel such effects. `--isolate` gives each scenario and library a process of its own, so every measurement follows only its own validation and priming. Code that runs beside a library changes its speed in other scenarios too: with all libraries and scenarios in one process, every library ran slower, by different amounts, so the summary reports isolated results for every scenario and keeps the shared-process runs as a control. +- Objects with hundreds of properties are sensitive to the object shapes V8 keeps from the code that ran earlier in the same process. Inserting a property into the 1,000-property object took the hand-written reducer 1.7 µs when it ran after the libraries and 124–130 µs when it ran first; Immer took 2.7–2.8 µs after the hand-written reducer and 146–148 µs after Mutative. Alternating the library order between runs does not cancel such effects. `--isolate` gives each scenario and library a process of its own, so every measurement follows only its own validation and priming. Code that runs beside a library changes its speed in other scenarios too: with all libraries and scenarios in one process, every library ran slower, by different amounts, so the summary reports isolated results for every scenario and keeps the shared-process runs as a control. - Auto-freeze **off** means unfrozen inputs and payloads for both libraries. - Auto-freeze **on** means deeply pre-frozen inputs and payloads, and actual output freezing enabled for both libraries. This measures ongoing immutable state updates; it excludes the first full-graph freeze of cold external data. Map and Set inputs are frozen like Immer's results: replaced mutators and `Object.freeze`. The candidate freezes collection instances too. The pinned Mutative v1 baseline only replaces their mutators, and the checks accept either form. - Every trial owns its input graph. Each timed sample starts at the same immutable base. Reuse and mixed scenarios evolve the returned state only within that sample; state does not grow across measurement samples. From e4918f72fe63f17bfedf44096218f2f2c5ae7896 Mon Sep 17 00:00:00 2001 From: unadlib Date: Sat, 10 Oct 2026 09:53:45 +0800 Subject: [PATCH 5/6] docs: refresh the Mutative bundle sizes in the README and the installation guide --- README.md | 4 ++-- website/content/docs/getting-started/installation.mdx | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index febeba02..a0f00042 100644 --- a/README.md +++ b/README.md @@ -103,8 +103,8 @@ Mutative ships patches, `Map`/`Set` support and native array methods built in; I | Bundle | Immer | Mutative | | ------------------------------------------- | -----: | -------: | -| `produce` / `create` only | 3.6 kB | 7.7 kB | -| With patches, `Map`/`Set` and array methods | 6.4 kB | 8.2 kB | +| `produce` / `create` only | 3.6 kB | 7.8 kB | +| With patches, `Map`/`Set` and array methods | 6.4 kB | 8.4 kB | Mutative's `create` includes patches, `Map`/`Set` support and the native array methods even when a recipe does not use them: they are part of `create`, not separate imports, so bundlers cannot drop them. The difference buys the draft fast paths and the native array methods measured in the [performance summary](./perf-testing/reports/SUMMARY.md), which also records the artifact sizes of each measured source. See the [array methods FAQ](#faqs) for the supported fast paths and their contract, and the [Immer regression cases](./test/immer-array-methods.md) for the behavior of its array-method plugin. diff --git a/website/content/docs/getting-started/installation.mdx b/website/content/docs/getting-started/installation.mdx index 5aaaa4c3..cdd54ca3 100644 --- a/website/content/docs/getting-started/installation.mdx +++ b/website/content/docs/getting-started/installation.mdx @@ -10,7 +10,7 @@ Mutative is a npm package that can be used in Node.js and browsers, it is also c npm install mutative ``` -A production bundle of `create`, `apply`, `current` and `original` is about 8.2 kB after Brotli compression, including patches, `Map`/`Set` support and native array methods. +A production bundle of `create`, `apply`, `current` and `original` is about 8.4 kB after Brotli compression, including patches, `Map`/`Set` support and native array methods. ## Using Mutative from CDN From 8317f1cfc88d48f3b3f7a6f0cd6c9784931396b5 Mon Sep 17 00:00:00 2001 From: unadlib Date: Sat, 10 Oct 2026 17:33:46 +0800 Subject: [PATCH 6/6] docs: add rawReturn benchmark comparison with Immer defaults --- perf-testing/README.md | 17 +++++++++++++++++ perf-testing/reports/SUMMARY.md | 16 ++++++++++++++++ 2 files changed, 33 insertions(+) diff --git a/perf-testing/README.md b/perf-testing/README.md index 252b9ede..82bbe884 100644 --- a/perf-testing/README.md +++ b/perf-testing/README.md @@ -58,6 +58,23 @@ The runner never upgrades dependencies implicitly. Reports record the actual ins `--patches off|on|both` defaults to `off`, preserving the original plain-update benchmark. `benchmark:immer:check` checks both patch modes and both freeze modes (all selected scenarios with all four libraries, each in the modes it supports). `--library all` (the default) measures candidate, pinned v1, Immer, and the hand-written reducer; `--library both` retains the original candidate/Immer comparison. `--patches on` measures patch generation with automatic freezing off and on; `--patches both` additionally measures the plain-update baseline in the same processes. The JSON schema is version 2, with `enablePatches` on each trial and summary; the summarizer still reads the original version 1 archives as patches off. +### Replacement returns with default freeze settings + +The [replacement-return comparison](./reports/SUMMARY.md#replacement-returns-with-rawreturn-and-default-freeze-settings) pairs Mutative's `return-replace-raw` with auto-freeze off and Immer's `return-replace` with auto-freeze on, both with patches off. `rawReturn()` explicitly promises that the whole returned value contains no drafts. Immer receives an ordinary replacement value and pre-frozen inputs and payloads; the first freeze of external data is outside timing. + +To reproduce this comparison and its ordinary-return controls, build once and run the three sizes sequentially: + +```sh +pnpm build +pnpm benchmark:immer:build +for rows in 100 1000 10000; do + node --expose-gc --enable-source-maps perf-testing/dist/immutability-benchmarks.mjs --check --library both --freeze both --patches off --filter '^return-replace(-raw)?$' --array-size "$rows" + node perf-testing/run-benchmarks.mjs --isolate --runs 3 --library both --freeze both --patches off --filter '^return-replace(-raw)?$' --array-size "$rows" --output "perf-testing/results/return-defaults-$rows.json" +done +``` + +Each generated report includes both scenarios with matched freeze modes. For the comparison across default settings, select `summary`'s `return-replace-raw` entry with `autoFreeze: false` and `enablePatches: false`, then read `libraries.mutative.medianMeanNs`; select `return-replace` with `autoFreeze: true` and `enablePatches: false`, then read `libraries.immer.medianMeanNs`. Divide Immer's time by Mutative's for the speed ratio, and calculate `100 * (1 - Mutative / Immer)` for the percentage of time saved. The reports' existing `immerOverMutative` fields compare the same scenario and freeze mode and therefore describe a different comparison. + ## Workloads and units | Group | Scenarios | Reducer calls per iteration | diff --git a/perf-testing/reports/SUMMARY.md b/perf-testing/reports/SUMMARY.md index d0329149..c2434a5c 100644 --- a/perf-testing/reports/SUMMARY.md +++ b/perf-testing/reports/SUMMARY.md @@ -51,6 +51,22 @@ Against Immer by workload group, faster / within 5% / slower and geometric mean: The 2 cells slower than Immer are `class-wide-update` with freeze on at 1,000 rows (415 µs and 410 µs against 237 µs and 241 µs); see the limits below. With each library's defaults, the candidate is slower only in `return-replace`, at every size: without auto-freeze it searches the returned state for drafts, which Immer's auto-freeze skips because the state is frozen (26.2 µs, 257 µs and 2.61 ms against 0.78–0.79 µs). The 3 cells slower than Mutative 1.3.0, by 7–16%, are `add` and `update-largeObject1` with freeze on at 100 rows (19.5 µs against 18.3 µs, and 141 µs and 148 µs against 129 µs and 128 µs), cells in which the three processes of each library overlapped: 1.3.0 measured 17.4–21.8 µs and 117–152 µs. +### Replacement returns with `rawReturn()` and default freeze settings + +For a replacement value that contains no drafts, Mutative's `rawReturn()` skips the search through that value. The following comparison uses each library's default freeze setting: candidate `return-replace-raw` with auto-freeze off, and Immer `return-replace` with auto-freeze on. Both have patches off, and Immer's array-method plugin is off. Both recipes return a new root with replaced rows; only the Mutative recipe wraps it in `rawReturn()`. + +Microseconds per call, using the same batch's medians of three independent-process means. The source cells are in `isolated-rest-default.json`, `isolated-1000.json`, and `isolated-10000.json`. Ratios and time reductions are calculated from the unrounded values. + +| Rows | Mutative `rawReturn()`, freeze off | Immer ordinary return, freeze on | Immer/Mutative | Mutative time reduction | +| ---: | ---: | ---: | ---: | ---: | +| 100 | 0.258 | 0.777 | 3.01 | 66.7% | +| 1,000 | 0.259 | 0.781 | 3.02 | 66.9% | +| 10,000 | 0.259 | 0.786 | 3.03 | 67.0% | + +The candidate's time stays approximately constant as the payload grows because it skips the draft search. Ordinary `return-replace` with auto-freeze off takes 26.2 µs, 257 µs, and 2.61 ms at these sizes. `rawReturn()` requires the entire returned value to contain no drafts; it cannot replace ordinary returns that include draft references. Immer's inputs and payloads are pre-frozen, so its times exclude the first freeze of external data. The candidate's inputs and payloads are unfrozen. + +This comparison reuses existing cells and keeps the ordinary-return results and aggregate counts above unchanged. The [focused reproduction commands](../README.md#replacement-returns-with-default-freeze-settings) measure both return scenarios and freeze modes at all three sizes. + ### Large arrays Against Immer without its array-method plugin, by size and mode, over the 30 scaling scenarios: cells faster / within 5% / slower, and geometric means of Immer time over candidate time. The moves are `array-shift-nested`, `array-unshift-nested`, `array-splice-insert-nested`, `array-reverse-nested` and `array-reverse-primitive`.