v25.4.0-preview1 (mutations pipeline)

[matthewmcneely]

This preview release showcases the per-predicate mutation pipeline feature. This feature increases mutation performance for write heavy processes under certain conditions. See the PR for more information.

The image can be pulled from our "custom" Docker repo via dgraph/dgraph-custom:v25.4.0-preview1

IMPORTANT
A feature flag enables this functionality: --feature-flags="mutations-pipeline-threshold=N" — N=0 disables the pipeline entirely (legacy path), N=1 always uses it, N>1 only routes mutations with ≥ N edges to it. The pipeline pays a per-predicate goroutine spin-up cost, so tiny mutations are slightly slower on it; bulk multi-predicate mutations are faster (crossover ≈ 100 edges in in-process benches; live-loading the 1M dataset is ~1.5× faster end-to-end at threshold=1 vs. legacy). As shipped, the current setting is 1, meaning always ON. It's recommended that for your testing you set the value to somewhere around 100.

---

This discussion was created from the release v25.4.0-preview1.

[rahst12]

I'm going to try to get this built and into a docker image to test. Looking forward to your thoughts on this comment: #9467 (comment)

Looks like the negative effects of flag can be mitigated by proportionally providing the 64 goroutines the predicate batches, with the scope of the previous math. This improvement feels like a great preview2!

[rahst12]

Here's some initial test results.
The baseline tests done with Dgrpah v25.3.4.
All tests were: 3x Alphas (sharded), and 1x Zero, with Apache NiFi and the dgraph4j java client.

	Baseline 1	Baseline 2	Preview w N=1 Test 1	Preview w N=1 Test 2
Average	19,155/sec	20,288/sec	9,849/sec	10,385/sec
Min	3,564/sec	4,094/sec	2,528/sec	2,601/sec
Max	36,337/sec	40,897/sec	21,278/sec	25,406/sec

Another test for the preview is in progress (UPDATED, Test 2 above), still with N=1 to validate those numbers. We missed the comment that the default setting is always on and that the recommendation is set N=100 (only apply new logic if there's more than 100 edges or mutations?). We'll do this in a subsequent test. It might be helpful to set the default value to the recommended setting (like N=100). That way, users get the best possible experience right out of the box.

To clarify, so I'm not misunderstanding: An edge equals 1 rdf line written to dgraph. This feature flag enables Dgraph to batch process each predicate with in a transaction with 1 go thread. Eg.

Input:

_:user1     <name>      "Alice" .
_:user1     <age>       "28" .
_:user1     <friend>    _:user2 .
_:user1     <friend>    _:user3 .
...

Then:

Step 1: Group by predicate:
    friend:    9,000 edges  (90%)
    name:        500 edges  (5%)
    age:         500 edges  (5%)
Step 2: Spin up 1x go thread for each predicate.

[matthewmcneely]

Hi Ryan, OK, great. Progress. Yes, one edge is equivalent (more or less) to one rdf line. We will set the default threshold to some number when this ships, still trying to find the sweet spot. One question for you: are you also evaluating the correctness of the graph following the ingest? Thanks, /M

…

On Wed, Jun 17, 2026 at 10:00 PM Ryan ***@***.***> wrote: Here's some initial test results. The baseline tests done with Dgrpah v25.3.4. All tests were: 3x Alphas (sharded), and 1x Zero, with Apache NiFi and the dgraph4j java client. Baseline 1 Baseline 2 Preview w N=1 Average 19,155/sec 20,288/sec 9,849/sec Min 3,564/sec 4,094/sec 2,528/sec Max 36,337/sec 40,897/sec 21,278/sec Another test for the preview is in progress, still with N=1 to validate those numbers. We missed the comment that the default setting is always on and that the recommendation is set N=100 (only apply new logic if there's more than 100 edges or mutations?). We'll do this in a subsequent test. It might be helpful to set the default value to the recommended setting (like N=100). That way, users get the best possible experience right out of the box. To clarify, so I'm not misunderstanding: An edge equals 1 rdf line written to dgraph. This feature flag enables Dgraph to batch process each predicate with in a transaction with 1 go thread. Eg. Input: _:user1 <name> "Alice" . _:user1 <age> "28" . _:user1 <friend> _:user2 . _:user1 <friend> _:user3 . ... Then: Step 1: Group by predicate: friend: 9,000 edges (90%) name: 500 edges (5%) age: 500 edges (5%) Step 2: Spin up 1x go thread for each predicate. — Reply to this email directly, view it on GitHub <#9727?email_source=notifications&email_token=AAKQEVI5MWHVFIVV4OUOZNL5ANENDA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZTGQ2TKMBSUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVRTG633UMVZF6Y3MNFRWW#discussioncomment-17345502>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKQEVL7MIG32M3UK375C535ANENDAVCNFSNUABHKJSXA33TNF2G64TZHM2DCMZUHEYDGOJ3IRUXGY3VONZWS33OHMYTAMJWGIZTGNVBOYBA> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/AAKQEVNMYR4JSDUUPSGY6IL5ANENDA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZTGQ2TKMBSUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVJTG633UMVZF62LPOM> and Android <https://github.com/notifications/mobile/android/AAKQEVIDFONNJGZIJY6ZLJL5ANENDA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNZTGQ2TKMBSUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVZTG633UMVZF6YLOMRZG62LE>. Download it today! You are receiving this because you authored the thread.Message ID: ***@***.***>

-- Matthew McNeely m: +1603-216-7071 https://matthewmcneely.github.io https://github.com/matthewmcneely https://linkedin.com/in/matthewmcneely

[rahst12]

I have not validated the graph afterwards. I do have robust error tracking for input files and during the ingest process. Do you think these changes warrant graph validity checks post-ingest?

I kicked off another test with N=100. I should have results on Monday.

[matthewmcneely]

Hi Ryan,

OK, great. Progress.

Yes, one edge is equivalent (more or less) to one rdf line.

We will set the default mutations-pipeline-threshold to some number when this ships, still trying to find the sweet spot.

One question for you: are you also evaluating the correctness of the graph following the ingest?