Introduce network batching

[Rich-T-kid]

TODO

[Rich-T-kid]

running distributed benchmarks, will share results

[Rich-T-kid]

Results from my branch

Query | p50 Time (ms)
-- | --
q1 | 487
q2 | 795
q3 | 943
q4 | 388
q5 | 1016
q6 | 299
q7 | 1100
q8 | 1357
q9 | 1538
q10 | 1283
q11 | 632
q12 | 640
q13 | 756
q14 | 578
q15 | 868
q16 | 428
q17 | 943
q18 | 1185
q19 | 576
q20 | 984
q21 | 1243
q22 | 359

results from main branch

Query | p50 Time (ms)
-- | --
q1 | 526
q2 | 874
q3 | 982
q4 | 504
q5 | 1109
q6 | 368
q7 | 1310
q8 | 1587
q9 | 1621
q10 | 1299
q11 | 605
q12 | 535
q13 | 777
q14 | 553
q15 | 1013
q16 | 465
q17 | 1021
q18 | 1453
q19 | 604
q20 | 1127
q21 | 1276
q22 | 442

Diff

=== Comparing tpch_sf10 results from engine 'datafusion-distributed-introduce-network-batching' [prev] with 'datafusion-distributed-main' [new] ===
      q1: prev= 487 ms, new= 526 ms, diff=1.08 slower ✖
      q2: prev= 795 ms, new= 874 ms, diff=1.10 slower ✖
      q3: prev= 943 ms, new= 982 ms, diff=1.04 slower ✖
      q4: prev= 388 ms, new= 504 ms, diff=1.30 slower ❌
      q5: prev=1016 ms, new=1109 ms, diff=1.09 slower ✖
      q6: prev= 299 ms, new= 368 ms, diff=1.23 slower ❌
      q7: prev=1100 ms, new=1310 ms, diff=1.19 slower ✖
      q8: prev=1357 ms, new=1587 ms, diff=1.17 slower ✖
      q9: prev=1538 ms, new=1621 ms, diff=1.05 slower ✖
     q10: prev=1283 ms, new=1299 ms, diff=1.01 slower ✖
     q11: prev= 632 ms, new= 605 ms, diff=1.04 faster ✔
     q12: prev= 640 ms, new= 535 ms, diff=1.20 faster ✔
     q13: prev= 756 ms, new= 777 ms, diff=1.03 slower ✖
     q14: prev= 578 ms, new= 553 ms, diff=1.05 faster ✔
     q15: prev= 868 ms, new=1013 ms, diff=1.17 slower ✖
     q16: prev= 428 ms, new= 465 ms, diff=1.09 slower ✖
     q17: prev= 943 ms, new=1021 ms, diff=1.08 slower ✖
     q18: prev=1185 ms, new=1453 ms, diff=1.23 slower ❌
     q19: prev= 576 ms, new= 604 ms, diff=1.05 slower ✖
     q20: prev= 984 ms, new=1127 ms, diff=1.15 slower ✖
     q21: prev=1243 ms, new=1276 ms, diff=1.03 slower ✖
     q22: prev= 359 ms, new= 442 ms, diff=1.23 slower ❌
   TOTAL: prev=18398 ms, new=20051 ms, diff=1.09 slower ✖

for reference this is with the flight_send_buffer_factor set to 4 so ~32k rows

[Rich-T-kid]

closing this in favor of #543