REMOP: Remote-Memory-aware Operator Optimization

REMOP is an operator optimization framework for database query processing under memory disaggregation. When local memory is limited and an operator must swap pages to a remote memory pool, each swap event pays a fixed network round-trip cost. REMOP minimizes the number of these transfer rounds by tuning how operators partition their internal buffers.

REMOP is implemented in DuckDB v1.0.0 with REMON (Remote Memory over the Network) as the remote memory backend.

Repository Layout

├── cdb/                     # REMON submodule
│   ├── compute_node/        #   compute node backend
│   └── memory_node/         #   remote memory node server
├── src/
│   ├── execution/
│   │   ├── operator/join/   # Optimized blocked NLJ and external hash join operators
│   │   ├── operator/order/  # Optimized k-way merge sort operator
│   │   ├── operator_memory_policy.cpp
│   │   └── physical_plan/   # Plan-level integration
│   ├── common/sort/         # K-way merge sorter, partitioner, merge tree
│   └── common/prefetch_buffer.hpp  # Per-operator prefetch double buffer
└── scripts/
    └── build_remon.sh

Prerequisites

• OS: Ubuntu 20.04+
• Compiler: GCC/G++ 9.3.0+
• CMake ≥ 3.16.3

sudo apt-get install -y build-essential cmake git libssl-dev

Quick Start

REMON Submodule

To add the REMON submodule, clone REMOP with --recursive, or run the following after cloning:

git submodule update --init

Build Without REMON

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -- -j "$(nproc)"
./build/duckdb

Build With REMON

./scripts/build_remon.sh --clean

Or manually:

# 1. Build libremon.so
cmake -S cdb/compute_node -B cdb/compute_node/build-release \
  -DCMAKE_BUILD_TYPE=Release
cmake --build cdb/compute_node/build-release -- -j "$(nproc)"

# 2. Build DuckDB with REMON
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_WITH_REMON=ON
cmake --build build -- -j "$(nproc)"

Run With REMON

Before starting DuckDB, set up and run the memory node on the remote host. For REMON configuration (memory node setup, config file, two-node deployment), please see the REMON README.

export LD_LIBRARY_PATH=cdb/compute_node/build-release:$LD_LIBRARY_PATH
./build/duckdb

REMOP Runtime Settings

REMOP exposes its operator policies as DuckDB runtime settings:

SET threads = 16;
SET operator_memory_budget = '64MB';

-- Blocked nested-loop join and k-way merge sort
SET operator_nlj_input_buffer_ratio = 0.4;
SET operator_nlj_short_table_buffer_ratio = 0.6;
SET operator_kway_merge_k = 16;
SET operator_sort_input_buffer_ratio = 0.6;

-- External hash join
SET operator_hj_radix_bits = 4;
SET operator_hj_build_partition_min_handles = 8;
SET operator_hj_probe_partition_buffer_rows = 512;

-- Per-operator prefetch double buffering
SET operator_bnlj_prefetch_enabled = true;
SET operator_ems_prefetch_enabled = true;
SET operator_hj_prefetch_enabled = true;

SET operator_memory_optimizations = true;

Enabling operator_memory_optimizations makes the query planner substitute the default nested-loop join, sort, and hash join operators with REMOP's optimized variants; per-operator prefetching is controlled via the *_prefetch_enabled settings.

License

REMOP inherits DuckDB's MIT License.