Metis

Overview of Metis.

Metis is a workload-aware cardinality estimation system based on single-column histograms and multidimensional histograms (MD-Hists). It builds statistics from a training workload and uses them to estimate cardinalities for unseen predicates.

This repository contains the Java implementation for histogram construction and estimation, plus a PostgreSQL extension under pg_metis/.

Requirements

• Java 21
• Maven 3.9+
• Gurobi Optimizer 13.0.0 with a valid local license

Metis uses Gurobi to solve optimization problems during histogram construction. Before running Metis, install Gurobi Optimizer 13.0.0 and configure a valid license on your machine. Academic users may apply for a free academic license through Gurobi's official website. This repository does not redistribute Gurobi binaries or a prebuilt Metis jar that contains Gurobi classes.

After installing Gurobi, register its Java API jar in the local Maven repository:

export GUROBI_HOME=/path/to/gurobi1300/linux64

mvn install:install-file \
  -Dfile="$GUROBI_HOME/lib/gurobi.jar" \
  -DgroupId=gurobi \
  -DartifactId=gurobi \
  -Dversion=13.0.0 \
  -Dpackaging=jar

Build Metis from source:

mvn -DskipTests package

The executable jar is generated at:

target/metis.jar

Data and Workloads

The validation setup uses three single-table datasets:

Expr_Data/census.csv
Expr_Data/date.csv
Expr_Data/order_wide_clean.csv

and four workload-aware train/test scenarios for each dataset:

Data_workloadAware/Census/
Data_workloadAware/DMV/
Data_workloadAware/Instacart/

Data and workload files are intentionally ignored by git. The full datasets and workloads used in the paper are available on Google Drive. After cloning the repository, download or copy them into the paths above. Workload files are named W0_train.json, W0_test.json, ..., W3_train.json, W3_test.json.

Run Metis

Command format:

java -jar target/metis.jar \
  <train.json> \
  <test.json> \
  <table.csv> \
  [--withHeader] \
  [--statsDir=<dir>] \
  [--limit=<size_kb>]

DMV W0 end-to-end example:

LC_ALL=C java -jar target/metis.jar \
  Data_workloadAware/DMV/W0_train.json \
  Data_workloadAware/DMV/W0_test.json \
  Expr_Data/date.csv \
  --withHeader \
  --statsDir=metis_stats/dmv/current

The run writes single-column statistics, MD-Hist JSON files, constraint metadata, and Q-error logs under --statsDir and qErrorResult/.

For a quick environment and command-path check, append --limit=1. Use the full DMV command above when generating statistics for PostgreSQL integration.

Run examples for the supported datasets:

# Census
LC_ALL=C java -jar target/metis.jar Data_workloadAware/Census/W0_train.json Data_workloadAware/Census/W0_test.json Expr_Data/census.csv

# DMV
LC_ALL=C java -jar target/metis.jar Data_workloadAware/DMV/W0_train.json Data_workloadAware/DMV/W0_test.json Expr_Data/date.csv --withHeader

# Instacart
LC_ALL=C java -jar target/metis.jar Data_workloadAware/Instacart/W0_train.json Data_workloadAware/Instacart/W0_test.json Expr_Data/order_wide_clean.csv --withHeader

PostgreSQL Integration

Integrating Metis with PostgreSQL.

Metis supports two PostgreSQL implementations through the same extension-based planner-hook integration path:

• out-of-process implementation: the PostgreSQL extension constructs an estimator request and invokes the Java Metis estimator through a service interface;
• in-process implementation: the PostgreSQL extension executes the C Metis estimator directly inside the PostgreSQL server process.

The full setup, including Docker image pull, container naming, DMV table import, extension build, and both implementations, is in docs/POSTGRESQL_INTEGRATION.md.

License

This project is licensed under the terms of the LICENSE file.