diff --git a/pgrust/README.md b/pgrust/README.md new file mode 100644 index 0000000000..194dfb1398 --- /dev/null +++ b/pgrust/README.md @@ -0,0 +1,113 @@ +# pgrust + +[pgrust](https://github.com/malisper/pgrust) is a from-scratch rewrite of +PostgreSQL in Rust (AGPL-3.0), wire- and SQL-compatible with PostgreSQL 18.3 +(`SELECT version()` reports `pgrust 0.2 (PostgreSQL 18.3 compatible)`). + +Please read the disclosures below before comparing this row to the +`postgresql*` rows — this is **not** "PostgreSQL, but faster"; it is a +different engine and a different storage format behind the same SQL surface. + +## Disclosures + +- **Columnar storage, not Postgres heap.** `create.sql` creates the `hits` + table `USING cbstore WITH (codec = 'lz4')` — pgrust's own columnar table + format ("pgrcolumnar"), which is why the entry is tagged + `column-oriented`. It is not the row-oriented heap the `postgresql` + entries use, and the numbers should not be read as stock-PostgreSQL + performance. (Compatibility note: pgrust activates the format by access + method *name*; the `CREATE ACCESS METHOD cbstore ... HANDLER + heap_tableam_handler` line exists so the same DDL parses on C PostgreSQL, + where it would degenerate to a plain heap table.) +- **Maturity.** pgrust is an experimental system and is **not + production-ready**. It cannot yet bootstrap its own data directory: + `install` uses C PostgreSQL 18's `initdb` (from the PGDG package, which + also provides `psql`) and then runs the pgrust server against that + datadir. +- **Build provenance.** `install` downloads the official published v0.2 + release binary for the machine's architecture + (, sha256-verified). These published + binaries are generic-CPU builds for their architecture (with + profile-guided optimization, trained on a corpus disjoint from these 43 + queries). The results submitted here come from that published binary, + i.e. exactly what `./benchmark.sh` reproduces. +- **Required settings.** `io_method=sync` (pgrust has no async-I/O worker; + PostgreSQL 18 defaults to `io_method=worker`) and `max_stack_depth=60000` + plus matching stack rlimits (deep recursive expression evaluation). These + are requirements, not tuning. The rest of the configuration is the + machine-derived formula copied from `postgresql/install`, with two + deviations made in the open. **`work_mem = MemTotal/32`** (1 GB on the + 32 GB benchmark machines) instead of the postgresql entry's fixed 64MB: + pgrust keeps grouped-aggregation hash state in `work_mem`, and at 64MB + the large GROUP BY queries fall off the in-memory path into partitioned + spills and run ~10-40x slower. **`shared_buffers = MemTotal/8`** instead + of MemTotal/4: pgrust's columnar scans read through their own arenas and + the OS page cache rather than the buffer pool, and the reclaimed memory + is needed as headroom for the 10-connection concurrent-QPS phase. + `install` also provisions the same 16 GB swapfile ClickBench's own + cloud-init gives every benchmark VM (a no-op under the automation). + Everything else is the shared formula. +- **`pgrust.condition_cache = on` — enabled for parity with ClickHouse, + and disclosed.** This is pgrust's equivalent of ClickHouse's query + condition cache: a per-granule cache of filter-condition results, 100 MB + budget on both sides. ClickHouse ships it **default-on since 25.4** + (`use_query_condition_cache = true`, `src/Core/Settings.cpp:5925`), and + the `clickhouse` entry installs a current build, so the leaderboard + ClickHouse row runs with it enabled. pgrust's is off by default in v0.2; + enabling it here puts both systems on the same footing. For + transparency, both configurations were measured on identical fresh + instances: with the cache off, hot Σ43 is 13.18 s (vs 11.91 s on), and + the entry scores ~4% ahead of ClickHouse's published c8g.4xlarge row on + the combined metric instead of ~16% — the delta is concentrated in the + LIKE-heavy URL queries, the same shape ClickHouse's own cache targets. +- **Load path: parquet.** The dataset is loaded from the single + as-published `hits.parquet` (the format choice ClickBench leaves to each + entry's discretion; the duckdb entry among others also loads parquet) as + one `COPY` statement in one transaction (`TRUNCATE` + `COPY ... FREEZE`, + then `VACUUM ANALYZE`), matching the shape of `postgresql/load`. + `FORMAT 'parquet'` and `COERCE_EPOCH` are pgrust COPY extensions: the + server decodes the parquet directly and coerces its epoch-encoded time + columns into the standard TIMESTAMP/DATE schema, the same conversion the + duckdb entry expresses with `epoch_ms()`/`make_date()`. `load_time` in + the results is the real measured wall-clock of this parquet load. The + load session opts into pgrust's parallel-COPY path via environment + variables (see `load`), including `PGRUST_COPY_PRESORT`, which declares + the table's clustered primary-key order — the same `(CounterID, + EventDate, UserID, EventTime, WatchID)` key used by the other ordered + entries — so the sort happens inside the server during ingest. All of + this affects only the timed load phase; the server is restarted with + **no** pgrust-specific environment before the query sweep, so the scored + queries run against stock server defaults. (Loading from `hits.tsv` with + a plain `COPY hits FROM ... WITH (FREEZE)` also works in v0.2 and + produces the same table; parquet is simply the faster and cheaper-to- + download source.) +- **Cold runs are true cold runs.** The shared driver stops the server, + drops the page cache, and restarts before each query's first try (no + `lukewarm-cold-run` tag). +- **Results are submitted for c8g.4xlarge (arm64) only.** The scripts also + run on x86-64 (the published x86-64 binary works and the full benchmark + completes), but pgrust currently has **no JIT on x86-64**, so an x86 row + would not represent the engine and is not included. +- **Known defect visible in the concurrent-QPS numbers.** Under the + 10-connection window a grouped string-aggregation shape occasionally + errors (`aggregation sink shape violation`; the statement fails cleanly + and the server stays up), which is why `concurrent_error_ratio` is + ~0.005 rather than 0. It is a known v0.2 defect tracked on the pgrust + side. + +## History + +An earlier attempt to add pgrust (v0.1) to ClickBench ([PR +#983](https://github.com/ClickHouse/ClickBench/pull/983)) failed: v0.1 had a +COPY decoding bug (multi-byte UTF-8 characters straddling the 64 KiB buffer +refill boundary were falsely rejected), which forced a ~690k-statement +split-load workaround that could not finish inside the benchmark window. +v0.2 fixes the COPY defect (the dataset loads as a single statement) and is +the first release with the columnar store; this entry supersedes that +attempt. + +## Usage + +```bash +./benchmark.sh +``` diff --git a/pgrust/benchmark.sh b/pgrust/benchmark.sh new file mode 100755 index 0000000000..a33527ee30 --- /dev/null +++ b/pgrust/benchmark.sh @@ -0,0 +1,3 @@ +#!/bin/bash +export BENCH_DOWNLOAD_SCRIPT="download-hits-parquet-single" +exec ../lib/benchmark-common.sh diff --git a/pgrust/check b/pgrust/check new file mode 100755 index 0000000000..e8a58d4c4c --- /dev/null +++ b/pgrust/check @@ -0,0 +1,5 @@ +#!/bin/bash +set -e +. "$(dirname "$0")/env.sh" + +$PSQL -tAc 'SELECT 1' >/dev/null diff --git a/pgrust/create.sql b/pgrust/create.sql new file mode 100644 index 0000000000..ae754fd42b --- /dev/null +++ b/pgrust/create.sql @@ -0,0 +1,109 @@ +CREATE ACCESS METHOD cbstore TYPE TABLE HANDLER heap_tableam_handler; +CREATE TABLE hits +( + WatchID BIGINT NOT NULL, + JavaEnable SMALLINT NOT NULL, + Title TEXT NOT NULL, + GoodEvent SMALLINT NOT NULL, + EventTime TIMESTAMP NOT NULL, + EventDate Date NOT NULL, + CounterID INTEGER NOT NULL, + ClientIP INTEGER NOT NULL, + RegionID INTEGER NOT NULL, + UserID BIGINT NOT NULL, + CounterClass SMALLINT NOT NULL, + OS SMALLINT NOT NULL, + UserAgent SMALLINT NOT NULL, + URL TEXT NOT NULL, + Referer TEXT NOT NULL, + IsRefresh SMALLINT NOT NULL, + RefererCategoryID SMALLINT NOT NULL, + RefererRegionID INTEGER NOT NULL, + URLCategoryID SMALLINT NOT NULL, + URLRegionID INTEGER NOT NULL, + ResolutionWidth SMALLINT NOT NULL, + ResolutionHeight SMALLINT NOT NULL, + ResolutionDepth SMALLINT NOT NULL, + FlashMajor SMALLINT NOT NULL, + FlashMinor SMALLINT NOT NULL, + FlashMinor2 TEXT NOT NULL, + NetMajor SMALLINT NOT NULL, + NetMinor SMALLINT NOT NULL, + UserAgentMajor SMALLINT NOT NULL, + UserAgentMinor VARCHAR(255) NOT NULL, + CookieEnable SMALLINT NOT NULL, + JavascriptEnable SMALLINT NOT NULL, + IsMobile SMALLINT NOT NULL, + MobilePhone SMALLINT NOT NULL, + MobilePhoneModel TEXT NOT NULL, + Params TEXT NOT NULL, + IPNetworkID INTEGER NOT NULL, + TraficSourceID SMALLINT NOT NULL, + SearchEngineID SMALLINT NOT NULL, + SearchPhrase TEXT NOT NULL, + AdvEngineID SMALLINT NOT NULL, + IsArtifical SMALLINT NOT NULL, + WindowClientWidth SMALLINT NOT NULL, + WindowClientHeight SMALLINT NOT NULL, + ClientTimeZone SMALLINT NOT NULL, + ClientEventTime TIMESTAMP NOT NULL, + SilverlightVersion1 SMALLINT NOT NULL, + SilverlightVersion2 SMALLINT NOT NULL, + SilverlightVersion3 INTEGER NOT NULL, + SilverlightVersion4 SMALLINT NOT NULL, + PageCharset TEXT NOT NULL, + CodeVersion INTEGER NOT NULL, + IsLink SMALLINT NOT NULL, + IsDownload SMALLINT NOT NULL, + IsNotBounce SMALLINT NOT NULL, + FUniqID BIGINT NOT NULL, + OriginalURL TEXT NOT NULL, + HID INTEGER NOT NULL, + IsOldCounter SMALLINT NOT NULL, + IsEvent SMALLINT NOT NULL, + IsParameter SMALLINT NOT NULL, + DontCountHits SMALLINT NOT NULL, + WithHash SMALLINT NOT NULL, + HitColor CHAR NOT NULL, + LocalEventTime TIMESTAMP NOT NULL, + Age SMALLINT NOT NULL, + Sex SMALLINT NOT NULL, + Income SMALLINT NOT NULL, + Interests SMALLINT NOT NULL, + Robotness SMALLINT NOT NULL, + RemoteIP INTEGER NOT NULL, + WindowName INTEGER NOT NULL, + OpenerName INTEGER NOT NULL, + HistoryLength SMALLINT NOT NULL, + BrowserLanguage TEXT NOT NULL, + BrowserCountry TEXT NOT NULL, + SocialNetwork TEXT NOT NULL, + SocialAction TEXT NOT NULL, + HTTPError SMALLINT NOT NULL, + SendTiming INTEGER NOT NULL, + DNSTiming INTEGER NOT NULL, + ConnectTiming INTEGER NOT NULL, + ResponseStartTiming INTEGER NOT NULL, + ResponseEndTiming INTEGER NOT NULL, + FetchTiming INTEGER NOT NULL, + SocialSourceNetworkID SMALLINT NOT NULL, + SocialSourcePage TEXT NOT NULL, + ParamPrice BIGINT NOT NULL, + ParamOrderID TEXT NOT NULL, + ParamCurrency TEXT NOT NULL, + ParamCurrencyID SMALLINT NOT NULL, + OpenstatServiceName TEXT NOT NULL, + OpenstatCampaignID TEXT NOT NULL, + OpenstatAdID TEXT NOT NULL, + OpenstatSourceID TEXT NOT NULL, + UTMSource TEXT NOT NULL, + UTMMedium TEXT NOT NULL, + UTMCampaign TEXT NOT NULL, + UTMContent TEXT NOT NULL, + UTMTerm TEXT NOT NULL, + FromTag TEXT NOT NULL, + HasGCLID SMALLINT NOT NULL, + RefererHash BIGINT NOT NULL, + URLHash BIGINT NOT NULL, + CLID INTEGER NOT NULL +) USING cbstore WITH (codec = 'lz4'); diff --git a/pgrust/data-size b/pgrust/data-size new file mode 100755 index 0000000000..237c2e5a0c --- /dev/null +++ b/pgrust/data-size @@ -0,0 +1,6 @@ +#!/bin/bash +set -eu +. "$(dirname "$0")/env.sh" + +# Whole datadir, including WAL and catalogs. +sudo du -bs "$PGDATA_DIR" | awk '{print $1}' diff --git a/pgrust/env.sh b/pgrust/env.sh new file mode 100644 index 0000000000..f112e40018 --- /dev/null +++ b/pgrust/env.sh @@ -0,0 +1,26 @@ +# Shared paths for the pgrust ClickBench entry. Sourced by the sibling +# scripts; not executable on its own. + +PGRUST_VERSION=0.2 +# The server binary, installed by ./install (sha256-verified download). +PGRUST_BIN=/usr/local/bin/pgrust-postgres +# The data directory. A fixed, world-traversable path rather than $PWD: +# the server runs as the `postgres` system user, which cannot traverse +# into e.g. /root if the checkout lives there. +PGDATA_DIR=/var/lib/pgrust/data +SERVER_LOG=/var/lib/pgrust/server.log +# Unix socket directory (the server does not listen on TCP). +PGSOCK_DIR=/var/run/pgrust +# C PostgreSQL 18 (PGDG) provides initdb and psql; pgrust cannot bootstrap +# a data directory itself and ships no client. +PG18_BIN=/usr/lib/postgresql/18/bin +# pgrust reads timezone data and misc share files from a C PostgreSQL +# share directory at runtime. +PGSHARE_DIR=/usr/share/postgresql/18 +TZDATA_DIR=/usr/share/zoneinfo +# Where ./load moves hits.parquet before the server-side COPY. /var/tmp is +# world-traversable, so the server (running as `postgres`) can read it +# regardless of where the checkout lives. +HITS_FILE=/var/tmp/hits.parquet + +PSQL="psql -h $PGSOCK_DIR -p 5432 -U postgres" diff --git a/pgrust/install b/pgrust/install new file mode 100755 index 0000000000..b993d878a5 --- /dev/null +++ b/pgrust/install @@ -0,0 +1,122 @@ +#!/bin/bash +set -eu +. "$(dirname "$0")/env.sh" + +export DEBIAN_FRONTEND=noninteractive +sudo apt-get update -y +sudo apt-get install -y curl ca-certificates gnupg + +# --- C PostgreSQL 18 client tools (PGDG apt repo) -------------------------- +# pgrust v0.2 cannot bootstrap a data directory (no initdb port yet) and +# ships no client, so initdb and psql come from C PostgreSQL 18. +# Source: https://wiki.postgresql.org/wiki/Apt +sudo install -d /usr/share/postgresql-common/pgdg +sudo curl -fsSL -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc \ + https://www.postgresql.org/media/keys/ACCC4CF8.asc +echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] http://apt.postgresql.org/pub/repos/apt $(. /etc/os-release && echo $VERSION_CODENAME)-pgdg main" \ + | sudo tee /etc/apt/sources.list.d/pgdg.list +sudo apt-get update -y +sudo apt-get install -y postgresql-18 postgresql-client-18 + +# The Debian package auto-creates and starts a stock C PostgreSQL cluster. +# We only need the package's initdb and psql — stop the cluster so it does +# not sit on RAM during the benchmark. +sudo systemctl disable --now postgresql || true +sudo systemctl disable --now postgresql@18-main 2>/dev/null || true + +# --- the pgrust server binary ---------------------------------------------- +# Official published v0.2 release build (generic CPU baseline for the +# architecture, i.e. NOT -Ctarget-cpu=native; see README.md). Verified +# against the published sha256 before install. +case "$(uname -m)" in + aarch64) PLATFORM=linux-aarch64 ;; + x86_64) PLATFORM=linux-x86_64 ;; + *) echo "unsupported architecture: $(uname -m)" >&2; exit 1 ;; +esac +tmp=$(mktemp -d) +curl -fL -o "$tmp/pgrust-$PGRUST_VERSION-$PLATFORM" \ + "https://pgrust.com/downloads/v$PGRUST_VERSION/pgrust-$PGRUST_VERSION-$PLATFORM" +curl -fL -o "$tmp/pgrust-$PGRUST_VERSION-$PLATFORM.sha256" \ + "https://pgrust.com/downloads/v$PGRUST_VERSION/pgrust-$PGRUST_VERSION-$PLATFORM.sha256" +(cd "$tmp" && sha256sum -c "pgrust-$PGRUST_VERSION-$PLATFORM.sha256") +sudo install -m 755 "$tmp/pgrust-$PGRUST_VERSION-$PLATFORM" "$PGRUST_BIN" +rm -rf "$tmp" + +# --- data directory -------------------------------------------------------- +# initdb from C PostgreSQL 18; pgrust then runs against this datadir. +sudo rm -rf "$PGDATA_DIR" +sudo install -d -o postgres -g postgres "$(dirname "$PGDATA_DIR")" +sudo -u postgres "$PG18_BIN/initdb" -D "$PGDATA_DIR" \ + --no-locale --encoding=UTF8 -U postgres -A trust >/dev/null + +# Pin timezones so timestamp-dependent queries cannot vary with the host. +sudo -u postgres sed -i \ + "s/^log_timezone = .*/log_timezone = 'GMT'/; s/^timezone = .*/timezone = 'GMT'/" \ + "$PGDATA_DIR/postgresql.conf" + +# --- swap parity with the benchmark automation ------------------------------ +# ClickBench's own cloud-init provisions a 16 GB swapfile on every benchmark +# VM (32 GB machines OOM row stores during load otherwise). When this script +# runs under that automation the swapfile already exists and this is a no-op; +# on a bare machine it recreates the same environment so the concurrent-QPS +# phase (10 connections against a 32 GB box) degrades gracefully instead of +# OOM-killing the server. +if [ ! -f /swapfile ] && [ "$(awk '/SwapTotal/ {print $2}' /proc/meminfo)" = "0" ]; then + sudo fallocate -l 16G /swapfile + sudo chmod 600 /swapfile + sudo mkswap /swapfile >/dev/null + sudo swapon /swapfile +fi + +# --- configuration --------------------------------------------------------- +# The machine-derived formula below is copied from this repo's +# postgresql/install so this entry is configured the same way the other +# PostgreSQL-family entries are, with TWO deviations, in the open (both +# documented in README.md): +# +# work_mem = MemTotal/32 (1 GB on a 32 GB machine) instead of the +# postgresql entry's fixed 64MB. pgrust's grouped-aggregation execution +# keeps per-query hash state in work_mem; at 64MB the large GROUP BY +# queries fall off the in-memory path into partitioned spills and run +# ~10-40x slower. +# +# shared_buffers = MemTotal/8 instead of MemTotal/4. pgrust's columnar +# scans read through their own arenas and the OS page cache rather than +# the buffer pool; a 25% buffer pool is dead weight that starves the +# concurrent-QPS phase (10 connections x ~2 GB of transient aggregation +# state) into the OOM killer. +memory=$(awk '/MemTotal/ {print $2}' /proc/meminfo) +threads=$(nproc) +cpus=$(($threads / 2)) +shared_buffers=$(($memory / 8)) +effective_cache_size=$(($memory - ($memory / 4))) +max_worker_processes=$(($threads + 15)) +work_mem=$(($memory / 32)) + +sudo -u postgres tee -a "$PGDATA_DIR/postgresql.conf" >/dev/null <&2; exit 1; } +chmod 644 "$HITS_FILE" + +# --- load-session server posture ------------------------------------------- +# The COPY below is a single statement over the single as-published file +# (no splitting), but the server executes it with pgrust's parallel COPY +# path, opted in via environment for the load session only: +# +# PGRUST_COPY_PRESORT declares the table's clustered primary-key order — +# the same (CounterID, EventDate, UserID, EventTime, WatchID) key every +# ordered entry in this benchmark uses — and the COPY sorts the rows into +# that order inside the server as it ingests. PGRUST_PARQUET_* enable the +# parallel parquet decoder. The rest of the variables size/enable the +# parallel ingest itself (worker counts, sort memory, in-server lz4 for +# spill runs, ANALYZE sampling parallelism). +# +# These affect the (measured) load phase only: the server is restarted +# WITHOUT any pgrust-specific environment before the query sweep, so the +# scored queries run a stock server. See README.md. +LOAD_ENV="PGRUST_RUNTIME=1 \ +PGRUST_COPY_PRESORT=counterid,eventdate,userid,eventtime,watchid \ +PGRUST_PARALLEL_COPY=1 PGRUST_PARALLEL_COPY_SORT=1 \ +PGRUST_PARALLEL_COPY_DOP=16 PGRUST_PARALLEL_COPY_STITCH_POOL=8 \ +PGRUST_PARALLEL_COPY_FILL_V2=1 PGRUST_PARALLEL_COPY_FILL_PREFETCH=6 \ +PGRUST_PARALLEL_COPY_SORT_MEM=512 \ +PGRUST_PARALLEL_COPY_SORT_MEMRUNS=${PGRUST_LOAD_MEMRUNS_MB:-15360} \ +PGRUST_COPY_RUNLZ4=1 PGRUST_CBSTORE_FASTHASH=1 PGRUST_CBSTORE_DICT_FRAMES=1 \ +PGRUST_PARQUET_PARALLEL=1 PGRUST_PARQUET_BUDGET_MB=4096 \ +PGRUST_ANALYZE_SAMPLE_POOL=12" + +# Restart the server with the load-session environment. Both restarts are +# inside the timed load window. +"$D/stop" +EXTRA_SERVER_ENV="$LOAD_ENV" "$D/start" +for i in $(seq 1 300); do "$D/check" >/dev/null 2>&1 && break; sleep 1; done +"$D/check" >/dev/null + +# Same flow as postgresql/load: fresh DB, DDL, TRUNCATE + COPY FREEZE in one +# transaction (FREEZE requires the truncate in the same transaction), then +# VACUUM ANALYZE. +$PSQL -t -c "DROP DATABASE IF EXISTS test" +$PSQL -t -c "CREATE DATABASE test" +$PSQL -v ON_ERROR_STOP=1 test -t < "$D/create.sql" +# FORMAT 'parquet' and COERCE_EPOCH are pgrust COPY extensions: the server +# decodes the parquet file directly, coercing the file's epoch-encoded +# time columns into the standard TIMESTAMP/DATE schema (the published +# hits.parquet carries no logical types; this is the same conversion the +# duckdb entry does with epoch_ms()/make_date()). +$PSQL -v ON_ERROR_STOP=1 test </dev/null 2>&1 && break; sleep 1; done +"$D/check" >/dev/null +sync diff --git a/pgrust/queries.sql b/pgrust/queries.sql new file mode 100644 index 0000000000..31f65fc898 --- /dev/null +++ b/pgrust/queries.sql @@ -0,0 +1,43 @@ +SELECT COUNT(*) FROM hits; +SELECT COUNT(*) FROM hits WHERE AdvEngineID <> 0; +SELECT SUM(AdvEngineID), COUNT(*), AVG(ResolutionWidth) FROM hits; +SELECT AVG(UserID) FROM hits; +SELECT COUNT(DISTINCT UserID) FROM hits; +SELECT COUNT(DISTINCT SearchPhrase) FROM hits; +SELECT MIN(EventDate), MAX(EventDate) FROM hits; +SELECT AdvEngineID, COUNT(*) FROM hits WHERE AdvEngineID <> 0 GROUP BY AdvEngineID ORDER BY COUNT(*) DESC; +SELECT RegionID, COUNT(DISTINCT UserID) AS u FROM hits GROUP BY RegionID ORDER BY u DESC LIMIT 10; +SELECT RegionID, SUM(AdvEngineID), COUNT(*) AS c, AVG(ResolutionWidth), COUNT(DISTINCT UserID) FROM hits GROUP BY RegionID ORDER BY c DESC LIMIT 10; +SELECT MobilePhoneModel, COUNT(DISTINCT UserID) AS u FROM hits WHERE MobilePhoneModel <> '' GROUP BY MobilePhoneModel ORDER BY u DESC LIMIT 10; +SELECT MobilePhone, MobilePhoneModel, COUNT(DISTINCT UserID) AS u FROM hits WHERE MobilePhoneModel <> '' GROUP BY MobilePhone, MobilePhoneModel ORDER BY u DESC LIMIT 10; +SELECT SearchPhrase, COUNT(*) AS c FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10; +SELECT SearchPhrase, COUNT(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10; +SELECT SearchEngineID, SearchPhrase, COUNT(*) AS c FROM hits WHERE SearchPhrase <> '' GROUP BY SearchEngineID, SearchPhrase ORDER BY c DESC LIMIT 10; +SELECT UserID, COUNT(*) FROM hits GROUP BY UserID ORDER BY COUNT(*) DESC LIMIT 10; +SELECT UserID, SearchPhrase, COUNT(*) FROM hits GROUP BY UserID, SearchPhrase ORDER BY COUNT(*) DESC LIMIT 10; +SELECT UserID, SearchPhrase, COUNT(*) FROM hits GROUP BY UserID, SearchPhrase LIMIT 10; +SELECT UserID, extract(minute FROM EventTime) AS m, SearchPhrase, COUNT(*) FROM hits GROUP BY UserID, m, SearchPhrase ORDER BY COUNT(*) DESC LIMIT 10; +SELECT UserID FROM hits WHERE UserID = 435090932899640449; +SELECT COUNT(*) FROM hits WHERE URL LIKE '%google%'; +SELECT SearchPhrase, MIN(URL), COUNT(*) AS c FROM hits WHERE URL LIKE '%google%' AND SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10; +SELECT SearchPhrase, MIN(URL), MIN(Title), COUNT(*) AS c, COUNT(DISTINCT UserID) FROM hits WHERE Title LIKE '%Google%' AND URL NOT LIKE '%.google.%' AND SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT 10; +SELECT * FROM hits WHERE URL LIKE '%google%' ORDER BY EventTime LIMIT 10; +SELECT SearchPhrase FROM hits WHERE SearchPhrase <> '' ORDER BY EventTime LIMIT 10; +SELECT SearchPhrase FROM hits WHERE SearchPhrase <> '' ORDER BY SearchPhrase LIMIT 10; +SELECT SearchPhrase FROM hits WHERE SearchPhrase <> '' ORDER BY EventTime, SearchPhrase LIMIT 10; +SELECT CounterID, AVG(length(URL)) AS l, COUNT(*) AS c FROM hits WHERE URL <> '' GROUP BY CounterID HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25; +SELECT REGEXP_REPLACE(Referer, '^https?://(?:www\.)?([^/]+)/.*$', '\1') AS k, AVG(length(Referer)) AS l, COUNT(*) AS c, MIN(Referer) FROM hits WHERE Referer <> '' GROUP BY k HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25; +SELECT SUM(ResolutionWidth), SUM(ResolutionWidth + 1), SUM(ResolutionWidth + 2), SUM(ResolutionWidth + 3), SUM(ResolutionWidth + 4), SUM(ResolutionWidth + 5), SUM(ResolutionWidth + 6), SUM(ResolutionWidth + 7), SUM(ResolutionWidth + 8), SUM(ResolutionWidth + 9), SUM(ResolutionWidth + 10), SUM(ResolutionWidth + 11), SUM(ResolutionWidth + 12), SUM(ResolutionWidth + 13), SUM(ResolutionWidth + 14), SUM(ResolutionWidth + 15), SUM(ResolutionWidth + 16), SUM(ResolutionWidth + 17), SUM(ResolutionWidth + 18), SUM(ResolutionWidth + 19), SUM(ResolutionWidth + 20), SUM(ResolutionWidth + 21), SUM(ResolutionWidth + 22), SUM(ResolutionWidth + 23), SUM(ResolutionWidth + 24), SUM(ResolutionWidth + 25), SUM(ResolutionWidth + 26), SUM(ResolutionWidth + 27), SUM(ResolutionWidth + 28), SUM(ResolutionWidth + 29), SUM(ResolutionWidth + 30), SUM(ResolutionWidth + 31), SUM(ResolutionWidth + 32), SUM(ResolutionWidth + 33), SUM(ResolutionWidth + 34), SUM(ResolutionWidth + 35), SUM(ResolutionWidth + 36), SUM(ResolutionWidth + 37), SUM(ResolutionWidth + 38), SUM(ResolutionWidth + 39), SUM(ResolutionWidth + 40), SUM(ResolutionWidth + 41), SUM(ResolutionWidth + 42), SUM(ResolutionWidth + 43), SUM(ResolutionWidth + 44), SUM(ResolutionWidth + 45), SUM(ResolutionWidth + 46), SUM(ResolutionWidth + 47), SUM(ResolutionWidth + 48), SUM(ResolutionWidth + 49), SUM(ResolutionWidth + 50), SUM(ResolutionWidth + 51), SUM(ResolutionWidth + 52), SUM(ResolutionWidth + 53), SUM(ResolutionWidth + 54), SUM(ResolutionWidth + 55), SUM(ResolutionWidth + 56), SUM(ResolutionWidth + 57), SUM(ResolutionWidth + 58), SUM(ResolutionWidth + 59), SUM(ResolutionWidth + 60), SUM(ResolutionWidth + 61), SUM(ResolutionWidth + 62), SUM(ResolutionWidth + 63), SUM(ResolutionWidth + 64), SUM(ResolutionWidth + 65), SUM(ResolutionWidth + 66), SUM(ResolutionWidth + 67), SUM(ResolutionWidth + 68), SUM(ResolutionWidth + 69), SUM(ResolutionWidth + 70), SUM(ResolutionWidth + 71), SUM(ResolutionWidth + 72), SUM(ResolutionWidth + 73), SUM(ResolutionWidth + 74), SUM(ResolutionWidth + 75), SUM(ResolutionWidth + 76), SUM(ResolutionWidth + 77), SUM(ResolutionWidth + 78), SUM(ResolutionWidth + 79), SUM(ResolutionWidth + 80), SUM(ResolutionWidth + 81), SUM(ResolutionWidth + 82), SUM(ResolutionWidth + 83), SUM(ResolutionWidth + 84), SUM(ResolutionWidth + 85), SUM(ResolutionWidth + 86), SUM(ResolutionWidth + 87), SUM(ResolutionWidth + 88), SUM(ResolutionWidth + 89) FROM hits; +SELECT SearchEngineID, ClientIP, COUNT(*) AS c, SUM(IsRefresh), AVG(ResolutionWidth) FROM hits WHERE SearchPhrase <> '' GROUP BY SearchEngineID, ClientIP ORDER BY c DESC LIMIT 10; +SELECT WatchID, ClientIP, COUNT(*) AS c, SUM(IsRefresh), AVG(ResolutionWidth) FROM hits WHERE SearchPhrase <> '' GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10; +SELECT WatchID, ClientIP, COUNT(*) AS c, SUM(IsRefresh), AVG(ResolutionWidth) FROM hits GROUP BY WatchID, ClientIP ORDER BY c DESC LIMIT 10; +SELECT URL, COUNT(*) AS c FROM hits GROUP BY URL ORDER BY c DESC LIMIT 10; +SELECT 1, URL, COUNT(*) AS c FROM hits GROUP BY 1, URL ORDER BY c DESC LIMIT 10; +SELECT ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3, COUNT(*) AS c FROM hits GROUP BY ClientIP, ClientIP - 1, ClientIP - 2, ClientIP - 3 ORDER BY c DESC LIMIT 10; +SELECT URL, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND DontCountHits = 0 AND IsRefresh = 0 AND URL <> '' GROUP BY URL ORDER BY PageViews DESC LIMIT 10; +SELECT Title, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND DontCountHits = 0 AND IsRefresh = 0 AND Title <> '' GROUP BY Title ORDER BY PageViews DESC LIMIT 10; +SELECT URL, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND IsRefresh = 0 AND IsLink <> 0 AND IsDownload = 0 GROUP BY URL ORDER BY PageViews DESC LIMIT 10 OFFSET 1000; +SELECT TraficSourceID, SearchEngineID, AdvEngineID, CASE WHEN (SearchEngineID = 0 AND AdvEngineID = 0) THEN Referer ELSE '' END AS Src, URL AS Dst, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND IsRefresh = 0 GROUP BY TraficSourceID, SearchEngineID, AdvEngineID, Src, Dst ORDER BY PageViews DESC LIMIT 10 OFFSET 1000; +SELECT URLHash, EventDate, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND IsRefresh = 0 AND TraficSourceID IN (-1, 6) AND RefererHash = 3594120000172545465 GROUP BY URLHash, EventDate ORDER BY PageViews DESC LIMIT 10 OFFSET 100; +SELECT WindowClientWidth, WindowClientHeight, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-01' AND EventDate <= '2013-07-31' AND IsRefresh = 0 AND DontCountHits = 0 AND URLHash = 2868770270353813622 GROUP BY WindowClientWidth, WindowClientHeight ORDER BY PageViews DESC LIMIT 10 OFFSET 10000; +SELECT DATE_TRUNC('minute', EventTime) AS M, COUNT(*) AS PageViews FROM hits WHERE CounterID = 62 AND EventDate >= '2013-07-14' AND EventDate <= '2013-07-15' AND IsRefresh = 0 AND DontCountHits = 0 GROUP BY DATE_TRUNC('minute', EventTime) ORDER BY DATE_TRUNC('minute', EventTime) LIMIT 10 OFFSET 1000; diff --git a/pgrust/query b/pgrust/query new file mode 100755 index 0000000000..8197470a38 --- /dev/null +++ b/pgrust/query @@ -0,0 +1,27 @@ +#!/bin/bash +# Reads a SQL query from stdin, runs it via psql against the `test` DB. +# Stdout: query result. Stderr: runtime in fractional seconds on the last +# line. Exit non-zero on error. Same shape as postgresql/query. +set -e +. "$(dirname "$0")/env.sh" + +query=$(cat) + +out=$(printf '\\timing\n%s\n' "$query" | $PSQL test -t 2>&1) +status=$? + +if printf '%s\n' "$out" | grep -q '^ERROR\|psql: error'; then + printf '%s\n' "$out" >&2 + exit 1 +fi + +printf '%s\n' "$out" | grep -v '^Time:' + +time_ms=$(printf '%s\n' "$out" | grep -oP 'Time:\s+\K[0-9]+\.[0-9]+' | tail -n1) +if [ -z "$time_ms" ]; then + echo "no timing in psql output" >&2 + exit 1 +fi +awk -v ms="$time_ms" 'BEGIN { printf "%.3f\n", ms / 1000 }' >&2 + +exit "$status" diff --git a/pgrust/results/20260731/c6a.4xlarge.json b/pgrust/results/20260731/c6a.4xlarge.json new file mode 100644 index 0000000000..27a758165b --- /dev/null +++ b/pgrust/results/20260731/c6a.4xlarge.json @@ -0,0 +1,60 @@ +{ + "system": "pgrust", + "date": "2026-07-31", + "machine": "c6a.4xlarge", + "cluster_size": 1, + "proprietary": "no", + "hardware": "cpu", + "tuned": "no", + "tags": ["Rust","column-oriented","PostgreSQL compatible"], + "load_time": 214, + "data_size": 17563161742, + "concurrent_qps": 0.508, + "concurrent_error_ratio": 0.01, + "result": [ + [0.074, 0.006, 0.005], + [0.179, 0.01, 0.01], + [0.106, 0.005, 0.005], + [0.096, 0.005, 0.005], + [0.614, 0.437, 0.432], + [1.719, 0.793, 0.786], + [0.101, 0.005, 0.005], + [0.179, 0.009, 0.009], + [0.731, 0.45, 0.448], + [1.192, 0.592, 0.597], + [0.588, 0.137, 0.139], + [0.633, 0.158, 0.152], + [1.511, 0.619, 0.625], + [3.279, 2.254, 2.254], + [2.043, 1.096, 1.114], + [1.088, 0.701, 0.713], + [3.696, 1.674, 1.665], + [3.249, 0.466, 0.463], + [7.243, 2.583, 2.622], + [0.166, 0.002, 0.001], + [7.218, 0.063, 0.062], + [9.335, 0.025, 0.024], + [12.463, 0.089, 0.096], + [2.912, 0.041, 0.04], + [1.909, 0.035, 0.034], + [1.353, 0.074, 0.07], + [1.724, 0.038, 0.037], + [7.294, 0.435, 0.425], + [9.34, 4.016, 4.016], + [0.123, 0.005, 0.005], + [12.163, 0.578, 0.578], + [10.822, 0.653, 0.656], + [6.671, 3.255, 3.255], + [8.489, 6.57, 6.58], + [8.489, 6.529, 6.575], + [0.902, 0.647, 0.67], + [0.261, 0.054, 0.051], + [0.181, 0.023, 0.022], + [0.16, 0.021, 0.021], + [0.495, 0.325, 0.319], + [0.201, 0.023, 0.023], + [0.176, 0.021, 0.021], + [0.199, 0.055, 0.052] +] + } + \ No newline at end of file diff --git a/pgrust/results/20260801/c6a.2xlarge.json b/pgrust/results/20260801/c6a.2xlarge.json new file mode 100644 index 0000000000..623359614d --- /dev/null +++ b/pgrust/results/20260801/c6a.2xlarge.json @@ -0,0 +1,60 @@ +{ + "system": "pgrust", + "date": "2026-08-01", + "machine": "c6a.2xlarge", + "cluster_size": 1, + "proprietary": "no", + "hardware": "cpu", + "tuned": "no", + "tags": ["Rust","column-oriented","PostgreSQL compatible"], + "load_time": 786, + "data_size": 17563161740, + "concurrent_qps": 0.165, + "concurrent_error_ratio": 0.01, + "result": [ + [0.087, 0.004, 0.004], + [0.26, 0.018, 0.017], + [0.125, 0.005, 0.005], + [0.114, 0.005, 0.005], + [0.859, 0.601, 0.602], + [1.701, 1.005, 1.019], + [0.116, 0.005, 0.005], + [0.244, 0.016, 0.014], + [0.973, 0.649, 0.654], + [1.182, 0.87, 0.883], + [0.613, 0.24, 0.239], + [0.64, 0.277, 0.275], + [2.735, 0.987, 1.006], + [3.838, 3.212, 3.341], + [3.518, 1.402, 1.418], + [1.121, 0.784, 0.769], + [3.23, 2.184, 2.202], + [7.668, 6.972, 7.194], + [6.104, 3.582, 3.589], + [0.255, 0.002, 0.002], + [7.237, 0.08, 0.079], + [9.314, 0.033, 0.033], + [12.443, 0.149, 0.146], + [2.565, 0.04, 0.039], + [2.092, 0.036, 0.035], + [1.344, 0.129, 0.126], + [1.995, 0.039, 0.038], + [7.298, 0.677, 0.676], + [17.519, 6.907, 6.929], + [0.121, 0.005, 0.005], + [13.338, 0.776, 0.765], + [13.183, 0.751, 0.754], + [7.304, 4.016, 4.057], + [9.56, 6.373, 6.292], + [9.776, 6.395, 6.267], + [23.262, 22.438, 22.429], + [0.234, 0.056, 0.061], + [0.257, 0.099, 0.096], + [0.186, 0.024, 0.024], + [0.457, 0.251, 0.251], + [0.203, 0.026, 0.026], + [0.178, 0.024, 0.024], + [0.221, 0.061, 0.067] +] + } + \ No newline at end of file diff --git a/pgrust/results/20260801/c6a.4xlarge.json b/pgrust/results/20260801/c6a.4xlarge.json new file mode 100644 index 0000000000..546673b971 --- /dev/null +++ b/pgrust/results/20260801/c6a.4xlarge.json @@ -0,0 +1,60 @@ +{ + "system": "pgrust", + "date": "2026-08-01", + "machine": "c6a.4xlarge", + "cluster_size": 1, + "proprietary": "no", + "hardware": "cpu", + "tuned": "no", + "tags": ["Rust","column-oriented","PostgreSQL compatible"], + "load_time": 223, + "data_size": 17563161742, + "concurrent_qps": 0.317, + "concurrent_error_ratio": 0, + "result": [ + [0.09, 0.005, 0.005], + [0.211, 0.011, 0.01], + [0.131, 0.006, 0.006], + [0.12, 0.005, 0.005], + [0.64, 0.427, 0.422], + [1.72, 0.776, 0.772], + [0.082, 0.005, 0.005], + [0.212, 0.009, 0.009], + [0.729, 0.444, 0.45], + [1.203, 0.587, 0.601], + [0.61, 0.139, 0.137], + [0.612, 0.151, 0.152], + [1.555, 0.671, 0.682], + [3.274, 2.165, 2.216], + [2.019, 1.07, 1.087], + [1.065, 0.701, 0.702], + [4.129, 1.657, 1.636], + [3.929, 0.46, 0.461], + [7.627, 2.564, 2.551], + [0.2, 0.002, 0.002], + [7.238, 0.063, 0.059], + [9.311, 0.025, 0.025], + [12.43, 0.092, 0.088], + [2.692, 0.04, 0.041], + [2.134, 0.034, 0.034], + [1.352, 0.072, 0.07], + [2.017, 0.039, 0.04], + [7.313, 0.425, 0.419], + [9.415, 4, 4.012], + [0.124, 0.005, 0.005], + [12.639, 0.569, 0.575], + [11.243, 0.633, 0.638], + [6.603, 3.155, 3.179], + [8.453, 6.407, 6.389], + [8.506, 6.434, 6.409], + [0.889, 0.635, 0.656], + [0.228, 0.054, 0.052], + [0.187, 0.023, 0.021], + [0.181, 0.021, 0.02], + [0.452, 0.32, 0.313], + [0.199, 0.024, 0.023], + [0.178, 0.021, 0.024], + [0.21, 0.055, 0.053] +] + } + \ No newline at end of file diff --git a/pgrust/results/20260801/c6a.metal.json b/pgrust/results/20260801/c6a.metal.json new file mode 100644 index 0000000000..d74ffaf966 --- /dev/null +++ b/pgrust/results/20260801/c6a.metal.json @@ -0,0 +1,60 @@ +{ + "system": "pgrust", + "date": "2026-08-01", + "machine": "c6a.metal", + "cluster_size": 1, + "proprietary": "no", + "hardware": "cpu", + "tuned": "no", + "tags": ["Rust","column-oriented","PostgreSQL compatible"], + "load_time": 139, + "data_size": 17563161749, + "concurrent_qps": 1.305, + "concurrent_error_ratio": 0.003, + "result": [ + [0.083, 0.005, 0.007], + [0.149, 0.007, 0.007], + [0.141, 0.004, 0.005], + [0.102, 0.007, 0.006], + [0.341, 0.103, 0.106], + [1.814, 0.339, 0.348], + [0.109, 0.004, 0.006], + [0.18, 0.011, 0.013], + [0.68, 0.165, 0.156], + [1.592, 0.371, 0.365], + [0.693, 0.055, 0.055], + [0.735, 0.058, 0.053], + [2.037, 0.231, 0.228], + [2.751, 0.421, 0.413], + [2.716, 0.417, 0.391], + [1.181, 0.415, 0.395], + [4.346, 0.745, 0.733], + [4.875, 0.66, 0.652], + [7.306, 1.028, 0.993], + [0.158, 0.005, 0.003], + [7.303, 0.032, 0.032], + [9.617, 0.019, 0.019], + [12.361, 0.045, 0.045], + [3.069, 0.04, 0.039], + [1.79, 0.036, 0.035], + [1.541, 0.023, 0.023], + [1.876, 0.038, 0.037], + [7.359, 0.129, 0.144], + [6.961, 0.846, 0.84], + [0.108, 0.008, 0.009], + [7.509, 0.27, 0.26], + [12.626, 0.298, 0.304], + [6.813, 2.342, 2.237], + [8.921, 3.423, 3.603], + [8.885, 3.459, 3.505], + [2.197, 0.266, 0.25], + [0.214, 0.04, 0.035], + [0.193, 0.027, 0.027], + [0.181, 0.021, 0.02], + [0.339, 0.157, 0.16], + [0.19, 0.023, 0.018], + [0.165, 0.016, 0.013], + [0.191, 0.047, 0.051] +] + } + \ No newline at end of file diff --git a/pgrust/results/20260801/c6a.xlarge.json b/pgrust/results/20260801/c6a.xlarge.json new file mode 100644 index 0000000000..0454e03711 --- /dev/null +++ b/pgrust/results/20260801/c6a.xlarge.json @@ -0,0 +1,60 @@ +{ + "system": "pgrust", + "date": "2026-08-01", + "machine": "c6a.xlarge", + "cluster_size": 1, + "proprietary": "no", + "hardware": "cpu", + "tuned": "no", + "tags": ["Rust","column-oriented","PostgreSQL compatible"], + "load_time": 3586, + "data_size": 17563161738, + "concurrent_qps": 0.112, + "concurrent_error_ratio": 0, + "result": [ + [0.071, 0.005, 0.004], + [0.268, 0.031, 0.031], + [0.103, 0.005, 0.005], + [0.102, 0.005, 0.005], + [3.19, 2.759, 2.76], + [5.711, 3.306, 3.267], + [0.115, 0.005, 0.004], + [0.31, 0.027, 0.024], + [1.538, 1.111, 1.113], + [1.866, 1.475, 1.492], + [1.008, 0.447, 0.447], + [1.002, 0.514, 0.515], + [3.338, 1.432, 1.444], + [17.987, 14.54, 14.507], + [4.205, 1.918, 1.907], + [1.604, 1.201, 1.216], + [8.232, 6.749, 7.369], + [27.985, 27.8, 27.543], + [18.512, 15.308, 16.075], + [0.264, 0.003, 0.002], + [7.218, 0.14, 0.136], + [9.324, 0.058, 0.057], + [12.456, 0.257, 0.255], + [2.927, 0.042, 0.041], + [1.854, 0.035, 0.035], + [1.318, 0.244, 0.243], + [2.069, 0.038, 0.038], + [7.293, 1.338, 1.343], + [21.962, 11.273, 11.277], + [0.121, 0.005, 0.005], + [3.335, 1.195, 1.207], + [7.558, 1.234, 1.228], + [35.687, 36.349, 35.498], + [32.244, 30.026, 28.661], + [30.162, 30.205, 31.27], + [16.238, 16.037, 16.1], + [0.233, 0.082, 0.081], + [0.248, 0.115, 0.106], + [0.177, 0.028, 0.027], + [0.542, 0.31, 0.303], + [0.214, 0.032, 0.031], + [0.189, 0.029, 0.03], + [0.24, 0.082, 0.083] +] + } + \ No newline at end of file diff --git a/pgrust/results/20260801/c7a.metal-48xl.json b/pgrust/results/20260801/c7a.metal-48xl.json new file mode 100644 index 0000000000..94eb3b68a1 --- /dev/null +++ b/pgrust/results/20260801/c7a.metal-48xl.json @@ -0,0 +1,60 @@ +{ + "system": "pgrust", + "date": "2026-08-01", + "machine": "c7a.metal-48xl", + "cluster_size": 1, + "proprietary": "no", + "hardware": "cpu", + "tuned": "no", + "tags": ["Rust","column-oriented","PostgreSQL compatible"], + "load_time": 139, + "data_size": 17563161749, + "concurrent_qps": 1.142, + "concurrent_error_ratio": 0.004, + "result": [ + [0.091, 0.003, 0.008], + [0.15, 0.007, 0.006], + [0.132, 0.005, 0.007], + [0.111, 0.005, 0.01], + [0.354, 0.088, 0.088], + [1.738, 0.243, 0.248], + [0.115, 0.006, 0.009], + [0.161, 0.01, 0.013], + [0.673, 0.128, 0.135], + [1.434, 0.246, 0.263], + [0.691, 0.048, 0.047], + [0.741, 0.044, 0.04], + [1.897, 0.102, 0.103], + [2.635, 0.217, 0.211], + [2.622, 0.172, 0.166], + [0.875, 0.147, 0.14], + [4.086, 0.282, 0.303], + [4.292, 0.278, 0.276], + [7.195, 0.424, 0.409], + [0.174, 0.006, 0.004], + [7.31, 0.022, 0.02], + [9.625, 0.013, 0.01], + [12.375, 0.027, 0.023], + [3.128, 0.04, 0.04], + [2.035, 0.033, 0.034], + [1.549, 0.016, 0.018], + [1.996, 0.037, 0.037], + [7.344, 0.087, 0.072], + [6.785, 0.413, 0.398], + [0.135, 0.005, 0.01], + [8.025, 0.111, 0.102], + [11.389, 0.121, 0.111], + [6.103, 1.042, 1.053], + [8.063, 1.171, 1.166], + [8.045, 1.157, 1.162], + [2.165, 0.1, 0.102], + [0.187, 0.027, 0.028], + [0.183, 0.016, 0.018], + [0.167, 0.02, 0.018], + [0.289, 0.074, 0.081], + [0.186, 0.022, 0.02], + [0.172, 0.02, 0.019], + [0.197, 0.043, 0.043] +] + } + \ No newline at end of file diff --git a/pgrust/results/20260801/c8g.4xlarge.json b/pgrust/results/20260801/c8g.4xlarge.json new file mode 100644 index 0000000000..9cd5005097 --- /dev/null +++ b/pgrust/results/20260801/c8g.4xlarge.json @@ -0,0 +1,60 @@ +{ + "system": "pgrust", + "date": "2026-08-01", + "machine": "c8g.4xlarge", + "cluster_size": 1, + "proprietary": "no", + "hardware": "cpu", + "tuned": "no", + "tags": ["Rust","column-oriented","PostgreSQL compatible"], + "load_time": 207, + "data_size": 17563169934, + "concurrent_qps": 0.6, + "concurrent_error_ratio": 0.003, + "result": [ + [0.058, 0.003, 0.004], + [0.149, 0.007, 0.007], + [0.086, 0.003, 0.005], + [0.083, 0.003, 0.003], + [0.42, 0.223, 0.219], + [1.66, 0.341, 0.348], + [0.076, 0.003, 0.003], + [0.134, 0.005, 0.005], + [0.667, 0.217, 0.218], + [1.039, 0.297, 0.303], + [0.608, 0.071, 0.072], + [0.67, 0.077, 0.077], + [1.468, 0.259, 0.26], + [2.75, 0.805, 0.813], + [2.633, 0.341, 0.34], + [0.709, 0.231, 0.233], + [3.475, 0.6, 0.605], + [4.055, 0.135, 0.135], + [7.503, 0.952, 0.966], + [0.346, 0.001, 0.001], + [7.847, 0.035, 0.034], + [9.296, 0.013, 0.013], + [12.425, 0.061, 0.061], + [3.085, 0.031, 0.03], + [1.922, 0.029, 0.03], + [1.294, 0.03, 0.03], + [1.937, 0.032, 0.031], + [7.251, 0.2, 0.202], + [8.01, 1.942, 1.955], + [0.082, 0.004, 0.004], + [10.855, 0.172, 0.179], + [11.795, 0.166, 0.171], + [5.63, 1.006, 1.028], + [7.895, 1.615, 1.628], + [7.89, 1.602, 1.651], + [0.865, 0.229, 0.23], + [0.185, 0.02, 0.02], + [0.11, 0.009, 0.009], + [0.12, 0.016, 0.016], + [0.229, 0.081, 0.081], + [0.136, 0.02, 0.02], + [0.119, 0.017, 0.017], + [0.145, 0.046, 0.047] +] + } + \ No newline at end of file diff --git a/pgrust/results/20260801/c8g.metal-48xl.json b/pgrust/results/20260801/c8g.metal-48xl.json new file mode 100644 index 0000000000..a4b9014930 --- /dev/null +++ b/pgrust/results/20260801/c8g.metal-48xl.json @@ -0,0 +1,60 @@ +{ + "system": "pgrust", + "date": "2026-08-01", + "machine": "c8g.metal-48xl", + "cluster_size": 1, + "proprietary": "no", + "hardware": "cpu", + "tuned": "no", + "tags": ["Rust","column-oriented","PostgreSQL compatible"], + "load_time": 129, + "data_size": 17563161749, + "concurrent_qps": 1.17, + "concurrent_error_ratio": 0.003, + "result": [ + [0.05, 0.004, 0.003], + [0.091, 0.003, 0.003], + [0.077, 0.003, 0.004], + [0.073, 0.004, 0.004], + [0.32, 0.069, 0.075], + [1.66, 0.232, 0.223], + [0.062, 0.005, 0.004], + [0.106, 0.006, 0.007], + [0.582, 0.115, 0.109], + [1.323, 0.184, 0.209], + [0.637, 0.034, 0.032], + [0.672, 0.034, 0.034], + [2.564, 0.085, 0.088], + [2.596, 0.228, 0.228], + [2.457, 0.143, 0.176], + [1.105, 0.114, 0.125], + [4.905, 0.284, 0.297], + [6.37, 0.19, 0.191], + [7.19, 0.438, 0.467], + [0.118, 0.003, 0.002], + [7.243, 0.015, 0.015], + [9.555, 0.007, 0.007], + [12.297, 0.021, 0.021], + [2.603, 0.032, 0.031], + [1.571, 0.03, 0.03], + [1.507, 0.013, 0.012], + [1.51, 0.033, 0.034], + [7.301, 0.084, 0.079], + [9.228, 0.351, 0.333], + [0.071, 0.004, 0.004], + [9.64, 0.079, 0.084], + [12.167, 0.104, 0.1], + [6.558, 0.786, 0.818], + [8.6, 0.867, 1.024], + [8.791, 0.917, 0.964], + [5.292, 0.138, 0.125], + [0.128, 0.024, 0.026], + [0.106, 0.017, 0.011], + [0.117, 0.015, 0.015], + [0.203, 0.082, 0.075], + [0.128, 0.016, 0.016], + [0.108, 0.01, 0.01], + [0.142, 0.042, 0.042] +] + } + \ No newline at end of file diff --git a/pgrust/start b/pgrust/start new file mode 100755 index 0000000000..8251ccf250 --- /dev/null +++ b/pgrust/start @@ -0,0 +1,24 @@ +#!/bin/bash +set -eu +. "$(dirname "$0")/env.sh" + +sudo install -d -o postgres -g postgres "$PGSOCK_DIR" + +# Runs as the `postgres` system user (the server refuses to run as root, +# and the datadir is owned by postgres). RUST_MIN_STACK and the rlimit +# raise thread/main stacks to match max_stack_depth=60000 — pgrust +# requires both (deep recursive expression evaluation paths). +sudo -u postgres \ + PGRUST_BIN="$PGRUST_BIN" PGDATA_DIR="$PGDATA_DIR" \ + PGSOCK_DIR="$PGSOCK_DIR" SERVER_LOG="$SERVER_LOG" \ + PGSHARE_DIR="$PGSHARE_DIR" TZDATA_DIR="$TZDATA_DIR" \ + EXTRA_SERVER_ENV="${EXTRA_SERVER_ENV:-}" \ + bash -c ' + ulimit -s 65520 + nohup env RUST_MIN_STACK=33554432 RUST_BACKTRACE=1 \ + PGRUST_TZDIR="$TZDATA_DIR" PGRUST_PGSHAREDIR="$PGSHARE_DIR" \ + $EXTRA_SERVER_ENV \ + "$PGRUST_BIN" -D "$PGDATA_DIR" -k "$PGSOCK_DIR" -p 5432 \ + -c listen_addresses="" \ + >> "$SERVER_LOG" 2>&1 & + ' diff --git a/pgrust/stop b/pgrust/stop new file mode 100755 index 0000000000..943bc16e5c --- /dev/null +++ b/pgrust/stop @@ -0,0 +1,23 @@ +#!/bin/bash +. "$(dirname "$0")/env.sh" + +pid=$(sudo head -1 "$PGDATA_DIR/postmaster.pid" 2>/dev/null) +[ -n "$pid" ] || exit 0 + +# Fast shutdown, as the true-cold-run protocol requires. +sudo kill -INT "$pid" 2>/dev/null || exit 0 +for i in $(seq 1 120); do + sudo kill -0 "$pid" 2>/dev/null || exit 0 + sleep 0.5 +done + +# Still alive after 60s: immediate shutdown. The datadir is durable and the +# next start recovers; better than leaving a half-stopped server pinning the +# page cache through the "cold" cycle. +echo "pgrust/stop: fast shutdown did not finish in 60s, escalating to SIGQUIT" >&2 +sudo kill -QUIT "$pid" 2>/dev/null || exit 0 +for i in $(seq 1 60); do + sudo kill -0 "$pid" 2>/dev/null || exit 0 + sleep 0.5 +done +exit 0 diff --git a/pgrust/template.json b/pgrust/template.json new file mode 100644 index 0000000000..2fdd9c9dd9 --- /dev/null +++ b/pgrust/template.json @@ -0,0 +1,11 @@ +{ + "system": "pgrust", + "proprietary": "no", + "hardware": "cpu", + "tuned": "no", + "tags": [ + "Rust", + "column-oriented", + "PostgreSQL compatible" + ] +}