diff --git a/products/managed-postgres/benchmarks.mdx b/products/managed-postgres/benchmarks.mdx deleted file mode 100644 index 5c2c9809..00000000 --- a/products/managed-postgres/benchmarks.mdx +++ /dev/null @@ -1,187 +0,0 @@ ---- -slug: /cloud/managed-postgres/benchmarks -sidebarTitle: 'Benchmarks' -title: 'Performance benchmarks' -description: 'Performance benchmarks comparing Postgres managed by ClickHouse with AWS Aurora, RDS, and other managed PostgreSQL services' -keywords: ['postgres benchmarks', 'performance', 'pgbench', 'aurora', 'rds', 'tps', 'latency', 'nvme performance'] -doc_type: 'guide' ---- - -import BetaBadge from "/snippets/components/BetaBadge/BetaBadge.jsx"; -import { Image } from "/snippets/components/Image.jsx"; - - - - -**TL;DR** - -- **Benchmarked** Postgres managed by ClickHouse against AWS RDS (16k provisioned IOPS) and Aurora IO Optimized using standard [`pgbench`](https://www.postgresql.org/docs/current/pgbench.html) tests -- **Performance**: ClickHouse's NVMe-backed Postgres delivers **4.3-9x faster performance** for IO-intensive workloads and **12% faster** for CPU-bound scenarios -- **Ideal for fast-growing AI-driven workloads** that demand high transaction rates, low-latency data access, and predictable performance without IO bottlenecks - - -## Benchmark overview {#overview} - -We conducted comprehensive performance testing using `pgbench`, the standard PostgreSQL benchmarking tool, to evaluate workload performance under both moderate and high-concurrency scenarios. - -## Benchmarks {#benchmarks} - -All performance tests were conducted using a client VM with the same compute capacity, collocated in the same region and availability zone as the PostgreSQL database to ensure fair comparison. - -### Test 1: IO Intensive - Read+Write (500 GB dataset) {#test1} - -IO Intensive Read+Write benchmark results - -**Performance improvement over RDS (16k IOPS):** -- **326% higher TPS** (4.3x faster) - -**Performance improvement over Aurora IO Optimized:** -- **345% higher TPS** (4.5x faster) - -**Analysis**: Mixed read/write workloads showcase the most dramatic performance advantages of NVMe storage and represent the **most realistic scenario for fast-growing AI-driven workloads** that require both high-throughput data ingestion and low-latency reads. **Postgres managed by ClickHouse achieved 19.8K TPS with higher concurrency**, demonstrating how NVMe storage scales effectively under load. This is **4.3-4.5x faster than RDS and Aurora**. Network-attached storage solutions struggled with write-heavy operations, with RDS and Aurora maxing out at 4.4K-4.6K TPS despite provisioned capacity and even with Aurora's IO Optimized configuration. - -#### Setup {#test1-setup} - -This test evaluates mixed read/write performance with a large 500 GB dataset, stressing both read and write paths of the storage subsystem. - -**Instance configuration:** - -| Configuration | Postgres managed by ClickHouse | RDS with 16k IOPS | Aurora IO Optimized | -|----------------|--------------------------------|--------------------------------|---------------------------------| -| **PG Version** | 17 | 17 | 17 | -| **vCPUs** | 16 | 16 | 16 | -| **RAM** | 64 GB | 64 GB | 128 GB | -| **Disk Size** | 1 TB | 1 TB | 1 TB | -| **Disk Type** | NVMe (unlimited IOPS) | Network-attached (16,000 IOPS) | Network-attached (IO Optimized) | - -**Test configuration:** - -```bash -# Initialize database (500 GB dataset) -pgbench -i -s 34247 - -# Read+Write benchmark -pgbench -c 256 -j 16 -T 600 -M prepared -P 30 -``` - -### Test 2: IO Intensive - Read-Only (500 GB dataset) {#test2} - -IO Intensive Read-Only benchmark results - -**Performance improvement over RDS (16k IOPS):** -- **802% higher TPS** (9.0x faster) - -**Analysis**: The performance gap widens dramatically for read-intensive workloads that are IO Bound. **Postgres managed by ClickHouse delivered 84.8K TPS**, while RDS with 16,000 provisioned IOPS achieved only 9.4K TPS despite having equivalent compute resources. The key difference: ClickHouse's NVMe storage scales with higher concurrency, while network-attached storage remains constrained by provisioned IOPS limits. Even with provisioned IOPS, RDS was still 9x slower than ClickHouse, demonstrating the critical importance of storage architecture for IO-intensive workloads. - -#### Setup {#test2-setup} - -This test evaluates read performance with a large 500 GB dataset that doesn't fit in memory, stressing disk I/O capabilities. - -**Instance configuration:** - -| Configuration | Postgres managed by ClickHouse | RDS with 16k IOPS | -|----------------|--------------------------------|--------------------------------| -| **PG Version** | 17 | 17 | -| **vCPUs** | 16 | 16 | -| **RAM** | 64 GB | 64 GB | -| **Disk Size** | 1 TB | 1 TB | -| **Disk Type** | NVMe (unlimited IOPS) | Network-attached (16,000 IOPS) | - -**Test configuration:** - -```bash -# Initialize database (500 GB dataset) -pgbench -i -s 34247 - -# Read-only benchmark -pgbench -c 256 -j 16 -T 600 -M prepared -P 30 -S -``` - -### Test 3: CPU Intensive (data fits in memory) {#test3} - -CPU Intensive benchmark results - -**Performance improvement:** -- **12.3% higher TPS** than RDS PostgreSQL - -**Analysis**: Even in CPU-bound scenarios where disk I/O is minimal, **Postgres managed by ClickHouse led the pack with 36.5K TPS**. Despite both services hitting 100% CPU utilization, ClickHouse's NVMe storage delivered superior performance with better cache hit rates. The 12% advantage over RDS demonstrates the efficiency of the underlying infrastructure even when workloads are primarily CPU-bound. - -#### Setup {#test3-setup} - -This test evaluates CPU performance when the working set fits entirely in memory, minimizing disk I/O impact. - -**Instance configuration:** - -| Configuration | Postgres managed by ClickHouse | RDS PostgreSQL | -|----------------|--------------------------------|------------------------| -| **PG Version** | 17 | 17 | -| **vCPUs** | 2 | 2 | -| **RAM** | 8 GB | 8 GB | -| **Disk Type** | NVMe | Network-attached (gp3) | - -**Test configuration:** - -```bash -# Initialize database (2 GB dataset) -pgbench -i -s 136 - -# Warm-up run to load dataset into memory -pgbench -c 1 -T 60 -S -M prepared - -# Run benchmark (read-only, prepared statements) -pgbench -c 32 -j 16 -T 300 -S -M prepared -P 30 -``` - -## Performance summary {#summary} - -### Key findings {#key-findings} - -Across all three benchmark scenarios, Postgres managed by ClickHouse consistently delivered superior performance: - -1. **IO-intensive read+write workloads**: 4.3-4.5x higher TPS compared to RDS (16k IOPS) and Aurora IO Optimized -2. **IO-intensive read workloads**: 9x higher TPS compared to RDS with 16k IOPS -3. **CPU-bound workloads**: 12% higher TPS than RDS - -### When Postgres by ClickHouse excels {#when-it-excels} - -Postgres by ClickHouse is ideal for applications that: -- **Power fast-growing AI-driven workloads** requiring high-throughput data ingestion with frequent upserts, real-time feature updates, and analytics out of the box through seamless integration with ClickHouse for OLAP workloads -- Perform frequent writes, updates, or mixed read/write operations -- Need predictable, high-performance storage -- Are currently constrained by IOPS limits on traditional managed Postgres services - -**If you expect analytics later** and anticipate deeper ClickHouse integration—common in modern AI workloads where transactional data feeds real-time dashboards, feature stores, and ML pipelines—**Postgres by ClickHouse should be your default go-to choice**. The native integration eliminates complex ETL pipelines and enables seamless data flow between your operational database and analytical queries. - -### NVMe architecture advantage {#nvme-advantage} - -The performance advantage comes from the fundamental architectural difference: - -| Aspect | NVMe Storage (Managed Postgres) | Network-Attached Storage (Provisioned IOPS) | -|-------------------------|-------------------------------------|----------------------------------------------------| -| **IOPS** | 100k to virtually unlimited | 16,000 provisioned | -| **Network hops** | Zero (local device) | Every disk operation requires network round trip | -| **Performance scaling** | Scales linearly with concurrency | Limited by provisioned IOPS | - -For more details on the performance benefits of NVMe storage, see [NVMe-powered performance](/products/managed-postgres/overview#nvme-performance). - -## Cost-effectiveness {#cost-effectiveness} - -Beyond raw performance, Postgres managed by ClickHouse offers superior price-performance: - -- **Higher throughput per dollar**: Achieve 4-9x more TPS compared to RDS with 16k provisioned IOPS and Aurora IO Optimized -- **Predictable costs**: No need to provision additional IOPS capacity—unlimited local IOPS included -- **Lower compute requirements**: Achieve target performance with smaller instance sizes due to efficient I/O -- **Reduced need for read replicas**: Higher single-instance throughput reduces need for horizontal scaling - -For workloads currently constrained by IOPS limits, switching to Managed Postgres can eliminate the need for expensive provisioned IOPS or IO-optimized configurations while delivering significantly better performance. - -## References {#references} - -The complete benchmark data, configurations, and detailed metrics are available in our [benchmark results spreadsheet](https://docs.google.com/spreadsheets/d/17TLWmwNKZb3Ie1vSQqvjtqByHskvoX6CF2eQ_FRx1cA/edit?gid=845104392#gid=845104392). - -## Additional resources {#resources} - -- [PeerDB: Comparing Postgres managed services](https://blog.peerdb.io/comparing-postgres-managed-services-aws-azure-gcp-and-supabase) -- [pgbench documentation](https://www.postgresql.org/docs/current/pgbench.html) -- [Managed Postgres overview](/products/managed-postgres/overview) -- [Scaling your Postgres instance](/products/managed-postgres/scaling) diff --git a/products/managed-postgres/navigation.json b/products/managed-postgres/navigation.json index f38202c7..51b109fd 100644 --- a/products/managed-postgres/navigation.json +++ b/products/managed-postgres/navigation.json @@ -24,12 +24,6 @@ "products/managed-postgres/openapi" ] }, - { - "group": "Benchmarks", - "pages": [ - "products/managed-postgres/benchmarks" - ] - }, { "group": "ClickHouse Integration", "pages": [ diff --git a/products/managed-postgres/overview.mdx b/products/managed-postgres/overview.mdx index fb278afb..591e89b2 100644 --- a/products/managed-postgres/overview.mdx +++ b/products/managed-postgres/overview.mdx @@ -30,6 +30,10 @@ Managed Postgres uses NVMe storage that is physically attached to the same serve For Postgres workloads that are primarily throttled by disk IOPS and latency, this translates to faster ingestion, quicker vacuums, lower tail latency, and more predictable performance under load. + +Learn how fast Postgres is on NVMe disks from [performance benchmarks](https://clickhouse.com/blog/postgresbench). + + ## Native ClickHouse integration {#clickhouse-integration} Managed Postgres integrates natively with ClickHouse to bring transactions and analytics together without complex ETL pipelines.