Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions content/develop/clients/go/autopipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,10 +154,10 @@ commands instead of 200. As soon as you supply `AutoPipelineOptions`, either on
the client or to `AutoPipelineWithOptions()`, that preset no longer applies, and
a `MaxBatchSize` you leave unset means 200.

Connection and buffer tuning is not part of `AutoPipelineOptions`. Batches use
the client's pipeline connections, which you size with the
`PipelineReadBufferSize`, `PipelineWriteBufferSize`, and `PipelinePoolSize`
fields of the client's options.
Connection and buffer tuning is not part of `AutoPipelineOptions`. Batches run
on the client's separate pipeline connection pool, which you size with the
fields described in
[Connection pooling]({{< relref "/develop/clients/go/produsage#connection-pooling" >}}).

Each client holds at most two autopipeliners: one for the blocking method and
one for the asynchronous method. Each of them is a
Expand Down
67 changes: 66 additions & 1 deletion content/develop/clients/go/produsage.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@ progress in implementing the recommendations.
- [ ] [Monitor performance and errors](#monitor-performance-and-errors)
- [ ] [Retries](#retries)
- [ ] [Timeouts](#timeouts)
- [ ] [Smart client handoffs](#seamless-client-experience)
- [ ] [Connection pooling](#connection-pooling)
- [ ] [Smart client handoffs](#smart-client-handoffs)
```

## Recommendations
Expand Down Expand Up @@ -127,6 +128,70 @@ might retry commands that would have succeeded if given more time. However,
if they are too long, your app might hang unnecessarily while waiting for a
response that will never arrive.

### Connection pooling

`go-redis` manages connections for you with a
[connection pool]({{< relref "/develop/clients/pools-and-muxing" >}}), so your
app doesn't have to cache and reuse open connections itself. The `PoolSize`
field of `Options` sets the number of
connections the main pool keeps (with a default of ten times the value of `GOMAXPROCS`). `MinIdleConns` sets how many
connections to open before your app asks for them, `MaxActiveConns` caps the
total number of connections, and `PoolTimeout` sets how long a command waits
for a free connection. By default, no connections are opened in advance, the
total is uncapped, and a command waits one second longer than `ReadTimeout`.

By default, pipelines don't use the main pool. Every client also creates a
separate *pipeline pool* that
[pipelines and transactions]({{< relref "/develop/clients/go/transpipe" >}}) and
[automatic pipelining]({{< relref "/develop/clients/go/autopipeline" >}}) use
for each batch they run. This prevents a burst of batches from competing with your
ordinary commands for the same connections.

{{< note >}}The pipeline pool is available in `github.com/redis/go-redis/v9` vX.Y.Z or later.
<!--DOC-6996: replace vX.Y.Z with the release that ships go-redis PR #3959 (merged 2026-08-24, unreleased as of v9.22.0) when this branch is unparked.-->
{{< / note >}}
Use the following fields of `Options` to size the pipeline pool:

| Field | Description |
| :---- | :---------- |
| `PipelinePoolSize` | Number of connections the pipeline pool keeps. Defaults to 10. Set it to `0` to use the default number of connections, or to `-1` to disable the separate pipeline pool (which means connections are allocated to pipelines from the main pool). |
| `PipelineReadBufferSize` | Size of the read buffer for each pipeline connection. Defaults to the larger of `ReadBufferSize` and 64 KiB, because a batch of commands can return many replies in a single round trip. If the value is set smaller than the minimum required by the [RESP3]({{< relref "/develop/reference/protocol-spec#resp-versions" >}}) protocol for push messages then that minimum is used instead. |
| `PipelineWriteBufferSize` | Size of the write buffer for each pipeline connection. Defaults to the larger of `WriteBufferSize` and 64 KiB. |

The pipeline pool costs you nothing while you are not using pipelining. It never opens
connections in advance regardless of the value of `MinIdleConns`, so an idle pipeline
pool holds no connections at all. Also, it doesn't use
`MaxActiveConns` as its upper limit (which would double the number of connections your client can
open). The limit is instead the value of `MaxActiveConns` plus `PipelinePoolSize`.
`ClusterClient` and `Ring` create a pipeline pool for each node, so this limit
applies for each node rather than for each client.

If every pipeline connection is busy, a batch will wait for the number of milliseconds
specified in `PoolTimeout` (up to a maximum of 100 milliseconds) and then
run on the main pool rather than queuing for a pipeline connection. The main
pool still applies the `MaxActiveConns` limit when a batch is allocated a connection in
this way, and a `Limiter` counts such a batch once rather than twice.

You can find out the current size of the pipeline pool using the `PipelineStats`
field from the client's pool statistics:

```go
stats := client.PoolStats()
fmt.Printf("Main pool hits: %d, misses: %d, timeouts: %d\n",
stats.Hits, stats.Misses, stats.Timeouts)
if ps := stats.PipelineStats; ps != nil {
fmt.Printf("Pipeline pool connections: %d, hits: %d, timeouts: %d\n",
ps.TotalConns, ps.Hits, ps.Timeouts)
}
```

The `PipelineStats.Timeouts` field indicates how many pipelines have timed out while
waiting for a connection and have consequently run on the main pool. Increase
`PipelinePoolSize` if the number of timeouts is growing rapidly.
Note that `PipelineStats` is `nil` when you disable the pipeline pool (by setting
`PipelinePoolSize` to `-1`). It also combines
the figures from every node for `ClusterClient` and `Ring` into the same count.

### Smart client handoffs

*Smart client handoffs (SCH)* is a feature of Redis Cloud and
Expand Down
6 changes: 6 additions & 0 deletions content/develop/clients/go/transpipe.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,12 @@ If you want the client to batch concurrent commands into pipelines for you
without writing any pipeline code, see
[Automatic pipelining]({{< relref "/develop/clients/go/autopipeline" >}}).

Both types of batch run on a separate pool of connections that the client keeps
for them, so a burst of batches doesn't compete with your ordinary commands for
the same connections. See
[Connection pooling]({{< relref "/develop/clients/go/produsage#connection-pooling" >}})
for how to size that pool.

## Execute a pipeline

To execute commands in a pipeline, you first create a pipeline object
Expand Down
Loading