-
Notifications
You must be signed in to change notification settings - Fork 167
NGF: Add document for upstream http2 #2021
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: ngf-release-2.7
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,196 @@ | ||
| --- | ||
| title: Configure upstream HTTP/2 through Service appProtocol | ||
| toc: true | ||
| weight: 1600 | ||
| f5-content-type: how-to | ||
| f5-product: FABRIC | ||
| f5-docs: DOCS-0000 | ||
| description: Configure NGINX Gateway Fabric to use HTTP/2 for upstream connections by setting appProtocol to kubernetes.io/h2c on a Kubernetes Service port. | ||
| f5-keywords: NGINX Gateway Fabric, HTTP/2, upstream HTTP/2, appProtocol, kubernetes.io/h2c, proxy_http_version, h2c, upstream connections, Service appProtocol, Gateway API, HTTPRoute, GRPCRoute | ||
| f5-summary: This guide describes how to configure NGINX Gateway Fabric to proxy requests to upstream services over HTTP/2 by setting appProtocol to kubernetes.io/h2c on a Kubernetes Service port. Using HTTP/2 for upstream connections enables multiplexing and reduces latency for services that support it. This guide is for operators and developers who have NGINX Gateway Fabric installed and are familiar with Kubernetes Services and the Gateway API. | ||
| --- | ||
|
|
||
| Learn how to configure NGINX Gateway Fabric to use HTTP/2 when proxying requests to upstream services using the Service port's `appProtocol` field. | ||
|
|
||
| ## Overview | ||
|
|
||
| The appProtocol field on a Kubernetes Service port provides a way to specify an application protocol. Controllers such as NGINX Gateway Fabric may use this field to enable protocol-specific functionality for supported protocols. For more information, view the official [Kubernetes Service Documentation](https://kubernetes.io/docs/concepts/services-networking/service/#application-protocol). | ||
|
|
||
| When a Kubernetes Service port has `appProtocol` set to `kubernetes.io/h2c`, NGINX Gateway Fabric configures the corresponding NGINX location to use HTTP/2 for upstream connections by setting the [`proxy_http_version`](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_http_version) directive to `2` in the NGINX configuration. | ||
|
|
||
| ## Before you begin | ||
|
|
||
| - [Install]({{< ref "/ngf/install/" >}}) NGINX Gateway Fabric. | ||
|
|
||
| ## Set up | ||
|
|
||
| Create the **coffee** application in Kubernetes by copying and pasting the following block into your terminal: | ||
|
|
||
| ```yaml | ||
| kubectl apply -f - <<EOF | ||
| apiVersion: apps/v1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: coffee | ||
| spec: | ||
| replicas: 1 | ||
| selector: | ||
| matchLabels: | ||
| app: coffee | ||
| template: | ||
| metadata: | ||
| labels: | ||
| app: coffee | ||
| spec: | ||
| containers: | ||
| - name: coffee | ||
| image: nginxdemos/nginx-hello:plain-text | ||
| ports: | ||
| - containerPort: 8080 | ||
| --- | ||
| apiVersion: v1 | ||
| kind: Service | ||
| metadata: | ||
| name: coffee | ||
| spec: | ||
| ports: | ||
| - port: 80 | ||
| targetPort: 8080 | ||
| protocol: TCP | ||
| name: http | ||
| appProtocol: kubernetes.io/h2c | ||
| selector: | ||
| app: coffee | ||
| EOF | ||
| ``` | ||
|
|
||
| Setting `appProtocol: kubernetes.io/h2c` on the Service port tells NGINX Gateway Fabric to use HTTP/2 for upstream connections to this Service. | ||
|
|
||
| Run the following command to verify the resources were created: | ||
|
|
||
| ```shell | ||
| kubectl get pods,svc | ||
| ``` | ||
|
|
||
| Your output should include a **coffee** pod and the **coffee** service: | ||
|
|
||
| ```text | ||
| NAME READY STATUS RESTARTS AGE | ||
| pod/coffee-654ddf664b-q28ps 1/1 Running 0 10s | ||
|
|
||
| NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE | ||
| service/coffee ClusterIP 10.96.30.58 <none> 80/TCP 10s | ||
| ``` | ||
|
|
||
| ## Create the Gateway API resources | ||
|
|
||
| Create a Gateway: | ||
|
|
||
| ```yaml | ||
| kubectl apply -f - <<EOF | ||
| apiVersion: gateway.networking.k8s.io/v1 | ||
| kind: Gateway | ||
| metadata: | ||
| name: gateway | ||
| spec: | ||
| gatewayClassName: nginx | ||
| listeners: | ||
| - name: http | ||
| port: 80 | ||
| protocol: HTTP | ||
| EOF | ||
| ``` | ||
|
|
||
| Verify the gateway is created: | ||
|
|
||
| ```shell | ||
| kubectl describe gateways.gateway.networking.k8s.io gateway | ||
| ``` | ||
|
|
||
| Verify the status is `Accepted`: | ||
|
|
||
| ```text | ||
| Status: | ||
| Addresses: | ||
| Type: IPAddress | ||
| Value: 10.96.69.11 | ||
| Attached Listener Sets: 0 | ||
| Conditions: | ||
| Last Transition Time: 2026-06-03T05:21:38Z | ||
| Message: The Gateway is accepted | ||
| Observed Generation: 1 | ||
| Reason: Accepted | ||
| Status: True | ||
| Type: Accepted | ||
| Last Transition Time: 2026-06-03T05:21:38Z | ||
| Message: The Gateway is programmed | ||
| Observed Generation: 1 | ||
| Reason: Programmed | ||
| Status: True | ||
| Type: Programmed | ||
| ``` | ||
|
|
||
| Create the **coffee** HTTPRoute: | ||
|
|
||
| ```yaml | ||
| kubectl apply -f - <<EOF | ||
| apiVersion: gateway.networking.k8s.io/v1 | ||
| kind: HTTPRoute | ||
| metadata: | ||
| name: coffee | ||
| spec: | ||
| parentRefs: | ||
| - name: gateway | ||
| hostnames: | ||
| - "cafe.example.com" | ||
| rules: | ||
| - matches: | ||
| - path: | ||
| type: PathPrefix | ||
| value: /coffee | ||
| backendRefs: | ||
| - name: coffee | ||
| port: 80 | ||
| EOF | ||
| ``` | ||
|
|
||
| ## Verify the NGINX configuration | ||
|
|
||
| Inspect the NGINX configuration to confirm that `proxy_http_version 2` is set for the **coffee** location: | ||
|
|
||
| ```shell | ||
| kubectl exec -it deployments/gateway-nginx -- nginx -T | ||
| ``` | ||
|
|
||
| Look for the location block that routes traffic to the **coffee** upstream. It should contain `proxy_http_version 2`: | ||
|
|
||
| ```nginx | ||
| location /coffee { | ||
| ... | ||
| proxy_http_version 2; | ||
| proxy_pass http://default_coffee_80; | ||
| ... | ||
| } | ||
| ``` | ||
|
|
||
| ## Important Notes | ||
|
|
||
| - `kubernetes.io/h2c` is supported on HTTPRoutes and GRPCRoutes. It isn't supported on TLSRoutes. | ||
| - For NGINX to set `proxy_http_version 2` for a location, all valid backend references in the routing rule must have `appProtocol: kubernetes.io/h2c` set on their Service ports. If any valid backend doesn't use `kubernetes.io/h2c`, NGINX falls back to the default HTTP/1.1. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. i think this could be in the troubleshooting section
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would that be in a different file? I feel like this is relevant enough to this feature that I would want it all contained in this single file. I can put it in a different section in the file if you think that would be better
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. no, just after other supported protocols section
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. After some thought, I think its more suited for where it is currently. The statement isn't really the highlighting of a potential issue rather it is highlighting an important note / requirement to avoid an issue. Additionally, creating a new troubleshooting section with only one bullet point feels incomplete.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We have a precedence of having troubleshooting section in our individual guides so its not really anything out of standard. Its a troubleshooting pointer, it should be part of that section.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree with you having troubleshooting sections is not anything out of standard. However, I don't believe its a troubleshooting pointer.
The troubleshooting sections are usually "If you have these problems... here is the fix". This statement is more of a "Make sure you do these steps so you don't run into an issue".
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. And my statement of
still stands.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The troubleshooting entry is actually validated by the steps themselves, if you skip setting the protocol, you hit some error. That's how you figured it required for this step I am not convinced with the argument of not having single-item troubleshooting section.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. maybe I am not framing my concern right here, but your statement here.
It sounds like you ran into some issue to come to this conclusion. Is there a specific error you see when your egress and ingress connections have different http2 (enabled and disabled mode) ? If yes, maybe add that pointer in the trouble shooting section as it maybe helpful I have the same question about the same above |
||
| - When NGINX Gateway Fabric detects `kubernetes.io/h2c`, it also omits the `proxy_set_header Upgrade` and `proxy_set_header Connection` directives from the location block, because those headers are HTTP/1.1-specific and aren't used in HTTP/2 connections. | ||
|
|
||
| ### Other supported appProtocols | ||
|
|
||
| In addition to the `kubernetes.io/h2c` `appProtocol`, NGINX Gateway Fabric recognizes `kubernetes.io/ws` and `kubernetes.io/wss` as described in [RFC 6455](https://www.rfc-editor.org/info/rfc6455/). | ||
|
|
||
| These `appProtocols` reference WebSocket over cleartext and WebSocket over TLS respectively and are supported natively by our default NGINX configuration. `kubernetes.io/ws` is only supported on HTTPRoutes and `kubernetes.io/wss` is only supported on TLSRoutes or on HTTPRoutes with an associated BackendTLSPolicy. | ||
|
|
||
| If an `appProtocol` on a Service port is referenced by an unsupported Route type, that backendRef will be considered invalid and status will be written to the Route. | ||
|
|
||
| NGINX Gateway Fabric is conformant to the information in [GEP-1911](https://gateway-api.sigs.k8s.io/geps/gep-1911/), which should be referenced for more detailed information. | ||
|
|
||
| ## See also | ||
|
|
||
| - [Backend Protocol](https://gateway-api.sigs.k8s.io/guides/user-guides/backend-protocol/): the Gateway API guide for the `appProtocol` field. | ||
| - [GEP-1911](https://gateway-api.sigs.k8s.io/geps/gep-1911/): for more details on how NGINX Gateway Fabric interacts with `appProtocols`. | ||
| - [proxy_http_version NGINX directive](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_http_version). | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so the curl doesn't return the protocol? or a different response to identify a http2 connection?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The upstream application in this example doesn't respond back with http2 responses/ "doesn't speak http2" so it won't really work with a curl request. I don't really feel the need to show a response with a working http2-available backend, so I felt that just showing the nginx conf would suffice.
If you feel like its necessary to show a curl request, basically checking if the nginx conf
proxy_http_version 2;works, then we can consider that, but I don't feel the need.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A user guide implies end-to-end coverage. If this doesn't do that, it should be scoped down to a section within a larger doc, this feels incomplete to me.
could this be part of data plane configuration section as part of how to?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we could do a small section to enable HTTP2 on a service section
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, I like to have curl requests and have sections verifying that traffic flows correctly, but that is essentially verifying that the NGINX product works correctly. I also think end-to-end can mean Kubernetes yaml -> nginx configuration generation. Users of NGF don't necessarily always need to validate nginx configuration works, as our product is mainly involved with generating correct nginx configuration.
The data plane configuration document primarily discusses the
NginxProxyresource.So I think that would be out of place.
There already is precedence of the practice of just verifying nginx configuration in the Upstream settings policy document https://docs.nginx.com/nginx-gateway-fabric/traffic-management/upstream-settings/. In that document, the primary way the guide is showing to the user that the policies work, is by verifying nginx configuration.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, i think in that case we should improve that document.
I am not convinced it should be standalone document. We could also just convert the disable HTTP2 section in the data plane configuration to
HTTP2only. That could talk about both how to enable and disable it.Disabling using NginxProxy and enabled by adding a
appProtocol: h2cfield to a service and check for specific line in configuration.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we have separate views on this, we should get a second opinion and see what makes the most sense to majority.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is my understanding correct that disabling HTTP/2 at the NGINX level affects all services referencing the route, or does it only apply to incoming connections to NGINX itself?
I'd assume there's still some dependency between the protocol allowed on the ingress side and what gets forwarded upstream even if NGINX can translate between protocols, the incoming connection type likely influences the end-to-end behavior. Want to make sure I understand the dependency