From a18a20db24d5278f5952cc9e6bb3bec9cf595f94 Mon Sep 17 00:00:00 2001 From: Michal Fupso Date: Thu, 9 Jul 2026 12:23:04 -0700 Subject: [PATCH 1/2] Update l2 docs --- .../configuring/local-subnet-l2-reachability.mdx | 11 ++++++++++- .../configuring/local-subnet-l2-reachability.mdx | 11 ++++++++++- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/calico-enterprise/networking/configuring/local-subnet-l2-reachability.mdx b/calico-enterprise/networking/configuring/local-subnet-l2-reachability.mdx index 579482c06f..ea7d540cda 100644 --- a/calico-enterprise/networking/configuring/local-subnet-l2-reachability.mdx +++ b/calico-enterprise/networking/configuring/local-subnet-l2-reachability.mdx @@ -37,6 +37,8 @@ $[prodname] answers only for IPs that belong to an IP pool with no encapsulation - **One node per LoadBalancer VIP.** There is no active-active high availability. If the owning node goes down, the VIP is unreachable until its `Node` object is removed and a new owner is elected. - **Cloud provider ARP filtering.** Some cloud networks (for example, AWS VPC) filter ARP at the hypervisor. The responder will not work unless the VIP/pod IPs are assigned to the node's cloud interface. This is an infrastructure constraint outside $[prodname]'s control. - **Heterogeneous subnets.** Node selection for LoadBalancer VIPs hashes over all nodes without checking whether a node has an interface on the VIP's subnet. This is only a concern when nodes do not share the same subnets. +- **Only `externalTrafficPolicy: Cluster` is supported for LoadBalancer services.** $[prodname] elects the node that answers for a VIP with a consistent hash over all nodes, independent of where the service's backends run. Under `externalTrafficPolicy: Local` the elected node may have no local backend, so traffic that reaches it is dropped. Leave `externalTrafficPolicy` at its default of `Cluster`. +- **kube-proxy in IPVS mode requires `strictARP`.** In IPVS mode, kube-proxy binds every LoadBalancer VIP to the `kube-ipvs0` dummy interface on *all* nodes. Because the Linux kernel answers ARP for any IP configured on the host by default, every node replies for the VIP — not just the one $[prodname] elected — so external hosts receive conflicting replies from several MAC addresses. Either run kube-proxy in `iptables` or `nftables` mode, or keep IPVS mode and set `strictARP: true` (the `ipvs.strictARP` field of the kube-proxy `ConfigMap`) so the kernel stops answering ARP for the dummy-interface VIPs and leaves it to $[prodname]'s responder. :::note @@ -62,7 +64,7 @@ kubectl patch felixconfiguration default --type='merge' \ -p '{"spec": {"localSubnetL2Reachability": "PodsAndLoadBalancers"}}' ``` -The feature is `Disabled` by default. The setting is evaluated at dataplane startup, so **restart Felix (`calico-node`) for the change to take effect**. For the full field reference, see [Felix configuration resource](../../reference/resources/felixconfig.mdx). +The feature is `Disabled` by default. This setting is read when the dataplane starts, so changing it makes Felix (`calico-node`) **restart automatically** to pick up the new value — no manual restart is required. The feature takes effect once Felix finishes restarting. For the full field reference, see [Felix configuration resource](../../reference/resources/felixconfig.mdx). :::note @@ -70,6 +72,13 @@ Enabling the feature alone does nothing. $[prodname] only answers for IPs in a n ::: +**Optional: tune periodic re-announcement.** While the feature is enabled, Felix periodically re-announces every IP it proxies with a gratuitous ARP / unsolicited Neighbor Advertisement, keeping upstream neighbor caches and switch forwarding tables warm even when the set of proxied IPs has not changed. `localSubnetL2ReachabilityRefreshInterval` controls how often this happens (default `120s`). Lower it if your upstream devices age out ARP/NDP entries quickly, or set it to `0` to disable periodic re-announcement and keep only the one-shot announcement sent when an IP is first proxied: + +```bash +kubectl patch felixconfiguration default --type='merge' \ + -p '{"spec": {"localSubnetL2ReachabilityRefreshInterval": "30s"}}' +``` + ### Create a dedicated IP pool in the host subnet Create an IP pool whose CIDR lies inside your host interface's subnet, with no encapsulation. diff --git a/calico/networking/configuring/local-subnet-l2-reachability.mdx b/calico/networking/configuring/local-subnet-l2-reachability.mdx index a82a3ffe17..d07202cea2 100644 --- a/calico/networking/configuring/local-subnet-l2-reachability.mdx +++ b/calico/networking/configuring/local-subnet-l2-reachability.mdx @@ -37,6 +37,8 @@ $[prodname] answers only for IPs that belong to an IP pool with no encapsulation - **One node per LoadBalancer VIP.** There is no active-active high availability. If the owning node goes down, the VIP is unreachable until its `Node` object is removed and a new owner is elected. - **Cloud provider ARP filtering.** Some cloud networks (for example, AWS VPC) filter ARP at the hypervisor. The responder will not work unless the VIP/pod IPs are assigned to the node's cloud interface. This is an infrastructure constraint outside $[prodname]'s control. - **Heterogeneous subnets.** Node selection for LoadBalancer VIPs hashes over all nodes without checking whether a node has an interface on the VIP's subnet. This is only a concern when nodes do not share the same subnets. +- **Only `externalTrafficPolicy: Cluster` is supported for LoadBalancer services.** $[prodname] elects the node that answers for a VIP with a consistent hash over all nodes, independent of where the service's backends run. Under `externalTrafficPolicy: Local` the elected node may have no local backend, so traffic that reaches it is dropped. Leave `externalTrafficPolicy` at its default of `Cluster`. +- **kube-proxy in IPVS mode requires `strictARP`.** In IPVS mode, kube-proxy binds every LoadBalancer VIP to the `kube-ipvs0` dummy interface on *all* nodes. Because the Linux kernel answers ARP for any IP configured on the host by default, every node replies for the VIP — not just the one $[prodname] elected — so external hosts receive conflicting replies from several MAC addresses. Either run kube-proxy in `iptables` or `nftables` mode, or keep IPVS mode and set `strictARP: true` (the `ipvs.strictARP` field of the kube-proxy `ConfigMap`) so the kernel stops answering ARP for the dummy-interface VIPs and leaves it to $[prodname]'s responder. :::note @@ -62,7 +64,7 @@ kubectl patch felixconfiguration default --type='merge' \ -p '{"spec": {"localSubnetL2Reachability": "PodsAndLoadBalancers"}}' ``` -The feature is `Disabled` by default. The setting is evaluated at dataplane startup, so **restart Felix (`calico-node`) for the change to take effect**. For the full field reference, see [Felix configuration resource](../../reference/resources/felixconfig.mdx). +The feature is `Disabled` by default. This setting is read when the dataplane starts, so changing it makes Felix (`calico-node`) **restart automatically** to pick up the new value — no manual restart is required. The feature takes effect once Felix finishes restarting. For the full field reference, see [Felix configuration resource](../../reference/resources/felixconfig.mdx). :::note @@ -70,6 +72,13 @@ Enabling the feature alone does nothing. $[prodname] only answers for IPs in a n ::: +**Optional: tune periodic re-announcement.** While the feature is enabled, Felix periodically re-announces every IP it proxies with a gratuitous ARP / unsolicited Neighbor Advertisement, keeping upstream neighbor caches and switch forwarding tables warm even when the set of proxied IPs has not changed. `localSubnetL2ReachabilityRefreshInterval` controls how often this happens (default `120s`). Lower it if your upstream devices age out ARP/NDP entries quickly, or set it to `0` to disable periodic re-announcement and keep only the one-shot announcement sent when an IP is first proxied: + +```bash +kubectl patch felixconfiguration default --type='merge' \ + -p '{"spec": {"localSubnetL2ReachabilityRefreshInterval": "30s"}}' +``` + ### Create a dedicated IP pool in the host subnet Create an IP pool whose CIDR lies inside your host interface's subnet, with no encapsulation. From 2b53eadfb1f45070720694322c7fb1bddd46cb34 Mon Sep 17 00:00:00 2001 From: Michal Fupso Date: Fri, 10 Jul 2026 09:36:51 -0700 Subject: [PATCH 2/2] Review comments --- .../networking/configuring/local-subnet-l2-reachability.mdx | 6 +++--- .../networking/configuring/local-subnet-l2-reachability.mdx | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/calico-enterprise/networking/configuring/local-subnet-l2-reachability.mdx b/calico-enterprise/networking/configuring/local-subnet-l2-reachability.mdx index ea7d540cda..7d7c11351f 100644 --- a/calico-enterprise/networking/configuring/local-subnet-l2-reachability.mdx +++ b/calico-enterprise/networking/configuring/local-subnet-l2-reachability.mdx @@ -20,7 +20,7 @@ With `localSubnetL2Reachability` enabled, $[prodname] runs a userspace ARP/NDP r ### Userspace ARP/NDP responder -$[prodname] opens a raw socket on each host physical interface that has at least one pod IP or LoadBalancer VIP within its subnet, and replies to ARP/NDP requests for those IPs. The reply carries the node's MAC address, so the external host sends the workload's traffic to that node. From there $[prodname]'s normal dataplane forwards it the rest of the way — directly to a local pod for a pod IP, or load-balanced to a service backend for a LoadBalancer VIP. The responder only answers ARP/NDP; it does not change how traffic is forwarded once it reaches the node. +$[prodname] opens a raw socket on each host physical interface that has at least one pod IP or LoadBalancer VIP within its subnet, and replies to ARP/NDP requests for those IPs. The reply carries the node's MAC address, so the external host sends the workload's traffic to that node. From there $[prodname]'s normal data plane forwards it the rest of the way — directly to a local pod for a pod IP, or load-balanced to a service backend for a LoadBalancer VIP. The responder only answers ARP/NDP; it does not change how traffic is forwarded once it reaches the node. ### One node answers per LoadBalancer VIP @@ -38,7 +38,7 @@ $[prodname] answers only for IPs that belong to an IP pool with no encapsulation - **Cloud provider ARP filtering.** Some cloud networks (for example, AWS VPC) filter ARP at the hypervisor. The responder will not work unless the VIP/pod IPs are assigned to the node's cloud interface. This is an infrastructure constraint outside $[prodname]'s control. - **Heterogeneous subnets.** Node selection for LoadBalancer VIPs hashes over all nodes without checking whether a node has an interface on the VIP's subnet. This is only a concern when nodes do not share the same subnets. - **Only `externalTrafficPolicy: Cluster` is supported for LoadBalancer services.** $[prodname] elects the node that answers for a VIP with a consistent hash over all nodes, independent of where the service's backends run. Under `externalTrafficPolicy: Local` the elected node may have no local backend, so traffic that reaches it is dropped. Leave `externalTrafficPolicy` at its default of `Cluster`. -- **kube-proxy in IPVS mode requires `strictARP`.** In IPVS mode, kube-proxy binds every LoadBalancer VIP to the `kube-ipvs0` dummy interface on *all* nodes. Because the Linux kernel answers ARP for any IP configured on the host by default, every node replies for the VIP — not just the one $[prodname] elected — so external hosts receive conflicting replies from several MAC addresses. Either run kube-proxy in `iptables` or `nftables` mode, or keep IPVS mode and set `strictARP: true` (the `ipvs.strictARP` field of the kube-proxy `ConfigMap`) so the kernel stops answering ARP for the dummy-interface VIPs and leaves it to $[prodname]'s responder. +- **kube-proxy in IPVS mode requires `strictARP`.** In IPVS mode, kube-proxy binds every LoadBalancer VIP to the `kube-ipvs0` dummy interface on *all* nodes. Because the Linux kernel answers ARP for any IP configured on the host by default, every node replies for the VIP — not just the one $[prodname] elected — so external hosts receive conflicting replies from several MAC addresses. Either run kube-proxy in `iptables` or `nftables` mode, or keep IPVS mode and set `strictARP: true` in kube-proxy's configuration (the `ipvs.strictARP` field of the `KubeProxyConfiguration`, which is stored as YAML inside the `kube-proxy` `ConfigMap`) so the kernel stops answering ARP for the dummy-interface VIPs and leaves it to $[prodname]'s responder. :::note @@ -64,7 +64,7 @@ kubectl patch felixconfiguration default --type='merge' \ -p '{"spec": {"localSubnetL2Reachability": "PodsAndLoadBalancers"}}' ``` -The feature is `Disabled` by default. This setting is read when the dataplane starts, so changing it makes Felix (`calico-node`) **restart automatically** to pick up the new value — no manual restart is required. The feature takes effect once Felix finishes restarting. For the full field reference, see [Felix configuration resource](../../reference/resources/felixconfig.mdx). +The feature is `Disabled` by default. This setting is read when the data plane starts, so changing it makes Felix (`calico-node`) **restart automatically** to pick up the new value — no manual restart is required. The feature takes effect once Felix finishes restarting. For the full field reference, see [Felix configuration resource](../../reference/resources/felixconfig.mdx). :::note diff --git a/calico/networking/configuring/local-subnet-l2-reachability.mdx b/calico/networking/configuring/local-subnet-l2-reachability.mdx index d07202cea2..4fdbf03e90 100644 --- a/calico/networking/configuring/local-subnet-l2-reachability.mdx +++ b/calico/networking/configuring/local-subnet-l2-reachability.mdx @@ -20,7 +20,7 @@ With `localSubnetL2Reachability` enabled, $[prodname] runs a userspace ARP/NDP r ### Userspace ARP/NDP responder -$[prodname] opens a raw socket on each host physical interface that has at least one pod IP or LoadBalancer VIP within its subnet, and replies to ARP/NDP requests for those IPs. The reply carries the node's MAC address, so the external host sends the workload's traffic to that node. From there $[prodname]'s normal dataplane forwards it the rest of the way — directly to a local pod for a pod IP, or load-balanced to a service backend for a LoadBalancer VIP. The responder only answers ARP/NDP; it does not change how traffic is forwarded once it reaches the node. +$[prodname] opens a raw socket on each host physical interface that has at least one pod IP or LoadBalancer VIP within its subnet, and replies to ARP/NDP requests for those IPs. The reply carries the node's MAC address, so the external host sends the workload's traffic to that node. From there $[prodname]'s normal data plane forwards it the rest of the way — directly to a local pod for a pod IP, or load-balanced to a service backend for a LoadBalancer VIP. The responder only answers ARP/NDP; it does not change how traffic is forwarded once it reaches the node. ### One node answers per LoadBalancer VIP @@ -38,7 +38,7 @@ $[prodname] answers only for IPs that belong to an IP pool with no encapsulation - **Cloud provider ARP filtering.** Some cloud networks (for example, AWS VPC) filter ARP at the hypervisor. The responder will not work unless the VIP/pod IPs are assigned to the node's cloud interface. This is an infrastructure constraint outside $[prodname]'s control. - **Heterogeneous subnets.** Node selection for LoadBalancer VIPs hashes over all nodes without checking whether a node has an interface on the VIP's subnet. This is only a concern when nodes do not share the same subnets. - **Only `externalTrafficPolicy: Cluster` is supported for LoadBalancer services.** $[prodname] elects the node that answers for a VIP with a consistent hash over all nodes, independent of where the service's backends run. Under `externalTrafficPolicy: Local` the elected node may have no local backend, so traffic that reaches it is dropped. Leave `externalTrafficPolicy` at its default of `Cluster`. -- **kube-proxy in IPVS mode requires `strictARP`.** In IPVS mode, kube-proxy binds every LoadBalancer VIP to the `kube-ipvs0` dummy interface on *all* nodes. Because the Linux kernel answers ARP for any IP configured on the host by default, every node replies for the VIP — not just the one $[prodname] elected — so external hosts receive conflicting replies from several MAC addresses. Either run kube-proxy in `iptables` or `nftables` mode, or keep IPVS mode and set `strictARP: true` (the `ipvs.strictARP` field of the kube-proxy `ConfigMap`) so the kernel stops answering ARP for the dummy-interface VIPs and leaves it to $[prodname]'s responder. +- **kube-proxy in IPVS mode requires `strictARP`.** In IPVS mode, kube-proxy binds every LoadBalancer VIP to the `kube-ipvs0` dummy interface on *all* nodes. Because the Linux kernel answers ARP for any IP configured on the host by default, every node replies for the VIP — not just the one $[prodname] elected — so external hosts receive conflicting replies from several MAC addresses. Either run kube-proxy in `iptables` or `nftables` mode, or keep IPVS mode and set `strictARP: true` in kube-proxy's configuration (the `ipvs.strictARP` field of the `KubeProxyConfiguration`, which is stored as YAML inside the `kube-proxy` `ConfigMap`) so the kernel stops answering ARP for the dummy-interface VIPs and leaves it to $[prodname]'s responder. :::note @@ -64,7 +64,7 @@ kubectl patch felixconfiguration default --type='merge' \ -p '{"spec": {"localSubnetL2Reachability": "PodsAndLoadBalancers"}}' ``` -The feature is `Disabled` by default. This setting is read when the dataplane starts, so changing it makes Felix (`calico-node`) **restart automatically** to pick up the new value — no manual restart is required. The feature takes effect once Felix finishes restarting. For the full field reference, see [Felix configuration resource](../../reference/resources/felixconfig.mdx). +The feature is `Disabled` by default. This setting is read when the data plane starts, so changing it makes Felix (`calico-node`) **restart automatically** to pick up the new value — no manual restart is required. The feature takes effect once Felix finishes restarting. For the full field reference, see [Felix configuration resource](../../reference/resources/felixconfig.mdx). :::note