link_up() fails with EHOSTUNREACH on dual-stack devices after link_down()

[Frando]

Summary

Iface::link_up() fails with No route to host (os error 113) on any dual-stack
device. A down/up cycle, which should be a routine fault-injection step, is
therefore impossible on dual-stack topologies, and that includes every device
built from RouterPreset::Home, Corporate, Cloud, IspCgnat or Public.

IpSupport::V4Only devices are unaffected.

Found in patchbay 0.7.0 while porting the iroh-gossip netsim tests from 0.1.

Repro

use patchbay::{IpSupport, Lab, RouterPreset};

#[ctor::ctor(unsafe)]
fn userns_ctor() {
    unsafe { patchbay::init_userns_for_ctor(); }
}

// fails: Received a netlink error message No route to host (os error 113)
#[tokio::test]
async fn dualstack_link_flap() -> anyhow::Result<()> {
    let lab = Lab::new().await?;
    let home = lab.add_router("home").preset(RouterPreset::Home).build().await?;
    let dev = lab.add_device("dev").uplink(home.id()).build().await?;
    let eth0 = dev.iface("eth0").unwrap();
    eth0.link_down().await?;
    eth0.link_up().await?;
    Ok(())
}

// passes
#[tokio::test]
async fn v4only_link_flap() -> anyhow::Result<()> {
    let lab = Lab::new().await?;
    let home = lab
        .add_router("home")
        .preset(RouterPreset::Home)
        .ip_support(IpSupport::V4Only)
        .build()
        .await?;
    let dev = lab.add_device("dev").uplink(home.id()).build().await?;
    let eth0 = dev.iface("eth0").unwrap();
    eth0.link_down().await?;
    eth0.link_up().await?;
    Ok(())
}

running 2 tests
test dualstack_link_flap ... Error: Received a netlink error message No route to host (os error 113)
FAILED
test v4only_link_flap ... ok

Cause

Linux flushes an interface's IPv6 addresses when the link goes down, unless
net.ipv6.conf.<if>.keep_addr_on_down is set. The kernel default is 0.

Iface::link_up (src/iface.rs:379) re-adds the IPv4 default route and then the
IPv6 one:

nl.replace_default_route_v4(&ifname_route, gw_ip).await?;
nl.set_default_route_v6(&ifname_route, primary_v6).await

By that point eth0 has no IPv6 address, so the gateway
(fd10:0:0:2::1 in the repro) is not on-link and the kernel rejects the route
with EHOSTUNREACH. link_down never emits the corresponding
replace_default_route_* teardown and link_up never re-adds the addresses, so
the two operations are not symmetric for IPv6 the way they are for IPv4.

Relevant trace from the failing run:

patchbay::netlink: set link up ifname=eth0
patchbay::netlink: replace default route v4 ifname=eth0 via=10.0.2.1
patchbay::netlink: replace default route v6 ifname=eth0 via=fd10:0:0:2::1
Error: Received a netlink error message No route to host (os error 113)

Suggested fix

Either of:

1. Set net.ipv6.conf.<if>.keep_addr_on_down=1 when creating a routed
   interface, alongside the existing DAD sysctls. This matches the
   Ipv6DadMode::Disabled philosophy of making test topologies deterministic,
   and the addresses then survive the transition untouched.
2. Re-add the interface's configured IPv6 addresses in link_up before
   installing the v6 default route.

Option 1 is a one-line sysctl and keeps link_up simple.

Workaround

Setting the sysctl from the test before taking the link down is enough:

let path = format!("/proc/sys/net/ipv6/conf/{}/keep_addr_on_down", iface.name());
device.run_sync(move || { std::fs::write(&path, "1")?; Ok(()) })?;
iface.link_down().await?;

Confirmed working: with this in place the dual-stack repro above passes.

Environment

• patchbay 0.7.0 (crates.io)
• Linux 7.1.3-arch2-2, rootless (unprivileged user namespace)