Skip to content

Mgmt firewall, cable plan and Ansible export, a click-together topology builder, and pods for leaf-spine-superspine fabrics. - #17

Closed
muhittink wants to merge 27 commits into
mainfrom
mgmt-firewall-and-ansible-export
Closed

muhittink wants to merge 27 commits into
mainfrom
mgmt-firewall-and-ansible-export

Conversation

@muhittink

Copy link
Copy Markdown

Dont merge, just to show some ideas

What changes for an existing plan

Opening a plan you already have, without touching anything:

  • The BOM orders more.
    • Two mgmt firewalls in a redundant management network, one otherwise. Each is a vendor-neutral 1U server with 6x 1G RJ45, running pfSense or Debian with FRR. The Teltonika RUTXR1 used so far is end of life, last order 2027-02-03 per Teltonika's EOL list.
    • RJ45 cables change. Each mgmt server now has one cable, to its own mgmt spine, instead of one to every mgmt spine. Each mgmt firewall adds three, as in the deployment guide's routed out-of-band network.
  • 25G breakouts are counted per leaf. A breakout serves one switch port, but a group's ports spread over the leaf pair. Groups whose node count is not a multiple of 4 were ordered short: 9 nodes need 6 breakouts, not 5. The leaf capacity check had the same error. Groups of 8 nodes, as in the templates, are unaffected.
  • Plan format 2. Files exported now are refused by older planners instead of losing data silently. A stored plan the app cannot read is kept under metal-stack-planner/plan-unreadable before it is replaced.
  • Leaf-spine-superspine plans get pods. A plan without pods gets an implicit Pod 1 holding every rack, plus the central rack's own spines. That doubles the spines in the BOM, and validation warns that a single pod gains nothing over leaf-spine. See below.
  • Clicking the topology selects instead of jumping to the Plan tab. "Open in plan editor" in the inspector still jumps. Clicks on the diagram used to not arrive at all in Chromium, see the pointer fix below.

Management network

  • Mgmt server i, mgmt firewall i and mgmt spine i form a chain. The firewall links to the server, to the server's BMC and to the mgmt spine's eth0.
  • Each spine hands its single eth0 to one mgmt spine, in turn.
  • Mgmt links are colored by the port they land on: red for a switch's eth0, green between front-panel ports.

Topology builder

  • A palette adds partitions, racks, rack groups, server groups, spines, exits, routers and external networks at the current selection.
  • An inspector reuses the Plan tab's section editors. They now size their grids by the section (container queries), so they also fit the narrow inspector.
  • Server groups can be dragged onto another rack. The move uses moveChassis from Sven-Ric's drag-and-drop-rack branch: only the model function, its tests and the store action, in a commit under his name.
  • ZoomPane captured the pointer on every press, so no click on a diagram element ever arrived. It now captures only once a press moves past 4 px.

Cable plan and Ansible export

derive/deploy/cabling.ts names both ends of every cable in a partition:

  • Hostnames follow a production inventory layout: <partition>-r01leaf01, -spine01, -mgmtspine01, -mgmtserver01, -mgmtfw01.
  • SONiC port names come from sonic-buildimage port_config.ini for the AS7726-32X and AS4630-54TE. A model without a port map is reported, never guessed.

Its tests compare the cable counts per type with deriveBom() for the default plan, every template, and plans with every cabled feature.

Export Ansible downloads <plan>-inventory.yaml:

  • Layout: a group per partition with role child groups and the generic role groups production playbooks target.
  • Variable names are the ones metal-roles v0.35.0 reads: sonic-config, metal-core, mgmt-server, dhcp, pixiecore, ztp.
  • Addresses: loopbacks, mgmt interfaces and PXE subnets come from the IP plan. The IP plan gains mgmt loopbacks and four /30 mgmt links per side.
  • DHCP: dhcp_subnets on the mgmt servers cover every network the switches relay, each server with its own share of the range. The ONIE and SONiC ZTP options are included.
  • ASNs: production-like, see the header of inventory.ts.
  • Cable plan: planner_ports documents the cables per device.
  • Missing inputs stay undefined, never a placeholder, so the roles' asserts stop a run. The file header lists them: control plane endpoints, release, DNS/NTP, BMC credentials, upstream peers, keys, switch MACs.
npm run dev   # Plan tab: "Export Ansible"; Topology tab: palette, inspector, drag & drop
ansible-inventory -i <plan>-inventory.yaml --list

Leaf-spine-superspine: pods and planes

Before, every leaf linked to every spine and every spine to every superspine, so the superspine tier carried no traffic. Now it follows the 5-stage Clos of RFC 7938:

  • A pod's racks uplink only to the pod's own spines (Figure 3, section 3.2.3).
  • Spine j of every pod connects to plane j of the superspines (section 3.2.4).
  • The central rack's spines are the border pod for exits, storage leaves and a control-plane rack, adapted from section 5.2.4.
  • ASNs follow section 5.2.1: one for the superspines, one per pod's spines.

derive/pods.ts is the single source for this, and spinePorts() in bom.ts the single rule for spine links. They drive:

  • the BOM and the port budgets, incl. superspines,
  • the busiest spine's oversubscription ratio,
  • the rack layout and the IP plan,
  • the cable plan and the Ansible export.

Pods are edited in the central rack section, and each rack picks its pod. The diagram draws each pod as a box with its spines above its racks.

Knob Where Meaning
Partition.pods central rack section the compute pods; none means an implicit Pod 1
Rack.podId rack section the pod a rack belongs to; unknown or absent means the first pod
spineCount "Spines per pod" spines of every pod, the central rack included
superspineCount "Superspines" must be a multiple of spineCount: one plane per spine

Verification

  • npm run test (368), lint, format check and build pass.
  • New assertions were shown to fail once against the old code or a deliberate mutation. For example, a full spine-superspine mesh in the cable plan fails the cable-vs-BOM comparison with 44 against 32 trunks.
  • The generated inventories were read by ansible-inventory with 0 warnings (it exits 0 even on YAML errors, so warnings were counted). Variable names were matched against metal-roles at v0.35.0-2-g75c0fbe. sonic-config's frr.conf.j2 and sonic-config.yaml.j2 and dhcp's dhcpd.conf.j2 were rendered with the generated host variables.
  • These were clicked through in headless Chromium with real mouse events: the builder, drag & drop, the pod editor, and the stored-plan migration and backup.
  • A second session reviewed the pod work and re-ran its own mutation probes at 8802e79.

Known limits

  • Leaf-spine-superspine is a planning design after RFC 7938, not a metal-stack reference architecture. It has not run on hardware or in containerlab, EVPN over two spine tiers included. Validation says so on every such partition, and the Ansible export marks it "UNTESTED in a lab". Validate it in a lab before ordering.
  • Nothing in the export was run against a live fabric. Rendering templates and parsing the inventory proves structure, not behaviour. The ASN scheme and port conventions are this planner's choices, modelled on one production inventory.
  • Pod spine hostnames are positional (p01spine01). Removing or reordering a pod renames them in the export.
  • Older planner builds on the same origin still replace a format-2 stored plan with an empty one. The backup exists only from this build on.
  • Not generated or cabled:
    • an L2 management network,
    • a control-plane rack of its own,
    • the AS4625-54T (no upstream port map),
    • internet routers in the inventory,
    • an end-of-row switch for mgmt leaf eth0,
    • the mgmt firewall's uplink.
  • Builder gaps: no "Pod" item in the palette, new racks land in the first pod, no keyboard alternative for drag & drop. Clicking was tested in Chromium only.
  • pfSense caveat: pfsensible has no module for FRR/BGP or numbered DHCP options, so a pfSense mgmt firewall is not covered by Ansible.

🤖 Generated with Claude Code

muhittink and others added 27 commits September 26, 2026 23:32
The deployment guide's routed out-of-band network is two chains:
mgmt firewall i links to mgmt server i (interface and BMC) and to mgmt
spine i's own mgmt port, and mgmt server i uplinks to mgmt spine i only.
The planner drew and counted every mgmt server against every mgmt spine
and had no mgmt firewall at all.

The firewall is a vendor-neutral 1U server with 6x 1G RJ45 (pfSense or
Debian with FRR); the Teltonika RUTXR1 of earlier setups is end-of-life
(last order 2027-02-03). Its 100 W power draw is an estimate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every switch has a single management interface, but the topology drew a
line from each spine to every mgmt spine. The spines now take turns on
the mgmt spines, as in the deployment guide (spine01 on mgmt-spine01).
The BOM already counted one cable per spine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A link from a switch's eth0 into the management network looked exactly
like a front-panel data link. TopoLink.mgmtPort now marks the links with
an end on a dedicated management interface (leaf to mgmt leaf, spine to
mgmt spine, mgmt firewall to mgmt spine); the diagram draws those red and
the other management links green. Production stays blue.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ZoomPane captured the pointer on every press, which retargets the click
to the pan container: no click on a diagram box or node ever arrived, so
"click a rack to open its editor" did nothing. Capture now starts after
the pointer has moved PAN_THRESHOLD pixels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A palette adds partitions, racks, rack groups, server groups, spines,
exits, routers and external networks at the current selection, and an
inspector shows the selected section's editor. Every palette item is an
existing store action, so the topology, BOM and validation stay derived
from the Plan and every add is undoable. The selection is view state and
never part of the Plan. The placement rules live in views/topology/build.ts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A 4x25G breakout serves four ports of a single switch port, but a
dual-attached group spreads its ports over the leaf pair. The BOM and
the leaf capacity check rounded the group's ports up to fours as if one
breakout could span both leaves, so any group whose node count is not a
multiple of four came out short: 9 nodes need 3 + 3 breakouts, not 5;
10 nodes need 6, not 5.

nodeSwitchPorts() is now the single rule for BOM lines (servers and
on-prem control plane nodes) and leafPortsNeeded(). Groups of eight
MicroCloud nodes, as in the templates, are unaffected. Found by laying
out the physical cable plan and comparing it with the BOM.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sonicPorts maps a switch model's front-panel ports to SONiC interface
names, per port group in the order of `ports`. Taken from sonic-buildimage
port_config.ini for the AS7726-32X (Ethernet0..124, step 4) and the
AS4630-54TE (Ethernet0..47 1G, 48..51 25G, 52 and 56 100G). Models
without an upstream port map (AS4625-54T) have none, so nothing guesses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The routed management network needs addresses the IP plan did not size:
a loopback per mgmt spine and mgmt leaf, and four /30 per side of the
management network (firewall to mgmt server, to its BMC and to the mgmt
spine's mgmt port, mgmt server to mgmt spine), as in the deployment
guide's routed out-of-band network.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
deriveCabling() names both ends of every cable in a partition: hostnames
modelled on a production inventory (<partition>-r01leaf01, -spine01,
-mgmtspine01, -mgmtserver01, -mgmtfw01), SONiC port names from the
catalog, 4x25G breakouts, and the deployment guide's port names for mgmt
firewall and mgmt server. Its tests hold the cable counts per type
against deriveBom() for the default plan, every template and a plan with
every cabled feature.

Superspines, an L2 management network, a control-plane rack of its own
and switch models without a port map are reported, not cabled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
toYaml() writes plain data with every string as a JSON string, which is a
valid YAML double-quoted scalar, so "no", "on" and "1.10" stay strings and
Jinja expressions survive. Undefined values are left out rather than
written as a placeholder. No new dependency.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
deriveInventory() turns the cable plan and the IP plan's infrastructure
ranges into an inventory in the layout of a production partition: a group
per partition with role child groups (spines, exits, leaves, storage
leaves, mgmt spines, mgmt leaves, mgmt servers, mgmt firewalls) and the
generic role groups above them. Hosts carry the variables metal-roles
v0.35.0 reads (sonic-config, metal-core, mgmt-server, pixiecore, ztp):
loopbacks, ASNs, mgmt interfaces, BGP ports, breakouts, PXE subnets and
DHCP relays; planner_ports documents the cable plan per device.

Inputs a plan cannot know (control plane endpoints, release, DNS/NTP,
BMC credentials, upstream peers, keys) stay undefined and are listed in
the file header, so the roles' own asserts stop a run instead of running
with a placeholder. Checked with ansible-inventory, a variable-name match
against metal-roles, and by rendering sonic-config's frr.conf and
sonic-config.yaml templates for spine, exit, leaf, mgmt spine and mgmt
leaf.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The switches relay PXE (vlan4000) and management DHCP to the mgmt
servers, but the inventory gave those servers no dhcp_subnets. The dhcp
role has no asserts, so a run would have succeeded with no leases for
PXE, BMCs or switch ZTP. Each mgmt server now serves every relayed
network with the relaying SVI as router and its own share of the dynamic
range (two dhcpd without failover must not lease the same address), plus
the ONIE and SONiC ZTP options from the dhcp and ztp role examples.
Checked by rendering the dhcp role's dhcpd.conf template.

A rack with more nodes than its leaves' PXE subnets hold is reported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Production partition playbooks address some roles by other group names:
sonic_leaves, mgmt_spine_switches, mgmt_leaf_switches, sonic_mgmtleaves
and mgmt_servers. The generic groups now include them, so those plays
find their hosts instead of skipping with an unmatched-pattern warning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Downloads <plan>-inventory.yaml, or explains in a toast why the plan
cannot be exported yet. Client-side only, like the other exports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The plan editor sections switched to four columns at the md window
breakpoint. Reused in the topology builder's 28rem inspector, that
squeezed four fields into a narrow card and cut off select values
("Leaf-spi", "Redunda"). The sections are now CSS containers and their
grids switch at the @2XL container breakpoint (42rem): four columns in
the Plan tab as before, two in the inspector.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Taken unchanged from 3073fa6 on drag-and-drop-rack (the model function,
its tests and the store action, without the Racks view drag-and-drop),
so the topology builder can move server groups with the same rule.
Nodes merge into a matching group in the target rack; a group moved in
full disappears from its source rack.
A server group node can be dragged onto another rack of its partition:
a ghost follows the pointer, the rack under it is outlined as the drop
target, and releasing there moves all of the group's nodes with
moveChassis, merging into a matching group in the target rack. The
target rack is selected afterwards and the move is undoable. A press
only becomes a drag after DRAG_THRESHOLD, so a click still selects, and
the hover fade is off while dragging so the drop targets stay visible.
moveGroupArgs() holds the drop rules: same partition, another plan rack,
never the central or the control-plane rack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A leaf-spine-superspine partition needs pods to be a 5-stage Clos at all:
with every leaf on every spine, the superspines carried no traffic.
Partition.pods and Rack.podId describe them; derive/pods.ts is the one
place that says which racks a pod holds (racks without a known pod go to
the first, and a partition without pods has an implicit Pod 1), how many
spines there are (spineCount per pod plus the border pod in the central
rack) and how many superspines each plane gets, following RFC 7938
section 3.2.3.

Version 1 files need no reshaping. The format still goes to version 2
so that an older planner refuses a file with pods instead of silently
dropping them.
In a leaf-spine-superspine partition every leaf was linked to every
spine and every spine to every superspine, so the superspine tier
carried no traffic. The topology now follows RFC 7938 section 3.2.3: a
rack's leaves uplink only to its pod's spines, and spine j of every pod,
including the border pod in the central rack, connects to plane j of the
superspines only. Exits and storage leaves stay on the border pod's
spines, the dedicated cluster for external connectivity (section 5.2.4).
Leaf-spine partitions are unchanged.

The diagram does not lay out pods yet; that follows separately.
With pods, the BOM ordered spineCount spines in all and cabled every leaf
to them; the superspine tier was sized against the whole partition. Now:

- spinePorts() in bom.ts is the one rule for the links a spine
  terminates, per group: the central rack's spines (exits, storage
  leaves, control-plane rack leaves) and each pod's (its own leaves),
  each plus one link per superspine of its plane. The BOM and the spine
  port budget both use it.
- The BOM, the central rack elevation, the IP plan's loopbacks and mgmt
  addresses count every pod's spines (spinesTotal).
- The spine tier ratio compares the busiest pod's leaf links against its
  plane's superspines.
- Validation reports a superspine count that does not split into one
  plane per spine, superspines without a port for every pod, pods without
  racks, and on every superspine partition that the design follows
  RFC 7938 and is not a metal-stack reference architecture.

podPlan.fixture.ts is the shared hand-computed test plan.
The cable plan and the Ansible export refused leaf-spine-superspine
partitions. The cable plan now lays out pods and planes: a pod's racks
uplink to the pod's own spines (<partition>-p01spine01), spine j of the
central rack and of every pod to plane j of the superspines from its top
ports, and each superspine takes one port per pod. The cabling test adds
a two-pod plan, so the independent cable-plan-vs-BOM comparison now
covers five stages as well.

The inventory gains a superspines group and follows RFC 7938 section
5.2.1 for ASNs: the superspines share one, the spines of each pod share
their own, the central rack's spines keep theirs. A new test checks that
every switch of the cable plan is a host; without it, missing
superspines went unnoticed by the per-host checks. Checked with
ansible-inventory (24 hosts, no warnings) and by rendering sonic-config's
frr.conf for a superspine, a pod spine and a pod leaf.
With pods the diagram had no place for the pod spines. Each pod is now a
box above its racks that holds the pod's spines: a leaf's uplinks stay
inside its pod, and only spine j of each pod rises to plane j of the
superspines in the central rack. Leaf-spine partitions and the
management and central views, which show no pods, are laid out as
before.
The central rack section of a leaf-spine-superspine partition lists its
pods: rename, remove (its racks move back to the first pod) and "Add
pod", whose first use turns the implicit Pod 1 into a real one so the
existing racks stay put. Each rack picks its pod next to its rack type.
Every change is a store action and undoable.

The spine and superspine info texts described the old full mesh ("each
spine connects once to every superspine"); they now describe pods and
planes, and the superspine text says it follows RFC 7938 and is not part
of the metal-stack documentation.
Review findings on the pod model:

- The spine tier ratio ignored the central rack's spines, which carry
  every pod's storage traffic (exits, storage leaves and control-plane
  rack leaves on their way to the superspines). It now takes the busiest
  spine of all groups; requiredSuperspines follows.
- The mgmt spine copper check counted every central device against a
  single mgmt spine, although the cable plan spreads them over the mgmt
  spines in turn. With pods that turned into false errors (a plan the
  cable plan lays out with 28 cables per mgmt spine was refused for 49).
  It now counts each mgmt spine's share plus its own mgmt server, and
  includes on-prem control plane nodes, which the cable plan puts there.
- New warnings: a superspine partition with only one pod gains nothing
  over leaf-spine, and a rack whose podId names no pod (a hand-edited or
  imported file) silently counts to the first pod.
The pod spines sit in the central rack, so the central view now keeps
the pods and their spines instead of dropping them. The pods.ts header
cited RFC 7938 section 3.2.3 for "Every Tier 2 device connects to a
single group of Tier 1 devices"; that sentence is in section 3.2.4
(3.2.3 holds Figure 3). The border pod is now described as adapted from
section 5.2.4, whose Tier 3 are WAN routers.
On rehydrate, a stored plan that does not migrate or parse (for example
one written by a newer planner) was replaced by an empty plan and that
was persisted: the original was gone without a message. With plan format
2 this becomes reachable as soon as an older build opens storage written
by this one. merge now copies the original to
metal-stack-planner/plan-unreadable first.

The key and helper are defined above the store: rehydration runs while
the module is still being evaluated, and a const further down was in its
temporal dead zone, which the try/catch swallowed. Checked in Chromium:
a stored v1 plan keeps its name and racks; a stored format 3 plan is
replaced and kept under the backup key.
An exported inventory with a leaf-spine-superspine partition says in its
header that the design follows RFC 7938, is not a metal-stack reference
architecture, and is untested in a lab. The cabling test gains a plan
that combines pods with a rack group, storage leaves, two links per leaf
and spine pair and an odd node count; mutating the plane mapping in the
cable plan makes the cable-vs-BOM comparison fail (44 vs 32 trunks).
@muhittink muhittink closed this Oct 3, 2026
@muhittink
muhittink deleted the mgmt-firewall-and-ansible-export branch October 3, 2026 06:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

2 participants