diff --git a/README.md b/README.md index 9de036b..4dd093b 100644 --- a/README.md +++ b/README.md @@ -4,20 +4,16 @@ [![Release](https://github.com/netclab/function-avd/actions/workflows/release.yml/badge.svg)](https://github.com/netclab/function-avd/actions/workflows/release.yml) **Arista AVD configs, kept live by Kubernetes.** Instead of running Ansible to produce -device configs once, a fabric is modelled as a Crossplane composite resource: edit the -`Fabric`, and every affected device's config re-renders on reconcile — including the -devices you didn't touch, because AVD's facts are fabric-wide. +device configs once, a fabric is a Crossplane composite resource: edit it, and every +affected device re-renders on reconcile — including the devices you did not touch, +because AVD's facts are fabric-wide. -The engine is [pyavd](https://pypi.org/project/pyavd/) (pure Python, no Ansible), -wrapped in a Crossplane composite function. Built against +The engine is [pyavd](https://pypi.org/project/pyavd/) (pure Python, no Ansible), wrapped +in a Crossplane composite function. Built against [Arista AVD](https://github.com/aristanetworks/avd) v6.3.0, pinned as a submodule under `avd/` — a read-only reference, and the source of the golden configs the tests diff against. -> The layout follows [function-template-python](https://github.com/crossplane/function-template-python): -> the code lives in a flat `function/` package, with `main.py` as the gRPC -> entrypoint and `fn.py` as the FunctionRunner. - ## Quick start Needs `docker`, `kind`, `kubectl`, `helm`, the `crossplane` CLI, and @@ -29,231 +25,139 @@ kubectl --context kind-avd apply -f examples/fabric/single-dc-l3ls.yaml kubectl --context kind-avd -n default get fabric,device ``` -One `Fabric` fans out to 8 `Device`s, each composing a ConfigMap that holds its AVD +One `Fabric` fans out to 8 `Device`s, each composing a ConfigMap with that device's AVD structured config and rendered `eos.cfg`: ```bash -kubectl --context kind-avd -n default get cm -l avd.netclab.dev/device kubectl --context kind-avd -n default get cm -l avd.netclab.dev/device=dc1-leaf1a \ -o jsonpath='{.items[0].data.eos\.cfg}' ``` -To see the model actually live, patch a spine's `bgp_as` under `spec.design` — the -change reaches that spine *and* its leaves' BGP neighbours, because the pipeline -recomputes fabric-wide facts. `scripts/kind-down.sh` tears it all down. +Patch a spine's `bgp_as` to see the model live: the change reaches that spine *and* its +leaves' BGP neighbours. `scripts/kind-down.sh` tears it all down. -### The lab +## The API -`WITH_NETCLAB=1` additionally brings up containerised cEOS nodes, cabled from the fabric -itself: AVD resolves the peering, so `avd-topology` derives the netclab topology from the -same model that renders the configs, and the two cannot drift apart. +Six composite kinds, one function image dispatching on `kind`: -The default subset's topology is generated once and committed as -[`examples/lab/topology.yaml`](examples/lab/topology.yaml), so the lab boots a reviewed -file and an AVD upgrade that changes the cabling fails `test_topology_golden` instead of -passing unnoticed. Regenerate it with the command in that test's docstring. +| Kind | What it is | +|------|------------| +| `Fabric` | one AVD fabric. Design comes from `spec.design` inline, from the inputs `spec.requires` names, or both. Composes one `Device` per host. | +| `Device` | one switch: validates and renders its own config, composes the ConfigMap, and with `spec.push` a provider-http `Request` that holds the box in sync over eAPI. | +| `NodeSet` | node-type blocks, and the devices they declare — the fabric's device list. | +| `NetworkServiceSet` | tenants, VRFs, SVIs. | +| `ConnectedEndpointSet` | servers, ports, port profiles. | +| `SettingSet` | everything fabric-wide: BGP peer groups, AAA, NTP, addressing pools. | -```bash -WITH_NETCLAB=1 scripts/kind-up.sh # + CNI plugins, Multus, netclab-chart, cEOS nodes -kubectl --context kind-avd apply -k examples/lab/ -``` +`spec.requires` is an ordered list; per host, the inputs that apply are layered in that +order with `dict.update()` — Ansible's `hash_behaviour=replace`. Nothing is merged, so +two `NodeSet`s carrying the same node-type key never meet. `spec.appliesTo` scopes an +input (`all`, `nodeSets`, `hosts`, `matchHostnames`); on a `NodeSet`, left out, it means +the devices that `NodeSet` declares. -cEOS is licensed and cannot be pulled — import it once and it outlives teardown in the -registry's data volume; `kind-up.sh` says how if it is missing. `LAB_HOSTS` picks the -subset, defaulting to the two that make one link, because all eight is 16Gi of cEOS. -Setting it to anything else regenerates the topology, since `--hosts` is an input to -generation rather than a filter applied to the result — a link survives only when both -of its ends are selected. +⚠ **A Fabric whose named inputs have not arrived renders nothing** and names what it is +waiting for. Requirements are answered on the *next* reconcile, so the first always +arrives empty — and a push replaces a device's whole configuration. -`examples/lab/` is that same fabric with eAPI bound to the default VRF, which is the one -thing a lab genuinely needs and production does not: AVD binds eAPI to VRF MGMT on -Management1, and a cEOS pod has neither. It also sets `spec.push`, so each lab Device -composes a provider-http `Request` that keeps the box itself in sync over eAPI: +`examples/fabric/` holds a `Fabric` per AVD example using `spec.design`; +`examples/fabric/inputs/` holds the same fabrics through `spec.requires`. Neither names a +namespace, so `kubectl -n apply -f ` decides where they land. -```bash -kubectl --context kind-avd -n default get requests.http.m.crossplane.io,device -kubectl --context kind-avd -n default exec dc1-spine1 -- Cli -p 15 -c "show running-config digest" -``` +## Migrating an AVD inventory -The Request's OBSERVE reads the device's own `show running-config digest`; a mismatch -- -whether the model changed or someone edited the box by hand -- makes the provider replay -a full `configure session` + `rollback clean-config` replace. Drift is reclaimed on the -provider's poll interval, the same rhythm that paces the rest of the model. - -**No cluster?** The engine and the function both run locally: +`avd-migrate` turns an Ansible inventory into a `Fabric` plus its input XRs — one fabric +per eos_designs play, one input per ownership fragment per category. ```bash -uv run avd-verify # pyavd vs AVD's own golden structured configs -uv run avd-verify-xr # same, through the Fabric-XR fold -crossplane render examples/fabric/single-dc-l3ls.yaml \ - apis/fabric/composition.yaml dev/function-render.yaml +uv run avd-migrate avd/ansible_collections/arista/avd/examples/single-dc-l3ls +uv run avd-migrate --emit /tmp/xrs # write them out as manifests ``` -## How it works - -Two composite types, served by one function image that dispatches on `kind`: - -**`Fabric`** — one AVD fabric, meaning a single `fabric_name`. `spec.design` carries a -fabric-wide `eos_designs` document (structurally open, validated by pyavd); device roles -resolve inside it via `default_node_types`. The function runs the fabric-wide pyavd -pipeline (`validate_inputs → get_avd_facts → get_device_structured_config`) and composes -one `Device` per host, carrying that device's config inline in `spec.structuredConfig`. -Multi-DC topologies are still one fabric when they share a `fabric_name` (DCs stitched -via `evpn_gateway`). - -**`Device`** — validates and renders its own config (`validate_structured_config` + -`get_device_config`), reports per-device status (`configHash`, `managementAddress`, -`nodeType`, and `Validated`/`Rendered` conditions), and composes the artifact: a -ConfigMap with `structured_config.yaml` + `eos.cfg`. With `spec.push` set (the Fabric -propagates it from its own `spec.push`, lab-only today), the Device also composes a -namespaced provider-http `Request` that pushes `eos.cfg` to the device over eAPI and -holds it there — `Deployed` condition, `status.push.digest`, and readiness then include -the box itself, not just the rendered artifact (see `push.py` for the protocol). - -Why the split? Facts are fabric-wide, so the fabric has to be the reconcile unit — one -spine's `bgp_as` must reach its peers. But the config push to the device (or CVP) -attaches *per device*, so a Device has to be a real reconcile unit with its own status, -not a passive ConfigMap. The e2e test pins exactly that contract: pause the Fabric and -patch a Device directly, and it still renders on its own; unpause, and the Fabric takes -the change back. The Device is live, but not authoritative. - -## Status - -The full path works on a real cluster: applying two `Fabric` XRs in one namespace -reconciles to `Synced=True Ready=True`, fanning out to 24 `Device`s → 24 ConfigMaps, all -`Ready`, contents matching AVD golden. - -Liveness is fabric-wide, not just local: bumping a spine's `bgp_as` in `spec.design` -re-renders that spine *and* rewrites `remote-as` on each of the four L3 leaves that peer -with it, because the pipeline recomputes AVD facts for the whole fabric. What paces this -is Crossplane's poll interval, not the render — measured on kind, the touched spine went -in ~5s and its leaves within ~45s (see [Gotchas](#gotchas)). - -The loop closes on the lab: with `spec.push` set, the same `bgp_as` bump reached the -touched spine's *running config* in ~47s and rewrote `remote-as` on its peering leaf's -running config in ~63s — and the reverse direction holds too, a `vlan 999` added by hand -on the box was replaced with the byte-identical golden config (EOS's own -`show running-config digest` certifying both states) on the next provider poll. -Deleting a Device orphans the box rather than wiping it: an empty switch is not a -desired state. +It reimplements no part of Ansible: `ansible-inventory --list --export` says which group +carries which variable, an ad-hoc `debug` run says what a play resolved it to. The one +rule Ansible does not print is group order (depth, then name) — the migration layers +fragments with it and **refuses to emit anything** unless the result equals what Ansible +reports. -### Engine fidelity +What it cannot carry it names rather than dropping. Four flags carry more: -Before wrapping pyavd in Crossplane, the point was to prove the pyavd path reproduces -AVD's own output. Each bundled AVD example is rebuilt into `all_inputs` the way Ansible -would (per-host group_vars merge, in precedence order), run through pyavd, and deep-diffed -against the example's checked-in `intended/structured_configs`. +| Flag | What it does | +|------|--------------| +| `--namespace NS` | pin the manifests to one namespace instead of naming none | +| `--emit-pool-seed` | carry the node-ID assignments as a ConfigMap the Fabric seeds from | +| `--compat-ip-addressing` | point `ip_addressing` at `function.avd_compat` where the pinned templates are the ones it transcribes | +| `--drop-description-templates` | drop interface-description templates so the rest renders | -| Example | Devices | From Ansible inputs | Folded into one `Fabric` | -|---------|--------:|:-------------------:|:------------------------:| -| single-dc-l3ls | 8 | ✅ | ✅ | -| single-dc-l3ls-ipv6 | 8 | ✅ | ✅ | -| single-dc-multipod-l3ls | 10 | ✅ | ✅ | -| dual-dc-l3ls | 16 | ✅ | ✅ | -| l2ls-fabric | 6 | ✅ | ✅ | -| isis-ldp-ipvpn | 9 | ✅ | ✅ | -| campus-fabric | 10 | ✅ | ⏸ deferred | -| cv-pathfinder | 17 | ⏸ deferred | ⏸ deferred | +Measured against AVD's own corpus: hostvars byte-identical to Ansible for all 8 bundled +examples and 19 molecule scenarios, up to 501 devices in one play; rendered configs +matching AVD's checked-in golden for **8 of 8 examples on a cluster**, every device. -Zero diffs everywhere it is ticked, across both group_vars layouts (per-group directories -and flat files) and explicit and implicit `all` inventories. The fold -(`function/xr.py`) unions each DC's node-type blocks and pushes per-DC/per-pod -`defaults` down to node_groups/nodes (which override defaults in AVD), so multi-DC -fabrics collapse losslessly into one document. +## The lab -The two deferrals are understood, not mysterious: +`WITH_NETCLAB=1` additionally brings up containerised cEOS nodes, cabled from the fabric +itself — `avd-topology` derives the netclab topology from the same model that renders the +configs, so the two cannot drift apart. -- **campus-fabric** — leaves carry RADIUS in `aaa_settings`, spines don't: a - fabric-global key that genuinely differs by role, with no node-scoped equivalent. -- **cv-pathfinder** — SD-WAN multi-site (a WAN gateway across 2 routers) plus - ansible-vault secrets. +```bash +WITH_NETCLAB=1 scripts/kind-up.sh +kubectl --context kind-avd apply -k examples/lab/ +``` -They live in `verify_xr.DEFERRED` as *strict* expected failures: if one starts folding, -the suite fails and says to remove it, so a deferral can't quietly rot. +`examples/lab/` is that same fabric with eAPI bound to the default VRF — the one thing a +lab needs and production does not — and with `spec.push` set, so each Device keeps the +box itself in sync. A change made by hand on the device is replaced with the golden +config on the next provider poll; EOS's own `show running-config digest` certifies both +states. Deleting a Device orphans the box rather than wiping it: an empty switch is not a +desired state. + +cEOS is licensed and cannot be pulled; import it once and it outlives teardown in the +registry's data volume (`kind-up.sh` says how). `LAB_HOSTS` picks the subset, defaulting +to the two that make one link, because all eight is 16Gi of cEOS. ## Testing ```bash -uv run pytest # offline: engine fidelity, the XR fold, the Struct gotcha (~7s) -uv run pytest -m e2e # live cluster: needs kind-up.sh + an applied fabric (~2min) +uv run pytest # offline, no cluster (~90s) +uv run pytest -m corpus # the whole molecule corpus: 501 devices, 71 plays (~4min) +uv run pytest -m e2e # live cluster: needs kind-up.sh (~6min) ``` | Path | Covers | -|------|---------| -| `tests/test_engine_fidelity.py` | the examples above reproduce golden structured config | -| `tests/test_xr_fold.py` | the Ansible→XR fold, over every discovered example | -| `tests/test_normalize_numbers.py` | the protobuf-`Struct` double→int coercion | -| `tests/test_push.py` | the eAPI push Request builders: session contents, digest provenance | -| `tests/test_e2e_device_layer.py` | drift/reclaim + steady-state idempotency, on a cluster | -| `tests/test_e2e_push_layer.py` | on-box drift reclaim + at-rest quiet, on the netclab lab (skips without it) | - -CI gates every push and PR on the offline suite (with `submodules: true` — the golden -configs live there). The e2e job builds the function image and installs Crossplane, so it -is `workflow_dispatch` only. +|------|--------| +| `tests/test_kinds_equivalence.py` | the inputs resolve exactly as Ansible does, and render golden | +| `tests/test_fabric_collect.py` | collection and the gate, driving `RunFunction` directly | +| `tests/test_nulls.py` | an explicit null survives an API server that prunes it | +| `tests/test_categories.py` | which kind a key belongs to still matches AVD's own schema | +| `tests/test_avd_compat.py` | the v2.x addressing class against AVD's golden | +| `tests/test_e2e_migrated_corpus.py` | all 8 examples migrated, applied, rendered, diffed — on a cluster | +| `tests/test_e2e_node_id_pool.py` | the node-ID pool, seeded and kept, on a cluster | +| `tests/test_e2e_device_layer.py` | drift/reclaim + steady-state idempotency (needs an applied fabric) | +| `tests/test_e2e_push_layer.py` | on-box drift reclaim on the netclab lab (skips without it) | + +CI gates every push and PR on the offline suite (with `submodules: true` — the goldens +live there). The e2e job builds the image and installs Crossplane, so it is +`workflow_dispatch` only. ## Releasing -`[project].version` in `pyproject.toml` is the only version number: the release workflow -publishes under it and `kind-up.sh` tags the local image with it, so a locally built -image and a released package can't mean different code under the same tag. Cutting a -release is a bump, a commit, and a matching tag: +`[project].version` in `pyproject.toml` is the only version number; a tag that disagrees +fails the run rather than publishing under either number. ⚠ `uv.lock` carries the project +version too, and CI runs `uv sync --locked` — bump both in one commit. ```bash -uv version 0.2.0 # edits pyproject.toml +uv version 0.2.0 && uv sync git commit -am 'Release v0.2.0' && git tag v0.2.0 git push origin main v0.2.0 ``` -A tag that disagrees with `pyproject.toml` fails the run rather than publishing under -either number. `workflow_dispatch` publishes too, taking no input — it releases whatever -version the checked-out tree declares. - -The workflow reruns the offline suite, builds the runtime image for `linux/amd64` and -`linux/arm64`, embeds each into an xpkg, and pushes both as one multi-platform package. -It also builds a **second package** — `configuration-avd`, the API this function serves -— and pushes it separately, under the same version: - -| Destination | When | -|-------------|------| -| `ghcr.io/netclab/function-avd:` | always — the workflow's `GITHUB_TOKEN` is enough | -| `ghcr.io/netclab/configuration-avd:` | always | -| `xpkg.upbound.io/netclab/function-avd:` | only when the `UPBOUND_TOKEN` secret is set | -| `xpkg.upbound.io/netclab/configuration-avd:` | only when the `UPBOUND_TOKEN` secret is set | - -The second one is what feeds -[marketplace.upbound.io/functions/netclab](https://marketplace.upbound.io/functions/netclab): -the Marketplace indexes `xpkg.upbound.io`, so a GHCR-only release is installable but not -listed. `UPBOUND_TOKEN` is an Upbound robot or personal access token with write access to -the `netclab` organization; without it the release still succeeds and logs a warning. -`up xpkg push --create` makes the repository on first push, but pushing is not publishing -— see the Marketplace gotcha below for the one-time step that actually produces a public -listing. - -Either registry makes the function installable without the local-registry dance in -`kind-up.sh`. Pick a tag from -[releases](https://github.com/netclab/function-avd/releases) — there is no `latest`, -deliberately: a function whose version can move under a running cluster is not one you -can reason about. - -```yaml -apiVersion: pkg.crossplane.io/v1 -kind: Function -metadata: - name: function-avd -spec: - package: ghcr.io/netclab/function-avd: -``` - -### The Configuration package - -A Function package **cannot carry XRDs or Compositions** — `crossplane xpkg build` -rejects both with `object is not a CRD`, because a Function package takes only the CRDs -describing its input, and this function has none (the XRs are its API). So -`apis/fabric/` and `apis/device/` ship as their own Configuration, built from `apis/` -with `apis/crossplane.yaml` as its metadata. +The workflow builds the runtime image for `linux/amd64` and `linux/arm64`, embeds each +into an xpkg, and pushes both as one multi-platform package — plus a **second package**, +`configuration-avd`, the API this function serves. Both go to `ghcr.io/netclab/` always, +and to `xpkg.upbound.io/netclab/` when the `UPBOUND_TOKEN` secret is set. Only the latter +feeds [the Marketplace](https://marketplace.upbound.io/functions/netclab). -Installing it pulls the function in as a dependency, so this is the one line a consumer -needs: +Installing the Configuration pulls the function in as a dependency, so this is the one +line a consumer needs: ```yaml apiVersion: pkg.crossplane.io/v1 @@ -264,117 +168,34 @@ spec: package: ghcr.io/netclab/configuration-avd: ``` -Crossplane names a dependency-installed function `-`, so the Function object -lands as `netclab-function-avd` — which is exactly the name both Compositions reference -and the name `kind-up.sh` installs under, so the dev cluster and a real install agree. - -## Gotchas - -The non-obvious things this repo encodes, each of which cost a debugging session: - -- **Pushing to Upbound is not publishing, and `PUBLIC` doesn't mean listed.** A repository - created by `up xpkg push --create` gets the publishing policy `draft`. Nothing in the - release output says so, and as the repository's owner you can open its Marketplace page - while logged in — so it looks published when anonymously there is no listing at all. - The `PUBLIC` column is a separate axis: it governs anonymous *pull* from the registry, - which works fine throughout. `up repository list` is the ground truth; fix it once with - `up repository update --private=false --publish function-avd` (both flags are required, - so `--private` has to be given explicitly). The policy is a repository attribute, not a - version one, so correcting it never needs a new release. -- **A bare 401 from a registry doesn't mean private.** Both `ghcr.io` and `xpkg.upbound.io` - answer an unauthenticated manifest request with `401` and a - `www-authenticate: Bearer realm=…` header — the normal Docker token flow. Exchange for - an anonymous token at that realm first, then judge visibility by whether *that* succeeds. - Reading the 401 directly is how you conclude a public package is private. -- **`Struct` has no integers.** Crossplane passes resources as protobuf `Struct`, whose - only numeric type is `double`, so VLAN ids and ASNs arrive as floats and pyavd's schema - rejects them. `fn._normalize_numbers` coerces whole-number floats back to - `int` (leaving bools and genuine fractionals alone). -- **Never write a value that changes every reconcile.** An unconditional timestamp causes - a perpetual reconcile and a watch storm. `lastRenderedTime` is only bumped when - `configHash` actually changes. -- **Readiness does not propagate by itself.** With function pipelines Crossplane does - *not* derive a composed resource's readiness from its `Ready` condition — the Fabric - function reads each observed Device's readiness and sets `ready` explicitly, so - `Fabric.Ready` reflects real per-device state. -- **Two image pullers, one reference.** The function image must be pulled by the - Crossplane pod (cluster network) *and* the node's containerd (node network, which can't - resolve cluster DNS). Referencing the registry by its kind-network IP over plain HTTP - satisfies both; containerd is told the registry is insecure. -- **Composed names are XR-scoped**, not `fabric_name`-scoped, so two fabrics with the same - `fabric_name` and overlapping hostnames never collide over a Device or a ConfigMap. -- **Non-root runtime.** The image entrypoint runs the installed console script directly, - not `uv run`, which would need a writable cache under `$HOME`. -- **Propagation waits on the poll interval, not on the render.** Crossplane polls every - 60s by default, so a change that doesn't raise a prompt watch event just sits until the - next pass. Measured on a fresh kind cluster: a direct `Device` patch renders in 2-41s, - while the Fabric reclaiming that drift takes ~127s — roughly two chained intervals, - since the Fabric has to notice first and the Device re-render after. A re-render that - "didn't happen" has usually just not happened *yet*; the e2e budgets allow for it. -- **A pushed config has to keep its own transport alive.** `configure replace` is a - *full* replace, so whatever eAPI the device was reached over is gone unless the pushed - config re-states it. AVD renders `management api http-commands / protocol https` bound - to the VRF from `mgmt_interface_vrf` (MGMT) — so a lab that bootstraps eAPI on plain - HTTP, or reaches the device outside VRF MGMT, loses it on the first successful push. - `examples/lab/` binds eAPI to the default VRF for exactly this reason; the netclab - chart bootstraps the same https/443 AVD renders, so bootstrap and steady state agree. - The same rule holds for *credentials*: the chart bootstraps `arista`/`arista`, and the - model's `arista` user carries the sha512 of that very password — one Secret works - before and after the first push. Change either side alone and the push locks itself out. -- **A config session must not have a fixed name.** EOS keeps exactly **one** completed - session in history (`show configuration sessions`), so `configure session avd-` - works once — and then every retry of the same revision, which is precisely the - drift-reclaim re-push, fails with `already completed`. The push uses an *unnamed* - session (the device picks a fresh name each attempt) and carries its revision in the - JSON-RPC `id` instead. -- **eAPI reports failure inside an HTTP 200.** A failed command comes back as a JSON - `error` member on a perfectly successful HTTP response, so provider-http counts the - push as a success and `Synced` stays `True`. The loop still converges — the next - OBSERVE sees the digest mismatch and retries — but the *only* honest failure signal - is the Request's `status.response.body`, not its conditions. -- **Record the golden digest only from a response that proves its revision.** The - running-config digest cannot be predicted from `eos.cfg` (EOS canonicalizes), so it is - recorded from the device — but the Request's status can lag the spec by a reconcile. - A push response is trusted via the revision-scoped JSON-RPC `id` it echoes; an observe - response only via the pushed `alias avd_cfg_` marker, and even then only while - no digest is recorded for that revision — otherwise manual drift observed at the wrong - moment would be blessed as the new golden state and the model change would never land. -- **`management_eapi` is all-or-nothing.** Absent, it defaults to enabled (https, VRF - MGMT) — which is what golden shows. But set *any* of it without `enabled: true` and the - block defaults to disabled, rendering no `management_api_http` at all: a config that - locks you out of the device it is pushed to. -- **`Responsive=False WatchCircuitOpen` looks like the culprit and isn't.** Devices report - it ("Too many watch events from ConfigMap/…") for long stretches after the create burst, - right next to every slow re-render — but with the breaker open, a direct Device patch - still rendered in 2s. Blaming it costs you an afternoon; the poll interval above is the - real pacing. At rest the model is idempotent and rewrites nothing regardless. +There is no `latest`, deliberately: a function whose version can move under a running +cluster is not one you can reason about. ## Layout | Path | Purpose | |------|---------| | `function/main.py` | gRPC entrypoint (`avd-function`, the image's ENTRYPOINT) | -| `function/fn.py` | the Crossplane composite function (FunctionRunner: Fabric + Device) | -| `function/engine.py` | pyavd pipeline wrapper; `render_fabric_design` is the function's core | +| `function/fn.py` | the composite function: dispatch, the six kinds, composition | +| `function/engine.py` | pyavd pipeline wrapper | +| `function/kinds.py` | the input model: `Input`, `resolve`, which kind a key belongs to | +| `function/pools.py` | the node-ID pool a Fabric composes and seeds from | +| `function/nulls.py` | an explicit null carried past an API server that prunes it | | `function/push.py` | eAPI push protocol: the provider-http `Request` builders | -| `function/xr.py` | fold an Ansible example into a `Fabric` document (block union + defaults push-down) | -| `function/ansible_inputs.py` | rebuild `all_inputs` from an Ansible example (inventory + group_vars merge) | -| `function/verify_example.py`, `verify_xr.py` | golden-diff harnesses (`avd-verify`, `avd-verify-xr`) | -| `apis/fabric/`, `apis/device/` | XRD + Composition for each layer | -| `apis/crossplane.yaml` | Configuration package metadata — `apis/` is that package's root | -| `dev/` | `Function` manifests for local use (kind install, `crossplane render`); outside `apis/` so the Configuration build needs no exclusions | -| `examples/fabric/` | example `Fabric` XRs (each reproduces golden) | -| `examples/lab/` | kustomize overlay: the same fabric as run on the netclab lab | -| `Dockerfile`, `package/crossplane.yaml` | function runtime image + package metadata | +| `function/avd_compat.py` | AVD behaviours pyavd cannot reach, as the classes AVD asks for | +| `function/ansible_cli.py` | Ansible asked rather than reimplemented (dev-only) | +| `function/migrate.py` | an AVD inventory → `Fabric` + input XRs (`avd-migrate`) | +| `apis/` | XRD + Composition per kind; `apis/crossplane.yaml` is the Configuration's metadata | +| `dev/` | `Function` manifests for local use; outside `apis/` so the build needs no exclusions | +| `examples/fabric/`, `examples/fabric/inputs/` | the same fabrics, by design and by inputs | +| `examples/lab/` | kustomize overlay: that fabric as run on the netclab lab | | `scripts/kind-up.sh`, `kind-down.sh` | reproducible cluster bring-up / teardown | -| `.github/workflows/` | `ci.yml` (offline suite + dispatchable e2e), `release.yml` (GHCR + Upbound) | | `avd/` | AVD v6.3.0 submodule (read-only) | -Versions are pinned deliberately: `pyavd` matches the `avd` submodule tag, because the -golden configs come from the submodule — the two only ever move together, which is why -Renovate leaves both alone (`renovate.json`). +`pyavd` matches the `avd` submodule tag — the goldens come from the submodule, so the two +only ever move together, which is why Renovate leaves both alone. ## License Apache-2.0. Builds on [Arista AVD](https://github.com/aristanetworks/avd), also -Apache-2.0; the example fabrics are folded from AVD's own published examples. +Apache-2.0; the example fabrics are derived from AVD's own published examples. diff --git a/apis/connectedendpointset/composition.yaml b/apis/connectedendpointset/composition.yaml new file mode 100644 index 0000000..32167a8 --- /dev/null +++ b/apis/connectedendpointset/composition.yaml @@ -0,0 +1,16 @@ +# Composition for ConnectedEndpointSet: validate this fragment and report on its own status. +# Uses the same function image as Fabric and Device, which dispatches on the +# composite kind. +apiVersion: apiextensions.crossplane.io/v1 +kind: Composition +metadata: + name: connectedendpointset-avd +spec: + compositeTypeRef: + apiVersion: avd.netclab.dev/v1alpha1 + kind: ConnectedEndpointSet + mode: Pipeline + pipeline: + - step: validate-input + functionRef: + name: netclab-function-avd diff --git a/apis/connectedendpointset/xrd.yaml b/apis/connectedendpointset/xrd.yaml new file mode 100644 index 0000000..ada1e89 --- /dev/null +++ b/apis/connectedendpointset/xrd.yaml @@ -0,0 +1,106 @@ +# CompositeResourceDefinition for what connects to the fabric. +# +# Carries `connected_endpoints_keys.key` lists -- `servers`, `firewalls` and the +# rest -- plus `port_profiles` and `network_ports`. Its own kind for the same +# reason as NetworkServiceSet: whoever attaches servers is rarely whoever owns +# the fabric, and RBAC is granted per kind. +# +# Named ConnectedEndpointSet, not Endpoints: `Endpoints` is a core/v1 kind, so +# `kubectl get endpoints` would be ambiguous. The `Set` suffix is the family's -- +# every input kind is a set of design entries of one category, carrying a scope -- +# and it keeps the kind singular, so `connectedendpointset` and +# `connectedendpointsets` say different things the way `nodeset` does. +apiVersion: apiextensions.crossplane.io/v2 +kind: CompositeResourceDefinition +metadata: + name: connectedendpointsets.avd.netclab.dev +spec: + scope: Namespaced + group: avd.netclab.dev + names: + kind: ConnectedEndpointSet + plural: connectedendpointsets + categories: + - crossplane + - netclab + defaultCompositionRef: + name: connectedendpointset-avd + versions: + - name: v1alpha1 + served: true + referenceable: true + schema: + openAPIV3Schema: + type: object + properties: + spec: + type: object + properties: + design: + type: object + description: >- + The endpoints this input contributes. Structurally open: + the top-level key names come from connected_endpoints_keys, + so they are decided by the document's own content and no + OpenAPI schema can describe them. + x-kubernetes-preserve-unknown-fields: true + appliesTo: + type: object + description: >- + Devices that see this input. Defaults to every device in the + fabric. + properties: + all: + type: boolean + nodeSets: + type: array + description: Devices declared by the named NodeSets. + items: + type: string + hosts: + type: array + description: Devices named directly. + items: + type: string + matchHostnames: + type: array + description: >- + Regular expressions matched against device names. Same + semantics as AVD's own default_node_types, which matches + hostnames this way: the pattern is anchored for you, so + `dc1-leaf.*` matches the whole name. A pattern matching + no device is an error, not an empty set -- a pattern is + silent about both matching nothing and matching too much, + and the render is pushed as a full config replacement. + items: + type: string + required: + - design + status: + type: object + properties: + keys: + type: array + description: Top-level eos_designs keys this input contributes. + items: + type: string + devices: + type: array + description: >- + Devices this input resolved to. Reported rather than left to + be inferred, because appliesTo may be a pattern and a pattern + does not say what it matched. + items: + type: string + validation: + type: object + description: >- + Result of validating this fragment. Reported here rather than + on the Fabric so the team that owns this object sees its own + error on its own object. + properties: + ok: + type: boolean + message: + type: string + x-kubernetes-preserve-unknown-fields: true diff --git a/apis/device/xrd.yaml b/apis/device/xrd.yaml index 8c85079..1f46d8c 100644 --- a/apis/device/xrd.yaml +++ b/apis/device/xrd.yaml @@ -19,6 +19,9 @@ spec: names: kind: Device plural: devices + categories: + - crossplane + - netclab # Devices are composed by the Fabric function without an explicit composition # selector; pin the default so selection is deterministic. defaultCompositionRef: diff --git a/apis/fabric/xrd.yaml b/apis/fabric/xrd.yaml index 8f02562..310bcfb 100644 --- a/apis/fabric/xrd.yaml +++ b/apis/fabric/xrd.yaml @@ -22,6 +22,9 @@ spec: names: kind: Fabric plural: fabrics + categories: + - crossplane + - netclab defaultCompositionRef: name: fabric-avd versions: @@ -46,8 +49,74 @@ spec: description: >- Fabric-wide AVD eos_designs input document (node-type blocks, default_node_types, tenants, connected endpoints, ...). - Validated by pyavd; violations reported on status. + Validated by pyavd; violations reported on status. A fabric + may instead be assembled from input objects listed in + spec.requires, in which case this carries only what is + fabric-wide and the inputs carry the rest. x-kubernetes-preserve-unknown-fields: true + requires: + type: array + description: >- + The input objects composing this fabric. Only objects listed + here take part in the render, however they are labelled -- + which is what makes the rendered document a function of this + Fabric rather than of whatever else exists in the namespace, + and it matters because the render is pushed as a full config + replacement. An entry that does not resolve leaves the Fabric + not ready, naming the object it could not find. List order is + the merge order: a later input replaces an earlier one's keys, + which is how a setting is narrowed to part of the fabric. + items: + type: object + properties: + kind: + type: string + enum: + - NodeSet + - NetworkServiceSet + - ConnectedEndpointSet + - SettingSet + - Secret + name: + type: string + minLength: 1 + namespace: + type: string + description: >- + Namespace holding the object. Defaults to the Fabric's + own namespace. + required: + - kind + - name + nodeIdPool: + type: object + description: >- + Where this fabric's node IDs come from when the design sets + fabric_numbering.node_id.algorithm to pool_manager. AVD then + hands out IDs instead of reading them off each node, and + keeps the assignments in a file. There is no such file in a + cluster, so the Fabric composes a ConfigMap and reads it back + on the next reconcile. Nothing here is needed for a fabric + that numbers its nodes itself. + properties: + seedConfigMapName: + type: string + description: >- + A ConfigMap holding assignments to start from, read once + when this fabric has no pool of its own yet. It exists + for a fabric that was already running elsewhere: AVD + generated its IDs and they are in a file, and a fabric + that starts over assigns different ones - which reaches + every device as a full configuration replacement. Not + needed for a new fabric, and ignored once the composed + pool exists, so it seeds rather than overrides. + minLength: 1 + seedKey: + type: string + description: >- + Key within that ConfigMap. Defaults to the key the + composed pool uses. + minLength: 1 push: type: object description: >- @@ -86,9 +155,14 @@ spec: description: cEOS serves a self-signed cert; default true. required: - credentialsSecretName + # `design` is NOT required: a fabric may be assembled entirely from + # the inputs named in `spec.requires`, which is what `avd-migrate` + # emits and what `fn.py` has always accepted ("design or requires"). + # Requiring it here made every migrated Fabric unapplyable, and no + # offline test could see it -- they drive RunFunction directly and + # never meet the API server's schema. required: - fabricName - - design status: type: object properties: diff --git a/apis/networkserviceset/composition.yaml b/apis/networkserviceset/composition.yaml new file mode 100644 index 0000000..b6b4b1a --- /dev/null +++ b/apis/networkserviceset/composition.yaml @@ -0,0 +1,16 @@ +# Composition for NetworkServiceSet: validate this fragment and report on its own status. +# Uses the same function image as Fabric and Device, which dispatches on the +# composite kind. +apiVersion: apiextensions.crossplane.io/v1 +kind: Composition +metadata: + name: networkserviceset-avd +spec: + compositeTypeRef: + apiVersion: avd.netclab.dev/v1alpha1 + kind: NetworkServiceSet + mode: Pipeline + pipeline: + - step: validate-input + functionRef: + name: netclab-function-avd diff --git a/apis/networkserviceset/xrd.yaml b/apis/networkserviceset/xrd.yaml new file mode 100644 index 0000000..274a587 --- /dev/null +++ b/apis/networkserviceset/xrd.yaml @@ -0,0 +1,104 @@ +# CompositeResourceDefinition for a set of a fabric's network services. +# +# Carries the tenants -- `network_services_keys.name`, `tenants` by default -- +# with their VRFs, SVIs and L2 VLANs. Its own kind because network services are +# owned by whoever runs the services, not by whoever owns the spines, and RBAC +# is granted per kind. +# +# It normally applies to every device: AVD decides per node which services land +# there, through `filter.tenants` and `filter.tags` on the node, so this input +# does not have to be scoped by hand. +apiVersion: apiextensions.crossplane.io/v2 +kind: CompositeResourceDefinition +metadata: + name: networkservicesets.avd.netclab.dev +spec: + scope: Namespaced + group: avd.netclab.dev + names: + kind: NetworkServiceSet + plural: networkservicesets + categories: + - crossplane + - netclab + defaultCompositionRef: + name: networkserviceset-avd + versions: + - name: v1alpha1 + served: true + referenceable: true + schema: + openAPIV3Schema: + type: object + properties: + spec: + type: object + properties: + design: + type: object + description: >- + The tenants this input contributes. Structurally open: + the top-level key names come from network_services_keys, so + they are decided by the document's own content and no OpenAPI + schema can describe them. + x-kubernetes-preserve-unknown-fields: true + appliesTo: + type: object + description: >- + Devices that see this input. Defaults to every device in the + fabric. + properties: + all: + type: boolean + nodeSets: + type: array + description: Devices declared by the named NodeSets. + items: + type: string + hosts: + type: array + description: Devices named directly. + items: + type: string + matchHostnames: + type: array + description: >- + Regular expressions matched against device names. Same + semantics as AVD's own default_node_types, which matches + hostnames this way: the pattern is anchored for you, so + `dc1-leaf.*` matches the whole name. A pattern matching + no device is an error, not an empty set -- a pattern is + silent about both matching nothing and matching too much, + and the render is pushed as a full config replacement. + items: + type: string + required: + - design + status: + type: object + properties: + keys: + type: array + description: Top-level eos_designs keys this input contributes. + items: + type: string + devices: + type: array + description: >- + Devices this input resolved to. Reported rather than left to + be inferred, because appliesTo may be a pattern and a pattern + does not say what it matched. + items: + type: string + validation: + type: object + description: >- + Result of validating this fragment. Reported here rather than + on the Fabric so the team that owns this object sees its own + error on its own object. + properties: + ok: + type: boolean + message: + type: string + x-kubernetes-preserve-unknown-fields: true diff --git a/apis/nodeset/composition.yaml b/apis/nodeset/composition.yaml new file mode 100644 index 0000000..936031d --- /dev/null +++ b/apis/nodeset/composition.yaml @@ -0,0 +1,16 @@ +# Composition for NodeSet: validate this fragment and report on its own status. +# Uses the same function image as Fabric and Device, which dispatches on the +# composite kind. +apiVersion: apiextensions.crossplane.io/v1 +kind: Composition +metadata: + name: nodeset-avd +spec: + compositeTypeRef: + apiVersion: avd.netclab.dev/v1alpha1 + kind: NodeSet + mode: Pipeline + pipeline: + - step: validate-input + functionRef: + name: netclab-function-avd diff --git a/apis/nodeset/xrd.yaml b/apis/nodeset/xrd.yaml new file mode 100644 index 0000000..ee40c07 --- /dev/null +++ b/apis/nodeset/xrd.yaml @@ -0,0 +1,131 @@ +# CompositeResourceDefinition for a set of fabric nodes. +# +# A NodeSet carries one node-type block's share of the eos_designs input -- the +# shape of a single group_vars file: `.defaults`, `.nodes[]`, +# `.node_groups[]`. One NodeSet is what `DC1_L3_LEAVES.yml` is in an AVD +# inventory, which is normally several `node_groups`, not one. +# +# It is the only kind that brings devices into existence. `spec.declares` is the +# fabric's device list, and there is no second list: a node named in a block the +# fabric does not declare is not a device. That is deliberate -- AVD inventories +# keep the inventory and the model as two lists that may disagree, and a device +# list in two places with nothing reconciling them is how `spec.push.hosts` +# went wrong. +# +# `spec.appliesTo` is a separate question from what a NodeSet declares: it says +# which devices *see* this input. The two coincide in simple topologies and +# diverge in a 5-stage CLOS, where a DC's super_spine block names four devices +# but is visible to every device of that DC. +apiVersion: apiextensions.crossplane.io/v2 +kind: CompositeResourceDefinition +metadata: + name: nodesets.avd.netclab.dev +spec: + scope: Namespaced + group: avd.netclab.dev + names: + kind: NodeSet + plural: nodesets + categories: + - crossplane + - netclab + defaultCompositionRef: + name: nodeset-avd + versions: + - name: v1alpha1 + served: true + referenceable: true + schema: + openAPIV3Schema: + type: object + properties: + spec: + type: object + properties: + design: + type: object + description: >- + This input's share of the eos_designs document, in the shape + of a group_vars file. Structurally open: the top-level key + names come from node_type_keys, so they are decided by the + document's own content and no OpenAPI schema can describe + them. Validated by pyavd; violations reported on status. + x-kubernetes-preserve-unknown-fields: true + declares: + type: array + description: >- + Devices this NodeSet brings into the fabric. Defaults to the + devices its node-type blocks name. Set it explicitly to + declare devices no block mentions -- their node type then + comes from default_node_types -- or to exclude a node the + blocks name but the fabric does not contain. + items: + type: string + appliesTo: + type: object + description: >- + Devices that see this input. Defaults to the devices this + NodeSet declares, which is the common case -- a node-type + block is read by its own group, the way a group_vars file + is. Set it to widen that: in a 5-stage CLOS a DC's + super_spine block declares four devices and is seen by every + device of that DC. Every other kind defaults to the whole + fabric instead, having nothing of its own to be scoped to. + properties: + all: + type: boolean + nodeSets: + type: array + description: Devices declared by the named NodeSets. + items: + type: string + hosts: + type: array + description: Devices named directly. + items: + type: string + matchHostnames: + type: array + description: >- + Regular expressions matched against device names. Same + semantics as AVD's own default_node_types, which matches + hostnames this way: the pattern is anchored for you, so + `dc1-leaf.*` matches the whole name. A pattern matching + no device is an error, not an empty set -- a pattern is + silent about both matching nothing and matching too much, + and the render is pushed as a full config replacement. + items: + type: string + required: + - design + status: + type: object + properties: + keys: + type: array + description: Top-level eos_designs keys this input contributes. + items: + type: string + devices: + type: array + description: >- + Devices this input resolved to. Reported rather than left to + be inferred, because appliesTo may be a pattern and a pattern + does not say what it matched. + items: + type: string + deviceCount: + type: integer + description: Devices this input declares. + validation: + type: object + description: >- + Result of validating this fragment. Reported here rather than + on the Fabric so the team that owns this object sees its own + error on its own object. + properties: + ok: + type: boolean + message: + type: string + x-kubernetes-preserve-unknown-fields: true diff --git a/apis/settingset/composition.yaml b/apis/settingset/composition.yaml new file mode 100644 index 0000000..0b900c6 --- /dev/null +++ b/apis/settingset/composition.yaml @@ -0,0 +1,16 @@ +# Composition for SettingSet: validate this fragment and report on its own status. +# Uses the same function image as Fabric and Device, which dispatches on the +# composite kind. +apiVersion: apiextensions.crossplane.io/v1 +kind: Composition +metadata: + name: settingset-avd +spec: + compositeTypeRef: + apiVersion: avd.netclab.dev/v1alpha1 + kind: SettingSet + mode: Pipeline + pipeline: + - step: validate-input + functionRef: + name: netclab-function-avd diff --git a/apis/settingset/xrd.yaml b/apis/settingset/xrd.yaml new file mode 100644 index 0000000..ded6ac7 --- /dev/null +++ b/apis/settingset/xrd.yaml @@ -0,0 +1,103 @@ +# CompositeResourceDefinition for a set of settings at any scope. +# +# Everything that is not node-scoped and not a service: routing protocol choices, +# `bgp_peer_groups`, `default_interfaces`, `aaa_settings`, `dns_settings`, +# `ntp_settings`, `management_eapi`. Fabric-wide settings live on the Fabric +# itself; this kind carries the same keys narrowed to part of the fabric, which +# is what an AVD inventory expresses by putting them in a DC or role group. +# +# The key categories across the input kinds are a convention, not a partition the +# schema could enforce -- eos_designs' top-level key names come from its own +# content. What the kinds separate is ownership. +apiVersion: apiextensions.crossplane.io/v2 +kind: CompositeResourceDefinition +metadata: + name: settingsets.avd.netclab.dev +spec: + scope: Namespaced + group: avd.netclab.dev + names: + kind: SettingSet + plural: settingsets + categories: + - crossplane + - netclab + defaultCompositionRef: + name: settingset-avd + versions: + - name: v1alpha1 + served: true + referenceable: true + schema: + openAPIV3Schema: + type: object + properties: + spec: + type: object + properties: + design: + type: object + description: >- + The settings this input contributes, in the shape of a + group_vars file. Structurally open, like every input's design. + x-kubernetes-preserve-unknown-fields: true + appliesTo: + type: object + description: >- + Devices that see this input. Defaults to every device in the + fabric. + properties: + all: + type: boolean + nodeSets: + type: array + description: Devices declared by the named NodeSets. + items: + type: string + hosts: + type: array + description: Devices named directly. + items: + type: string + matchHostnames: + type: array + description: >- + Regular expressions matched against device names. Same + semantics as AVD's own default_node_types, which matches + hostnames this way: the pattern is anchored for you, so + `dc1-leaf.*` matches the whole name. A pattern matching + no device is an error, not an empty set -- a pattern is + silent about both matching nothing and matching too much, + and the render is pushed as a full config replacement. + items: + type: string + required: + - design + status: + type: object + properties: + keys: + type: array + description: Top-level eos_designs keys this input contributes. + items: + type: string + devices: + type: array + description: >- + Devices this input resolved to. Reported rather than left to + be inferred, because appliesTo may be a pattern and a pattern + does not say what it matched. + items: + type: string + validation: + type: object + description: >- + Result of validating this fragment. Reported here rather than + on the Fabric so the team that owns this object sees its own + error on its own object. + properties: + ok: + type: boolean + message: + type: string + x-kubernetes-preserve-unknown-fields: true diff --git a/examples/fabric/inputs/campus-fabric.yaml b/examples/fabric/inputs/campus-fabric.yaml new file mode 100644 index 0000000..8d93549 --- /dev/null +++ b/examples/fabric/inputs/campus-fabric.yaml @@ -0,0 +1,526 @@ +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc1 +spec: + design: + aaa_settings: + authorization: + exec: + default: local + local_users: + - name: admin + no_password: true + privilege: 15 + role: network-admin + - name: arista + privilege: 15 + role: network-admin + sha512_password: $6$Enl0WfE32FthwyiJ$yTyGaEJ2uPKLU.F7314YtB7J1jrzrMi7ogXIRTEHQfLdLgKWWmr1UvNlZLN6AyuxET7G5aH3AI9OYRzxVTkB1. + dns_settings: + servers: + - ip_address: 8.8.4.4 + - ip_address: 8.8.8.8 + mgmt_gateway: 172.16.100.1 + mgmt_interface_settings: + interface: Management0 + ntp_settings: + server_vrf: use_mgmt_interface_vrf + servers: + - name: time.google.com + - name: pool.ntp.org + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-fabric +spec: + design: + l2leaf: + defaults: + inband_mgmt_subnet: 10.10.10.0/24 + inband_mgmt_vlan: 10 + mlag_peer_ipv4_pool: 192.168.0.0/24 + platform: cEOSLab + spanning_tree_mode: mstp + spanning_tree_priority: 16384 + node_groups: + - filter: + tags: + - '110' + - '120' + - '130' + group: IDF1 + mlag: true + mlag_interfaces: + - Ethernet53 + - Ethernet54 + nodes: + - id: 3 + mgmt_ip: 172.16.100.103/24 + name: LEAF1A + uplink_switch_interfaces: + - Ethernet1 + uplink_switches: + - SPINE1 + - id: 4 + mgmt_ip: 172.16.100.104/24 + name: LEAF1B + uplink_switch_interfaces: + - Ethernet1 + uplink_switches: + - SPINE2 + uplink_interfaces: + - Ethernet51 + - filter: + tags: + - '210' + - '220' + - '230' + group: IDF2 + mlag: false + nodes: + - id: 5 + mgmt_ip: 172.16.100.105/24 + name: LEAF2A + uplink_interfaces: + - Ethernet1/1 + - Ethernet1/3 + uplink_switch_interfaces: + - Ethernet49/1 + - Ethernet49/1 + uplink_switches: + - SPINE1 + - SPINE2 + platform: 720XP + - filter: + tags: + - '310' + - '320' + - '330' + group: IDF3_AGG + mlag: true + mlag_interfaces: + - Ethernet98/3 + - Ethernet98/4 + nodes: + - id: 6 + mgmt_ip: 172.16.100.106/24 + name: LEAF3A + uplink_switch_interfaces: + - Ethernet50/1 + - Ethernet50/1 + - id: 7 + mgmt_ip: 172.16.100.107/24 + name: LEAF3B + uplink_switch_interfaces: + - Ethernet51/1 + - Ethernet51/1 + uplink_interfaces: + - Ethernet97/1 + - Ethernet97/2 + uplink_switches: + - SPINE1 + - SPINE2 + - group: IDF3_3C + mlag: false + nodes: + - id: 8 + mgmt_ip: 172.16.100.108/24 + name: LEAF3C + uplink_switch_interfaces: + - Ethernet97/3 + - Ethernet97/3 + uplink_interfaces: + - Ethernet97/1 + - Ethernet97/2 + uplink_switches: + - LEAF3A + - LEAF3B + - group: IDF3_3D + mlag: false + nodes: + - id: 9 + mgmt_ip: 172.16.100.109/24 + name: LEAF3D + uplink_switch_interfaces: + - Ethernet97/4 + - Ethernet97/4 + uplink_interfaces: + - Ethernet97/1 + - Ethernet97/2 + uplink_switches: + - LEAF3A + - LEAF3B + - group: IDF3_3E + mlag: false + nodes: + - id: 10 + mgmt_ip: 172.16.100.110/24 + name: LEAF3E + uplink_switch_interfaces: + - Ethernet98/1 + - Ethernet98/1 + uplink_interfaces: + - Ethernet97/1 + - Ethernet97/2 + uplink_switches: + - LEAF3A + - LEAF3B + l3spine: + defaults: + loopback_ipv4_pool: 172.16.1.0/24 + mlag_interfaces: + - Ethernet55/1 + - Ethernet56/1 + mlag_peer_ipv4_pool: 192.168.0.0/24 + mlag_peer_l3_ipv4_pool: 10.1.1.0/24 + platform: cEOSLab + spanning_tree_mode: mstp + spanning_tree_priority: 4096 + virtual_router_mac_address: 00:1c:73:00:dc:01 + node_groups: + - group: SPINES + nodes: + - id: 1 + mgmt_ip: 172.16.100.101/24 + name: SPINE1 + - id: 2 + mgmt_ip: 172.16.100.102/24 + name: SPINE2 + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc1-fabric-settings +spec: + design: + core_interfaces: + p2p_links: + - include_in_underlay_protocol: true + interfaces: + - Ethernet52/1 + - Ethernet1/1 + ip: + - 10.0.0.3/31 + - 10.0.0.2/31 + nodes: + - SPINE1 + - WAN + - include_in_underlay_protocol: true + interfaces: + - Ethernet52/1 + - Ethernet1/1 + ip: + - 10.0.0.5/31 + - 10.0.0.4/31 + nodes: + - SPINE2 + - WAN + fabric_name: DC1_FABRIC + management_eapi: + enabled: true + underlay_routing_protocol: ospf + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: ConnectedEndpointSet +metadata: + name: dc1-network-ports +spec: + design: + network_ports: + - description: IDF1 Standard Port + dot1x: + authentication_failure: + action: allow + allow_vlan: 130 + mode: trunk phone + native_vlan: 110 + phone_trunk_mode: untagged + phone_vlan: 120 + profile: PP-DOT1X + switch_ports: + - Ethernet1-48 + switches: + - LEAF1[AB] + - description: IDF2 Standard Port + dot1x: + authentication_failure: + action: allow + allow_vlan: 230 + mode: trunk phone + native_vlan: 210 + phone_trunk_mode: untagged + phone_vlan: 220 + profile: PP-DOT1X + switch_ports: + - Ethernet3-7/1-48 + switches: + - LEAF2A + - description: IDF3 Standard Port + dot1x: + authentication_failure: + action: allow + allow_vlan: 330 + mode: trunk phone + native_vlan: 310 + phone_trunk_mode: untagged + phone_vlan: 320 + profile: PP-DOT1X + switch_ports: + - Ethernet1-96 + switches: + - LEAF3[ABCDE] + port_profiles: + - dot1x: + host_mode: + mode: multi-host + multi_host_authenticated: true + mac_based_authentication: + enabled: true + pae: + mode: authenticator + port_control: auto + reauthentication: true + reauthorization_request_limit: 3 + timeout: + reauth_period: server + tx_period: 3 + mode: trunk phone + poe: + limit: + class: 4 + link_down: + action: maintain + priority: critical + reboot: + action: maintain + shutdown: + action: power-off + profile: PP-DOT1X + spanning_tree_bpduguard: enabled + spanning_tree_portfast: edge + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NetworkServiceSet +metadata: + name: dc1-network-services +spec: + design: + tenants: + - name: MY_FABRIC + vrfs: + - name: default + svis: + - enabled: true + id: 110 + ip_virtual_router_addresses: + - 10.1.10.1 + name: IDF1-Data + nodes: + - ip_address: 10.1.10.2/23 + node: SPINE1 + - ip_address: 10.1.10.3/23 + node: SPINE2 + tags: + - '110' + - enabled: true + id: 120 + ip_virtual_router_addresses: + - 10.1.20.1 + name: IDF1-Voice + nodes: + - ip_address: 10.1.20.2/23 + node: SPINE1 + - ip_address: 10.1.20.3/23 + node: SPINE2 + tags: + - '120' + - enabled: true + id: 130 + ip_virtual_router_addresses: + - 10.1.30.1 + name: IDF1-Guest + nodes: + - ip_address: 10.1.30.2/23 + node: SPINE1 + - ip_address: 10.1.30.3/23 + node: SPINE2 + tags: + - '130' + - enabled: true + id: 210 + ip_virtual_router_addresses: + - 10.2.10.1 + name: IDF2-Data + nodes: + - ip_address: 10.2.10.2/23 + node: SPINE1 + - ip_address: 10.2.10.3/23 + node: SPINE2 + tags: + - '210' + - enabled: true + id: 220 + ip_virtual_router_addresses: + - 10.2.20.1 + name: IDF2-Voice + nodes: + - ip_address: 10.2.20.2/23 + node: SPINE1 + - ip_address: 10.2.20.3/23 + node: SPINE2 + tags: + - '220' + - enabled: true + id: 230 + ip_virtual_router_addresses: + - 10.2.30.1 + name: IDF2-Guest + nodes: + - ip_address: 10.2.30.2/23 + node: SPINE1 + - ip_address: 10.2.30.3/23 + node: SPINE2 + tags: + - '230' + - enabled: true + id: 310 + ip_virtual_router_addresses: + - 10.3.10.1 + name: IDF3-Data + nodes: + - ip_address: 10.3.10.2/23 + node: SPINE1 + - ip_address: 10.3.10.3/23 + node: SPINE2 + tags: + - '310' + - enabled: true + id: 320 + ip_virtual_router_addresses: + - 10.3.20.1 + name: IDF3-Voice + nodes: + - ip_address: 10.3.20.2/23 + node: SPINE1 + - ip_address: 10.3.20.3/23 + node: SPINE2 + tags: + - '320' + - enabled: true + id: 330 + ip_virtual_router_addresses: + - 10.3.30.1 + name: IDF3-Guest + nodes: + - ip_address: 10.3.30.2/23 + node: SPINE1 + - ip_address: 10.3.30.3/23 + node: SPINE2 + tags: + - '330' + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-leafs +spec: + design: + type: l2leaf + declares: + - LEAF1A + - LEAF1B + - LEAF2A + - LEAF3A + - LEAF3B + - LEAF3C + - LEAF3D + - LEAF3E +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc1-leafs-settings +spec: + design: + aaa_settings: + authorization: + exec: + default: local + local_users: + - name: admin + no_password: true + privilege: 15 + role: network-admin + - name: arista + privilege: 15 + role: network-admin + sha512_password: $6$Enl0WfE32FthwyiJ$yTyGaEJ2uPKLU.F7314YtB7J1jrzrMi7ogXIRTEHQfLdLgKWWmr1UvNlZLN6AyuxET7G5aH3AI9OYRzxVTkB1. + radius: + servers: + - cleartext_key: super-secret-key + host: agni.arista.com + dot1x_settings: + enabled: true + appliesTo: + nodeSets: + - dc1-leafs +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-spines +spec: + design: + type: l3spine + declares: + - SPINE1 + - SPINE2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc1-spines-settings +spec: + design: + underlay_ospf_graceful_restart: false + appliesTo: + nodeSets: + - dc1-spines +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: Fabric +metadata: + name: campus-fabric +spec: + fabricName: DC1_FABRIC + requires: + - kind: SettingSet + name: dc1 + - kind: NodeSet + name: dc1-fabric + - kind: SettingSet + name: dc1-fabric-settings + - kind: ConnectedEndpointSet + name: dc1-network-ports + - kind: NetworkServiceSet + name: dc1-network-services + - kind: NodeSet + name: dc1-leafs + - kind: SettingSet + name: dc1-leafs-settings + - kind: NodeSet + name: dc1-spines + - kind: SettingSet + name: dc1-spines-settings diff --git a/examples/fabric/inputs/dual-dc-l3ls.yaml b/examples/fabric/inputs/dual-dc-l3ls.yaml new file mode 100644 index 0000000..8f7e39b --- /dev/null +++ b/examples/fabric/inputs/dual-dc-l3ls.yaml @@ -0,0 +1,540 @@ +apiVersion: avd.netclab.dev/v1alpha1 +kind: ConnectedEndpointSet +metadata: + name: connected-endpoints +spec: + design: + servers: + - adapters: + - endpoint_ports: + - PCI1 + - PCI2 + mode: trunk + native_vlan: 4092 + port_channel: + description: PortChannel dc1-leaf1-server1 + mode: active + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + - Ethernet5 + switches: + - dc1-leaf1a + - dc1-leaf1b + vlans: 11-12,21-22 + - endpoint_ports: + - iLO + mode: access + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + switches: + - dc1-leaf1c + vlans: 11 + name: dc1-leaf1-server1 + - adapters: + - endpoint_ports: + - PCI1 + - PCI2 + mode: trunk + native_vlan: 4092 + port_channel: + mode: active + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + - Ethernet5 + switches: + - dc1-leaf2a + - dc1-leaf2b + vlans: 11-12,21-22 + - endpoint_ports: + - iLO + mode: access + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + switches: + - dc1-leaf2c + vlans: 11 + name: dc1-leaf2-server1 + - adapters: + - endpoint_ports: + - PCI1 + - PCI2 + mode: trunk + native_vlan: 4092 + port_channel: + endpoint_port_channel: Bond1 + mode: active + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + - Ethernet5 + switches: + - dc2-leaf1a + - dc2-leaf1b + vlans: 11-12,21-22 + - endpoint_ports: + - iLO + mode: access + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + switches: + - dc2-leaf1c + vlans: 11 + name: dc2-leaf1-server1 + - adapters: + - endpoint_ports: + - PCI1 + - PCI2 + mode: trunk + native_vlan: 4092 + port_channel: + mode: active + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + - Ethernet5 + switches: + - dc2-leaf2a + - dc2-leaf2b + vlans: 11-12,21-22 + - endpoint_ports: + - iLO + mode: access + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + switches: + - dc2-leaf2c + vlans: 11 + name: dc2-leaf2-server1 + appliesTo: + nodeSets: + - dc1-l2-leafs + - dc1-l3-leafs + - dc2-l2-leafs + - dc2-l3-leafs +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: fabric +spec: + design: + aaa_settings: + local_users: + - name: admin + no_password: true + privilege: 15 + role: network-admin + - name: arista + privilege: 15 + role: network-admin + sha512_password: $6$Enl0WfE32FthwyiJ$yTyGaEJ2uPKLU.F7314YtB7J1jrzrMi7ogXIRTEHQfLdLgKWWmr1UvNlZLN6AyuxET7G5aH3AI9OYRzxVTkB1. + bgp_peer_groups: + evpn_overlay_peers: + password: Q4fqtbqcZ7oQuKfuWtNGRQ== + ipv4_underlay_peers: + password: 7x4B4rnJhZB438m9+BrBfQ== + mlag_ipv4_underlay_peer: + password: 4b21pAdCvWeAqpcKDFMdWw== + default_interfaces: + - downlink_interfaces: + - Ethernet1-8 + platforms: + - default + types: + - spine + uplink_interfaces: + - Ethernet1-2 + - downlink_interfaces: + - Ethernet8 + mlag_interfaces: + - Ethernet3-4 + platforms: + - default + types: + - l3leaf + uplink_interfaces: + - Ethernet1-8 + - platforms: + - default + types: + - l2leaf + uplink_interfaces: + - Ethernet1-2 + eos_designs_documentation: + p2p_links_csv: true + topology_csv: true + fabric_name: FABRIC + l3_edge: + p2p_links: + - id: 1 + interfaces: + - Ethernet6 + - Ethernet6 + nodes: + - dc1-leaf2a + - dc2-leaf2a + profile: DCI_profile + - id: 2 + interfaces: + - Ethernet6 + - Ethernet6 + nodes: + - dc1-leaf2b + - dc2-leaf2b + profile: DCI_profile + p2p_links_ip_pools: + - ipv4_pool: 172.16.100.0/24 + name: DCI_IP_pool + p2p_links_profiles: + - as: + - 65102 + - 65202 + include_in_underlay_protocol: true + ip_pool: DCI_IP_pool + name: DCI_profile + management_eapi: + enabled: true + overlay_routing_protocol: ebgp + underlay_routing_protocol: ebgp + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NetworkServiceSet +metadata: + name: network-services +spec: + design: + tenants: + - l2vlans: + - id: 3401 + name: L2_VLAN3401 + - id: 3402 + name: L2_VLAN3402 + mac_vrf_vni_base: 10000 + name: TENANT1 + vrfs: + - name: VRF10 + svis: + - enabled: true + id: 11 + ip_address_virtual: 10.10.11.1/24 + name: VRF10_VLAN11 + - enabled: true + id: 12 + ip_address_virtual: 10.10.12.1/24 + name: VRF10_VLAN12 + vrf_vni: 10 + vtep_diagnostic: + loopback: 10 + loopback_ip_range: 10.255.10.0/27 + - name: VRF11 + svis: + - enabled: true + id: 21 + ip_address_virtual: 10.10.21.1/24 + name: VRF11_VLAN21 + - enabled: true + id: 22 + ip_address_virtual: 10.10.22.1/24 + name: VRF11_VLAN22 + vrf_vni: 11 + vtep_diagnostic: + loopback: 11 + loopback_ip_range: 10.255.11.0/27 + appliesTo: + nodeSets: + - dc1-l2-leafs + - dc1-l3-leafs + - dc2-l2-leafs + - dc2-l3-leafs +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc1 +spec: + design: + mgmt_gateway: 172.16.1.1 + appliesTo: + nodeSets: + - dc1-l2-leafs + - dc1-l3-leafs + - dc1-spines +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc2 +spec: + design: + mgmt_gateway: 172.16.1.1 + appliesTo: + nodeSets: + - dc2-l2-leafs + - dc2-l3-leafs + - dc2-spines +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-l2-leafs +spec: + design: + l2leaf: + defaults: + platform: cEOSLab + spanning_tree_mode: mstp + node_groups: + - group: DC1_L2_LEAF1 + nodes: + - id: 1 + mgmt_ip: 172.16.1.151/24 + name: dc1-leaf1c + uplink_switches: + - dc1-leaf1a + - dc1-leaf1b + - group: DC1_L2_LEAF2 + nodes: + - id: 1 + mgmt_ip: 172.16.1.152/24 + name: dc1-leaf2c + uplink_switches: + - dc1-leaf2a + - dc1-leaf2b + type: l2leaf + declares: + - dc1-leaf1c + - dc1-leaf2c +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-l3-leafs +spec: + design: + l3leaf: + defaults: + loopback_ipv4_offset: 2 + loopback_ipv4_pool: 10.255.0.0/27 + mlag_peer_ipv4_pool: 10.255.1.64/27 + mlag_peer_l3_ipv4_pool: 10.255.1.96/27 + platform: cEOSLab + spanning_tree_mode: mstp + spanning_tree_priority: 4096 + uplink_ipv4_pool: 10.255.255.0/26 + uplink_switches: + - dc1-spine1 + - dc1-spine2 + virtual_router_mac_address: 00:1c:73:00:00:99 + vtep_loopback_ipv4_pool: 10.255.1.0/27 + node_groups: + - bgp_as: 65101 + group: DC1_L3_LEAF1 + nodes: + - id: 1 + mgmt_ip: 172.16.1.101/24 + name: dc1-leaf1a + - id: 2 + mgmt_ip: 172.16.1.102/24 + name: dc1-leaf1b + - bgp_as: 65102 + evpn_gateway: + evpn_l2: + enabled: true + evpn_l3: + enabled: true + inter_domain: true + group: DC1_L3_LEAF2 + nodes: + - evpn_gateway: + remote_peers: + - hostname: dc2-leaf2a + id: 3 + mgmt_ip: 172.16.1.103/24 + name: dc1-leaf2a + - evpn_gateway: + remote_peers: + - hostname: dc2-leaf2b + id: 4 + mgmt_ip: 172.16.1.104/24 + name: dc1-leaf2b + type: l3leaf + declares: + - dc1-leaf1a + - dc1-leaf1b + - dc1-leaf2a + - dc1-leaf2b +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-spines +spec: + design: + spine: + defaults: + bgp_as: 65100 + loopback_ipv4_pool: 10.255.0.0/27 + platform: cEOSLab + nodes: + - id: 1 + mgmt_ip: 172.16.1.11/24 + name: dc1-spine1 + - id: 2 + mgmt_ip: 172.16.1.12/24 + name: dc1-spine2 + type: spine + declares: + - dc1-spine1 + - dc1-spine2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc2-l2-leafs +spec: + design: + l2leaf: + defaults: + platform: cEOSLab + spanning_tree_mode: mstp + node_groups: + - group: DC2_L2_LEAF1 + nodes: + - id: 1 + mgmt_ip: 172.16.1.161/24 + name: dc2-leaf1c + uplink_switches: + - dc2-leaf1a + - dc2-leaf1b + - group: DC2_L2_LEAF2 + nodes: + - id: 1 + mgmt_ip: 172.16.1.162/24 + name: dc2-leaf2c + uplink_switches: + - dc2-leaf2a + - dc2-leaf2b + type: l2leaf + declares: + - dc2-leaf1c + - dc2-leaf2c +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc2-l3-leafs +spec: + design: + l3leaf: + defaults: + loopback_ipv4_offset: 2 + loopback_ipv4_pool: 10.255.128.0/27 + mlag_peer_ipv4_pool: 10.255.129.64/27 + mlag_peer_l3_ipv4_pool: 10.255.129.96/27 + platform: cEOSLab + spanning_tree_mode: mstp + spanning_tree_priority: 4096 + uplink_ipv4_pool: 10.255.255.64/26 + uplink_switches: + - dc2-spine1 + - dc2-spine2 + virtual_router_mac_address: 00:1c:73:00:00:99 + vtep_loopback_ipv4_pool: 10.255.129.0/27 + node_groups: + - bgp_as: 65201 + group: DC2_L3_LEAF1 + nodes: + - id: 1 + mgmt_ip: 172.16.1.111/24 + name: dc2-leaf1a + - id: 2 + mgmt_ip: 172.16.1.112/24 + name: dc2-leaf1b + - bgp_as: 65202 + evpn_gateway: + evpn_l2: + enabled: true + evpn_l3: + enabled: true + inter_domain: true + group: DC2_L3_LEAF2 + nodes: + - evpn_gateway: + remote_peers: + - hostname: dc1-leaf2a + id: 3 + mgmt_ip: 172.16.1.113/24 + name: dc2-leaf2a + - evpn_gateway: + remote_peers: + - hostname: dc1-leaf2b + id: 4 + mgmt_ip: 172.16.1.114/24 + name: dc2-leaf2b + type: l3leaf + declares: + - dc2-leaf1a + - dc2-leaf1b + - dc2-leaf2a + - dc2-leaf2b +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc2-spines +spec: + design: + spine: + defaults: + bgp_as: 65200 + loopback_ipv4_pool: 10.255.128.0/27 + platform: cEOSLab + nodes: + - id: 1 + mgmt_ip: 172.16.1.21/24 + name: dc2-spine1 + - id: 2 + mgmt_ip: 172.16.1.22/24 + name: dc2-spine2 + type: spine + declares: + - dc2-spine1 + - dc2-spine2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: Fabric +metadata: + name: dual-dc-l3ls +spec: + fabricName: FABRIC + requires: + - kind: ConnectedEndpointSet + name: connected-endpoints + - kind: SettingSet + name: fabric + - kind: NetworkServiceSet + name: network-services + - kind: SettingSet + name: dc1 + - kind: SettingSet + name: dc2 + - kind: NodeSet + name: dc1-l2-leafs + - kind: NodeSet + name: dc1-l3-leafs + - kind: NodeSet + name: dc1-spines + - kind: NodeSet + name: dc2-l2-leafs + - kind: NodeSet + name: dc2-l3-leafs + - kind: NodeSet + name: dc2-spines diff --git a/examples/fabric/inputs/eos-designs-twodc-5stage-clos-twodc-5stage-clos.yaml b/examples/fabric/inputs/eos-designs-twodc-5stage-clos-twodc-5stage-clos.yaml new file mode 100644 index 0000000..a95637b --- /dev/null +++ b/examples/fabric/inputs/eos-designs-twodc-5stage-clos-twodc-5stage-clos.yaml @@ -0,0 +1,1859 @@ +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: all +spec: + design: + root_dir: /home/student/function-avd/avd/ansible_collections/arista/avd/extensions/molecule/eos_designs-twodc-5stage-clos + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: ConnectedEndpointSet +metadata: + name: servers-endpoints +spec: + design: + port_profiles: + - parent_profile: TENANT_A + port_channel: + raw_eos_cli: 'comment + + Comment created from raw_eos_cli under port_channel on profile NESTED_TENANT_A + + EOF + + ' + profile: NESTED_TENANT_A + raw_eos_cli: 'comment + + Comment created from raw_eos_cli under profile NESTED_TENANT_A + + EOF + + ' + - mode: access + profile: TENANT_A + qos_profile: foo + raw_eos_cli: 'comment + + Comment created from raw_eos_cli under profile TENANT_A + + EOF + + ' + vlans: '110' + servers: + - adapters: + - endpoint_ports: + - Eth1 + - Eth2 + port_channel: + description: PortChannel + enabled: true + mode: active + profile: TENANT_A + qos_profile: bar + switch_ports: + - Ethernet16 + - Ethernet16 + switches: + - DC1.POD1.LEAF2A + - DC1-POD1-LEAF2B + - endpoint_ports: + - Eth3 + - Eth4 + port_channel: + description: PortChannel + enabled: true + mode: active + structured_config: + description: Set using structured_config on server adapter port-channel + profile: TENANT_A + raw_eos_cli: 'comment + + Comment created from raw_eos_cli under adapter for switch Eth17 + + EOF + + ' + structured_config: + description: Set using structured_config on server adapter + switch_ports: + - Ethernet17 + - Ethernet17 + switches: + - DC1.POD1.LEAF2A + - DC1-POD1-LEAF2B + - endpoint_ports: + - Eth5 + - Eth6 + port_channel: + description: PortChannel + enabled: true + mode: active + profile: NESTED_TENANT_A + switch_ports: + - Ethernet18 + - Ethernet18 + switches: + - DC1.POD1.LEAF2A + - DC1-POD1-LEAF2B + - endpoint_ports: + - Eth7 + - Eth8 + port_channel: + description: PortChannel + enabled: true + mode: active + raw_eos_cli: 'comment + + Comment created from raw_eos_cli under adapter port_channel for switch + Po19 + + EOF + + ' + profile: NESTED_TENANT_A + switch_ports: + - Ethernet19 + - Ethernet19 + switches: + - DC1.POD1.LEAF2A + - DC1-POD1-LEAF2B + - endpoint_ports: + - Eth9 + poe: + reboot: + action: maintain + switch_ports: + - Ethernet2 + switches: + - DC1-POD1-L2LEAF1A + name: server-1 + rack: RackB + - adapters: + - endpoint_ports: + - Eth1 + - Eth2 + profile: TENANT_A + switch_ports: + - Ethernet26 + - Ethernet26 + switches: + - DC1.POD1.LEAF2A + - DC1-POD1-LEAF2B + name: server-2 + type: SpecialServer + appliesTo: + nodeSets: + - dc1-pod1-l2leaf1a + - dc1-pod1-l2leaf2a + - dc1-pod1-l2leaf2b + - dc1-pod1-leaf1a + - dc1-pod1-leaf1b + - dc1-pod1-leaf2a + - dc1-pod1-leaf2b + - dc1-pod2-leaf1a + - dc2-pod1-l2leaf1a + - dc2-pod1-l2leaf2a + - dc2-pod1-leaf1a + - dc2-pod1-leaf2a +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: servers-settings +spec: + design: + connected_endpoints: + - adapters: + - endpoint_ports: + - Eth1 + - Eth2 + profile: TENANT_A + switch_ports: + - Ethernet27 + - Ethernet27 + switches: + - DC1.POD1.LEAF2A + - DC1-POD1-LEAF2B + name: server-3 + - adapters: + - endpoint_ports: + - Eth5 + - Eth6 + port_channel: + description: PortChannel + enabled: true + mode: active + profile: NESTED_TENANT_A + switch_ports: + - Ethernet28 + - Ethernet28 + switches: + - DC1.POD1.LEAF2A + - DC1-POD1-LEAF2B + name: server-4 + type: SpecialEndpoint + appliesTo: + nodeSets: + - dc1-pod1-l2leaf1a + - dc1-pod1-l2leaf2a + - dc1-pod1-l2leaf2b + - dc1-pod1-leaf1a + - dc1-pod1-leaf1b + - dc1-pod1-leaf2a + - dc1-pod1-leaf2b + - dc1-pod2-leaf1a + - dc2-pod1-l2leaf1a + - dc2-pod1-l2leaf2a + - dc2-pod1-leaf1a + - dc2-pod1-leaf2a +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NetworkServiceSet +metadata: + name: tenants-networks-services +spec: + design: + tenants: + - enable_mlag_ibgp_peering_vrfs: false + l2vlans: + - id: 2500 + name: web-l2-vlan + tags: + - web + vni_override: 2500 + - bgp: + raw_eos_cli: 'comment + + comment created from raw_eos_cli under router bgp l2vlan 2600 + + EOF + + ' + structured_config: + redistribute_routes: + - router-mac system + id: 2600 + name: web-l2-vlan-2 + rt_override: 32600 + tags: + - web + - id: 2601 + name: l2vlan_with_no_vxlan + tags: + - web + vxlan: false + mac_vrf_id_base: 20000 + mac_vrf_vni_base: 10000 + name: Tenant_A + vrfs: + - bgp: + raw_eos_cli: 'comment + + Comment created from raw_eos_cli under BGP for VRF Common_VRF + + EOF + + ' + name: Common_VRF + raw_eos_cli: "interface Loopback1000\n description Loopback created from\ + \ raw_eos_cli under VRF Common_VRF\n" + structured_config: + domain_list: + - structured-config.set.under.vrf.common-vrf + svis: + - enabled: true + id: 110 + ip_address_virtual: 10.1.10.1/24 + name: Tenant_A_OP_Zone_1 + nodes: + - node: DC1.POD1.LEAF2A + structured_config: + description: set from structured_config on svi for DC1.POD1.LEAF2A (was + Tenant_A_OP_Zone_1) + profile: TEST_PROFILE_STRUCT_CFG + rt_override: 99110 + structured_config: + description: set from structured_config on svi (was Tenant_A_OP_Zone_1) + tags: + - opzone + - bgp: + raw_eos_cli: 'comment + + comment created from raw_eos_cli under router bgp svi 111 + + EOF + + ' + structured_config: + redistribute_routes: + - router-mac system + id: 111 + ip_address_virtual: 10.1.11.1/24 + name: Tenant_A_OP_Zone_2 + tags: + - opzone + vni_override: 50111 + - enabled: true + id: 112 + ip_address_virtual: 10.1.12.1/24 + name: Tenant_A_OP_Zone_3 + nodes: + - node: DC1-POD1-LEAF1A + raw_eos_cli: 'comment + + Comment created from raw_eos_cli under SVI 112 in VRF Common_VRF overridden + on DC1-POD1-LEAF1A + + EOF + + ' + profile: TEST_PARENT_PROFILE_STRUCT_CFG + raw_eos_cli: 'comment + + Comment created from raw_eos_cli under SVI 112 in VRF Common_VRF + + EOF + + ' + tags: + - opzone + - enabled: true + id: 113 + ip_address_virtual: 10.10.13.1/24 + name: SVI_with_no_vxlan + tags: + - opzone + vxlan: false + vrf_vni: 1025 + - name: vrf_with_loopbacks_from_overlapping_pool + svis: + - enabled: true + id: 1100 + ip_address_virtual: 10.100.100.1/24 + name: test_svi + tags: + - opzone + - web + vxlan: false + vrf_vni: 1100 + vtep_diagnostic: + loopback: 100 + loopback_ip_range: 10.100.0.0/24 + - name: vrf_with_loopbacks_from_pod_pools + svis: + - enabled: true + id: 1101 + ip_address_virtual: 10.101.100.1/24 + name: test_svi + tags: + - opzone + - web + vxlan: false + vrf_vni: 1101 + vtep_diagnostic: + loopback: 101 + loopback_ip_pools: + - ipv4_pool: 10.101.101.0-10.101.101.15, 10.101.101.16/28, 10.101.101.32-10.101.101.255 + ipv6_pool: 2001:db8:1::-2001:db8:1:7fff::, 2001:db8:1:8000::/49 + pod: DC1_POD1 + - ipv4_pool: 10.101.102.0/24 + pod: DC1_POD2 + - ipv4_pool: 10.101.201.0/24 + pod: DC2_POD1 + - ipv6_pool: 2001:db8:2::/48 + pod: DC2_POD2 + - name: vrf_with_loopbacks_dc1_pod1_only + svis: + - enabled: true + id: 1102 + ip_address_virtual: 10.102.100.1/24 + name: test_svi + tags: + - opzone + - web + vxlan: false + vrf_vni: 1102 + vtep_diagnostic: + loopback: 102 + loopback_ip_pools: + - ipv4_pool: 10.102.101.0/24 + pod: DC1_POD1 + appliesTo: + nodeSets: + - dc1-pod1-l2leaf1a + - dc1-pod1-l2leaf2a + - dc1-pod1-l2leaf2b + - dc1-pod1-leaf1a + - dc1-pod1-leaf1b + - dc1-pod1-leaf2a + - dc1-pod1-leaf2b + - dc1-pod2-leaf1a + - dc2-pod1-l2leaf1a + - dc2-pod1-l2leaf2a + - dc2-pod1-leaf1a + - dc2-pod1-leaf2a +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: tenants-networks-settings +spec: + design: + svi_profiles: + - bgp: + raw_eos_cli: 'comment + + comment created from raw_eos_cli under router bgp svis inherited from svi + parent profile + + EOF + + ' + structured_config: + redistribute_routes: + - router-mac system + profile: PARENT_PROFILE_STRUCT_CFG + - parent_profile: PARENT_PROFILE_STRUCT_CFG + profile: TEST_PARENT_PROFILE_STRUCT_CFG + - bgp: + raw_eos_cli: 'comment + + comment created from raw_eos_cli under router bgp svis inherited from svi + profile + + EOF + + ' + structured_config: + redistribute_routes: + - router-mac system + profile: TEST_PROFILE_STRUCT_CFG + appliesTo: + nodeSets: + - dc1-pod1-l2leaf1a + - dc1-pod1-l2leaf2a + - dc1-pod1-l2leaf2b + - dc1-pod1-leaf1a + - dc1-pod1-leaf1b + - dc1-pod1-leaf2a + - dc1-pod1-leaf2b + - dc1-pod2-leaf1a + - dc2-pod1-l2leaf1a + - dc2-pod1-l2leaf2a + - dc2-pod1-leaf1a + - dc2-pod1-leaf2a +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: twodc-5stage-clos +spec: + design: + node_type_keys: + - default_evpn_role: server + ip_addressing: + python_module: function.avd_compat + python_class_name: AvdIpAddressingV2Spine + key: spine + type: spine + - connected_endpoints: true + default_evpn_role: client + key: l3leaf + mlag_support: true + network_services: + l2: true + l3: true + type: l3leaf + vtep: true + - connected_endpoints: true + key: l2leaf + mlag_support: true + network_services: + l2: true + type: l2leaf + underlay_router: false + uplink_type: port-channel + - key: super_spine + type: super-spine + - key: overlay_controller + type: overlay-controller + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: twodc-5stage-clos-settings +spec: + design: + aaa_settings: + local_users: + - name: admin + no_password: true + privilege: 15 + role: network-admin + sha512_password: $6$eJ5TvI8oru5i9e8G$R1X/SbtGTk9xoEHEBQASc7SC2nHYmi.crVgp2pXuCXwxsXEA81e4E0cXgQ6kX08fIeQzauqhv2kS.RGJFCon5/ + custom_platform_settings: + - digital_twin: + platform: CloudEOS + feature_support: + poe: true + platforms: + - CUSTOM-PLATFORM + - digital_twin: + act_node_type: tools-server + platforms: + - TOOLS-SERVER + reload_delay: + mlag: 300 + non_mlag: 330 + custom_structured_configuration_list_merge: append + cv_topology: + - hostname: DC1-RS1 + interfaces: + - name: Ethernet1 + neighbor: DC1-SUPER-SPINE1 + neighbor_interface: Ethernet5 + - name: Ethernet2 + neighbor: DC1-POD1-SPINE1 + neighbor_interface: Ethernet6 + - name: Ethernet3 + neighbor: DC1-POD1-LEAF1A + neighbor_interface: Ethernet4 + platform: vEOS-LAB + - hostname: DC1.POD1.LEAF2A + interfaces: + - name: Ethernet1 + neighbor: DC1-POD1-SPINE1 + neighbor_interface: Ethernet4 + - name: Ethernet11 + neighbor: DC1-POD1-SPINE1 + neighbor_interface: Ethernet7 + - name: Ethernet2 + neighbor: DC1-POD1-SPINE2 + neighbor_interface: Ethernet4 + - name: Ethernet12 + neighbor: DC1-POD1-SPINE2 + neighbor_interface: Ethernet7 + - name: Ethernet5 + neighbor: DC1-POD1-LEAF2B + neighbor_interface: Ethernet5 + - name: Ethernet6 + neighbor: DC1-POD1-LEAF2B + neighbor_interface: Ethernet6 + platform: vEOS-LAB + - hostname: DC1-POD1-LEAF2B + interfaces: + - name: Ethernet1 + neighbor: DC1-POD1-SPINE1 + neighbor_interface: Ethernet5 + - name: Ethernet11 + neighbor: DC1-POD1-SPINE1 + neighbor_interface: Ethernet8 + - name: Ethernet2 + neighbor: DC1-POD1-SPINE2 + neighbor_interface: Ethernet5 + - name: Ethernet12 + neighbor: DC1-POD1-SPINE2 + neighbor_interface: Ethernet8 + - name: Ethernet5 + neighbor: DC1.POD1.LEAF2A + neighbor_interface: Ethernet5 + - name: Ethernet6 + neighbor: DC1.POD1.LEAF2A + neighbor_interface: Ethernet6 + platform: vEOS-LAB + cv_topology_levels: + - level: 1 + type: super-spine + - level: 2 + type: spine + - level: 3 + type: l3leaf + - level: 4 + type: l2leaf + - level: 5 + type: overlay-controller + eos_designs_documentation: + connected_endpoints: true + p2p_links_csv: true + topology_csv: true + evpn_ebgp_multihop: 5 + evpn_overlay_bgp_rtc: true + fabric_name: TWODC_5STAGE_CLOS + fabric_numbering: + node_id: + algorithm: pool_manager + pools_file: intended/data/test-ids.yml + l3_edge: + p2p_links: + - as: + - 65100 + - 65200 + id: 1 + interfaces: + - Ethernet6 + - Ethernet4 + ip_pool: pool-super-spine + macsec_profile: MACSEC_PROFILE + nodes: + - DC1-SUPER-SPINE1 + - DC2-SUPER-SPINE1 + profile: generic-profile + qos_profile: avd.netclab.dev/null + - as: + - 65100 + - 65200 + id: 2 + interfaces: + - Ethernet6 + - Ethernet4 + ip_pool: pool-super-spine + mtu: 1500 + nodes: + - DC1-SUPER-SPINE2 + - DC2-SUPER-SPINE2 + qos_profile: avd.netclab.dev/null + - as: + - 65120 + - 65210 + id: 10 + interfaces: + - Ethernet5 + - Ethernet5 + ip_pool: pool-spine + mtu: 1500 + nodes: + - DC1-POD2-SPINE1 + - DC2-POD1-SPINE1 + qos_profile: avd.netclab.dev/null + - as: + - 65112 + - 65210 + interfaces: + - Ethernet4 + - Ethernet5 + ip: + - 200.200.200.101/24 + - 200.200.200.201/24 + mtu: 1498 + nodes: + - DC1-POD2-SPINE2 + - DC2-POD1-SPINE2 + profile: generic-profile + qos_profile: avd.netclab.dev/null + - as: + - '65112.100' + - 65211 + interfaces: + - Ethernet7 + - Ethernet6 + ip: + - 100.100.100.101/24 + - 100.100.100.201/24 + mtu: 1500 + nodes: + - DC1.POD1.LEAF2A + - DC2-POD1-LEAF1A + qos_profile: avd.netclab.dev/null + - as: + - 65120 + - 65211 + bfd: true + id: 20 + interfaces: + - Ethernet7 + - Ethernet7 + ip_pool: pool-leaf + nodes: + - DC1-POD1-LEAF2B + - DC2-POD1-LEAF1A + profile: generic-profile + qos_profile: avd.netclab.dev/null + p2p_links_ip_pools: + - ipv4_pool: 11.1.0.0/24 + name: pool-leaf + - ipv4_pool: 11.1.1.0/24 + name: pool-spine + - ipv4_pool: 11.1.2.0/24 + name: pool-super-spine + p2p_links_profiles: + - bfd: false + mtu: 1499 + name: generic-profile + ptp: + enabled: true + mgmt_destination_networks: + - 0.0.0.0/0 + mgmt_gateway: 192.168.1.254 + p2p_uplinks_mtu: 1500 + p2p_uplinks_qos_profile: QOS-PROFILE + platform_settings: + - digital_twin: + platform: vEOS-LAB + platforms: + - default + reload_delay: + mlag: 300 + non_mlag: 330 + - digital_twin: + act_node_type: veos + feature_support: + per_interface_mtu: false + platforms: + - vEOS-LAB + raw_eos_cli: "interface Loopback1111\n description Loopback created from raw_eos_cli\ + \ under platform_settings vEOS-LAB\n" + reload_delay: + mlag: 300 + non_mlag: 330 + - digital_twin: + act_node_type: cloudeos + platforms: + - CloudEOS + snmp_settings: + location: true + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1 +spec: + design: + overlay_controller: + defaults: + bgp_defaults: + - distance bgp 20 200 200 + loopback_ipv4_pool: 172.16.10.0/24 + max_uplink_switches: 4 + uplink_bfd: true + uplink_ipv4_pool: 172.17.10.0/24 + nodes: + - bgp_as: 65101 + digital_twin: + mgmt_ip: 192.168.1.3/24 + evpn_role: server + evpn_route_servers: + - DC2-RS1 + - DC2-SUPER-SPINE1 + - DC2-POD1-SPINE1 + - DC2-POD1-LEAF1A + name: DC1-RS1 + - bgp_as: 65102 + evpn_role: server + evpn_route_servers: + - DC2-RS1 + - DC2-SUPER-SPINE1 + - DC2-POD1-SPINE1 + - DC2-POD1-LEAF1A + mgmt_ip: 192.168.1.4/24 + name: DC1-RS2 + platform: vEOS-LAB + uplink_interfaces: + - Ethernet1 + - Ethernet2 + - Ethernet3 + uplink_switch_interfaces: + - Ethernet5 + - Ethernet4 + - Ethernet3 + uplink_switches: + - DC1-SUPER-SPINE2 + - DC1-POD2-SPINE1 + - DC1-POD2-LEAF1A + super_spine: + defaults: + bgp_as: 65100 + bgp_defaults: + - distance bgp 20 200 200 + loopback_ipv4_pool: 172.16.100.0/24 + platform: vEOS-LAB + nodes: + - digital_twin: + mgmt_ip: 192.168.1.1/24 + name: DC1-SUPER-SPINE1 + - mgmt_ip: 192.168.1.2/24 + name: DC1-SUPER-SPINE2 + appliesTo: + nodeSets: + - dc1-pod1-l2leaf1a + - dc1-pod1-l2leaf2a + - dc1-pod1-l2leaf2b + - dc1-pod1-leaf1a + - dc1-pod1-leaf1b + - dc1-pod1-leaf2a + - dc1-pod1-leaf2b + - dc1-pod1-spine1 + - dc1-pod1-spine2 + - dc1-pod2-leaf1a + - dc1-pod2-spine1 + - dc1-pod2-spine2 + - dc1-rs1 + - dc1-rs2 + - dc1-super-spine1 + - dc1-super-spine2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc1-settings +spec: + design: + bgp_peer_groups: + evpn_overlay_peers: + password: q+VNViP5i4rVjW1cxFv2wA== + ipv4_underlay_peers: + password: AQQvKeimxJu+uGQ/yYvv9w== + mlag_ipv4_underlay_peer: + password: vnEaG8gMeQf3d3cN6PktXQ== + dc_name: DC1 + evpn_import_pruning: true + evpn_prevent_readvertise_to_server: true + appliesTo: + nodeSets: + - dc1-pod1-l2leaf1a + - dc1-pod1-l2leaf2a + - dc1-pod1-l2leaf2b + - dc1-pod1-leaf1a + - dc1-pod1-leaf1b + - dc1-pod1-leaf2a + - dc1-pod1-leaf2b + - dc1-pod1-spine1 + - dc1-pod1-spine2 + - dc1-pod2-leaf1a + - dc1-pod2-spine1 + - dc1-pod2-spine2 + - dc1-rs1 + - dc1-rs2 + - dc1-super-spine1 + - dc1-super-spine2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc2 +spec: + design: + overlay_controller: + defaults: + bgp_as: 65201 + bgp_defaults: + - distance bgp 20 200 200 + evpn_route_servers: + - DC1-RS1 + - DC1-SUPER-SPINE1 + - DC1-POD1-SPINE1 + - DC1-POD1-LEAF1A + loopback_ipv4_pool: 172.16.20.0/24 + max_uplink_switches: 4 + platform: vEOS-LAB + uplink_bfd: false + uplink_ipv4_pool: 172.17.20.0/24 + uplink_switches: + - DC2-SUPER-SPINE1 + - DC2-SUPER-SPINE1 + nodes: + - evpn_role: server + mgmt_ip: 192.168.1.18/24 + name: DC2-RS1 + uplink_interfaces: + - Ethernet1 + - Ethernet2 + uplink_switch_interfaces: + - Ethernet3 + - Ethernet6 + - evpn_role: none + mgmt_ip: 192.168.1.19/24 + name: DC2-RS2 + uplink_interfaces: + - Ethernet1 + - Ethernet2 + uplink_switch_interfaces: + - Ethernet5 + - Ethernet7 + super_spine: + defaults: + bgp_as: 65200 + bgp_defaults: + - distance bgp 20 200 200 + loopback_ipv4_pool: 172.16.200.0/24 + platform: vEOS-LAB + nodes: + - evpn_role: server + evpn_route_servers: + - DC1-RS1 + - DC1-SUPER-SPINE1 + - DC1-POD1-SPINE1 + - DC1-POD1-LEAF1A + mgmt_ip: 192.168.1.16/24 + name: DC2-SUPER-SPINE1 + - evpn_role: none + mgmt_ip: 192.168.1.17/24 + name: DC2-SUPER-SPINE2 + appliesTo: + nodeSets: + - dc2-pod1-l2leaf1a + - dc2-pod1-l2leaf2a + - dc2-pod1-leaf1a + - dc2-pod1-leaf2a + - dc2-pod1-spine1 + - dc2-pod1-spine2 + - dc2-rs1 + - dc2-rs2 + - dc2-super-spine1 + - dc2-super-spine2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc2-settings +spec: + design: + dc_name: DC2 + evpn_prevent_readvertise_to_server: false + appliesTo: + nodeSets: + - dc2-pod1-l2leaf1a + - dc2-pod1-l2leaf2a + - dc2-pod1-leaf1a + - dc2-pod1-leaf2a + - dc2-pod1-spine1 + - dc2-pod1-spine2 + - dc2-rs1 + - dc2-rs2 + - dc2-super-spine1 + - dc2-super-spine2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-pod1 +spec: + design: + l2leaf: + defaults: + digital_twin: + act_internet_access: true + inband_mgmt_description: L2LEAF_INBAND_MGMT + inband_mgmt_subnet: 172.21.110.0/24 + inband_mgmt_vlan: 4085 + inband_mgmt_vlan_name: L2LEAF_INBAND_MGMT + mlag_peer_ipv4_pool: 172.20.110.0/24 + mlag_peer_l3_ipv4_pool: 172.19.110.0/24 + node_groups: + - group: RACK2_SINGLE + mlag: false + nodes: + - digital_twin: + mgmt_ip: 192.168.1.10/24 + mgmt_ip: 192.168.1.10/24 + name: DC1-POD1-L2LEAF1A + platform: CUSTOM-PLATFORM + uplink_switch_interfaces: + - Ethernet3 + platform: vEOS-LAB + spanning_tree_mode: mstp + spanning_tree_priority: 8192 + uplink_interfaces: + - Ethernet1 + uplink_switches: + - DC1-POD1-LEAF1A + - group: RACK2_MLAG + mlag_interfaces: + - Ethernet3 + - Ethernet4 + nodes: + - digital_twin: + mgmt_ip: 192.168.1.11/24 + mgmt_ip: 192.168.1.11/24 + name: DC1-POD1-L2LEAF2A + platform: vEOS-LAB + uplink_switch_interfaces: + - Ethernet3 + - Ethernet3 + - mgmt_ip: 192.168.1.12/24 + name: DC1-POD1-L2LEAF2B + platform: TOOLS-SERVER + raw_eos_cli: "interface Loopback1003\n description Loopback created from\ + \ raw_eos_cli under l2leaf node DC1-POD1-L2LEAF2B\n" + uplink_switch_interfaces: + - Ethernet4 + - Ethernet4 + raw_eos_cli: "interface Loopback1002\n description Loopback created from\ + \ raw_eos_cli under l2leaf node-group RACK2_MLAG\n" + spanning_tree_mode: mstp + spanning_tree_priority: 8192 + uplink_interfaces: + - Ethernet1 + - Ethernet2 + uplink_switches: + - DC1.POD1.LEAF2A + - DC1-POD1-LEAF2B + l3leaf: + defaults: + bgp_as: '65555' + bgp_defaults: + - distance bgp 20 200 200 + inband_mgmt_description: L2LEAF_INBAND_MGMT + inband_mgmt_vlan_name: L2LEAF_INBAND_MGMT + loopback_ipv4_offset: 2 + loopback_ipv4_pool: 172.16.110.0/24 + mlag_peer_ipv4_pool: 172.20.110.0/24 + mlag_peer_l3_ipv4_pool: 172.19.110.0/24 + uplink_ipv4_pool: 172.17.110.0/24 + uplink_macsec: + profile: MACSEC_PROFILE + uplink_ptp: + enable: true + virtual_router_mac_address: 00:1c:73:00:dc:01 + vtep_loopback_ipv4_pool: 172.18.110.0/24 + node_groups: + - bgp_as: '65111.100' + evpn_role: server + evpn_route_servers: + - DC2-RS1 + - DC2-SUPER-SPINE1 + - DC2-POD1-SPINE1 + - DC2-POD1-LEAF1A + filter: + always_include_vrfs_in_tenants: + - all + tags: [] + tenants: [] + group: RACK1_SINGLE + mlag: false + nodes: + - digital_twin: + mgmt_ip: 192.168.1.7/24 + name: DC1-POD1-LEAF1A + raw_eos_cli: "interface Loopback1002\n description Loopback created from\ + \ raw_eos_cli under node DC1-POD1-LEAF1A\n" + uplink_switch_interfaces: + - Ethernet3 + - Ethernet3 + - evpn_role: none + filter: + tags: + - all + tenants: + - all + mgmt_ip: 192.168.1.26/24 + name: DC1-POD1-LEAF1B + uplink_switch_interfaces: + - Ethernet13 + - Ethernet13 + uplink_switches: + - DC1.POD1.LEAF2A + - DC1-POD1-LEAF2B + uplink_type: p2p-vrfs + vtep: false + platform: vEOS-LAB + raw_eos_cli: "interface Loopback1001\n description Loopback created from\ + \ raw_eos_cli under node-group RACK1_SINGLE\n" + spanning_tree_mode: none + uplink_interfaces: + - Ethernet1 + - Ethernet2 + uplink_switches: + - DC1-POD1-SPINE1 + - DC1-POD1-SPINE2 + - bgp_as: '65112.100' + evpn_route_servers: + - DC1-RS1 + - DC1-SUPER-SPINE1 + - DC1-POD1-SPINE1 + - DC1-POD1-LEAF1A + group: RACK2_MLAG + mlag_dual_primary_detection: true + mlag_peer_l3_vlan: 4094 + nodes: + - digital_twin: + mgmt_ip: 192.168.1.8/24 + evpn_gateway: + evpn_l2: + enabled: false + evpn_l3: + enabled: true + inter_domain: true + remote_peers: + - hostname: DC1-POD2-LEAF1A + mgmt_ip: 192.168.1.8/16 + name: DC1.POD1.LEAF2A + structured_config: + domain_list: + - structured-config.set.on.node + - evpn_gateway: + evpn_l2: + enabled: true + evpn_l3: + enabled: true + inter_domain: false + remote_peers: + - bgp_as: 1111 + hostname: HOSTNAME_NOT_IN_ANSIBLE + ip_address: 1.1.1.1 + mgmt_ip: 192.168.1.9/16 + name: DC1-POD1-LEAF2B + raw_eos_cli: "interface Loopback1002\n description Loopback created from\ + \ raw_eos_cli under l3leaf node-group RACK2_MLAG\n" + spanning_tree_mode: mstp + spanning_tree_priority: 4096 + spine: + defaults: + bgp_as: '65110.100' + bgp_defaults: + - distance bgp 20 200 200 + digital_twin: + act_internet_access: false + act_os_version: 4.33.1F + loopback_ipv4_pool: 172.16.110.0/24 + max_uplink_switches: 4 + platform: vEOS-LAB + uplink_interfaces: + - Ethernet1 + - Ethernet2 + uplink_ipv4_pool: 172.16.11.0/24 + uplink_macsec: + profile: MACSEC_PROFILE + uplink_ptp: + enable: true + uplink_switches: + - DC1-SUPER-SPINE1 + - DC1-SUPER-SPINE2 + nodes: + - digital_twin: + mgmt_ip: 192.168.1.5/24 + evpn_role: server + evpn_route_servers: + - DC2-RS1 + - DC2-SUPER-SPINE1 + - DC2-POD1-SPINE1 + - DC2-POD1-LEAF1A + name: DC1-POD1-SPINE1 + uplink_switch_interfaces: + - Ethernet1 + - Ethernet1 + - digital_twin: + act_internet_access: true + evpn_role: none + mgmt_ip: 192.168.1.6/24 + name: DC1-POD1-SPINE2 + serial_number: DEADBEEFC0FFEE + uplink_switch_interfaces: + - Ethernet2 + - Ethernet2 + appliesTo: + nodeSets: + - dc1-pod1-l2leaf1a + - dc1-pod1-l2leaf2a + - dc1-pod1-l2leaf2b + - dc1-pod1-leaf1a + - dc1-pod1-leaf1b + - dc1-pod1-leaf2a + - dc1-pod1-leaf2b + - dc1-pod1-spine1 + - dc1-pod1-spine2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc1-pod1-settings +spec: + design: + max_l3leaf_to_spine_links: 2 + pod_name: DC1_POD1 + underlay_filter_redistribute_connected: false + appliesTo: + nodeSets: + - dc1-pod1-l2leaf1a + - dc1-pod1-l2leaf2a + - dc1-pod1-l2leaf2b + - dc1-pod1-leaf1a + - dc1-pod1-leaf1b + - dc1-pod1-leaf2a + - dc1-pod1-leaf2b + - dc1-pod1-spine1 + - dc1-pod1-spine2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-pod2 +spec: + design: + l3leaf: + defaults: + bgp_as: 65555 + bgp_defaults: + - distance bgp 20 200 200 + loopback_ipv4_offset: 2 + loopback_ipv4_pool: 172.16.120.0/24 + mlag_peer_ipv4_pool: 172.20.120.0/24 + mlag_peer_l3_ipv4_pool: 172.19.120.0/24 + uplink_ipv4_pool: 172.17.120.0/24 + uplink_macsec: + profile: MACSEC_PROFILE + uplink_ptp: + enable: true + virtual_router_mac_address: 00:1c:73:00:dc:01 + vtep_loopback_ipv4_pool: 172.18.120.0/24 + node_groups: + - bgp_as: 65121 + group: RACK1_SINGLE + nodes: + - mgmt_ip: 192.168.1.15/24 + name: DC1-POD2-LEAF1A + uplink_switch_interfaces: + - Ethernet3 + - Ethernet3 + platform: vEOS-LAB + spanning_tree_mode: none + uplink_interfaces: + - Ethernet1 + - Ethernet2 + uplink_switches: + - DC1-POD2-SPINE1 + - DC1-POD2-SPINE2 + spine: + defaults: + bgp_as: 65120 + bgp_defaults: + - distance bgp 20 200 200 + loopback_ipv4_pool: 172.16.120.0/24 + max_uplink_switches: 4 + platform: vEOS-LAB + uplink_interfaces: + - Ethernet1 + - Ethernet2 + uplink_ipv4_pool: 172.16.12.0/24 + uplink_macsec: + profile: MACSEC_PROFILE + uplink_ptp: + enable: true + uplink_switches: + - DC1-SUPER-SPINE1 + - DC1-SUPER-SPINE2 + nodes: + - evpn_route_servers: + - DC1-SUPER-SPINE2 + mgmt_ip: 192.168.1.13/24 + name: DC1-POD2-SPINE1 + uplink_switch_interfaces: + - Ethernet3 + - Ethernet3 + - evpn_route_servers: + - DC1-SUPER-SPINE2 + mgmt_ip: 192.168.1.14/24 + name: DC1-POD2-SPINE2 + uplink_switch_interfaces: + - Ethernet4 + - Ethernet4 + appliesTo: + nodeSets: + - dc1-pod2-leaf1a + - dc1-pod2-spine1 + - dc1-pod2-spine2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc1-pod2-settings +spec: + design: + pod_name: DC1_POD2 + appliesTo: + nodeSets: + - dc1-pod2-leaf1a + - dc1-pod2-spine1 + - dc1-pod2-spine2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc2-pod1 +spec: + design: + l2leaf: + defaults: + inband_mgmt_subnet: 172.21.210.0/24 + mlag: false + platform: vEOS-LAB + raw_eos_cli: "interface Loopback1011\n description Loopback created from\ + \ raw_eos_cli under l2leaf defaults in DC2 POD1\n" + spanning_tree_mode: mstp + spanning_tree_priority: 8192 + uplink_interfaces: + - Ethernet1 + node_groups: + - group: RACK1_SINGLE + nodes: + - mgmt_ip: 192.168.1.23/24 + name: DC2-POD1-L2LEAF1A + uplink_switch_interfaces: + - Ethernet3 + uplink_switches: + - DC2-POD1-LEAF1A + - group: RACK2_SINGLE + nodes: + - mgmt_ip: 192.168.1.25/24 + name: DC2-POD1-L2LEAF2A + uplink_switch_interfaces: + - Ethernet3 + uplink_switches: + - DC2-POD1-LEAF2A + l3leaf: + defaults: + bgp_as: 65555 + bgp_defaults: + - distance bgp 20 200 200 + evpn_role: server + evpn_route_servers: + - DC1-RS1 + - DC1-SUPER-SPINE1 + - DC1-POD1-SPINE1 + - DC1-POD1-LEAF1A + filter: + always_include_vrfs_in_tenants: + - all + tags: [] + loopback_ipv4_offset: 2 + loopback_ipv4_pool: 172.16.210.0/24 + mlag: false + mlag_peer_ipv4_pool: 172.20.120.0/24 + mlag_peer_l3_ipv4_pool: 172.19.120.0/24 + platform: vEOS-LAB + raw_eos_cli: "interface Loopback1010\n description Loopback created from\ + \ raw_eos_cli under l3leaf defaults in DC2 POD1\n" + spanning_tree_mode: rstp + spanning_tree_priority: 4096 + uplink_interfaces: + - Ethernet1 + - Ethernet2 + uplink_ipv4_pool: 172.17.210.0/24 + uplink_ptp: + enable: true + uplink_switches: + - DC2-POD1-SPINE1 + - DC2-POD1-SPINE2 + virtual_router_mac_address: 00:1c:73:00:dc:01 + vtep_loopback_ipv4_pool: 172.18.210.0/24 + node_groups: + - bgp_as: 65211 + group: RACK1_SINGLE + nodes: + - mgmt_ip: 192.168.1.22/24 + name: DC2-POD1-LEAF1A + uplink_switch_interfaces: + - Ethernet3 + - Ethernet3 + - bgp_as: 65212 + evpn_role: client + evpn_route_servers: + - DC2-POD1-LEAF1A + evpn_services_l2_only: true + group: RACK2_SINGLE + nodes: + - mgmt_ip: 192.168.1.24/24 + name: DC2-POD1-LEAF2A + uplink_switch_interfaces: + - Ethernet4 + - Ethernet4 + spine: + defaults: + bgp_as: 65210 + bgp_defaults: + - distance bgp 20 200 200 + loopback_ipv4_pool: 172.16.210.0/24 + max_uplink_switches: 4 + platform: vEOS-LAB + raw_eos_cli: "interface Loopback1009\n description Loopback created from\ + \ raw_eos_cli under spine defaults in DC2 POD1\n" + uplink_interfaces: + - Ethernet1 + - Ethernet2 + uplink_ipv4_pool: 172.16.21.0/24 + uplink_ptp: + enable: true + uplink_switches: + - DC2-SUPER-SPINE1 + - DC2-SUPER-SPINE2 + nodes: + - evpn_role: server + evpn_route_servers: + - DC1-RS1 + - DC1-SUPER-SPINE1 + - DC1-POD1-SPINE1 + - DC1-POD1-LEAF1A + mgmt_ip: 192.168.1.20/24 + name: DC2-POD1-SPINE1 + uplink_switch_interfaces: + - Ethernet1 + - Ethernet1 + - evpn_role: none + mgmt_ip: 192.168.1.21/24 + name: DC2-POD1-SPINE2 + uplink_switch_interfaces: + - Ethernet2 + - Ethernet2 + appliesTo: + nodeSets: + - dc2-pod1-l2leaf1a + - dc2-pod1-l2leaf2a + - dc2-pod1-leaf1a + - dc2-pod1-leaf2a + - dc2-pod1-spine1 + - dc2-pod1-spine2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc2-pod1-settings +spec: + design: + pod_name: DC2_POD1 + appliesTo: + nodeSets: + - dc2-pod1-l2leaf1a + - dc2-pod1-l2leaf2a + - dc2-pod1-leaf1a + - dc2-pod1-leaf2a + - dc2-pod1-spine1 + - dc2-pod1-spine2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-pod1-l2leaf1a +spec: + design: + type: l2leaf + declares: + - DC1-POD1-L2LEAF1A +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-pod1-l2leaf2a +spec: + design: + type: l2leaf + declares: + - DC1-POD1-L2LEAF2A +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-pod1-l2leaf2b +spec: + design: + type: l2leaf + declares: + - DC1-POD1-L2LEAF2B +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-pod1-leaf1a +spec: + design: + type: l3leaf + declares: + - DC1-POD1-LEAF1A +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-pod1-leaf1b +spec: + design: + type: l3leaf + declares: + - DC1-POD1-LEAF1B +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-pod1-leaf2b +spec: + design: + type: l3leaf + declares: + - DC1-POD1-LEAF2B +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc1-pod1-leaf2b-settings +spec: + design: + use_cv_topology: true + appliesTo: + nodeSets: + - dc1-pod1-leaf2b +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-pod1-spine1 +spec: + design: + type: spine + declares: + - DC1-POD1-SPINE1 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc1-pod1-spine1-settings +spec: + design: + management_eapi: + enabled: false + appliesTo: + nodeSets: + - dc1-pod1-spine1 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-pod1-spine2 +spec: + design: + type: spine + declares: + - DC1-POD1-SPINE2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-pod2-leaf1a +spec: + design: + type: l3leaf + declares: + - DC1-POD2-LEAF1A +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-pod2-spine1 +spec: + design: + type: spine + declares: + - DC1-POD2-SPINE1 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-pod2-spine2 +spec: + design: + type: spine + declares: + - DC1-POD2-SPINE2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-rs1 +spec: + design: + type: overlay-controller + declares: + - DC1-RS1 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc1-rs1-settings +spec: + design: + use_cv_topology: true + appliesTo: + nodeSets: + - dc1-rs1 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-rs2 +spec: + design: + type: overlay-controller + declares: + - DC1-RS2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-super-spine1 +spec: + design: + type: super-spine + declares: + - DC1-SUPER-SPINE1 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-super-spine2 +spec: + design: + type: super-spine + declares: + - DC1-SUPER-SPINE2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-pod1-leaf2a +spec: + design: + type: l3leaf + declares: + - DC1.POD1.LEAF2A +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc1-pod1-leaf2a-settings +spec: + design: + use_cv_topology: true + appliesTo: + nodeSets: + - dc1-pod1-leaf2a +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc2-pod1-l2leaf1a +spec: + design: + type: l2leaf + declares: + - DC2-POD1-L2LEAF1A +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc2-pod1-l2leaf2a +spec: + design: + type: l2leaf + declares: + - DC2-POD1-L2LEAF2A +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc2-pod1-leaf1a +spec: + design: + type: l3leaf + declares: + - DC2-POD1-LEAF1A +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc2-pod1-leaf2a +spec: + design: + type: l3leaf + declares: + - DC2-POD1-LEAF2A +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc2-pod1-spine1 +spec: + design: + type: spine + declares: + - DC2-POD1-SPINE1 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc2-pod1-spine2 +spec: + design: + type: spine + declares: + - DC2-POD1-SPINE2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc2-rs1 +spec: + design: + type: overlay-controller + declares: + - DC2-RS1 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc2-rs2 +spec: + design: + type: overlay-controller + declares: + - DC2-RS2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc2-super-spine1 +spec: + design: + type: super-spine + declares: + - DC2-SUPER-SPINE1 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc2-super-spine2 +spec: + design: + type: super-spine + declares: + - DC2-SUPER-SPINE2 +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: eos-designs-twodc-5stage-clos-twodc-5stage-clos-id-seed +data: + node-id-pools.yml: "# This file is auto-generated by AVD eos_designs.\n# When there\ + \ is a merge conflict for this file, rebase the branch, accept all current changes\ + \ for this file, and re-run AVD.\nnode_id_pools:\n fabric_name=TWODC_5STAGE_CLOS/dc_name=DC1/pod_name=DC1_POD1/type=l2leaf:\n\ + \ hostname=DC1-POD1-L2LEAF1A: 1\n hostname=DC1-POD1-L2LEAF2A: 2\n hostname=DC1-POD1-L2LEAF2B:\ + \ 3\n fabric_name=TWODC_5STAGE_CLOS/dc_name=DC1/pod_name=DC1_POD1/type=l3leaf:\n\ + \ hostname=DC1-POD1-LEAF1A: 1\n hostname=DC1-POD1-LEAF1B: 2\n hostname=DC1-POD1-LEAF2B:\ + \ 3\n hostname=DC1.POD1.LEAF2A: 4\n fabric_name=TWODC_5STAGE_CLOS/dc_name=DC1/pod_name=DC1_POD1/type=spine:\n\ + \ hostname=DC1-POD1-SPINE1: 1\n hostname=DC1-POD1-SPINE2: 2\n fabric_name=TWODC_5STAGE_CLOS/dc_name=DC1/pod_name=DC1_POD2/type=l3leaf:\n\ + \ hostname=DC1-POD2-LEAF1A: 1\n fabric_name=TWODC_5STAGE_CLOS/dc_name=DC1/pod_name=DC1_POD2/type=spine:\n\ + \ hostname=DC1-POD2-SPINE1: 1\n hostname=DC1-POD2-SPINE2: 2\n fabric_name=TWODC_5STAGE_CLOS/dc_name=DC1/type=overlay-controller:\n\ + \ hostname=DC1-RS1: 1\n hostname=DC1-RS2: 2\n fabric_name=TWODC_5STAGE_CLOS/dc_name=DC1/type=super-spine:\n\ + \ hostname=DC1-SUPER-SPINE1: 1\n hostname=DC1-SUPER-SPINE2: 2\n fabric_name=TWODC_5STAGE_CLOS/dc_name=DC2/pod_name=DC2_POD1/type=l2leaf:\n\ + \ hostname=DC2-POD1-L2LEAF1A: 1\n hostname=DC2-POD1-L2LEAF2A: 2\n fabric_name=TWODC_5STAGE_CLOS/dc_name=DC2/pod_name=DC2_POD1/type=l3leaf:\n\ + \ hostname=DC2-POD1-LEAF1A: 1\n hostname=DC2-POD1-LEAF2A: 2\n fabric_name=TWODC_5STAGE_CLOS/dc_name=DC2/pod_name=DC2_POD1/type=spine:\n\ + \ hostname=DC2-POD1-SPINE1: 1\n hostname=DC2-POD1-SPINE2: 2\n fabric_name=TWODC_5STAGE_CLOS/dc_name=DC2/type=overlay-controller:\n\ + \ hostname=DC2-RS1: 1\n hostname=DC2-RS2: 2\n fabric_name=TWODC_5STAGE_CLOS/dc_name=DC2/type=super-spine:\n\ + \ hostname=DC2-SUPER-SPINE1: 1\n hostname=DC2-SUPER-SPINE2: 2\n" +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: Fabric +metadata: + name: eos-designs-twodc-5stage-clos-twodc-5stage-clos +spec: + fabricName: TWODC_5STAGE_CLOS + requires: + - kind: SettingSet + name: all + - kind: ConnectedEndpointSet + name: servers-endpoints + - kind: SettingSet + name: servers-settings + - kind: NetworkServiceSet + name: tenants-networks-services + - kind: SettingSet + name: tenants-networks-settings + - kind: NodeSet + name: twodc-5stage-clos + - kind: SettingSet + name: twodc-5stage-clos-settings + - kind: NodeSet + name: dc1 + - kind: SettingSet + name: dc1-settings + - kind: NodeSet + name: dc2 + - kind: SettingSet + name: dc2-settings + - kind: NodeSet + name: dc1-pod1 + - kind: SettingSet + name: dc1-pod1-settings + - kind: NodeSet + name: dc1-pod2 + - kind: SettingSet + name: dc1-pod2-settings + - kind: NodeSet + name: dc2-pod1 + - kind: SettingSet + name: dc2-pod1-settings + - kind: NodeSet + name: dc1-pod1-l2leaf1a + - kind: NodeSet + name: dc1-pod1-l2leaf2a + - kind: NodeSet + name: dc1-pod1-l2leaf2b + - kind: NodeSet + name: dc1-pod1-leaf1a + - kind: NodeSet + name: dc1-pod1-leaf1b + - kind: NodeSet + name: dc1-pod1-leaf2b + - kind: SettingSet + name: dc1-pod1-leaf2b-settings + - kind: NodeSet + name: dc1-pod1-spine1 + - kind: SettingSet + name: dc1-pod1-spine1-settings + - kind: NodeSet + name: dc1-pod1-spine2 + - kind: NodeSet + name: dc1-pod2-leaf1a + - kind: NodeSet + name: dc1-pod2-spine1 + - kind: NodeSet + name: dc1-pod2-spine2 + - kind: NodeSet + name: dc1-rs1 + - kind: SettingSet + name: dc1-rs1-settings + - kind: NodeSet + name: dc1-rs2 + - kind: NodeSet + name: dc1-super-spine1 + - kind: NodeSet + name: dc1-super-spine2 + - kind: NodeSet + name: dc1-pod1-leaf2a + - kind: SettingSet + name: dc1-pod1-leaf2a-settings + - kind: NodeSet + name: dc2-pod1-l2leaf1a + - kind: NodeSet + name: dc2-pod1-l2leaf2a + - kind: NodeSet + name: dc2-pod1-leaf1a + - kind: NodeSet + name: dc2-pod1-leaf2a + - kind: NodeSet + name: dc2-pod1-spine1 + - kind: NodeSet + name: dc2-pod1-spine2 + - kind: NodeSet + name: dc2-rs1 + - kind: NodeSet + name: dc2-rs2 + - kind: NodeSet + name: dc2-super-spine1 + - kind: NodeSet + name: dc2-super-spine2 + nodeIdPool: + seedConfigMapName: eos-designs-twodc-5stage-clos-twodc-5stage-clos-id-seed diff --git a/examples/fabric/inputs/isis-ldp-ipvpn.yaml b/examples/fabric/inputs/isis-ldp-ipvpn.yaml new file mode 100644 index 0000000..5adaf73 --- /dev/null +++ b/examples/fabric/inputs/isis-ldp-ipvpn.yaml @@ -0,0 +1,369 @@ +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: fabric +spec: + design: + aaa_settings: + local_users: + - name: admin + no_password: true + privilege: 15 + role: network-admin + - name: arista + privilege: 15 + role: network-admin + sha512_password: $6$Enl0WfE32FthwyiJ$yTyGaEJ2uPKLU.F7314YtB7J1jrzrMi7ogXIRTEHQfLdLgKWWmr1UvNlZLN6AyuxET7G5aH3AI9OYRzxVTkB1. + bgp_peer_groups: + mpls_overlay_peers: + password: $1c$G8BQN0ezkiJOX2cuAYpsEA== + rr_overlay_peers: + password: $1c$G8BQN0ezkiJOX2cuAYpsEA== + custom_structured_configuration_management_security: + password: + encryption_key_common: true + eos_designs_documentation: + p2p_links_csv: true + topology_csv: true + fabric_name: FABRIC + management_eapi: + enabled: true + overlay_routing_protocol: ibgp + underlay_routing_protocol: isis-ldp + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NetworkServiceSet +metadata: + name: network-services +spec: + design: + tenants: + - name: CUSTOMER1 + vrfs: + - address_families: + - vpn-ipv4 + l3_interfaces: + - description: C1_L3_SERVICE + enabled: true + interfaces: + - Ethernet3.10 + - Ethernet4.10 + - Ethernet2 + ip_addresses: + - 10.0.1.1/29 + - 10.0.1.2/29 + - 10.0.1.9/30 + nodes: + - pe1 + - pe2 + - pe3 + ospf: + enabled: true + name: C1_VRF1 + ospf: + enabled: true + nodes: + - pe1 + - pe2 + - pe3 + vrf_id: 10 + - name: CUSTOMER2 + vrfs: + - address_families: + - vpn-ipv4 + bgp_peers: + - description: C2_ROUTER1 + ip_address: 10.1.1.3 + maximum_routes: 100 + nodes: + - pe1 + - pe2 + remote_as: 65123 + send_community: standard + - description: C2_ROUTER2 + ip_address: 10.1.1.10 + maximum_routes: 100 + nodes: + - pe3 + remote_as: 65124 + send_community: standard + l3_interfaces: + - description: C2_L3_SERVICE + enabled: true + interfaces: + - Ethernet3.20 + - Ethernet4.20 + - Ethernet4 + ip_addresses: + - 10.1.1.1/29 + - 10.1.1.2/29 + - 10.1.1.9/30 + nodes: + - pe1 + - pe2 + - pe3 + name: C2_VRF1 + vrf_id: 20 + appliesTo: + nodeSets: + - wan1-pe-routers +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: wan1 +spec: + design: + p: + defaults: + isis_system_id_prefix: '0000.0000' + loopback_ipv4_pool: 10.255.0.0/27 + platform: cEOSLab + nodes: + - id: 1 + mgmt_ip: 172.16.1.11/24 + name: p1 + - id: 2 + mgmt_ip: 172.16.1.12/24 + name: p2 + - id: 3 + mgmt_ip: 172.16.1.13/24 + name: p3 + - id: 4 + mgmt_ip: 172.16.1.14/24 + name: p4 + pe: + defaults: + isis_system_id_prefix: '0000.0001' + loopback_ipv4_pool: 10.255.1.0/27 + mpls_route_reflectors: + - rr1 + - rr2 + platform: cEOSLab + spanning_tree_mode: none + virtual_router_mac_address: 00:1c:73:00:dc:00 + node_groups: + - group: WAN1-PE1-2 + nodes: + - id: 1 + mgmt_ip: 172.16.1.101/24 + name: pe1 + - id: 2 + mgmt_ip: 172.16.1.102/24 + name: pe2 + - group: WAN1-PE3 + nodes: + - id: 3 + mgmt_ip: 172.16.1.103/24 + name: pe3 + rr: + defaults: + isis_system_id_prefix: '0000.0002' + loopback_ipv4_pool: 10.255.2.0/27 + mpls_route_reflectors: + - rr1 + - rr2 + platform: cEOSLab + spanning_tree_mode: none + node_groups: + - group: WAN1_RR1-2 + nodes: + - id: 1 + mgmt_ip: 172.16.1.151/24 + name: rr1 + - id: 2 + mgmt_ip: 172.16.1.152/24 + name: rr2 + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: wan1-settings +spec: + design: + bgp_as: 65001 + bgp_distance: + external_routes: 20 + internal_routes: 200 + local_routes: 200 + core_interfaces: + p2p_links: + - id: 1 + interfaces: + - Ethernet1 + - Ethernet1 + nodes: + - pe1 + - p1 + profile: core_profile + - id: 2 + interfaces: + - Ethernet2 + - Ethernet2 + nodes: + - pe1 + - p2 + profile: core_profile + - id: 3 + interfaces: + - Ethernet1 + - Ethernet1 + nodes: + - pe2 + - p2 + profile: core_profile + - id: 4 + interfaces: + - Ethernet2 + - Ethernet2 + nodes: + - pe2 + - p1 + profile: core_profile + - id: 5 + interfaces: + - Ethernet4 + - Ethernet4 + nodes: + - p1 + - p2 + profile: core_profile + - id: 6 + interfaces: + - Ethernet3 + - Ethernet3 + nodes: + - rr1 + - p1 + profile: core_profile + - id: 7 + interfaces: + - Ethernet2 + - Ethernet2 + nodes: + - rr1 + - p3 + profile: core_profile + - id: 8 + interfaces: + - Ethernet4 + - Ethernet4 + nodes: + - rr1 + - rr2 + profile: core_profile + - id: 9 + interfaces: + - Ethernet3 + - Ethernet3 + nodes: + - rr2 + - p2 + profile: core_profile + - id: 10 + interfaces: + - Ethernet2 + - Ethernet2 + nodes: + - rr2 + - p4 + profile: core_profile + - id: 11 + interfaces: + - Ethernet4 + - Ethernet4 + nodes: + - p3 + - p4 + profile: core_profile + - id: 12 + interfaces: + - Ethernet1 + - Ethernet1 + nodes: + - pe3 + - p3 + profile: core_profile + - id: 13 + interfaces: + - Ethernet3 + - Ethernet3 + nodes: + - pe3 + - p4 + profile: core_profile + p2p_links_ip_pools: + - ipv4_pool: 10.255.3.0/24 + name: core_pool + p2p_links_profiles: + - ip_pool: core_pool + isis_authentication_key: $1c$sTNAlR6rKSw= + isis_authentication_mode: md5 + isis_circuit_type: level-2 + isis_metric: 50 + name: core_profile + mgmt_gateway: 172.16.1.1 + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: wan1-pe-routers +spec: + design: + type: pe + declares: + - pe1 + - pe2 + - pe3 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: wan1-p-routers +spec: + design: + type: p + declares: + - p1 + - p2 + - p3 + - p4 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: wan1-rr-routers +spec: + design: + type: rr + declares: + - rr1 + - rr2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: Fabric +metadata: + name: isis-ldp-ipvpn +spec: + fabricName: FABRIC + requires: + - kind: SettingSet + name: fabric + - kind: NetworkServiceSet + name: network-services + - kind: NodeSet + name: wan1 + - kind: SettingSet + name: wan1-settings + - kind: NodeSet + name: wan1-pe-routers + - kind: NodeSet + name: wan1-p-routers + - kind: NodeSet + name: wan1-rr-routers diff --git a/examples/fabric/inputs/l2ls-fabric.yaml b/examples/fabric/inputs/l2ls-fabric.yaml new file mode 100644 index 0000000..1f1ab8e --- /dev/null +++ b/examples/fabric/inputs/l2ls-fabric.yaml @@ -0,0 +1,279 @@ +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: fabric +spec: + design: + aaa_settings: + authentication: + policies: + local: + allow_nopassword: true + authorization: + exec: + default: local + local_users: + - name: admin + no_password: true + privilege: 15 + role: network-admin + - name: arista + privilege: 15 + role: network-admin + sha512_password: $6$Enl0WfE32FthwyiJ$yTyGaEJ2uPKLU.F7314YtB7J1jrzrMi7ogXIRTEHQfLdLgKWWmr1UvNlZLN6AyuxET7G5aH3AI9OYRzxVTkB1. + dns_settings: + servers: + - ip_address: 8.8.4.4 + - ip_address: 8.8.8.8 + eos_designs_documentation: + connected_endpoints: true + fabric_name: DC1 + ntp_settings: + server_vrf: use_mgmt_interface_vrf + servers: + - name: time.google.com + - name: pool.ntp.org + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc1 +spec: + design: + management_eapi: + enabled: true + mgmt_gateway: 172.16.100.1 + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: ConnectedEndpointSet +metadata: + name: dc1-endpoints +spec: + design: + connected_endpoints_keys: + - key: servers + type: server + - key: firewalls + type: firewall + - key: routers + type: router + firewalls: + - adapters: + - endpoint_ports: + - Eth1 + - Eth2 + port_channel: + mode: active + profile: PP-FIREWALL + switch_ports: + - Ethernet5 + - Ethernet5 + switches: + - SPINE1 + - SPINE2 + name: FIREWALL + port_profiles: + - profile: PP-DEFAULTS + spanning_tree_portfast: edge + - mode: access + parent_profile: PP-DEFAULTS + profile: PP-BLUE + vlans: '10' + - mode: access + parent_profile: PP-DEFAULTS + profile: PP-GREEN + vlans: '20' + - mode: access + parent_profile: PP-DEFAULTS + profile: PP-ORANGE + vlans: '30' + - mode: trunk + profile: PP-FIREWALL + vlans: 10,20,30 + servers: + - adapters: + - endpoint_ports: + - Eth1 + profile: PP-BLUE + switch_ports: + - Ethernet3 + switches: + - LEAF1 + name: HostA + rack: POD1 + - adapters: + - endpoint_ports: + - Eth1 + profile: PP-GREEN + switch_ports: + - Ethernet3 + switches: + - LEAF2 + name: HostB + rack: POD1 + - adapters: + - endpoint_ports: + - Eth1 + profile: PP-BLUE + switch_ports: + - Ethernet3 + switches: + - LEAF3 + name: HostC + rack: POD2 + - adapters: + - endpoint_ports: + - Eth1 + profile: PP-ORANGE + switch_ports: + - Ethernet3 + switches: + - LEAF4 + name: Host2 + rack: POD2 + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NetworkServiceSet +metadata: + name: dc1-network-services +spec: + design: + tenants: + - l2vlans: + - id: 10 + name: BLUE-NET + tags: + - bluezone + - id: 20 + name: GREEN-NET + tags: + - greenzone + - id: 30 + name: ORANGE-NET + tags: + - orangezone + name: MY_FABRIC + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-leafs +spec: + design: + l2leaf: + defaults: + mlag_interfaces: + - Ethernet47 + - Ethernet48 + mlag_peer_ipv4_pool: 192.168.0.0/24 + platform: cEOSLab + spanning_tree_mode: mstp + spanning_tree_priority: 16384 + uplink_interfaces: + - Ethernet1 + - Ethernet2 + uplink_switches: + - SPINE1 + - SPINE2 + node_groups: + - filter: + tags: + - bluezone + - greenzone + group: RACK1 + mlag: true + nodes: + - id: 1 + mgmt_ip: 172.16.100.105/24 + name: LEAF1 + uplink_switch_interfaces: + - Ethernet1 + - Ethernet1 + - id: 2 + mgmt_ip: 172.16.100.106/24 + name: LEAF2 + uplink_switch_interfaces: + - Ethernet2 + - Ethernet2 + - filter: + tags: + - bluezone + - orangezone + group: RACK2 + mlag: true + nodes: + - id: 3 + mgmt_ip: 172.16.100.107/24 + name: LEAF3 + uplink_switch_interfaces: + - Ethernet3 + - Ethernet3 + - id: 4 + mgmt_ip: 172.16.100.108/24 + name: LEAF4 + uplink_switch_interfaces: + - Ethernet4 + - Ethernet4 + type: l2leaf + declares: + - LEAF1 + - LEAF2 + - LEAF3 + - LEAF4 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-spines +spec: + design: + l2spine: + defaults: + mlag_interfaces: + - Ethernet47 + - Ethernet48 + mlag_peer_ipv4_pool: 192.168.0.0/24 + platform: cEOSLab + spanning_tree_mode: mstp + spanning_tree_priority: 4096 + node_groups: + - group: SPINES + nodes: + - id: 1 + mgmt_ip: 172.16.100.101/24 + name: SPINE1 + - id: 2 + mgmt_ip: 172.16.100.102/24 + name: SPINE2 + type: l2spine + declares: + - SPINE1 + - SPINE2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: Fabric +metadata: + name: l2ls-fabric +spec: + fabricName: DC1 + requires: + - kind: SettingSet + name: fabric + - kind: SettingSet + name: dc1 + - kind: ConnectedEndpointSet + name: dc1-endpoints + - kind: NetworkServiceSet + name: dc1-network-services + - kind: NodeSet + name: dc1-leafs + - kind: NodeSet + name: dc1-spines diff --git a/examples/fabric/inputs/single-dc-l3ls-ipv6.yaml b/examples/fabric/inputs/single-dc-l3ls-ipv6.yaml new file mode 100644 index 0000000..e2250a5 --- /dev/null +++ b/examples/fabric/inputs/single-dc-l3ls-ipv6.yaml @@ -0,0 +1,349 @@ +apiVersion: avd.netclab.dev/v1alpha1 +kind: ConnectedEndpointSet +metadata: + name: connected-endpoints +spec: + design: + servers: + - adapters: + - endpoint_ports: + - PCI1 + - PCI2 + mode: trunk + native_vlan: 4092 + port_channel: + endpoint_port_channel: Bond1 + mode: active + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + - Ethernet5 + switches: + - dc1-leaf1a + - dc1-leaf1b + vlans: 11-12,21-22 + - endpoint_ports: + - iLO + mode: access + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + switches: + - dc1-leaf1c + vlans: 11 + name: dc1-leaf1-server1 + - adapters: + - endpoint_ports: + - PCI1 + - PCI2 + mode: trunk + native_vlan: 4092 + port_channel: + mode: active + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + - Ethernet5 + switches: + - dc1-leaf2a + - dc1-leaf2b + vlans: 11-12,21-22 + - endpoint_ports: + - iLO + mode: access + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + switches: + - dc1-leaf2c + vlans: 11 + name: dc1-leaf2-server1 + appliesTo: + nodeSets: + - dc1-l2-leaves + - dc1-l3-leaves +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: fabric +spec: + design: + aaa_settings: + local_users: + - name: admin + no_password: true + privilege: 15 + role: network-admin + - name: arista + privilege: 15 + role: network-admin + sha512_password: $6$Enl0WfE32FthwyiJ$yTyGaEJ2uPKLU.F7314YtB7J1jrzrMi7ogXIRTEHQfLdLgKWWmr1UvNlZLN6AyuxET7G5aH3AI9OYRzxVTkB1. + bgp_peer_groups: + evpn_overlay_peers: + password: Q4fqtbqcZ7oQuKfuWtNGRQ== + ipv4_underlay_peers: + name: IPv6-UNDERLAY-PEERS + password: F0bdF6JLcJtagc6PCN8W0Q== + mlag_ipv4_underlay_peer: + name: MLAG-IPv6-UNDERLAY-PEER + password: GNtuK+/vTwZwSZiwj1FK/Q== + cv_settings: + onprem_clusters: + - name: primary + servers: + - name: 192.168.1.12 + terminattr: + disable_aaa: true + default_interfaces: + - downlink_interfaces: + - Ethernet1-8 + platforms: + - default + types: + - spine + uplink_interfaces: + - Ethernet1-2 + - downlink_interfaces: + - Ethernet8 + mlag_interfaces: + - Ethernet3-4 + platforms: + - default + types: + - l3leaf + uplink_interfaces: + - Ethernet1-2 + - platforms: + - default + types: + - l2leaf + uplink_interfaces: + - Ethernet1-2 + dns_settings: + servers: + - ip_address: 192.168.1.1 + eos_designs_documentation: + p2p_links_csv: true + topology_csv: true + fabric_ip_addressing: + loopback: + ipv6_prefix_length: 64 + mlag: + ipv6_prefix_length: 64 + p2p_uplinks: + ipv6_prefix_length: 64 + fabric_name: FABRIC + ntp_settings: + server_vrf: use_mgmt_interface_vrf + servers: + - name: 0.pool.ntp.org + overlay_routing_protocol: ebgp + underlay_ipv6: true + underlay_ipv6_numbered: true + underlay_routing_protocol: ebgp + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NetworkServiceSet +metadata: + name: network-services +spec: + design: + tenants: + - l2vlans: + - id: 3401 + name: L2_VLAN3401 + - id: 3402 + name: L2_VLAN3402 + mac_vrf_vni_base: 10000 + name: TENANT1 + vrfs: + - name: VRF10 + svis: + - enabled: true + id: 11 + ip_address_virtual: 10.10.11.1/24 + name: VRF10_VLAN11 + - enabled: true + id: 12 + ip_address_virtual: 10.10.12.1/24 + name: VRF10_VLAN12 + vrf_vni: 10 + vtep_diagnostic: + loopback: 10 + loopback_ip_range: 10.255.10.0/27 + - name: VRF11 + svis: + - enabled: true + id: 21 + ipv6_address_virtuals: + - 2001:DB8:21::1/48 + name: VRF11_VLAN21 + - enabled: true + id: 22 + ip_address_virtual: 10.10.22.1/24 + name: VRF11_VLAN22 + vrf_vni: 11 + vtep_diagnostic: + loopback: 11 + loopback_ip_range: 10.255.11.0/27 + - name: VRF12 + svis: + - enabled: true + id: 31 + ipv6_address_virtuals: + - 2001:DB8:31::1/48 + name: VRF12_VLAN31 + - enabled: true + id: 32 + ipv6_address_virtuals: + - 2001:DB8:32::1/48 + name: VRF12_VLAN32 + vrf_vni: 12 + vtep_diagnostic: + loopback: 12 + loopback_ip_range: 10.255.12.0/27 + appliesTo: + nodeSets: + - dc1-l2-leaves + - dc1-l3-leaves +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc1 +spec: + design: + management_eapi: + enabled: true + mgmt_gateway: 172.16.1.1 + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-l2-leaves +spec: + design: + l2leaf: + defaults: + platform: cEOSLab + spanning_tree_mode: mstp + node_groups: + - group: DC1_L2_LEAF1 + nodes: + - id: 1 + mgmt_ip: 172.16.1.151/24 + name: dc1-leaf1c + uplink_switches: + - dc1-leaf1a + - dc1-leaf1b + - group: DC1_L2_LEAF2 + nodes: + - id: 1 + mgmt_ip: 172.16.1.152/24 + name: dc1-leaf2c + uplink_switches: + - dc1-leaf2a + - dc1-leaf2b + type: l2leaf + declares: + - dc1-leaf1c + - dc1-leaf2c +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-l3-leaves +spec: + design: + l3leaf: + defaults: + loopback_ipv6_pool: 2001:DB8:1::/48 + mlag_peer_address_family: ipv6 + mlag_peer_ipv6_pool: 2001:DB8:3::/48 + mlag_peer_l3_ipv6_pool: 2001:DB8:4::/48 + platform: cEOSLab + router_id_pool: 10.255.1.0/24 + spanning_tree_mode: mstp + spanning_tree_priority: 4096 + uplink_ipv6_pool: 2001:DB8:2::/48 + uplink_switches: + - dc1-spine1 + - dc1-spine2 + virtual_router_mac_address: 00:1c:73:00:00:99 + vtep_loopback_ipv6_pool: 2001:DB8:5::/48 + node_groups: + - bgp_as: 65101 + group: DC1_L3_LEAF1 + nodes: + - id: 1 + mgmt_ip: 172.16.1.101/24 + name: dc1-leaf1a + - id: 2 + mgmt_ip: 172.16.1.102/24 + name: dc1-leaf1b + - bgp_as: 65102 + group: DC1_L3_LEAF2 + nodes: + - id: 3 + mgmt_ip: 172.16.1.103/24 + name: dc1-leaf2a + - id: 4 + mgmt_ip: 172.16.1.104/24 + name: dc1-leaf2b + type: l3leaf + declares: + - dc1-leaf1a + - dc1-leaf1b + - dc1-leaf2a + - dc1-leaf2b +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-spines +spec: + design: + spine: + defaults: + bgp_as: 65100 + loopback_ipv6_pool: 2001:DB8:0::/48 + platform: cEOSLab + router_id_pool: 10.255.0.0/24 + nodes: + - id: 1 + mgmt_ip: 172.16.1.11/24 + name: dc1-spine1 + - id: 2 + mgmt_ip: 172.16.1.12/24 + name: dc1-spine2 + type: spine + declares: + - dc1-spine1 + - dc1-spine2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: Fabric +metadata: + name: single-dc-l3ls-ipv6 +spec: + fabricName: FABRIC + requires: + - kind: ConnectedEndpointSet + name: connected-endpoints + - kind: SettingSet + name: fabric + - kind: NetworkServiceSet + name: network-services + - kind: SettingSet + name: dc1 + - kind: NodeSet + name: dc1-l2-leaves + - kind: NodeSet + name: dc1-l3-leaves + - kind: NodeSet + name: dc1-spines diff --git a/examples/fabric/inputs/single-dc-l3ls.yaml b/examples/fabric/inputs/single-dc-l3ls.yaml new file mode 100644 index 0000000..9153f82 --- /dev/null +++ b/examples/fabric/inputs/single-dc-l3ls.yaml @@ -0,0 +1,319 @@ +apiVersion: avd.netclab.dev/v1alpha1 +kind: ConnectedEndpointSet +metadata: + name: connected-endpoints +spec: + design: + servers: + - adapters: + - endpoint_ports: + - PCI1 + - PCI2 + mode: trunk + native_vlan: 4092 + port_channel: + endpoint_port_channel: Bond1 + mode: active + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + - Ethernet5 + switches: + - dc1-leaf1a + - dc1-leaf1b + vlans: 11-12,21-22 + - endpoint_ports: + - iLO + mode: access + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + switches: + - dc1-leaf1c + vlans: 11 + name: dc1-leaf1-server1 + - adapters: + - endpoint_ports: + - PCI1 + - PCI2 + mode: trunk + native_vlan: 4092 + port_channel: + mode: active + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + - Ethernet5 + switches: + - dc1-leaf2a + - dc1-leaf2b + vlans: 11-12,21-22 + - endpoint_ports: + - iLO + mode: access + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + switches: + - dc1-leaf2c + vlans: 11 + name: dc1-leaf2-server1 + appliesTo: + nodeSets: + - dc1-l2-leaves + - dc1-l3-leaves +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: fabric +spec: + design: + aaa_settings: + local_users: + - name: admin + no_password: true + privilege: 15 + role: network-admin + - name: arista + privilege: 15 + role: network-admin + sha512_password: $6$Enl0WfE32FthwyiJ$yTyGaEJ2uPKLU.F7314YtB7J1jrzrMi7ogXIRTEHQfLdLgKWWmr1UvNlZLN6AyuxET7G5aH3AI9OYRzxVTkB1. + bgp_peer_groups: + evpn_overlay_peers: + password: Q4fqtbqcZ7oQuKfuWtNGRQ== + ipv4_underlay_peers: + password: 7x4B4rnJhZB438m9+BrBfQ== + mlag_ipv4_underlay_peer: + password: 4b21pAdCvWeAqpcKDFMdWw== + cv_settings: + onprem_clusters: + - name: primary + servers: + - name: 192.168.1.12 + terminattr: + disable_aaa: true + default_interfaces: + - downlink_interfaces: + - Ethernet1-8 + platforms: + - default + types: + - spine + uplink_interfaces: + - Ethernet1-2 + - downlink_interfaces: + - Ethernet8 + mlag_interfaces: + - Ethernet3-4 + platforms: + - default + types: + - l3leaf + uplink_interfaces: + - Ethernet1-2 + - platforms: + - default + types: + - l2leaf + uplink_interfaces: + - Ethernet1-2 + dns_settings: + servers: + - ip_address: 192.168.1.1 + eos_designs_documentation: + p2p_links_csv: true + topology_csv: true + fabric_name: FABRIC + ntp_settings: + server_vrf: use_mgmt_interface_vrf + servers: + - name: 0.pool.ntp.org + overlay_routing_protocol: ebgp + underlay_routing_protocol: ebgp + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NetworkServiceSet +metadata: + name: network-services +spec: + design: + tenants: + - l2vlans: + - id: 3401 + name: L2_VLAN3401 + - id: 3402 + name: L2_VLAN3402 + mac_vrf_vni_base: 10000 + name: TENANT1 + vrfs: + - name: VRF10 + svis: + - enabled: true + id: 11 + ip_address_virtual: 10.10.11.1/24 + name: VRF10_VLAN11 + - enabled: true + id: 12 + ip_address_virtual: 10.10.12.1/24 + name: VRF10_VLAN12 + vrf_vni: 10 + vtep_diagnostic: + loopback: 10 + loopback_ip_range: 10.255.10.0/27 + - name: VRF11 + svis: + - enabled: true + id: 21 + ip_address_virtual: 10.10.21.1/24 + name: VRF11_VLAN21 + - enabled: true + id: 22 + ip_address_virtual: 10.10.22.1/24 + name: VRF11_VLAN22 + vrf_vni: 11 + vtep_diagnostic: + loopback: 11 + loopback_ip_range: 10.255.11.0/27 + appliesTo: + nodeSets: + - dc1-l2-leaves + - dc1-l3-leaves +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: dc1 +spec: + design: + management_eapi: + enabled: true + mgmt_gateway: 172.16.1.1 + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-l2-leaves +spec: + design: + l2leaf: + defaults: + platform: cEOSLab + spanning_tree_mode: mstp + node_groups: + - group: DC1_L2_LEAF1 + nodes: + - id: 1 + mgmt_ip: 172.16.1.151/24 + name: dc1-leaf1c + uplink_switches: + - dc1-leaf1a + - dc1-leaf1b + - group: DC1_L2_LEAF2 + nodes: + - id: 1 + mgmt_ip: 172.16.1.152/24 + name: dc1-leaf2c + uplink_switches: + - dc1-leaf2a + - dc1-leaf2b + type: l2leaf + declares: + - dc1-leaf1c + - dc1-leaf2c +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-l3-leaves +spec: + design: + l3leaf: + defaults: + loopback_ipv4_offset: 2 + loopback_ipv4_pool: 10.255.0.0/27 + mlag_peer_ipv4_pool: 10.255.1.64/27 + mlag_peer_l3_ipv4_pool: 10.255.1.96/27 + platform: cEOSLab + spanning_tree_mode: mstp + spanning_tree_priority: 4096 + uplink_ipv4_pool: 10.255.255.0/26 + uplink_switches: + - dc1-spine1 + - dc1-spine2 + virtual_router_mac_address: 00:1c:73:00:00:99 + vtep_loopback_ipv4_pool: 10.255.1.0/27 + node_groups: + - bgp_as: 65101 + group: DC1_L3_LEAF1 + nodes: + - id: 1 + mgmt_ip: 172.16.1.101/24 + name: dc1-leaf1a + - id: 2 + mgmt_ip: 172.16.1.102/24 + name: dc1-leaf1b + - bgp_as: 65102 + group: DC1_L3_LEAF2 + nodes: + - id: 3 + mgmt_ip: 172.16.1.103/24 + name: dc1-leaf2a + - id: 4 + mgmt_ip: 172.16.1.104/24 + name: dc1-leaf2b + type: l3leaf + declares: + - dc1-leaf1a + - dc1-leaf1b + - dc1-leaf2a + - dc1-leaf2b +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: dc1-spines +spec: + design: + spine: + defaults: + bgp_as: 65100 + loopback_ipv4_pool: 10.255.0.0/27 + platform: cEOSLab + nodes: + - id: 1 + mgmt_ip: 172.16.1.11/24 + name: dc1-spine1 + - id: 2 + mgmt_ip: 172.16.1.12/24 + name: dc1-spine2 + type: spine + declares: + - dc1-spine1 + - dc1-spine2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: Fabric +metadata: + name: single-dc-l3ls +spec: + fabricName: FABRIC + requires: + - kind: ConnectedEndpointSet + name: connected-endpoints + - kind: SettingSet + name: fabric + - kind: NetworkServiceSet + name: network-services + - kind: SettingSet + name: dc1 + - kind: NodeSet + name: dc1-l2-leaves + - kind: NodeSet + name: dc1-l3-leaves + - kind: NodeSet + name: dc1-spines diff --git a/examples/fabric/inputs/single-dc-multipod-l3ls.yaml b/examples/fabric/inputs/single-dc-multipod-l3ls.yaml new file mode 100644 index 0000000..7a44704 --- /dev/null +++ b/examples/fabric/inputs/single-dc-multipod-l3ls.yaml @@ -0,0 +1,315 @@ +apiVersion: avd.netclab.dev/v1alpha1 +kind: ConnectedEndpointSet +metadata: + name: connected-endpoints +spec: + design: + servers: + - adapters: + - mode: access + port_channel: + mode: active + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + - Ethernet5 + switches: + - dc1-leaf1a + - dc1-leaf1b + vlans: 10 + name: host1 + - adapters: + - mode: access + port_channel: + mode: active + spanning_tree_portfast: edge + switch_ports: + - Ethernet5 + - Ethernet5 + switches: + - dc1-leaf2a + - dc1-leaf2b + vlans: 20 + name: host2 + appliesTo: + hosts: + - dc1-leaf1a + - dc1-leaf1b + - dc1-leaf2a + - dc1-leaf2b +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NetworkServiceSet +metadata: + name: evpn-services +spec: + design: + tenants: + - mac_vrf_vni_base: 10000 + name: ACME + vrfs: + - name: VRF_A + svis: + - enabled: true + id: 10 + ip_address_virtual: 10.1.10.1/24 + name: DMZ + - enabled: true + id: 20 + ip_address_virtual: 10.1.20.1/24 + name: Internal + vrf_vni: 10 + appliesTo: + hosts: + - dc1-leaf1a + - dc1-leaf1b + - dc1-leaf2a + - dc1-leaf2b +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: SettingSet +metadata: + name: fabric +spec: + design: + aaa_settings: + local_users: + - name: admin + no_password: true + privilege: 15 + role: network-admin + - name: arista + privilege: 15 + role: network-admin + sha512_password: $6$Enl0WfE32FthwyiJ$yTyGaEJ2uPKLU.F7314YtB7J1jrzrMi7ogXIRTEHQfLdLgKWWmr1UvNlZLN6AyuxET7G5aH3AI9OYRzxVTkB1. + bfd_multihop: + interval: 1200 + min_rx: 1200 + multiplier: 3 + default_interfaces: + - downlink_interfaces: + - Ethernet1-4 + platforms: + - default + types: + - super-spine + - downlink_interfaces: + - Ethernet3-4 + platforms: + - default + types: + - spine + uplink_interfaces: + - Ethernet1-2 + - mlag_interfaces: + - Ethernet3-4 + platforms: + - default + types: + - l3leaf + uplink_interfaces: + - Ethernet1-2 + default_node_types: + - match_hostnames: + - .*-ss.* + node_type: super-spine + - match_hostnames: + - .*-spine.* + node_type: spine + - match_hostnames: + - .*-leaf.* + node_type: l3leaf + dns_settings: + servers: + - ip_address: 192.168.1.1 + eos_designs_documentation: + p2p_links_csv: true + topology_csv: true + evpn_vlan_aware_bundles: true + fabric_name: FABRIC + management_eapi: + enabled: true + mgmt_gateway: 172.16.1.1 + ntp_settings: + server_vrf: use_mgmt_interface_vrf + servers: + - name: 0.pool.ntp.org + appliesTo: + all: true +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: pod1 +spec: + design: + l3leaf: + defaults: + bgp_as: 65100-65199 + loopback_ipv4_pool: 192.168.101.0/24 + mlag_peer_ipv4_pool: 10.255.252.0/24 + mlag_peer_l3_ipv4_pool: 10.255.251.0/24 + platform: cEOSLab + spanning_tree_mode: mstp + spanning_tree_priority: 16384 + uplink_ipv4_pool: 192.168.103.0/24 + uplink_switches: + - dc1-spine1 + - dc1-spine2 + virtual_router_mac_address: 00:1c:73:00:00:99 + vtep_loopback_ipv4_pool: 192.168.102.0/24 + node_groups: + - group: dc1-leaf1 + nodes: + - id: 1 + mgmt_ip: 172.16.1.21/24 + name: dc1-leaf1a + uplink_switch_interfaces: + - Ethernet3 + - Ethernet3 + - id: 2 + mgmt_ip: 172.16.1.22/24 + name: dc1-leaf1b + uplink_switch_interfaces: + - Ethernet4 + - Ethernet4 + spine: + defaults: + bgp_as: 65001 + evpn_route_servers: + - dc1-ss1 + - dc1-ss2 + loopback_ipv4_pool: 192.168.101.0/24 + platform: cEOSLab + uplink_ipv4_pool: 192.168.103.0/24 + uplink_switches: + - dc1-ss1 + - dc1-ss2 + nodes: + - id: 11 + mgmt_ip: 172.16.1.11/24 + name: dc1-spine1 + uplink_switch_interfaces: + - Ethernet1 + - Ethernet1 + - id: 12 + mgmt_ip: 172.16.1.12/24 + name: dc1-spine2 + uplink_switch_interfaces: + - Ethernet2 + - Ethernet2 + declares: + - dc1-leaf1a + - dc1-leaf1b + - dc1-spine1 + - dc1-spine2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: pod2 +spec: + design: + l3leaf: + defaults: + bgp_as: 65100-65199 + loopback_ipv4_pool: 192.168.101.0/24 + mlag_peer_ipv4_pool: 10.255.252.0/24 + mlag_peer_l3_ipv4_pool: 10.255.251.0/24 + platform: cEOSLab + spanning_tree_mode: mstp + spanning_tree_priority: 16384 + uplink_ipv4_pool: 192.168.103.0/24 + uplink_switches: + - dc1-spine3 + - dc1-spine4 + virtual_router_mac_address: 00:1c:73:00:00:99 + vtep_loopback_ipv4_pool: 192.168.102.0/24 + node_groups: + - group: dc1-leaf2 + nodes: + - id: 3 + mgmt_ip: 172.16.1.23/24 + name: dc1-leaf2a + uplink_switch_interfaces: + - Ethernet3 + - Ethernet3 + - id: 4 + mgmt_ip: 172.16.1.24/24 + name: dc1-leaf2b + uplink_switch_interfaces: + - Ethernet4 + - Ethernet4 + spine: + defaults: + bgp_as: 65002 + evpn_route_servers: + - dc1-ss1 + - dc1-ss2 + loopback_ipv4_pool: 192.168.101.0/24 + platform: cEOSLab + uplink_ipv4_pool: 192.168.103.0/24 + uplink_switches: + - dc1-ss1 + - dc1-ss2 + nodes: + - id: 13 + mgmt_ip: 172.16.1.13/24 + name: dc1-spine3 + uplink_switch_interfaces: + - Ethernet3 + - Ethernet3 + - id: 14 + mgmt_ip: 172.16.1.14/24 + name: dc1-spine4 + uplink_switch_interfaces: + - Ethernet4 + - Ethernet4 + declares: + - dc1-leaf2a + - dc1-leaf2b + - dc1-spine3 + - dc1-spine4 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: NodeSet +metadata: + name: superspines +spec: + design: + super_spine: + defaults: + bgp_as: 65000 + evpn_role: server + loopback_ipv4_pool: 192.168.101.0/24 + platform: cEOSLab + nodes: + - id: 201 + mgmt_ip: 172.16.1.25/24 + name: dc1-ss1 + - id: 202 + mgmt_ip: 172.16.1.26/24 + name: dc1-ss2 + declares: + - dc1-ss1 + - dc1-ss2 +--- +apiVersion: avd.netclab.dev/v1alpha1 +kind: Fabric +metadata: + name: single-dc-multipod-l3ls +spec: + fabricName: FABRIC + requires: + - kind: ConnectedEndpointSet + name: connected-endpoints + - kind: NetworkServiceSet + name: evpn-services + - kind: SettingSet + name: fabric + - kind: NodeSet + name: pod1 + - kind: NodeSet + name: pod2 + - kind: NodeSet + name: superspines diff --git a/function/__init__.py b/function/__init__.py index 98fa0f1..6f462e3 100644 --- a/function/__init__.py +++ b/function/__init__.py @@ -1,11 +1,11 @@ """function-avd: a living AVD supermodel driven by Crossplane XRs. -Exposes the pyavd pipeline fed from AVD Ansible examples, which proves the -engine reproduces AVD's golden structured configs -- the same engine the -Crossplane composite function (`fn.py`) wraps. +Exposes the pyavd pipeline the Crossplane composite function (`fn.py`) wraps. +Nothing here reaches the migration harness: the runtime is handed input XRs and +never sees an inventory, so `ansible_cli` and `migrate` are imported by the +tools that need them and by nothing else. """ -from .ansible_inputs import build_all_inputs from .engine import render_structured_configs, validate_all -__all__ = ["build_all_inputs", "render_structured_configs", "validate_all"] +__all__ = ["render_structured_configs", "validate_all"] diff --git a/function/ansible_cli.py b/function/ansible_cli.py new file mode 100644 index 0000000..4e584a9 --- /dev/null +++ b/function/ansible_cli.py @@ -0,0 +1,357 @@ +"""Ansible, asked rather than reimplemented. + +Everything about an inventory that the migration needs is something Ansible +already states, so nothing here parses a ``group_vars`` tree, expands an +inventory pattern or opens a vault. Four questions, three subprocesses: + +``ansible-inventory --list --export`` + Which group carries which variable -- the ownership boundary an input XR + maps to. Inline ``vars:`` blocks, ``group_vars/.yml``, + ``group_vars//*.yml`` and their ``.yaml`` spellings are already merged + into one namespace per group, exactly as one XR carries one fragment. + +``ansible -m debug -a var=hostvars[inventory_hostname]`` + The merged per-host variables **after templating**. Not a source -- an + oracle. The migration layers the fragments itself and refuses to emit + anything unless the result equals this, so its precedence model can never be + silently wrong. + + An ad-hoc run rather than ``ansible-inventory --list`` because + ``ansible-inventory`` does not template and has no flag that makes it: + `{{ playbook_dir }}`, `{{ spine_bgp_defaults }}` and cv-pathfinder's + `bgp_password | arista.avd.encrypt` come out as literal strings, and pyavd + then meets a `Str` where it wants a `List`. A play templates; this is the + cheapest thing that is a play. `--list` remains available for a caller that + wants the raw text. + +``ansible-playbook --list-tasks`` / ``--list-hosts`` + Which play runs eos_designs, and which devices it runs on. AVD renders a + play, not an inventory, and Ansible resolves the host pattern with its own + engine -- including ``!`` exclusions and ``:&`` intersections, which reading + the playbook's ``hosts:`` string cannot do. + +Only the *harness* depends on this. The runtime takes XRs and never sees an +inventory, so ``ansible-core`` is a development dependency and reaches neither +the image nor the published package. +""" + +from __future__ import annotations + +import json +import os +import re +import shutil +import subprocess +import tempfile +from dataclasses import dataclass, field +from pathlib import Path + +ALL_GROUP = "all" +TIMEOUT = 900 + +#: Transport variables that make an ad-hoc `debug` fail although it never +#: connects. `ansible_connection` in the inventory outranks `-c`, and AVD's +#: examples set it to `ansible.netcommon.httpapi` with `become_method: enable`, +#: neither of which is installed here. Extra vars are the only precedence level +#: above inventory vars. +_ADHOC_OVERRIDES = ("ansible_connection=local", "ansible_become=false") + +# `play #3 (FABRIC): Build Configurations TAGS: []` +_PLAY = re.compile(r"^\s*play #(\d+) \((?P.*)\): (?P.*)\tTAGS:") +# ` arista.avd.eos_designs : Validate eos_designs inputs TAGS: []` +_TASK = re.compile(r"^\s+(?P[\w.]+) : .*\tTAGS:") + + +class AnsibleError(RuntimeError): + """An ansible CLI call failed. Carries what it printed.""" + + +@dataclass(frozen=True) +class Play: + """One play, as ``ansible-playbook`` reports it.""" + + playbook: str + index: int + name: str + pattern: str + hosts: tuple[str, ...] + roles: frozenset[str] + + @property + def runs_eos_designs(self) -> bool: + return any(role.endswith("eos_designs") for role in self.roles) + + +@dataclass +class Inventory: + """An inventory as Ansible describes it, at both levels.""" + + group_vars: dict[str, dict] = field(default_factory=dict) + host_vars: dict[str, dict] = field(default_factory=dict) + children: dict[str, set[str]] = field(default_factory=dict) + direct_hosts: dict[str, set[str]] = field(default_factory=dict) + #: merged per-host variables -- the oracle, never a source + hostvars: dict[str, dict] = field(default_factory=dict) + #: whether `hostvars` had its Jinja evaluated + templated: bool = False + + @property + def depth(self) -> dict[str, int]: + """Longest path from ``all``, which is what orders Ansible's groups. + + The one rule here that Ansible does not print. It is never trusted: + :func:`function.migrate.layer` reproduces ``hostvars`` with it, and the + migration refuses to emit XRs when that fails. + """ + out = {g: 0 for g in self.children} + changed = True + while changed: + changed = False + for parent, kids in self.children.items(): + for kid in kids: + if out.get(parent, 0) + 1 > out.get(kid, 0): + out[kid] = out[parent] + 1 + changed = True + return out + + def members(self, group: str) -> set[str]: + """Every host in the group, transitively.""" + seen: set[str] = set() + stack = [group] + hosts: set[str] = set() + while stack: + current = stack.pop() + if current in seen: + continue + seen.add(current) + hosts |= self.direct_hosts.get(current, set()) + stack.extend(self.children.get(current, ())) + return hosts + + def groups_of(self, host: str) -> list[str]: + """The host's groups, in Ansible's precedence order.""" + mine = {g for g in self.children if host in self.members(g)} | {ALL_GROUP} + depth = self.depth + return sorted(mine, key=lambda g: (depth.get(g, 0), g)) + + @property + def hosts(self) -> set[str]: + return self.members(ALL_GROUP) | set(self.host_vars) + + +def _strip(data: dict) -> dict: + """Drop Ansible's own transport variables -- not part of the AVD model.""" + return {k: v for k, v in data.items() if not k.startswith("ansible_")} + + +def _env(collections: Path | None) -> dict[str, str]: + env = dict(os.environ) + if collections is not None: + env["ANSIBLE_COLLECTIONS_PATH"] = str(collections) + return env + + +def _run(binary: str, args: list[str], cwd: Path, collections: Path | None) -> str: + """Run an ansible CLI in the inventory's own directory. + + The directory matters: ``ansible.cfg`` is read from the working directory, + and it is what points cv-pathfinder at its vault password file. + """ + path = shutil.which(binary) + if path is None: + raise AnsibleError( + f"{binary} not found. This is a development dependency; install it " + f"with `uv run --with ansible-core ...` or add it to the dev group." + ) + result = subprocess.run( + [path, *args], cwd=cwd, capture_output=True, text=True, + env=_env(collections), timeout=TIMEOUT, + ) + if result.returncode != 0: + raise AnsibleError( + f"{binary} {' '.join(args)} (in {cwd}) exited {result.returncode}:\n" + f"{(result.stderr or result.stdout).strip()[:2000]}" + ) + return result.stdout + + +def find_inventory(root: Path) -> Path: + """The inventory file, in either layout AVD ships. + + Absolute, always. Every call runs with ``cwd`` set to the inventory's own + directory, and a relative ``-i`` that does not resolve from there makes + ``ansible-inventory`` **exit 0 with an empty inventory** rather than fail -- + a silent wrong answer, not an error. + """ + root = Path(root).resolve() + for candidate in (root / "inventory.yml", root / "inventory" / "hosts.yml"): + if candidate.is_file(): + return candidate + raise AnsibleError(f"no inventory.yml or inventory/hosts.yml under {root}") + + +def templated_hostvars(root: Path, inventory: Path, collections: Path | None, + known: set[str]) -> dict[str, dict]: + """Per-host variables as a play sees them -- Jinja evaluated. + + ``--tree`` writes one JSON file per host, which is the only machine-readable + output an ad-hoc run offers. + + ⚠ **A value source, never a key source.** The dump carries Ansible's magic + variables (`groups`, `inventory_hostname`, `playbook_dir`, ...) beside the + inventory's own, and those are not part of any document. Rather than name + them -- a literal that would go stale the way every literal here has -- keep + only the keys the inventory itself declares, which ``--export`` already said. + """ + out: dict[str, dict] = {} + with tempfile.TemporaryDirectory() as tree: + _run( + "ansible", + ["all", "-i", str(inventory), "-m", "ansible.builtin.debug", + "-a", "var=hostvars[inventory_hostname]", + *[arg for override in _ADHOC_OVERRIDES for arg in ("-e", override)], + "--tree", tree], + root, collections, + ) + for path in sorted(Path(tree).iterdir()): + try: + body = json.loads(path.read_text()) + except (OSError, ValueError): + continue + data = body.get("hostvars[inventory_hostname]") + if isinstance(data, dict): + out[path.name] = {k: v for k, v in data.items() if k in known} + return out + + +def read_inventory(root: Path, inventory: Path | None = None, + collections: Path | None = None, templated: bool = True) -> Inventory: + """Both levels of the inventory, in two subprocesses. + + ``templated`` decides what the oracle is: what a play would hand AVD + (default), or the raw text ``ansible-inventory --list`` prints. Two calls + either way -- the templated oracle replaces ``--list`` rather than joining it. + """ + root = Path(root).resolve() + inventory = Path(inventory).resolve() if inventory else find_inventory(root) + args = ["-i", str(inventory), "--playbook-dir", str(root), "--list"] + export = json.loads(_run("ansible-inventory", [*args, "--export"], root, collections)) + merged = ( + {} if templated + else json.loads(_run("ansible-inventory", args, root, collections)) + ) + + inv = Inventory() + for group, body in export.items(): + if group == "_meta": + continue + inv.group_vars[group] = _strip(body.get("vars") or {}) + inv.children[group] = set(body.get("children") or []) + inv.direct_hosts[group] = set(body.get("hosts") or []) + for group in list(inv.children): + for kid in inv.children[group]: + inv.children.setdefault(kid, set()) + inv.direct_hosts.setdefault(kid, set()) + inv.children.setdefault(ALL_GROUP, set()) + + for host, data in (export.get("_meta", {}).get("hostvars") or {}).items(): + inv.host_vars[host] = _strip(data) + if templated: + known = {k for design in inv.group_vars.values() for k in design} + known |= {k for design in inv.host_vars.values() for k in design} + inv.hostvars = templated_hostvars(root, inventory, collections, known) + inv.templated = True + else: + for host, data in (merged.get("_meta", {}).get("hostvars") or {}).items(): + inv.hostvars[host] = _strip(data) + # A host carrying no variables at all is omitted by both; it still exists, + # and its merged view is empty. + for host in inv.hosts: + inv.hostvars.setdefault(host, {}) + return inv + + +def _parse_plays(text: str, playbook: str) -> dict[int, dict]: + """Split ``--list-hosts --list-tasks`` output into plays by index. + + The two flags combine, so one subprocess answers both questions: which + devices a play targets, and which roles it runs. Asking separately cost + twice as many processes and told us nothing more. + """ + plays: dict[int, dict] = {} + current: int | None = None + section: str | None = None + for line in text.splitlines(): + header = _PLAY.match(line) + if header: + current = int(header.group(1)) + section = None + plays[current] = { + "playbook": playbook, + "name": header.group("name").strip(), + "pattern": header.group("pattern"), + "hosts": [], + "tasks": [], + } + continue + if current is None: + continue + stripped = line.strip() + if stripped.startswith("hosts (") and stripped.endswith("):"): + section = "hosts" + continue + if stripped == "tasks:": + section = "tasks" + continue + if section == "hosts" and stripped and not stripped.endswith(":"): + plays[current]["hosts"].append(stripped) + elif section == "tasks": + task = _TASK.match(line) + if task: + plays[current]["tasks"].append(task.group("role")) + return plays + + +def plays(root: Path, collections: Path | None = None, + inventory: Path | None = None) -> list[Play]: + """Every play in every playbook beside the inventory, with hosts and roles. + + One subprocess per playbook. They can be passed together in a single call, + but a playbook that does not resolve then takes every other one down with + it -- AVD's examples ship `deploy.yml`, which needs the `arista.eos` + collection -- so the batch is not worth the failure it introduces. + + Playbooks that do not parse are skipped rather than fatal: a molecule + scenario keeps ``molecule.yml`` next to its playbooks and that is a config + file, not a play. + """ + root = Path(root).resolve() + inventory = Path(inventory).resolve() if inventory else find_inventory(root) + found: list[Play] = [] + for playbook in sorted(root.glob("*.yml")) + sorted(root.glob("*.yaml")): + if playbook.resolve() == inventory.resolve() or playbook.name == "molecule.yml": + continue + try: + output = _run( + "ansible-playbook", + ["-i", str(inventory), playbook.name, "--list-hosts", "--list-tasks"], + root, collections, + ) + except AnsibleError: + continue + for index, play in sorted(_parse_plays(output, playbook.name).items()): + found.append(Play( + playbook=playbook.name, + index=index, + name=play["name"], + pattern=play["pattern"], + hosts=tuple(play["hosts"]), + roles=frozenset(play["tasks"]), + )) + return found + + +def design_plays(root: Path, collections: Path | None = None, + inventory: Path | None = None) -> list[Play]: + """The plays that run eos_designs -- one fabric each.""" + return [p for p in plays(root, collections, inventory) if p.runs_eos_designs] diff --git a/function/ansible_inputs.py b/function/ansible_inputs.py deleted file mode 100644 index 2d79640..0000000 --- a/function/ansible_inputs.py +++ /dev/null @@ -1,148 +0,0 @@ -"""Reconstruct pyavd ``all_inputs`` from an AVD Ansible example directory. - -This replicates the parts of Ansible we rely on to feed :mod:`pyavd`: - -* ``inventory.yml`` provides the group hierarchy and host membership. -* ``group_vars//*.yml`` provide the layered AVD data model. - -Ansible merges group_vars **per host**, in precedence order (``all`` first, -then groups sorted by depth and name, host_vars last), with default -``hash_behaviour = replace`` (top-level keys override wholesale). We reproduce -exactly that so the resulting hostvars equal what ``ansible-playbook`` would -hand to AVD. - -The output is ``{hostname: hostvars}`` -- the ``all_inputs`` mapping consumed by -``pyavd.get_avd_facts``. -""" - -from __future__ import annotations - -from pathlib import Path - -import yaml - -ALL_GROUP = "all" - - -class _AnsibleLoader(yaml.SafeLoader): - """SafeLoader that tolerates Ansible-specific tags (e.g. ``!vault``). - - Vault-encrypted values are kept as opaque strings -- enough to parse the - file. Reproducing configs that embed them still needs the vault password. - """ - - -_AnsibleLoader.add_constructor( - "!vault", lambda loader, node: loader.construct_scalar(node) -) - - -def _yaml_load(path: Path): - return yaml.load(path.read_text(), Loader=_AnsibleLoader) or {} - - -class AnsibleInventory: - """Group hierarchy + host membership parsed from an ``inventory.yml`` tree.""" - - def __init__(self) -> None: - self.children: dict[str, set[str]] = {ALL_GROUP: set()} - self.direct_hosts: dict[str, set[str]] = {} - self.depth: dict[str, int] = {ALL_GROUP: 0} - - @classmethod - def from_file(cls, inventory_path: Path) -> "AnsibleInventory": - data = _yaml_load(inventory_path) - inv = cls() - if ALL_GROUP in data: - root = data[ALL_GROUP] or {} - else: - # No explicit `all:` root -> every top-level key is an implicit - # child group of `all` (standard Ansible inventory behaviour). - root = {"children": {k: v for k, v in data.items() if k != "_meta"}} - inv._walk(ALL_GROUP, root) - inv._compute_depths() - return inv - - def _walk(self, group: str, body: dict | None) -> None: - body = body or {} - self.children.setdefault(group, set()) - for host in (body.get("hosts") or {}): - self.direct_hosts.setdefault(group, set()).add(host) - for child, child_body in (body.get("children") or {}).items(): - self.children[group].add(child) - self._walk(child, child_body) - - def _compute_depths(self) -> None: - # Ansible depth = longest path from `all`. Iterate to a fixpoint. - for g in self.children: - self.depth.setdefault(g, 0) - changed = True - while changed: - changed = False - for parent, kids in self.children.items(): - for kid in kids: - d = self.depth[parent] + 1 - if d > self.depth.get(kid, 0): - self.depth[kid] = d - changed = True - - def hosts(self) -> set[str]: - return {h for hs in self.direct_hosts.values() for h in hs} - - def groups_for_host(self, host: str) -> list[str]: - """All groups the host belongs to (transitively), in Ansible merge order.""" - direct = {g for g, hs in self.direct_hosts.items() if host in hs} - groups: set[str] = {ALL_GROUP} - for g in direct: - groups.add(g) - groups |= self._ancestors(g) - return sorted(groups, key=lambda g: (self.depth.get(g, 0), g)) - - def _ancestors(self, group: str) -> set[str]: - parents = {p for p, kids in self.children.items() if group in kids} - result = set(parents) - for p in parents: - result |= self._ancestors(p) - return result - - -def _strip_ansible_keys(data: dict) -> dict: - """Drop Ansible transport vars (ansible_connection, ansible_user, ...). - - They are not part of the AVD data model and will not exist on our XRs. - """ - return {k: v for k, v in data.items() if not k.startswith("ansible_")} - - -def _load_group_vars(group_vars_dir: Path, group: str) -> dict: - """Merge every ``*.yml`` under ``group_vars//`` (alphabetically).""" - merged: dict = {} - dir_path = group_vars_dir / group - single_file = group_vars_dir / f"{group}.yml" - files: list[Path] = [] - if dir_path.is_dir(): - files = sorted(dir_path.glob("*.yml")) + sorted(dir_path.glob("*.yaml")) - elif single_file.is_file(): - files = [single_file] - for f in files: - data = _yaml_load(f) - merged.update(data) # hash_behaviour = replace - return merged - - -def build_all_inputs(example_dir: str | Path) -> dict[str, dict]: - """Return ``{hostname: hostvars}`` for a single-DC AVD example directory.""" - example_dir = Path(example_dir) - inventory = AnsibleInventory.from_file(example_dir / "inventory.yml") - group_vars_dir = example_dir / "group_vars" - - group_cache: dict[str, dict] = {} - all_inputs: dict[str, dict] = {} - for host in sorted(inventory.hosts()): - hostvars: dict = {} - for group in inventory.groups_for_host(host): - if group not in group_cache: - group_cache[group] = _load_group_vars(group_vars_dir, group) - hostvars.update(group_cache[group]) # replace semantics, in precedence order - all_inputs[host] = _strip_ansible_keys(hostvars) - return all_inputs diff --git a/function/avd_compat.py b/function/avd_compat.py new file mode 100644 index 0000000..aa1c352 --- /dev/null +++ b/function/avd_compat.py @@ -0,0 +1,59 @@ +"""AVD behaviours pyavd cannot reach, written as the classes AVD asks for. + +pyavd implements **no Jinja templating**: `get_device_structured_config` passes +`templar=None` and the call raises `NotImplementedError`, and no public entry +point accepts a templar. So a design pinning a `.j2` path cannot render here, +wherever the file is carried. + +The same schema blocks — `node_type_keys[].ip_addressing` and +`.interface_descriptions` — take `python_module` / `python_class_name` instead, +and **that route pyavd supports**: `load_python_class` imports the module by +dotted path and checks it against the public base class. A module shipped inside +this package is importable by dotted path, so pointing a design at +`function.avd_compat` loads no arbitrary code — it loads ours. + +This is not a general answer. It reproduces one specific, published scheme. +Anyone whose fabric uses a template of their own writes their own class and +builds their own function image on this one. +""" + +from __future__ import annotations + +import ipaddress + +from pyavd.api.ip_addressing import AvdIpAddressing + + +class AvdIpAddressingV2Spine(AvdIpAddressing): + """AVD v2.x spine-to-super-spine P2P addressing. + + A transcription of the two templates `eos_designs-twodc-5stage-clos` pins, + which say what they are: *"In AVD v2.x the spine to super-spine links used + this special IP addressing scheme. This file may still be used by older + inventories."* + + ⚠ The comment describes where they came from, not that they are inert. The + scheme divides the uplink pool by `max_uplink_switches` so that adding a + spine does not move existing addresses — which is why that fabric's golden + puts a spine's two super-spine uplinks 64 apart rather than adjacent. AVD's + native algorithm packs them contiguously and the two are not interchangeable. + + Everything else falls through to `AvdIpAddressing`, so a fabric selecting + this class changes only its P2P uplinks. + """ + + def _v2_p2p(self, uplink_switch_index: int, last: int) -> str: + pool = ipaddress.ip_network(self._uplink_ipv4_pool, strict=False) + offset = (self._id - 1) % self._max_parallel_uplinks + index = ( + (pool.num_addresses // self._max_uplink_switches) * int(uplink_switch_index) + + ((self._id - 1) * self._max_parallel_uplinks + offset) * 2 + + last + ) + return str(pool.network_address + index) + + def p2p_uplinks_ip(self, uplink_switch_index: int) -> str: + return self._v2_p2p(uplink_switch_index, 1) + + def p2p_uplinks_peer_ip(self, uplink_switch_index: int) -> str: + return self._v2_p2p(uplink_switch_index, 0) diff --git a/function/engine.py b/function/engine.py index e97970b..f91176a 100644 --- a/function/engine.py +++ b/function/engine.py @@ -85,19 +85,27 @@ def validate_all(all_inputs: dict[str, dict]) -> dict[str, list]: def render_structured_configs( - all_inputs: dict[str, dict], *, validate: bool = True + all_inputs: dict[str, dict], *, validate: bool = True, pool_manager: object = None ) -> dict[str, dict]: """Run facts + per-device structured config for the whole fabric. ``get_avd_facts`` is fabric-wide (needs every device at once); the structured config is then derived per device from those shared facts. + + ``pool_manager`` is required only by a fabric setting + ``fabric_numbering.node_id.algorithm: pool_manager``, which asks AVD to + assign node IDs from a pool instead of reading them off each node. ⚠ The + pool is **a file** (`pyavd.api.pool_manager.PoolManager(output_dir)`), and + the assignments have to survive between runs or every device is renumbered — + so nothing composes one yet, and a Fabric that asks for it fails with AVD's + own message until a Fabric has somewhere to keep it. """ if validate: violations = validate_all(all_inputs) if violations: raise InputValidationError(violations) - avd_facts = pyavd.get_avd_facts(all_inputs) + avd_facts = pyavd.get_avd_facts(all_inputs, pool_manager=pool_manager) return { hostname: pyavd.get_device_structured_config( hostname, inputs, avd_facts=avd_facts diff --git a/function/fn.py b/function/fn.py index 53897b5..b8cd86e 100644 --- a/function/fn.py +++ b/function/fn.py @@ -1,15 +1,27 @@ """Crossplane composite function for the AVD living model. -One function serves two composite kinds (dispatched on ``kind``): - -* ``Fabric`` -- runs the fabric-wide pyavd pipeline - (``engine.render_fabric_design``) and emits one ``Device`` XR per host, - carrying that device's structured config inline in ``spec.structuredConfig``. +One function serves six composite kinds (dispatched on ``kind``): + +* ``Fabric`` -- the design, either inline in ``spec.design`` or assembled from + the inputs ``spec.requires`` names. Runs the fabric-wide pyavd pipeline and + emits one ``Device`` XR per host, carrying that device's structured config + inline in ``spec.structuredConfig``. A fabric whose inputs have not arrived + renders nothing: a push replaces a device's whole config, so a premature + render reaches the switch. * ``Device`` -- validates and renders its own structured config (pyavd ``validate_structured_config`` + ``get_device_config``), reports per-device status/conditions, and emits a ConfigMap artifact (structured config + EOS CLI). With ``spec.push`` set it also composes a provider-http ``Request`` that keeps the device's running config in sync over eAPI (see push.py). +* ``NodeSet``, ``NetworkServiceSet``, ``ConnectedEndpointSet``, ``SettingSet`` + -- one category of design each. They compose nothing; a Fabric collects them + (see kinds.py). Their reconcile reports what the fragment contributes, on the + object its own team owns. + +Two things the transport loses on the way in, both restored here rather than +worked around downstream: protobuf ``Struct`` carries every number as a double +(``_normalize_numbers``), and an API server prunes an explicit null out of an +open field (``nulls``). The gRPC entrypoint lives in ``main.py`` (function-template-python layout). """ @@ -17,6 +29,7 @@ from __future__ import annotations import hashlib +import re from datetime import datetime, timezone import pyavd @@ -25,11 +38,20 @@ from crossplane.function.proto.v1 import run_function_pb2 as fnv1 from crossplane.function.proto.v1 import run_function_pb2_grpc as grpcv1 -from . import push +from . import nulls, pools, push from .engine import ( InputValidationError, device_roles_from_design, - render_fabric_design, + hostnames_from_design, + render_structured_configs, +) +from .kinds import ( + KINDS, + Input, + hosts_in_blocks, + overwrites, + resolve, + unmatched_patterns, ) API_VERSION = "avd.netclab.dev/v1alpha1" @@ -55,6 +77,19 @@ def _normalize_numbers(obj): return obj +def _dns_name(hostname: str) -> str: + """A hostname as a Kubernetes object name may spell it. + + AVD hostnames are free text and two of its eight bundled examples -- + `campus-fabric` and `l2ls-fabric` -- write them entirely in capitals. A + composed resource named after one is rejected outright: *"invalid name + ... Must be a valid RFC 1123 subdomain name"*, so the whole fabric fails to + compose. The hostname itself is untouched; only the object's name is spelled + this way. + """ + return re.sub(r"[^a-z0-9-]+", "-", hostname.lower()).strip("-") or "device" + + def _now() -> str: return datetime.now(timezone.utc).isoformat(timespec="seconds") @@ -94,10 +129,41 @@ async def RunFunction( # noqa: N802 (gRPC method name) self._reconcile_fabric(req, rsp, observed) elif kind == "Device": self._reconcile_device(req, rsp, observed) + elif kind in KINDS: + self._reconcile_input(rsp, observed) else: response.fatal(rsp, f"unsupported composite kind: {kind!r}") return rsp + # -- Inputs: a fragment of the design; they compose nothing --------------- + + def _reconcile_input(self, rsp: fnv1.RunFunctionResponse, observed: dict) -> None: + """Report what this fragment contributes, on its own object. + + An input composes nothing -- a Fabric collects it. This reconcile exists + so the team that owns the object sees its own shape here rather than + buried in someone else's Fabric status. + + It cannot report `status.devices`: an input does not know the fabric's + device list, so `appliesTo` only resolves where the inputs are collected. + The Fabric fills that in. Validation is deliberately not attempted + either -- whether pyavd can validate a fragment standalone is unsettled, + and a green validation that never ran is worse than none. + """ + spec = observed.get("spec") or {} + design = spec.get("design") or {} + status: dict = {"keys": sorted(design)} + + if observed.get("kind") == "NodeSet": + declared = spec.get("declares") + devices = set(declared) if declared is not None else hosts_in_blocks(design) + status["deviceCount"] = len(devices) + + resource.update_status(rsp.desired.composite, status) + response.normal( + rsp, f"{observed.get('kind')} contributes {len(design)} top-level key(s)" + ) + # -- Fabric: fabric-wide model -> one Device XR per host ------------------ def _reconcile_fabric( @@ -110,12 +176,64 @@ def _reconcile_fabric( namespace = meta.get("namespace", "default") xr_name = meta.get("name") or (fabric_name or "fabric").lower() - if not fabric_name or not design: - response.fatal(rsp, "spec.fabricName and spec.design are required") + requires = spec.get("requires") or [] + if not fabric_name or not (design or requires): + response.fatal(rsp, "spec.fabricName and spec.design or spec.requires are required") return + if requires: + all_inputs = self._collect(req, rsp, observed, requires, design, fabric_name) + if all_inputs is None: + return # gated -- _collect reported why + else: + # The released path: one fabric-wide document handed to every device, + # with AVD resolving roles from the node-type blocks. It never meets + # resolve(), so it restores its own explicit nulls. + document = nulls.restored(dict(design)) + document["fabric_name"] = fabric_name + all_inputs = {host: document for host in hostnames_from_design(document)} + + # A fabric that asks for pool-assigned node IDs needs its assignments + # back before it renders, or every device is renumbered on every pass. + keeps_a_pool = pools.wanted_by(all_inputs) + pool = "" try: - structured_configs = render_fabric_design(design, fabric_name) + if keeps_a_pool: + previous = pools.observed_pool(req.observed.resources) + if not previous.strip(): + # Only before this fabric has a pool of its own. Once it has, + # the seed is history and must not override it. + state, seeded = pools.seed(req, rsp, spec, namespace) + if state == "pending": + # The normal first pass, exactly as for the named inputs. + response.set_conditions( + rsp, + resource.Condition( + typ="InputsResolved", + status="False", + reason="WaitingForSeed", + message="waiting for the node-ID pool seed ConfigMap", + ), + ) + response.normal(rsp, "waiting for the node-ID pool seed") + return + if state == "missing": + response.fatal( + rsp, + "spec.nodeIdPool.seedConfigMapName names a ConfigMap that " + "does not exist; rendering without it would renumber every " + "device", + ) + return + previous = seeded + with pools.pool_manager(all_inputs, previous) as (manager, pool_file): + structured_configs = render_structured_configs( + all_inputs, pool_manager=manager + ) + manager.save_updated_pools() + pool = pool_file.read_text() if pool_file.is_file() else previous + else: + structured_configs = render_structured_configs(all_inputs) except InputValidationError as err: resource.update_status( rsp.desired.composite, @@ -127,6 +245,15 @@ def _reconcile_fabric( response.fatal(rsp, f"AVD render failed: {type(err).__name__}: {err}") return + if keeps_a_pool: + # Composed after the render, so a failed render never overwrites a + # good pool with a partial one. + resource.update( + rsp.desired.resources[pools.RESOURCE_NAME], + pools.configmap(xr_name, namespace, fabric_name, pool), + ) + rsp.desired.resources[pools.RESOURCE_NAME].ready = fnv1.READY_TRUE + push_spec = spec.get("push") or {} if push_spec and not push_spec.get("credentialsSecretName"): response.fatal(rsp, "spec.push.credentialsSecretName is required when push is set") @@ -139,7 +266,26 @@ def _reconcile_fabric( "urlTemplate", "https://{hostname}.{namespace}.svc/command-api" ) - roles = device_roles_from_design(design) + # Roles come from each device's own view: with inputs, a leaf in DC1 sees + # DC1's node-type block and nothing of DC2's. + roles = { + host: device_roles_from_design(hostvars).get(host) or hostvars.get("type", "") + for host, hostvars in all_inputs.items() + } + # Two hostnames that differ only in case, or only in a character a + # Kubernetes name cannot carry, would compose a single Device between + # them -- one config silently standing in for two switches. + spelled: dict[str, str] = {} + for hostname in sorted(structured_configs): + clash = spelled.setdefault(_dns_name(hostname), hostname) + if clash != hostname: + response.fatal( + rsp, + f"devices {clash!r} and {hostname!r} both need the object name " + f"{_dns_name(hostname)!r}; rename one", + ) + return + observed_devices = req.observed.resources # keyed by composition-resource-name (hostname) devices = [] for hostname, structured_config in structured_configs.items(): @@ -150,7 +296,7 @@ def _reconcile_fabric( "apiVersion": API_VERSION, "kind": "Device", "metadata": { - "name": resource.child_name(xr_name, hostname), + "name": resource.child_name(xr_name, _dns_name(hostname)), "namespace": namespace, "labels": { "avd.netclab.dev/fabric": fabric_name, @@ -195,6 +341,139 @@ def _reconcile_fabric( rsp, f"Composed {len(structured_configs)} Device(s) for fabric {fabric_name}" ) + # -- Collecting the inputs a Fabric names --------------------------------- + + def _collect( # noqa: PLR0913 + self, + req: fnv1.RunFunctionRequest, + rsp: fnv1.RunFunctionResponse, + observed: dict, + requires: list[dict], + design: dict, + fabric_name: str, + ) -> dict[str, dict] | None: + """Ask Crossplane for the named inputs; layer them once they arrive. + + Returns per-device inputs, or ``None`` when the fabric must not render -- + the gate. That gate is not only a guard against a slow operator: + requirements are answered on the *next* reconcile, so the first one + always arrives with nothing at all, and rendering then would push a + fabric short of its inputs as a full config replacement. + """ + namespace = (observed.get("metadata") or {}).get("namespace", "default") + + # State the requirements on every reconcile. Crossplane fetches what the + # latest response asked for, so leaving them out once drops the inputs. + keys: list[tuple[str, dict]] = [] + for index, entry in enumerate(requires): + kind = entry["kind"] + key = f"{index:03d}-{kind.lower()}-{entry['name']}" + keys.append((key, entry)) + response.require_resources( + rsp, + name=key, + api_version="v1" if kind == "Secret" else API_VERSION, + kind=kind, + match_name=entry["name"], + namespace=entry.get("namespace", namespace), + ) + + pending: list[str] = [] + absent: list[str] = [] + inputs: list[Input] = [] + for key, entry in keys: + named = f"{entry['kind']}/{entry.get('namespace', namespace)}/{entry['name']}" + if entry["kind"] == "Secret": + # In the API from the first version so the mechanism can land + # without a schema change, but not implemented. Refuse rather + # than render a fabric whose credentials are silently absent. + response.fatal(rsp, f"Secret inputs are not implemented yet: {named}") + return None + if key not in req.required_resources: + pending.append(named) + continue + items = req.required_resources[key].items + if not items: + # Crossplane looked and found nothing. The proto distinguishes + # this from "not fetched yet" by sending an empty Resources, and + # that is what lets a Fabric tell "waiting" from "missing" -- + # the one thing this design was previously unable to do. + absent.append(named) + continue + inputs.append( + Input.from_xr(_normalize_numbers(resource.struct_to_dict(items[0].resource))) + ) + + if pending or absent: + detail = [] + if absent: + detail.append(f"not found: {', '.join(absent)}") + if pending: + detail.append(f"not fetched yet: {', '.join(pending)}") + message = "; ".join(detail) + response.set_conditions( + rsp, + resource.Condition( + typ="InputsResolved", + status="False", + reason="InputsMissing" if absent else "WaitingForInputs", + message=message[:400], + ), + ) + resource.update_status( + rsp.desired.composite, + {"fabricName": fabric_name, "validation": {"ok": False, "message": message}}, + ) + # Missing is a real problem; not-fetched-yet is the normal first pass. + report = response.warning if absent else response.normal + report(rsp, f"fabric {fabric_name} is waiting on inputs -- {message}") + return None + + # The Fabric's own design is the first input: fabric-wide, seen by every + # device, and declaring whatever devices its own blocks name so a Fabric + # that carries both a design and a requires list still has its devices. + document = dict(design) + document["fabric_name"] = fabric_name + inputs.insert( + 0, + Input( + name="fabric", + kind="SettingSet", + design=document, + all_devices=True, + declares=sorted(hosts_in_blocks(document)), + ), + ) + + if stray := unmatched_patterns(inputs): + listed = ", ".join(f"{name}: {pattern!r}" for name, pattern in stray) + response.fatal( + rsp, + f"appliesTo.matchHostnames matched no device ({listed}) -- " + f"a pattern that matches nothing is silent, so it is refused", + ) + return None + + if replaced := overwrites(inputs): + shown = ", ".join(f"{key} on {host} ({first} -> {second})" + for host, key, first, second in replaced[:5]) + response.warning( + rsp, + f"{len(replaced)} value(s) replaced by a later input: {shown}" + + (" ..." if len(replaced) > 5 else ""), + ) + + response.set_conditions( + rsp, + resource.Condition( + typ="InputsResolved", + status="True", + reason="AllInputsResolved", + message=f"{len(inputs)} input(s)", + ), + ) + return resolve(inputs) + # -- Device: validate + render one device's config ----------------------- def _reconcile_device( diff --git a/function/kinds.py b/function/kinds.py new file mode 100644 index 0000000..b92794d --- /dev/null +++ b/function/kinds.py @@ -0,0 +1,332 @@ +"""The input-kind model: several XRs layered into per-device AVD inputs. + +A ``Fabric`` names its inputs in ``spec.requires``. Each input XR carries a +fragment of the eos_designs document in ``spec.design`` plus ``spec.appliesTo`` +saying which devices see it. Per device, the inputs that apply are layered in +``requires`` order with ``dict.update()`` -- Ansible's default +``hash_behaviour=replace``, which is what group_vars resolution does and what +``pyavd.get_avd_facts`` expects to be handed. + +**Nothing is merged.** Two NodeSets carrying the same node-type key never meet, +because no device sees both: in a dual-DC fabric a DC1 leaf sees DC1's +``l3leaf.defaults`` and a DC2 leaf sees DC2's. So there is no fabric-wide +document to assemble and no conflict to resolve. + +Two things are separate that look like one: + +* which devices an input *declares* (``spec.declares``, plus the nodes its + blocks name) -- the union of these is the fabric's device list, and there is + no second list; +* which devices *see* it (``spec.appliesTo``). They coincide in simple + topologies and diverge in a 5-stage CLOS, where a DC's ``super_spine`` block + names four devices but is visible to every device of that DC. + +Measured against AVD's own corpus: the hostvars this produces are byte-identical +to what **Ansible itself** reports -- all 8 bundled examples and 19 molecule +scenarios, up to 501 devices in one play. :mod:`function.migrate` builds the +inputs and :mod:`function.ansible_cli` supplies the reference. +""" + +from __future__ import annotations + +import re +from dataclasses import dataclass, field +from functools import lru_cache +from typing import Any + +from . import nulls + +KINDS = ("NodeSet", "NetworkServiceSet", "ConnectedEndpointSet", "SettingSet") + + +def matches(pattern: str, hostname: str) -> bool: + """AVD's own hostname-matching semantics, copied from the code not the docs. + + ``shared_utils/node_type.py`` resolves ``default_node_types`` with + ``search(f"^{regex}$", hostname)`` -- **AVD anchors the pattern for you**, so + ``dc1-leaf.*`` matches a whole name. The schema's description reads as though + the author must anchor it; the code does it for them. Copying the description + instead of the code would make the same pattern mean different things in the + two places. + """ + return re.search(f"^{pattern}$", hostname) is not None + + +def is_node_block(value: Any) -> bool: + """A node-type block is a dict carrying ``nodes`` and/or ``node_groups``.""" + return isinstance(value, dict) and ("nodes" in value or "node_groups" in value) + + +def hosts_in_blocks(design: dict) -> set[str]: + """Device names a design's node-type blocks mention.""" + hosts: set[str] = set() + for value in design.values(): + if not is_node_block(value): + continue + groups = list(value.get("node_groups") or []) + for nodes in [value.get("nodes") or []] + [g.get("nodes") or [] for g in groups]: + for node in nodes: + if isinstance(node, dict) and node.get("name"): + hosts.add(node["name"]) + return hosts + + +@lru_cache(maxsize=1) +def _default_vocabulary() -> "Vocabulary": + """The dynamic key names AVD invents when a document says nothing. + + Read from pyavd's own public schema rather than copied into a literal: the + three generators ship defaults (13 node types, 12 endpoint kinds, `tenants`), + and a hand-kept copy goes stale silently. It already had -- `cameras` was + added upstream and the literal this replaces never grew it. + """ + from pyavd.api.schemas import AVDDesign + + design = AVDDesign() + return Vocabulary( + node_types=frozenset(e.key for e in design.node_type_keys), + network_services=frozenset(e.name for e in design.network_services_keys), + connected_endpoints=frozenset(e.key for e in design.connected_endpoints_keys), + ) + + +@dataclass(frozen=True) +class Vocabulary: + """The top-level key names in force for a document. + + eos_designs generates key names from its own content -- `node_type_keys`, + `network_services_keys` and `connected_endpoints_keys` each name a family of + top-level keys. So no static map can classify every key, and this is that + map made per document instead: AVD's defaults, extended by whatever + generators the document carries. + """ + + node_types: frozenset[str] + network_services: frozenset[str] + connected_endpoints: frozenset[str] + + @classmethod + def default(cls) -> "Vocabulary": + return _default_vocabulary() + + def extend(self, design: dict) -> "Vocabulary": + """Add the key names this document's own generators declare.""" + + def named(source: str, field_name: str) -> frozenset[str]: + entries = design.get(source) + if not isinstance(entries, list): + return frozenset() + return frozenset( + str(e[field_name]) for e in entries + if isinstance(e, dict) and e.get(field_name) + ) + + return Vocabulary( + node_types=self.node_types | named("node_type_keys", "key") + | named("custom_node_type_keys", "key"), + network_services=self.network_services | named("network_services_keys", "name"), + connected_endpoints=self.connected_endpoints + | named("connected_endpoints_keys", "key") + | named("custom_connected_endpoints_keys", "key"), + ) + + +# The keys eos_designs names itself that are *not* settings. Everything else in +# its schema is, so only the exceptions are listed -- and `test_categories` +# checks each one against `documentation_options.table` in AVD's own schema, so +# an upstream recategorisation fails the suite instead of drifting. +_NODE_SET_KEYS = frozenset({ + "type", # table: type-setting -- the device's node type + "node_type_keys", # table: node-type-keys -- names the node families + "custom_node_type_keys", + "l3_interface_profiles", # table: node-type-l3-interfaces-configuration +}) +_NETWORK_SERVICE_KEYS = frozenset({ + "network_services", # table: network-services + "network_services_keys", + "evpn_vlan_bundles", # table: evpn-vlan-bundles + "l2vlan_profiles", # table: network-services-l2vlans-settings + "mlag_ibgp_peering_vrfs", # table: network-services-vrfs-settings +}) +_CONNECTED_ENDPOINT_KEYS = frozenset({ + "connected_endpoints_keys", # table: connected-endpoints-keys + "custom_connected_endpoints_keys", + "default_connected_endpoints_description", + "default_connected_endpoints_port_channel_description", + "default_network_ports_description", + "default_network_ports_port_channel_description", + # AVD tags neither of these; they are endpoint content by their own reading + # and this repo places them here. Nothing upstream contradicts it. + "port_profiles", + "network_ports", +}) + + +def kind_of(key: str, value: Any, vocabulary: "Vocabulary | None" = None) -> str: + """Which kind one top-level key belongs to.""" + vocabulary = vocabulary or Vocabulary.default() + if key in vocabulary.node_types or key in _NODE_SET_KEYS or is_node_block(value): + return "NodeSet" + if key in vocabulary.network_services or key in _NETWORK_SERVICE_KEYS: + return "NetworkServiceSet" + if key in vocabulary.connected_endpoints or key in _CONNECTED_ENDPOINT_KEYS: + return "ConnectedEndpointSet" + return "SettingSet" + + +def by_kind(design: dict, vocabulary: "Vocabulary | None" = None) -> dict[str, dict]: + """Partition a fragment into ``{kind: design}``, keys in their own order.""" + vocabulary = (vocabulary or Vocabulary.default()).extend(design) + parts: dict[str, dict] = {} + for key, value in design.items(): + parts.setdefault(kind_of(key, value, vocabulary), {})[key] = value + return {kind: parts[kind] for kind in KINDS if kind in parts} + + +def classify(design: dict, vocabulary: "Vocabulary | None" = None) -> str: + """The kind a whole fragment belongs to: the one holding most of its keys. + + Advisory. The kinds exist for ownership -- RBAC is granted per kind -- not + as a partition a schema could enforce, because eos_designs' top-level key + names come from its own content. A fragment spanning categories is split by + :func:`by_kind` rather than resolved by this. + """ + parts = by_kind(design, vocabulary) + if not parts: + return "SettingSet" + return max(parts, key=lambda kind: len(parts[kind])) + + +@dataclass +class Input: + """One input XR, reduced to what resolution needs.""" + + name: str + kind: str + design: dict + # spec.appliesTo -- the criteria are unioned; none set means every device + all_devices: bool = False + node_sets: list[str] = field(default_factory=list) + hosts: list[str] = field(default_factory=list) + match_hostnames: list[str] = field(default_factory=list) + # spec.declares -- devices this input brings into the fabric. Never a + # pattern: visibility may be matched, existence may not. A typo in a pattern + # would silently drop devices from the fabric. + declares: list[str] = field(default_factory=list) + + @classmethod + def from_xr(cls, xr: dict) -> "Input": + """Build from an XR as ``required_resources`` delivers it.""" + spec = xr.get("spec") or {} + applies = spec.get("appliesTo") or {} + design = spec.get("design") or {} + declares = list(spec.get("declares") or []) + kind = xr.get("kind") or classify(design) + if kind == "NodeSet" and not declares: + # A NodeSet that declares nothing explicitly declares what its + # blocks name -- the common case, where the two coincide. + declares = sorted(hosts_in_blocks(design)) + return cls( + name=(xr.get("metadata") or {}).get("name", ""), + kind=kind, + design=design, + all_devices=bool(applies.get("all")), + node_sets=list(applies.get("nodeSets") or []), + hosts=list(applies.get("hosts") or []), + match_hostnames=list(applies.get("matchHostnames") or []), + declares=declares, + ) + + def scope(self, declared_by: dict[str, set[str]], devices: set[str]) -> set[str]: + """Devices that see this input. The criteria are unioned. + + Omitting ``appliesTo`` means the whole fabric -- except on a ``NodeSet``, + where it means the devices that NodeSet declares. A node-type block seen + fabric-wide is not a thing Ansible can express: a ``group_vars`` file is + read by its group. Every NodeSet in AVD's 8 examples is scoped to exactly + what it declares, 26 of 26, so the default carries the common case and + ``appliesTo`` is left to say the uncommon one -- which is real: in a + 5-stage CLOS a DC's ``super_spine`` block declares 4 devices and is seen + by all 16 of that DC. + """ + if self.all_devices: + return devices + if not (self.node_sets or self.hosts or self.match_hostnames): + return devices & set(self.declares) if self.kind == "NodeSet" else devices + named: set[str] = set() + for name in self.node_sets: + named |= declared_by.get(name, set()) + named |= set(self.hosts) + named |= {h for h in devices for p in self.match_hostnames if matches(p, h)} + return devices & named + + +def resolve(inputs: list[Input]) -> dict[str, dict]: + """Layer ordered inputs into ``{hostname: hostvars}`` for ``get_avd_facts``. + + List order is precedence order: later inputs overwrite earlier ones key by + key, whole-key, exactly as Ansible resolves group_vars. An overwrite is + therefore intentional -- it is what the fabric owner declared by ordering -- + and belongs on status as a warning, never as an error. + + Explicit nulls come back here, once the layering is done: an input carries + them as :data:`nulls.MARKER`, because an API server prunes a real null out of + an open field. Restoring after the layering rather than before keeps the + marker comparable like any other value -- a later input overwriting a null + with a value, or the other way round, works because both are just values + until this point. + """ + declared_by = {i.name: set(i.declares) for i in inputs if i.declares} + devices: set[str] = set() + for hosts in declared_by.values(): + devices |= hosts + + out: dict[str, dict] = {host: {} for host in devices} + for inp in inputs: + for host in inp.scope(declared_by, devices): + out[host].update(inp.design) + return {host: nulls.restored(document) for host, document in out.items()} + + +def unmatched_patterns(inputs: list[Input]) -> list[tuple[str, str]]: + """``(input name, pattern)`` for every ``matchHostnames`` entry matching no device. + + A pattern is silent in both directions: a typo matches nothing and the input + quietly reaches no device, while a wide pattern quietly reaches devices it + was not meant to. The second is visible on status (`devices`); the first is + not, so the caller is expected to treat this as an error and refuse to + render -- the render is pushed as a full config replacement. + """ + devices: set[str] = set() + for inp in inputs: + devices |= set(inp.declares) + return [ + (inp.name, pattern) + for inp in inputs + for pattern in inp.match_hostnames + if not any(matches(pattern, host) for host in devices) + ] + + +def overwrites(inputs: list[Input]) -> list[tuple[str, str, str, str]]: + """``(device, key, earlier input, later input)`` for every value replaced. + + Ansible resolves these silently. Here the ordering is written down by a + person, so surfacing them is cheap and worth doing -- on status, as a + warning. + """ + declared_by = {i.name: set(i.declares) for i in inputs if i.declares} + devices: set[str] = set() + for hosts in declared_by.values(): + devices |= hosts + + seen: dict[tuple[str, str], tuple[str, Any]] = {} + found: list[tuple[str, str, str, str]] = [] + for inp in inputs: + for host in inp.scope(declared_by, devices): + for key, value in inp.design.items(): + previous = seen.get((host, key)) + if previous is not None and previous[1] != value: + found.append((host, key, previous[0], inp.name)) + seen[(host, key)] = (inp.name, value) + return found diff --git a/function/migrate.py b/function/migrate.py new file mode 100644 index 0000000..a73f94f --- /dev/null +++ b/function/migrate.py @@ -0,0 +1,849 @@ +"""Turn an AVD Ansible inventory into a Fabric and its input XRs. + +The translation reads nothing itself: :mod:`function.ansible_cli` asks Ansible +which group carries which variable, which devices a play runs on, and what the +merged result is. This module only decides how those fragments become XRs. + +Four rules, and the last is what makes the others safe: + +* **One fabric per play.** AVD renders a play, not an inventory. Where the two + differ the inventory is wider -- cv-pathfinder carries `cloudvision`, an API + server that is not a switch -- and a declared device with no node type fails + the *whole* fabric's render, not just its own. +* **One input per ownership fragment per category.** A group's variables are one + fragment because Ansible merges them into one namespace; splitting them by + file would preserve a boundary Ansible does not keep. Splitting the merged + fragment by category afterwards costs nothing and is what the kinds are for. +* **Values are what the play produced, not what the file says.** An XR has no + templating engine, no vault and no playbook directory, so anything Ansible + resolves at play time is resolved before it is written down. See + :func:`templated`. +* **The precedence model is checked, not trusted.** Group order is (depth, name), + the one rule the Ansible CLI does not print. :func:`migrate` layers the + fragments with it and refuses to emit anything unless the result equals the + per-device variables Ansible reports. A wrong order cannot reach an XR. +""" + +from __future__ import annotations + +import json +import re +from dataclasses import dataclass, field +from pathlib import Path + +from . import nulls +from .ansible_cli import ALL_GROUP, Inventory, Play, design_plays, plays, read_inventory +from .kinds import Input, KINDS, Vocabulary, by_kind, hosts_in_blocks, is_node_block + +#: kinds whose fragment gets a name suffix; a NodeSet keeps the plain name +SUFFIX = { + "NetworkServiceSet": "services", + "ConnectedEndpointSet": "endpoints", + "SettingSet": "settings", +} + + +class MigrationError(RuntimeError): + """The migration could not produce inputs it is able to stand behind.""" + + +def slug(name: str) -> str: + """A Kubernetes object name from an Ansible group or host name. + + Ansible group names are conventionally SHOUTED and use underscores; neither + survives RFC 1123. Collisions are the caller's to detect -- see + :func:`_unique`. + """ + out = re.sub(r"[^a-z0-9-]+", "-", name.lower()).strip("-") + return out or "unnamed" + + +@dataclass +class Fabric: + """One play, translated.""" + + name: str + devices: tuple[str, ...] + inputs: list[Input] = field(default_factory=list) + play: Play | None = None + #: AVD's own `fabric_name`, promoted to `spec.fabricName` + fabric_name: str = "" + #: things the XRs cannot express, said out loud rather than dropped + notes: list[str] = field(default_factory=list) + #: the pool's assignments, when asked to carry them (--emit-pool-seed) + pool_seed: str = "" + + @property + def seed_name(self) -> str: + """The ConfigMap the emitted Fabric seeds its pool from.""" + return f"{self.name}-id-seed" + + @property + def requires(self) -> list[tuple[str, str]]: + """``spec.requires`` as (kind, name), in precedence order.""" + return [(i.kind, i.name) for i in self.inputs] + + +@dataclass(frozen=True) +class Fragment: + """One ownership unit: a group's variables, or a host's.""" + + name: str + design: dict + #: devices that see it + scope: frozenset[str] + + +def _fragments(inv: Inventory, devices: frozenset[str]) -> list[Fragment]: + """Every fragment that reaches a device, in Ansible's precedence order. + + `all` first, then groups by (depth, name), then host variables. That is a + global total order, so restricting one flat list per device reproduces that + device's view -- which is why `spec.requires` can be a single list. + """ + depth = inv.depth + ordered = sorted( + (g for g, v in inv.group_vars.items() if v), + key=lambda g: (depth.get(g, 0), g), + ) + out = [ + Fragment(group, inv.group_vars[group], + frozenset(devices if group == ALL_GROUP else inv.members(group) & devices)) + for group in ordered + ] + out += [ + Fragment(host, inv.host_vars[host], frozenset({host})) + for host in sorted(inv.host_vars) + if host in devices and inv.host_vars[host] + ] + return out + + +def _winners(fragments: list[Fragment]) -> dict[tuple[int, str], set[str]]: + """``(fragment index, key) -> devices where that fragment's value survives``. + + Layering is last-wins, so a fragment's own value is only observable on the + devices no later fragment overrides it for. That is exactly where a + templated value may be read back from the play's output. + """ + last: dict[str, dict[str, int]] = {} + for index, fragment in enumerate(fragments): + for device in fragment.scope: + for key in fragment.design: + last.setdefault(device, {})[key] = index + out: dict[tuple[int, str], set[str]] = {} + for device, keys in last.items(): + for key, index in keys.items(): + out.setdefault((index, key), set()).add(device) + return out + + +def templated(fragments: list[Fragment], hostvars: dict[str, dict]) -> list[Fragment]: + """Replace each fragment value with what the play actually produced. + + An XR has no templating engine and no playbook directory, so whatever + Ansible resolves at play time has to be resolved before the value is + written down. The trigger is **not** "does this look like Jinja" -- it is + "did the play produce something else", which needs no pattern and catches + every mechanism at once: + + * Jinja -- `{{ spine_bgp_defaults }}` reaches pyavd as a `Str` where AVD + wants a `List`; + * ansible-vault -- ⚠ `ansible-inventory` does **not** decrypt. Both `--list` + and `--export` emit `{"__ansible_vault": "$ANSIBLE_VAULT;1.1;AES256..."}`, + which looks resolved in a diff and is not. A play decrypts it. + + Two rules keep it honest: + + * a value is only read back on devices where **this** fragment wins the key + (:func:`_winners`); reading it off a device some later fragment overrode + would copy the wrong fragment's value; + * measured over AVD's whole corpus, no group-level value resolves to + different things on different devices. An input XR carries one value for + many devices, so if that ever stops holding the fragment is not + expressible and this refuses rather than picking one. + """ + winners = _winners(fragments) + out: list[Fragment] = [] + for index, fragment in enumerate(fragments): + design = dict(fragment.design) + for key, value in fragment.design.items(): + raw = json.dumps(value, sort_keys=True, default=str) + seen: dict[str, object] = {} + for device in sorted(winners.get((index, key), ())): + if key in hostvars.get(device, {}): + seen[json.dumps(hostvars[device][key], sort_keys=True, default=str)] = ( + hostvars[device][key] + ) + if len(seen) > 1: + raise MigrationError( + f"{fragment.name}.{key} resolves to {len(seen)} different values " + f"across the devices that see it; one input XR carries one value" + ) + if seen and next(iter(seen)) != raw: + design[key] = next(iter(seen.values())) + out.append(Fragment(fragment.name, design, fragment.scope)) + return out + + +def layer(fragments: list[Fragment], devices: frozenset[str]) -> dict[str, dict]: + """Apply the fragments in order -- Ansible's ``hash_behaviour=replace``.""" + out: dict[str, dict] = {device: {} for device in devices} + for fragment in fragments: + for device in fragment.scope: + out[device].update(fragment.design) + return out + + +def _node_owner(fragments: list[Fragment], devices: frozenset[str], + vocabulary: Vocabulary) -> dict[str, str]: + """Which fragment declares each device: the narrowest one with node content. + + "Narrowest" is last in precedence order, which is what depth already sorts + by. A device typed only through `default_node_types` matches no fragment and + is left for the caller to place. + """ + owner: dict[str, str] = {} + for fragment in fragments: + parts = by_kind(fragment.design, vocabulary) + if "NodeSet" not in parts: + continue + named = hosts_in_blocks(fragment.design) & devices + for device in fragment.scope | named: + if device in devices: + owner[device] = fragment.name + return owner + + +def _unique(name: str, taken: set[str]) -> str: + candidate, n = name, 2 + while candidate in taken: + candidate, n = f"{name}-{n}", n + 1 + taken.add(candidate) + return candidate + + +def _applies_to(inp: Input, scope: frozenset[str], devices: frozenset[str], + declared_by: dict[str, set[str]]) -> None: + """Say which devices see the input, as narrowly as it can be said. + + Silence means the whole fabric -- except on a NodeSet, where it means the + devices it declares. Both defaults carry the common case, so `appliesTo` + appears only where the answer is unusual: a DC-wide node block seen by more + devices than it declares. + """ + if inp.kind == "NodeSet" and scope == set(inp.declares): + return + if scope == devices: + inp.all_devices = True + return + cover = sorted(n for n, hosts in declared_by.items() if hosts and hosts <= scope) + covered: set[str] = set() + for name in cover: + covered |= declared_by[name] + if covered == scope: + inp.node_sets = cover + else: + inp.hosts = sorted(scope) + + +def _inputs(fragments: list[Fragment], devices: frozenset[str], + vocabulary: Vocabulary) -> list[Input]: + """Every input XR for one fabric, in precedence order. + + Two passes, because `appliesTo` may name a NodeSet that comes later in the + order: a group at depth 1 is commonly scoped to node sets defined at depth 2. + """ + owner = _node_owner(fragments, devices, vocabulary) + declares: dict[str, set[str]] = {} + for device, name in owner.items(): + declares.setdefault(name, set()).add(device) + + taken: set[str] = set() + declared_by: dict[str, set[str]] = {} + planned: list[tuple[Fragment, Input]] = [] + + # A device no fragment types is reached by `default_node_types`, by pattern. + # It still has to exist, so give it a NodeSet of its own, named after the + # narrowest fragment that reaches it. + orphans: dict[str, set[str]] = {} + for device in sorted(devices - set(owner)): + reaching = [f.name for f in fragments if device in f.scope] + orphans.setdefault(reaching[-1] if reaching else device, set()).add(device) + for base, hosts in sorted(orphans.items()): + name = _unique(f"{slug(base)}-devices", taken) + inp = Input(name, "NodeSet", {}, declares=sorted(hosts)) + declared_by[name] = set(hosts) + planned.append((Fragment(base, {}, frozenset(hosts)), inp)) + + for fragment in fragments: + parts = by_kind(fragment.design, vocabulary) + if not parts: + continue + base = slug(fragment.name) + for kind, design in parts.items(): + plain = len(parts) == 1 or kind == "NodeSet" + name = _unique(base if plain else f"{base}-{SUFFIX[kind]}", taken) + inp = Input(name=name, kind=kind, design=design) + if kind == "NodeSet": + inp.declares = sorted(declares.get(fragment.name, ())) + declared_by[name] = set(inp.declares) + if not fragment.scope and not inp.declares: + # Reaches nobody -- and "nobody" is not expressible in + # appliesTo, where silence means everybody. Emitting it would + # invert its meaning. + taken.discard(name) + declared_by.pop(name, None) + continue + planned.append((fragment, inp)) + + for fragment, inp in planned: + _applies_to(inp, fragment.scope, devices, declared_by) + return [inp for _, inp in planned] + + +def _fabric_name(root: Path, play: Play, many: bool, taken: set[str]) -> str: + """A name per play, and never the same one twice. + + ⚠ The pattern does not always tell two plays apart: + `eos_designs-twodc-5stage-clos` runs eos_designs twice over the same + `hosts: TWODC_5STAGE_CLOS`, so naming by pattern gave both fabrics one name — + and `--emit` wrote one file, the second silently overwriting the first. + """ + if not many: + return _unique(slug(root.name), taken) + base = slug(f"{root.name}-{play.pattern}") or slug(root.name) + if base in taken: + base = slug(f"{base}-{play.playbook.removesuffix('.yml')}-{play.index}") + return _unique(base, taken) + + +def migrate(root: Path, collections: Path | None = None, inventory: Path | None = None, + inv: Inventory | None = None, drop_descriptions: bool = False, + compat_addressing: bool = False, + emit_pool_seed: bool = False) -> list[Fabric]: + """Translate every eos_designs play under ``root`` into a Fabric. + + Raises :class:`MigrationError` when the layered fragments disagree with the + per-device variables Ansible reports. That is the whole safety property: the + only rule this module supplies is the group order, and it is never allowed + to be wrong silently. + """ + root = Path(root).resolve() + # Reading an inventory costs two subprocesses; a caller that already has one + # (a harness measuring the whole corpus) may hand it over. + inv = inv if inv is not None else read_inventory(root, inventory, collections) + found = design_plays(root, collections, inventory) + if not found: + # No play runs eos_designs -- eos_cli_config_gen scenarios carry + # structured config directly and render at the device layer. Say so + # rather than inventing a fabric out of the inventory. + raise MigrationError( + f"{root.name}: no play runs eos_designs " + f"({len(plays(root, collections, inventory))} plays seen)" + ) + + vocabulary = Vocabulary.default() + fabrics: list[Fabric] = [] + named: set[str] = set() + for play in found: + devices = frozenset(play.hosts) + if not devices: + continue + fragments = _fragments(inv, devices) + if inv.templated: + fragments = templated(fragments, inv.hostvars) + + expected = {d: inv.hostvars.get(d, {}) for d in devices} + if layer(fragments, devices) != expected: + differing = sorted( + f"{d}.{k}" for d in devices + for k in set(expected[d]) | set(layer(fragments, devices)[d]) + if expected[d].get(k) != layer(fragments, devices)[d].get(k) + ) + raise MigrationError( + f"{root.name} play #{play.index}: layering the fragments does not " + f"reproduce what Ansible reports ({len(differing)} differences, " + f"first: {', '.join(differing[:5])})" + ) + + fabric = Fabric( + name=_fabric_name(root, play, len(found) > 1, named), + devices=tuple(sorted(devices)), + inputs=_inputs(fragments, devices, vocabulary), + play=play, + ) + _fabric_name_of(fabric, inv.hostvars) + _report_play_vars(fabric, root, play) + # Only now, with the translation proven faithful. Dropping anything + # before the comparison above would weaken the one gate this module has. + if emit_pool_seed: + _carry_pool_seed(fabric, root) + _report_unsupported(fabric, drop_descriptions, compat_addressing) + fabrics.append(fabric) + return fabrics + + +def _pooled_ids(design: dict) -> dict: + node_id = (design.get("fabric_numbering") or {}).get("node_id") + if isinstance(node_id, dict) and node_id.get("algorithm") == "pool_manager": + return node_id + return {} + + +def _carry_pool_seed(fabric: Fabric, root: Path) -> None: + """Read the pool's assignments so they can travel as a seed ConfigMap. + + The setting travels by itself; the state does not, and that asymmetry is + silent and expensive -- applied to a fabric that is already deployed, fresh + IDs renumber every device and the render reaches switches as a full + configuration replacement. `eos_designs-twodc-5stage-clos` cannot even + render without them: its v2.x addressing derives addresses from IDs, so a + fresh set collides and AVD refuses. + + Named and absent is fatal here for the same reason it is fatal in the + function: carrying on is the one outcome this exists to prevent. + """ + for inp in fabric.inputs: + pooled = _pooled_ids(inp.design) + if not pooled: + continue + named = pooled.get("pools_file") + if not named: + raise MigrationError( + f"{fabric.name} asks for pool-assigned node IDs but names no " + f"pools_file, so there are no assignments to carry" + ) + path = Path(named) + path = path if path.is_absolute() else root / path + if not path.is_file(): + raise MigrationError( + f"{fabric.name}: pools_file {named} does not exist under {root}; " + f"emitting without it would renumber every device" + ) + fabric.pool_seed = path.read_text() + return + + +def _report_play_vars(fabric: Fabric, root: Path, play: Play) -> None: + """Variables set on the play itself, which no input XR carries. + + ⚠ A source `ansible-inventory` cannot see, and one that changes the render: + `eos_designs-twodc-5stage-clos` runs eos_designs twice over the same hosts + and the second play sets `avd_digital_twin_mode: true`, producing a + different config into a different golden directory. Migrated without it, the + two fabrics come out identical and one of them is wrong. + + **Reported, not carried.** Play vars outrank group and host vars, but the + oracle this migration checks itself against is per-host hostvars, which do + not include them -- so carrying them would silently weaken the one gate this + module has. Whoever migrates such a play adds the keys to an input by hand. + """ + import yaml as _yaml + + playbook = root / play.playbook + try: + document = _yaml.safe_load(playbook.read_text()) + except (OSError, _yaml.YAMLError): + return + if not isinstance(document, list) or play.index > len(document): + return + variables = document[play.index - 1].get("vars") if isinstance( + document[play.index - 1], dict) else None + if isinstance(variables, dict) and variables: + fabric.notes.append( + f"{play.playbook} play #{play.index} sets {len(variables)} variable(s) on the " + f"play itself ({', '.join(sorted(variables))}); play vars outrank group and " + f"host vars and are NOT carried into any input" + ) + + +def _report_unsupported(fabric: Fabric, drop_descriptions: bool, + compat_addressing: bool = False) -> None: + """Note -- and optionally drop -- what pyavd will not honour.""" + # State, not settings. An inventory already running `pool_manager` keeps its + # node IDs in a file AVD generated; this translates the *setting* and leaves + # the *assignments* behind. Applied to a fabric that is already deployed, + # that renumbers every device -- and a render reaches a switch as a full + # configuration replacement. + for inp in fabric.inputs: + pooled = _pooled_ids(inp.design) + if pooled: + where = pooled.get("pools_file") or "/intended/data/-ids.yml" + if fabric.pool_seed: + fabric.notes.append( + f"node IDs come from a pool; its assignments travel as ConfigMap " + f"{fabric.seed_name}, read from {where}" + ) + else: + fabric.notes.append( + f"node IDs come from a pool; its assignments live in {where} and do " + f"NOT travel with this migration. Seed them into the fabric " + f"(spec.nodeIdPool.seedConfigMapName, --emit-pool-seed) or every " + f"device is renumbered" + ) + break + + # Carried, not dropped -- but a reader of the emitted file meets a value AVD + # never wrote, so the tool says where it came from. + marked = sum(nulls.count(inp.design) for inp in fabric.inputs) + if marked: + fabric.notes.append( + f"{marked} explicitly-null value(s) carried as {nulls.MARKER}; a real " + f"null is pruned from an open field by the API server" + ) + + # Before the unsupported sweep below, so a block this replaces is reported + # as replaced rather than as something pyavd cannot honour. + if compat_addressing: + swapped = [ + f"{inp.name}.{path}" + for inp in fabric.inputs + for path in use_compat_addressing(inp.design) + ] + if swapped: + fabric.notes.append( + f"{len(swapped)} ip_addressing template(s) replaced by " + f"{COMPAT_CLASS['python_module']}.{COMPAT_CLASS['python_class_name']}, " + f"which transcribes AVD's v2.x spine scheme: {swapped[0]}" + ) + + if drop_descriptions: + dropped = [ + f"{inp.name}.{path}" + for inp in fabric.inputs + for path in drop_description_templates(inp.design) + ] + if dropped: + fabric.notes.append( + f"dropped {len(dropped)} interface-description template(s); AVD's own " + f"descriptions apply instead: {', '.join(dropped[:3])}" + + (" ..." if len(dropped) > 3 else "") + ) + + found: dict[str, list[str]] = {} + for inp in fabric.inputs: + for owner, paths in unsupported(inp.design).items(): + found.setdefault(owner, []).extend(f"{inp.name}.{p}" for p in paths) + for owner, paths in sorted(found.items()): + what = ( + "descriptions only -- `--drop-description-templates` renders without them" + if owner == COSMETIC + else "this decides addresses, not wording; dropping it would emit a " + "different network" + if owner == "ip_addressing" + else "custom code; a function image is immutable and loads no arbitrary Python" + if owner == "python_module" + else "pyavd implements no Jinja templating" + ) + fabric.notes.append(f"{len(paths)} x {owner} pyavd cannot honour ({what}): {paths[0]}") + + +#: A value pyavd cannot honour, and whether losing it is cosmetic. +#: +#: pyavd implements no Jinja templating: `get_device_structured_config` passes +#: `templar=None` and the call raises `NotImplementedError`. AVD's own Ansible +#: action plugin reaches into pyavd's internal API precisely to hand in a +#: templar built from Ansible's own -- which needs Ansible at render time, and +#: there is none in a cluster. +#: +#: So a template path cannot travel. What differs is what is lost with it: +#: an `interface_descriptions` template decides a `description` string, while an +#: `ip_addressing` template decides an address. Measured on +#: `evpn_underlay_ebgp_overlay_ebgp`: dropping its description templates renders +#: all 16 devices and differs from AVD's golden in 168 places, **all of them a +#: `description` field**. Dropping an addressing template would silently emit a +#: different network. +COSMETIC = "interface_descriptions" + + +def unsupported(design: dict) -> dict[str, list[str]]: + """Values in a fragment that pyavd cannot honour, by the key that owns them. + + Detected by shape rather than by a list of key names: a Jinja template is a + string naming a `.j2` file, and custom logic is a `python_module`. Both are + code paths into a filesystem that a cluster does not have. + """ + found: dict[str, list[str]] = {} + + def walk(node: object, path: list[str], owner: str | None) -> None: + if isinstance(node, dict): + for key, value in node.items(): + walk(value, [*path, str(key)], + str(key) if str(key) in (COSMETIC, "ip_addressing") else owner) + if key == "python_module" and isinstance(value, str): + found.setdefault("python_module", []).append(".".join([*path, str(key)])) + elif isinstance(node, list): + for index, value in enumerate(node): + walk(value, [*path, f"[{index}]"], owner) + elif isinstance(node, str) and node.endswith(".j2"): + found.setdefault(owner or "template", []).append(".".join(path)) + + walk(design, [], None) + return found + + +def drop_description_templates(design: dict) -> list[str]: + """Remove interface-description templates so the rest can render. + + Returns the paths removed, which the caller is expected to report -- a + migration that quietly drops input is worse than one that refuses, because + the render is pushed to a device as a full configuration replacement. + + AVD falls back to its own built-in descriptions, so what is lost is exactly + the wording. Nothing else in the document depends on it. + """ + removed: list[str] = [] + + def walk(node: object, path: list[str]) -> None: + if isinstance(node, dict): + for key in list(node): + value = node[key] + if key == COSMETIC and isinstance(value, dict): + gone = [n for n, v in value.items() if isinstance(v, str) and v.endswith(".j2")] + for name in gone: + del value[name] + removed.append(".".join([*path, key, name])) + if not value: + del node[key] + continue + walk(value, [*path, str(key)]) + elif isinstance(node, list): + for index, value in enumerate(node): + walk(value, [*path, f"[{index}]"]) + + walk(design, []) + return removed + + +#: What `function.avd_compat` transcribes, and therefore the only templates this +#: substitution is entitled to replace. AVD's v2.x spine-to-super-spine scheme +#: divides the uplink pool by `max_uplink_switches`; its native algorithm packs +#: addresses contiguously, so the two are not interchangeable and swapping the +#: class in over somebody else's template would emit a different network. +COMPAT_CLASS = {"python_module": "function.avd_compat", + "python_class_name": "AvdIpAddressingV2Spine"} +COMPAT_TEMPLATES = {"p2p_uplinks_ip", "p2p_uplinks_peer_ip"} + + +def use_compat_addressing(design: dict) -> list[str]: + """Point `ip_addressing` at the class this image ships, where it fits. + + pyavd implements no Jinja templating, so a design pinning `.j2` addressing + cannot render at all -- not "renders differently". The same schema block + takes `python_module`, which pyavd does support, and this repo ships one + transcription: AVD's v2.x spine scheme. + + Replaces a block **only** when every template in it is one this class + reproduces, and returns what it replaced so the caller can say so. A block + naming anything else is left alone and stays reported as unsupported -- + silently substituting there would put our addresses on somebody else's + fabric. + """ + swapped: list[str] = [] + + def walk(node: object, path: list[str]) -> None: + if isinstance(node, dict): + for key in list(node): + value = node[key] + if key == "ip_addressing" and isinstance(value, dict): + templates = { + n for n, v in value.items() + if isinstance(v, str) and v.endswith(".j2") + } + if templates and templates <= COMPAT_TEMPLATES: + node[key] = dict(COMPAT_CLASS) + swapped.append(".".join([*path, key])) + continue + walk(value, [*path, str(key)]) + elif isinstance(node, list): + for index, value in enumerate(node): + walk(value, [*path, f"[{index}]"]) + + walk(design, []) + return swapped + + +def _fabric_name_of(fabric: Fabric, hostvars: dict[str, dict]) -> None: + """Fill ``spec.fabricName`` and note it when the devices disagree. + + `fn.py` writes one `fabric_name` into every device's document, so a Fabric + has exactly one. Ansible does not: `eos_designs_unit_tests` runs one + eos_designs play over 501 devices carrying **six** different `fabric_name` + values. That is not expressible, and quietly picking one would render every + device under a name AVD never gave it -- so it is named as a note. + """ + seen: dict[str, int] = {} + for device in fabric.devices: + name = hostvars.get(device, {}).get("fabric_name") + if isinstance(name, str) and name: + seen[name] = seen.get(name, 0) + 1 + if not seen: + return + fabric.fabric_name = max(seen, key=lambda n: seen[n]) + if len(seen) > 1: + others = ", ".join(f"{n} ({c})" for n, c in sorted(seen.items(), key=lambda kv: -kv[1])) + fabric.notes.append( + f"devices disagree on fabric_name and a Fabric carries one: {others}" + ) + + +def to_manifests(fabric: Fabric, namespace: str | None = None) -> list[dict]: + """The Fabric and its inputs, as manifests in ``spec.requires`` order. + + Explicitly-null values leave here as :data:`nulls.MARKER`, because an API + server prunes a real null out of an open field -- see :mod:`function.nulls`. + A document already carrying the marker is refused: encoding it would make + somebody else's string indistinguishable from a null on the way back. + + ⚠ **No namespace unless one is asked for.** Written without it, the whole set + goes wherever ``kubectl -n apply -f`` puts it: `spec.requires` needs + only kind and name, and a `Fabric` looks for an input it does not place in + its own namespace (`fn.py`). Proven on a cluster -- eight devices, Ready, in + a namespace named nowhere in the file. Pinning one instead makes the file + apply to exactly one place and collide with any other fabric that named the + same input there. + """ + group = "avd.netclab.dev/v1alpha1" + + def placed(name: str) -> dict: + return {"name": name} | ({"namespace": namespace} if namespace else {}) + + out: list[dict] = [] + for inp in fabric.inputs: + if nulls.carries(inp.design): + raise MigrationError( + f"{inp.kind}/{inp.name} already contains {nulls.MARKER!r} as a value; " + f"it is how an explicit null is carried, so this cannot be told apart" + ) + spec: dict = {"design": nulls.encoded(inp.design)} + if inp.declares: + spec["declares"] = inp.declares + applies: dict = {} + if inp.all_devices: + applies["all"] = True + if inp.node_sets: + applies["nodeSets"] = inp.node_sets + if inp.hosts: + applies["hosts"] = inp.hosts + if inp.match_hostnames: + applies["matchHostnames"] = inp.match_hostnames + if applies: + spec["appliesTo"] = applies + out.append({ + "apiVersion": group, "kind": inp.kind, + "metadata": placed(inp.name), + "spec": spec, + }) + spec_fabric: dict = { + "fabricName": fabric.fabric_name or fabric.name, + "requires": [{"kind": i.kind} | placed(i.name) for i in fabric.inputs], + } + if fabric.pool_seed: + # Not an XR, and the only object here that is not one. A pool is state: + # the assignments AVD already made, which decide addresses. Without them + # this fabric does not render differently, it does not render at all. + from . import pools + + out.append({ + "apiVersion": "v1", "kind": "ConfigMap", + "metadata": placed(fabric.seed_name), + "data": {pools.DATA_KEY: fabric.pool_seed}, + }) + spec_fabric["nodeIdPool"] = {"seedConfigMapName": fabric.seed_name} + out.append({ + "apiVersion": group, "kind": "Fabric", + "metadata": placed(fabric.name), + "spec": spec_fabric, + }) + return out + + +def _discover(root: Path) -> list[Path]: + """Every inventory under a directory, in either layout AVD ships.""" + if (root / "inventory.yml").is_file() or (root / "inventory" / "hosts.yml").is_file(): + return [root] + return sorted( + d for d in root.iterdir() + if d.is_dir() + and ((d / "inventory.yml").is_file() or (d / "inventory" / "hosts.yml").is_file()) + ) + + +def main() -> int: + """``avd-migrate [ROOT ...] [--emit DIR] [--namespace NS]``""" + import argparse + + import yaml + + parser = argparse.ArgumentParser( + prog="avd-migrate", + description="Translate AVD Ansible inventories into Fabric and input XRs.", + ) + parser.add_argument("roots", nargs="*", type=Path, + default=[Path("avd/ansible_collections/arista/avd/examples")]) + parser.add_argument("--emit", type=Path, help="write manifests under this directory") + parser.add_argument( + "--namespace", default=None, + help="pin the manifests to one namespace. Left out, they name none and " + "`kubectl -n apply -f` decides -- which is what makes one " + "emitted file usable in any namespace.", + ) + parser.add_argument( + "--emit-pool-seed", action="store_true", + help="carry the node-ID assignments too, as a ConfigMap the emitted " + "Fabric seeds from. Without them a fabric already running " + "pool_manager is renumbered on its first reconcile -- and the " + "render is pushed as a full configuration replacement.", + ) + parser.add_argument( + "--compat-ip-addressing", action="store_true", + help="point an `ip_addressing` block at function.avd_compat where it " + "pins the v2.x spine templates that module transcribes. pyavd " + "implements no Jinja templating, so such a design does not render " + "at all; any other template is left alone and stays reported.", + ) + parser.add_argument( + "--drop-description-templates", action="store_true", + help="drop interface-description templates so the rest renders. Cosmetic " + "by measurement -- on AVD's own corpus it costs `description` fields " + "and nothing else. Addressing templates are never dropped.", + ) + parser.add_argument("--collections", type=Path, + default=Path("avd"), help="ANSIBLE_COLLECTIONS_PATH") + args = parser.parse_args() + + collections = args.collections.resolve() if args.collections.is_dir() else None + failures = 0 + for top in args.roots: + for root in _discover(top): + try: + fabrics = migrate(root, collections=collections, + drop_descriptions=args.drop_description_templates, + compat_addressing=args.compat_ip_addressing, + emit_pool_seed=args.emit_pool_seed) + except MigrationError as err: + print(f"[skip] {root.name:38s} {str(err).split(': ', 1)[-1]}") + continue + except Exception as err: # noqa: BLE001 - surface any ansible failure + failures += 1 + print(f"[FAIL] {root.name:38s} {type(err).__name__}: " + f"{str(err).splitlines()[0][:70]}") + continue + for fabric in fabrics: + kinds = {k: sum(1 for i in fabric.inputs if i.kind == k) for k in KINDS} + shape = " ".join(f"{k[:-3] if k.endswith('Set') else k}={v}" + for k, v in kinds.items() if v) + print(f"[ ok ] {fabric.name:38s} {len(fabric.devices):4d} devices {shape}") + for note in fabric.notes: + print(f" ! {note}") + if args.emit: + target = args.emit / f"{fabric.name}.yaml" + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(yaml.safe_dump_all( + to_manifests(fabric, args.namespace), sort_keys=False)) + return 1 if failures else 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/function/nulls.py b/function/nulls.py new file mode 100644 index 0000000..ffbbefa --- /dev/null +++ b/function/nulls.py @@ -0,0 +1,61 @@ +"""An explicit ``null`` in a design, carried past an API server that prunes it. + +AVD reads a key set to ``null`` differently from an absent one, and the +difference decides configuration. Kubernetes does not keep it -- measured: + + scalar: null -> the key is gone + list_items: [a, null, b] -> [a, null, b] (kept) + marker: avd.netclab.dev/null -> avd.netclab.dev/null (kept) + +A null map value is pruned, a null list item is not, and there is nowhere to put +``nullable: true``: keys under an open design have no node in the schema. So the +value travels as a marker and becomes ``None`` again after layering, on both the +``spec.requires`` and the hand-written ``spec.design`` path. + +⚠ The marker is API -- a reader of a published example sees it. ``"None"`` is one +capital from a real AVD value (``spanning_tree_mode: none``, 116 uses in the +corpus) and ``"null"`` is two quotes from the silently-pruned null this module +exists to remove; a qualified marker cannot be typed by accident. +""" + +from __future__ import annotations + +MARKER = "avd.netclab.dev/null" + + +def encoded(node): + """A design with every explicitly-null map value replaced by the marker.""" + if isinstance(node, dict): + return {k: MARKER if v is None else encoded(v) for k, v in node.items()} + if isinstance(node, list): + # List items survive intact, so they are left alone -- including a + # genuine null item, which must stay a null item. + return [None if v is None else encoded(v) for v in node] + return node + + +def restored(node): + """A design with every marker turned back into ``None``.""" + if isinstance(node, dict): + return {k: None if v == MARKER else restored(v) for k, v in node.items()} + if isinstance(node, list): + return [None if v is None else restored(v) for v in node] + return node + + +def carries(node) -> bool: + """Whether the marker already appears as a value -- the one unsafe case.""" + if isinstance(node, dict): + return any(v == MARKER or carries(v) for v in node.values()) + if isinstance(node, list): + return any(v == MARKER or carries(v) for v in node) + return False + + +def count(node) -> int: + """How many map values are explicitly null -- what :func:`encoded` marks.""" + if isinstance(node, dict): + return sum(1 if v is None else count(v) for v in node.values()) + if isinstance(node, list): + return sum(count(v) for v in node if v is not None) + return 0 diff --git a/function/pools.py b/function/pools.py new file mode 100644 index 0000000..0d862e9 --- /dev/null +++ b/function/pools.py @@ -0,0 +1,154 @@ +"""Where a Fabric keeps its node-ID pool. + +`fabric_numbering.node_id.algorithm: pool_manager` asks AVD to hand out node IDs +instead of reading them off each node. AVD keeps those assignments in **a file**, +and the render is only reproducible while that file survives — lose it and every +device is renumbered, which reaches a switch as a full configuration replacement. + +There is no such file in a cluster, so the Fabric keeps the pool in a ConfigMap +it composes and reads back on the next reconcile. The pool is therefore an +**output that is also the next run's input** — the one place a render stops being +a pure function of its inputs. + +⚠ **`spec.design`'s own `pools_file` is overridden, not honoured.** It names a +path relative to a working directory, which is a statement about somebody's +laptop; the same design in a cluster has nowhere to point. The value is replaced +with a path inside a scratch directory that exists only for the length of one +reconcile, and the ConfigMap is the real home. +""" + +from __future__ import annotations + +import tempfile +from collections.abc import Iterator +from contextlib import contextmanager +from pathlib import Path +from typing import Any + +#: composition-resource-name of the ConfigMap, and the key inside it +RESOURCE_NAME = "id-pool" +DATA_KEY = "node-id-pools.yml" + +_ALGORITHM_PATH = ("fabric_numbering", "node_id") +_POOL_FILE = "pools.yml" + + +def wanted_by(all_inputs: dict[str, dict]) -> bool: + """Does any device ask for pool-assigned node IDs? + + Read per device rather than fabric-wide because the setting is an ordinary + input key: nothing stops one input from narrowing it to part of the fabric, + and one device asking is enough to need a pool. + """ + return any(_node_id(hostvars).get("algorithm") == "pool_manager" + for hostvars in all_inputs.values()) + + +def _node_id(hostvars: dict) -> dict: + node = hostvars + for key in _ALGORITHM_PATH: + node = node.get(key) if isinstance(node, dict) else None + if node is None: + return {} + return node if isinstance(node, dict) else {} + + +def observed_pool(observed_resources: Any) -> str: + """The pool as the cluster last saw it, or empty on the first reconcile.""" + entry = (observed_resources or {}).get(RESOURCE_NAME) + if entry is None: + return "" + from crossplane.function import resource + + data = (resource.struct_to_dict(entry.resource) or {}).get("data") or {} + return data.get(DATA_KEY) or "" + + +@contextmanager +def pool_manager(all_inputs: dict[str, dict], previous: str) -> Iterator[tuple[Any, Path]]: + """A ``PoolManager`` backed by ``previous``, and the file it ends up in. + + Rewrites every device's `pools_file` to the scratch copy first: AVD resolves + that value verbatim against the working directory, so leaving the design's + own value in place would read a path that does not exist here and silently + assign a fresh set of IDs. + """ + from pyavd.api.pool_manager import PoolManager + + with tempfile.TemporaryDirectory() as scratch: + path = Path(scratch) / _POOL_FILE + if previous.strip(): + path.write_text(previous) + for hostvars in all_inputs.values(): + node_id = _node_id(hostvars) + if node_id.get("algorithm") == "pool_manager": + node_id["pools_file"] = str(path) + yield PoolManager(Path(scratch)), path + + +#: requirement key for a seed ConfigMap named by the Fabric +SEED_NAME = "id-pool-seed" + + +def seed(req: Any, rsp: Any, spec: dict, namespace: str) -> tuple[str, str]: + """Assignments to start from, when this fabric has no pool of its own yet. + + Returns ``(state, text)`` where state is one of: + + * ``none`` -- no seed named; a new fabric assigns its own IDs; + * ``pending`` -- asked for, not delivered yet. ⚠ **The caller must not + render.** Requirements are answered on the *next* reconcile, so the first + one always arrives with nothing; rendering then would assign a fresh set + of IDs and compose them as the pool, and the seed would never be read; + * ``missing`` -- Crossplane looked and it is not there. Named and absent is + an error, not an empty pool: proceeding renumbers every device, which is + the one thing this field exists to prevent; + * ``ok`` -- the text. + """ + from crossplane.function import resource, response + + pool_spec = spec.get("nodeIdPool") or {} + name = pool_spec.get("seedConfigMapName") + if not name: + return "none", "" + + response.require_resources( + rsp, name=SEED_NAME, api_version="v1", kind="ConfigMap", + match_name=name, namespace=namespace, + ) + if SEED_NAME not in req.required_resources: + return "pending", "" + items = req.required_resources[SEED_NAME].items + if not items: + return "missing", "" + + data = (resource.struct_to_dict(items[0].resource) or {}).get("data") or {} + return "ok", data.get(pool_spec.get("seedKey") or DATA_KEY) or "" + + +def configmap(xr_name: str, namespace: str, fabric_name: str, pool: str) -> dict: + """The ConfigMap the Fabric composes to keep its assignments.""" + from crossplane.function import resource + + return { + "apiVersion": "v1", + "kind": "ConfigMap", + "metadata": { + "name": resource.child_name(xr_name, "id-pool"), + "namespace": namespace, + # ⚠ `fabric` alone does not find this: every ConfigMap the fabric + # composes carries it, so a selector returns the pool and one render + # per device -- 27 objects for a 26-device fabric, with the pool in + # no particular position. An object whose annotation says deleting it + # renumbers the fabric has to be addressable, so it says what it is. + "labels": { + "avd.netclab.dev/fabric": fabric_name, + "avd.netclab.dev/artifact": "node-id-pool", + }, + "annotations": { + "avd.netclab.dev/description": + "AVD node-ID assignments. Deleting this renumbers the fabric.", + }, + }, + "data": {DATA_KEY: pool}, + } diff --git a/function/verify_example.py b/function/verify_example.py deleted file mode 100644 index 686e0a4..0000000 --- a/function/verify_example.py +++ /dev/null @@ -1,106 +0,0 @@ -"""Milestone 1: prove the pyavd path reproduces AVD's golden output. - -Reads an AVD Ansible example, rebuilds ``all_inputs`` the way Ansible would, -runs the pyavd pipeline, and deep-diffs the resulting structured configs against -the example's checked-in ``intended/structured_configs/*.yml``. - -Green = the engine our Crossplane function will wrap is faithful to AVD. - -Usage: - uv run avd-verify [EXAMPLE_DIR] -""" - -from __future__ import annotations - -import sys -from pathlib import Path - -import yaml - -from .ansible_inputs import build_all_inputs -from .engine import render_structured_configs - -DEFAULT_EXAMPLE = ( - "avd/ansible_collections/arista/avd/examples/single-dc-l3ls" -) - - -def _diff(path: str, ours, gold, out: list[str]) -> None: - """Collect human-readable differences between two nested structures.""" - if type(ours) is not type(gold) and not ( - isinstance(ours, (int, float)) and isinstance(gold, (int, float)) - ): - out.append(f"{path}: type {type(ours).__name__} != {type(gold).__name__}") - return - if isinstance(gold, dict): - for k in sorted(set(ours) | set(gold)): - if k not in ours: - out.append(f"{path}.{k}: missing (only in golden)") - elif k not in gold: - out.append(f"{path}.{k}: extra (only in ours)") - else: - _diff(f"{path}.{k}", ours[k], gold[k], out) - elif isinstance(gold, list): - if len(ours) != len(gold): - out.append(f"{path}: list len {len(ours)} != {len(gold)}") - for i, (a, b) in enumerate(zip(ours, gold)): - _diff(f"{path}[{i}]", a, b, out) - elif ours != gold: - out.append(f"{path}: {ours!r} != {gold!r}") - - -def verify(example_dir: str | Path) -> int: - example_dir = Path(example_dir) - golden_dir = example_dir / "intended" / "structured_configs" - - all_inputs = build_all_inputs(example_dir) - rendered = render_structured_configs(all_inputs) - - # Guard: the set of rendered devices must match the golden set, otherwise a - # transcoder that produces zero devices would falsely report "all match". - golden_hosts = {p.stem for p in golden_dir.glob("*.yml")} - rendered_hosts = set(rendered) - if rendered_hosts != golden_hosts: - missing = sorted(golden_hosts - rendered_hosts) - extra = sorted(rendered_hosts - golden_hosts) - print( - f"DEVICE SET MISMATCH: rendered {len(rendered_hosts)} vs golden " - f"{len(golden_hosts)}" - ) - if missing: - print(f" not rendered (in golden): {missing}") - if extra: - print(f" rendered but no golden: {extra}") - print() - print("MISMATCH: device sets differ, cannot claim reproduction.") - return 1 - - total_diffs = 0 - for hostname in sorted(rendered): - golden_file = golden_dir / f"{hostname}.yml" - gold = yaml.safe_load(golden_file.read_text()) or {} - diffs: list[str] = [] - _diff(hostname, rendered[hostname], gold, diffs) - status = "OK " if not diffs else "DIFF" - print(f"[{status}] {hostname} ({len(diffs)} diffs)") - for d in diffs[:20]: - print(f" {d}") - if len(diffs) > 20: - print(f" ... and {len(diffs) - 20} more") - total_diffs += len(diffs) - - print() - if total_diffs == 0: - print(f"MATCH: all {len(rendered)} devices reproduce golden structured config.") - return 0 - print(f"MISMATCH: {total_diffs} total diffs across the fabric.") - return 1 - - -def main() -> int: - example = sys.argv[1] if len(sys.argv) > 1 else DEFAULT_EXAMPLE - return verify(example) - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/function/verify_xr.py b/function/verify_xr.py deleted file mode 100644 index bbfdeb4..0000000 --- a/function/verify_xr.py +++ /dev/null @@ -1,103 +0,0 @@ -"""Milestone 2 harness: prove the Fabric-XR path reproduces AVD's golden output. - -For each AVD example, fold it into a single fabric document (`spec.design`), -render it the way the composite function will (`render_fabric_design`), and -deep-diff against the checked-in `intended/structured_configs`. - -This is the XR-level analogue of `avd-verify`, and doubles as the regression net -for the Ansible->XR fold. - -Usage: - uv run avd-verify-xr [EXAMPLE_DIR ...] # default: every bundled example -""" - -from __future__ import annotations - -import sys -from pathlib import Path - -import yaml - -from .ansible_inputs import build_all_inputs -from .engine import render_fabric_design -from .verify_example import _diff -from .xr import fabric_design_from_inputs - -EXAMPLES_ROOT = Path("avd/ansible_collections/arista/avd/examples") - -# Examples that are known not to fold into a single fabric document, with the -# reason. Expected-failure semantics: a deferred example that fails is not a -# regression, but one that starts passing IS reported (the deferral is stale and -# should be removed), so this list can never silently hide a fixed example. -DEFERRED = { - "campus-fabric": "aaa_settings.radius differs by role; no node-scoped equivalent", - "cv-pathfinder": "SD-WAN: WAN gateway across 2 routers + ansible-vault secrets", -} - - -def _discover(root: Path) -> list[Path]: - return sorted( - d - for d in root.iterdir() - if (d / "inventory.yml").is_file() - and (d / "intended" / "structured_configs").is_dir() - ) - - -def verify_one(example_dir: Path) -> tuple[str, int]: - """Return (status, diff_count). status in {ok, diff, conflict, error}.""" - per_host = build_all_inputs(example_dir) - fabric_name, design, conflicts = fabric_design_from_inputs(per_host) - try: - rendered = render_fabric_design(design, fabric_name) - except Exception as exc: # noqa: BLE001 - surface any AVD/render failure - detail = f"conflicts={sorted(conflicts)} " if conflicts else "" - return f"error: {detail}{type(exc).__name__}: {str(exc)[:80]}", -1 - - golden_dir = example_dir / "intended" / "structured_configs" - total = 0 - for hostname in sorted(rendered): - gold = yaml.safe_load((golden_dir / f"{hostname}.yml").read_text()) or {} - out: list[str] = [] - _diff(hostname, rendered[hostname], gold, out) - total += len(out) - if total: - return f"diff ({total})", total - return "ok", total - - -def main() -> int: - args = [Path(a) for a in sys.argv[1:]] - examples = args or _discover(EXAMPLES_ROOT) - failures = 0 - deferred = 0 - for example_dir in examples: - status, diffs = verify_one(example_dir) - n = len(list((example_dir / "intended" / "structured_configs").glob("*.yml"))) - ok = status.startswith("ok") - reason = DEFERRED.get(example_dir.name) - if reason and not ok: - mark, deferred = "DEFER", deferred + 1 - status = f"deferred: {reason}" - elif reason and ok: - # Stale deferral: it folds now, so drop it from DEFERRED. - mark = "XPASS" - failures += 1 - status = "folds now -- remove from DEFERRED" - elif ok: - mark = "OK " - else: - mark = "FAIL" - failures += 1 - print(f"[{mark}] {example_dir.name:26s} devices={n:2d} {status}") - print() - expected = len(examples) - deferred - print( - f"{expected - failures}/{expected} expected examples reproduce golden via the " - f"Fabric-XR fold ({deferred} deferred)." - ) - return 1 if failures else 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/function/xr.py b/function/xr.py deleted file mode 100644 index be58fe1..0000000 --- a/function/xr.py +++ /dev/null @@ -1,165 +0,0 @@ -"""Build a ``Fabric`` XR (custom resource) from an AVD Ansible example. - -Bridges Milestone 1 (Ansible example -> proven pyavd output) to Milestone 2 -(Crossplane XR -> pyavd output): it folds the per-host Ansible inputs into a -single fabric-wide AVD design document -- the shape carried by -``Fabric.spec.design`` -- with device roles expressed via ``default_node_types`` -instead of Ansible's per-group ``type``. - -The resulting XR is a self-contained fixture for ``crossplane render`` and for -regression-testing the composite function. -""" - -from __future__ import annotations - -import copy -import re -from pathlib import Path - -import yaml - -from .ansible_inputs import build_all_inputs - -API_VERSION = "avd.netclab.dev/v1alpha1" -KIND = "Fabric" - - -def _is_node_type_block(value: object) -> bool: - return isinstance(value, dict) and ("nodes" in value or "node_groups" in value) - - -def _canon(value: object) -> str: - return yaml.safe_dump(value, sort_keys=True, default_flow_style=True) - - -def _conflicting_block_defaults(per_host: dict[str, dict]) -> set[tuple[str, str]]: - """Return ``{(block, default_key)}`` whose value differs across DCs. - - These are per-DC/per-pod settings (pools, ASNs, uplinks) kept in a group's - ``defaults``; they cannot share one block-level ``defaults`` and are instead - pushed down to the node_groups/nodes of their originating DC (which override - defaults in AVD, so effective values are unchanged). - """ - seen: dict[tuple[str, str], set[str]] = {} - for hostvars in per_host.values(): - for key, value in hostvars.items(): - if _is_node_type_block(value): - for dk, dv in (value.get("defaults") or {}).items(): - seen.setdefault((key, dk), set()).add(_canon(dv)) - return {kd for kd, values in seen.items() if len(values) > 1} - - -def _append_unique(dst_list: list, items, id_key: str, pushdown: dict) -> None: - """Append node/node_group dicts not already present, baking pushdown defaults.""" - seen = {e[id_key] for e in dst_list if isinstance(e, dict) and id_key in e} - for item in items or []: - if isinstance(item, dict) and item.get(id_key) not in seen: - item = copy.deepcopy(item) # group_vars dicts are shared across hosts - for pk, pv in pushdown.items(): - item.setdefault(pk, pv) - dst_list.append(item) - seen.add(item.get(id_key)) - - -def fabric_design_from_inputs( - per_host: dict[str, dict], -) -> tuple[str, dict, set[str]]: - """Fold ``{hostname: hostvars}`` into ``(fabric_name, design, conflicts)``. - - Per-host hostvars differ by ``type`` (re-expressed as ``default_node_types``) - and by per-DC node-type blocks. Node-type blocks are unioned; per-DC - ``defaults`` that disagree are pushed down to that DC's node_groups/nodes so a - single document stays lossless. Remaining ``conflicts`` are fabric-level keys - (e.g. ``aaa_settings``) that are not node-scoped and need an explicit call. - """ - fabric_name = "FABRIC" - design: dict = {} - roles: dict[str, list[str]] = {} - conflicts: set[str] = set() - conflict_defaults = _conflicting_block_defaults(per_host) - - for hostname, hostvars in per_host.items(): - for key, value in hostvars.items(): - if key == "type": - roles.setdefault(value, []).append(hostname) - elif key == "fabric_name": - fabric_name = value - elif _is_node_type_block(value): - block = design.setdefault(key, {}) - shared_defaults = block.setdefault("defaults", {}) - pushdown: dict = {} - for dk, dv in (value.get("defaults") or {}).items(): - if (key, dk) in conflict_defaults: - pushdown[dk] = dv # per-DC -> node_group/node level - else: - shared_defaults[dk] = dv # uniform -> stays shared - _append_unique(block.setdefault("nodes", []), value.get("nodes"), "name", pushdown) - _append_unique( - block.setdefault("node_groups", []), value.get("node_groups"), "group", pushdown - ) - for bk, bv in value.items(): - if bk not in ("defaults", "nodes", "node_groups"): - block[bk] = bv - else: - if key in design and _canon(design[key]) != _canon(value): - conflicts.add(key) - design[key] = value - - # Drop empty scaffolding left by setdefault. - for block in design.values(): - if isinstance(block, dict): - for empty_key in ("defaults", "nodes", "node_groups"): - if empty_key in block and not block[empty_key]: - del block[empty_key] - - # Roles come from Ansible's per-group `type`. If the example instead assigns - # roles fabric-wide via its own `default_node_types` (e.g. multipod), that - # value already flowed into `design` -- keep it rather than clobbering it. - if roles: - design["default_node_types"] = [ - { - "node_type": node_type, - "match_hostnames": [f"^{re.escape(h)}$" for h in sorted(hosts)], - } - for node_type, hosts in sorted(roles.items()) - ] - return fabric_name, design, conflicts - - -class FabricFoldConflict(Exception): - """Raised when per-host Ansible data cannot collapse into one fabric document. - - Carries the conflicting keys -- typically per-DC/per-pod settings kept in - group ``defaults`` that must be relocated to ``node_groups`` level to be - representable in a single ``spec.design``. - """ - - -def fabric_xr_from_example( - example_dir: str | Path, - *, - name: str | None = None, - namespace: str = "default", - strict: bool = True, -) -> dict: - """Return a ``Fabric`` XR dict for an AVD Ansible example directory. - - With ``strict`` (default), raises :class:`FabricFoldConflict` if the example's - Ansible group-vars cannot be folded losslessly into a single fabric document. - """ - per_host = build_all_inputs(example_dir) - fabric_name, design, conflicts = fabric_design_from_inputs(per_host) - if conflicts and strict: - raise FabricFoldConflict(sorted(conflicts)) - return { - "apiVersion": API_VERSION, - "kind": KIND, - "metadata": { - "name": name or fabric_name.lower().replace("_", "-"), - "namespace": namespace, - }, - "spec": { - "fabricName": fabric_name, - "design": design, - }, - } diff --git a/pyproject.toml b/pyproject.toml index 824f76a..b0cc558 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -14,19 +14,27 @@ dependencies = [ ] [project.scripts] -avd-verify = "function.verify_example:main" -avd-verify-xr = "function.verify_xr:main" +avd-migrate = "function.migrate:main" avd-function = "function.main:main" avd-topology = "function.netclab_topology:main" [dependency-groups] -dev = ["pytest>=8.0"] +# `ansible-core` is the migration harness's reference implementation of Ansible: +# `function/ansible_cli.py` asks the CLI what an inventory resolves to instead of +# reimplementing it. The runtime never imports it -- `fn.py` takes XRs and there +# is no inventory in a cluster -- so it reaches neither the image nor the +# published package. Keeping Ansible out of the *runtime* is the rule; keeping it +# out of the *harness* only bought a second, wrong implementation of Ansible. +dev = ["pytest>=8.0", "ansible-core>=2.18"] [tool.pytest.ini_options] testpaths = ["tests"] # e2e needs a live cluster, so it is opt-in: `uv run pytest -m e2e`. -addopts = "-m 'not e2e'" -markers = ["e2e: requires a cluster from scripts/kind-up.sh with a fabric applied"] +addopts = "-m 'not e2e and not corpus'" +markers = [ + "e2e: requires a cluster from scripts/kind-up.sh with a fabric applied", + "corpus: the whole molecule corpus -- 501 devices in one play, 71 plays in one scenario", +] [build-system] requires = ["uv_build>=0.11.7,<0.12.0"] diff --git a/scripts/kind-up.sh b/scripts/kind-up.sh index 6213d31..630cd31 100755 --- a/scripts/kind-up.sh +++ b/scripts/kind-up.sh @@ -101,7 +101,15 @@ done echo ">> build + push function image/xpkg (tag ${TAG})" docker build --provenance=false -t "${IMG}:${TAG}" . -crossplane xpkg build --package-root=package --embed-runtime-image="${IMG}:${TAG}" -o "function-avd-${TAG}.xpkg" +# --examples-root is stated even though `package/` is the package root: it +# defaults to ./examples, which is not "no examples" but *everything* under +# examples/ -- including examples/lab/topology.yaml, which is helm values with no +# `kind` and fails the build with "Object 'Kind' is missing". --ignore is no help, +# it does not reach --examples-root. CI and the release workflow have named it +# since v0.1.4 failed on exactly this; this script was the build path that did +# not, so a local bring-up broke the day the topology was committed. +crossplane xpkg build --package-root=package --examples-root=examples/fabric \ + --embed-runtime-image="${IMG}:${TAG}" -o "function-avd-${TAG}.xpkg" crossplane xpkg push -f "function-avd-${TAG}.xpkg" "localhost:${REG_PORT}/netclab/function-avd:${TAG}" echo ">> install Crossplane (chart ${XP_CHART})" @@ -122,10 +130,23 @@ spec: EOF kubectl --context "$CTX" wait --for=condition=Healthy function.pkg.crossplane.io/netclab-function-avd --timeout=180s -echo ">> install XRDs + Compositions (Fabric + Device)" -kubectl --context "$CTX" apply -f apis/fabric/xrd.yaml -f apis/device/xrd.yaml -kubectl --context "$CTX" wait --for=condition=Established xrd/fabrics.avd.netclab.dev xrd/devices.avd.netclab.dev --timeout=60s -kubectl --context "$CTX" apply -f apis/fabric/composition.yaml -f apis/device/composition.yaml +# Every API under apis/, not a named pair: the package root ships whatever is +# there, so a script naming two of them installs a cluster that does not match +# the package it was built from -- and the input kinds would be missing exactly +# where a Fabric that names them is being tested. +echo ">> install XRDs + Compositions (everything under apis/)" +# One -f per file: `apply -f apis/*/xrd.yaml` reads only the first path the +# glob expands to and passes the rest as positional args, which kubectl rejects. +# It worked while there were two API directories and broke silently at six -- +# and nothing caught it, because `e2e on kind` is workflow_dispatch-only and +# nothing else in CI runs this script. +for manifest in apis/*/xrd.yaml; do + kubectl --context "$CTX" apply -f "$manifest" +done +kubectl --context "$CTX" wait --for=condition=Established xrd --all --timeout=60s +for manifest in apis/*/composition.yaml; do + kubectl --context "$CTX" apply -f "$manifest" +done if [ "$WITH_NETCLAB" = "1" ]; then echo ">> provider-http ${PROVIDER_HTTP} (config push over eAPI)" diff --git a/tests/test_apis_consistency.py b/tests/test_apis_consistency.py new file mode 100644 index 0000000..bf4f09f --- /dev/null +++ b/tests/test_apis_consistency.py @@ -0,0 +1,81 @@ +"""The published API and the code that serves it do not drift apart. + +Offline. Cheap checks over `apis/`, each guarding a failure that is silent: + +* a new input kind gets an XRD but `fn.py` never learns to reconcile it (or the + reverse), and the XR sits unready with "unsupported composite kind"; +* an XRD ships without `categories`, so `kubectl get netclab` does not list it -- + the exact defect this repo carried in Fabric and Device until it was found by + running the command, not by reading the file; +* an XRD points `defaultCompositionRef` at a Composition that is not there, or at + one built for a different kind, so nothing selects it. + +None of these break a build. They break in a cluster, one release later. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest +import yaml + +from function.kinds import KINDS + +APIS = Path("apis") +# Fabric and Device are composed, not collected -- they are not input kinds. +COMPOSED_KINDS = {"Fabric", "Device"} + + +def _xrds() -> dict[str, dict]: + return {p.parent.name: yaml.safe_load(p.read_text()) for p in sorted(APIS.glob("*/xrd.yaml"))} + + +XRDS = _xrds() + + +def test_apis_directory_is_not_empty() -> None: + assert len(XRDS) >= 6, f"expected the six XRDs, found {sorted(XRDS)}" + + +@pytest.mark.parametrize("name", sorted(XRDS), ids=str) +def test_xrd_declares_categories(name: str) -> None: + """Every XRD is reachable by `kubectl get netclab` and `kubectl get crossplane`.""" + names = XRDS[name]["spec"]["names"] + assert names.get("categories") == ["crossplane", "netclab"], ( + f"{name}: categories are {names.get('categories')!r}; netclab-xp's twelve " + f"XRDs all carry ['crossplane', 'netclab'] and these must match" + ) + + +@pytest.mark.parametrize("name", sorted(XRDS), ids=str) +def test_xrd_has_its_composition(name: str) -> None: + """`defaultCompositionRef` resolves, and to a Composition for this kind.""" + xrd = XRDS[name]["spec"] + wanted = xrd["defaultCompositionRef"]["name"] + composition = yaml.safe_load((APIS / name / "composition.yaml").read_text()) + assert composition["metadata"]["name"] == wanted + assert composition["spec"]["compositeTypeRef"]["kind"] == xrd["names"]["kind"] + + +def test_input_kinds_match_the_function() -> None: + """The XRDs that exist and the kinds fn.py reconciles are the same set.""" + from_apis = {x["spec"]["names"]["kind"] for x in XRDS.values()} - COMPOSED_KINDS + assert from_apis == set(KINDS), ( + f"apis/ serves {sorted(from_apis)} but function.kinds.KINDS is " + f"{sorted(KINDS)} -- fn.py would answer 'unsupported composite kind'" + ) + + +def test_fabric_requires_accepts_every_input_kind_and_secret() -> None: + """A Fabric can name each input kind, plus a Secret carrying credentials. + + Secret is in the enum from the first version deliberately: adding it later + would be a schema change to a released API, and the mechanism it enables -- + a Secret layered like any other input -- needs no other schema footprint. + """ + spec = XRDS["fabric"]["spec"]["versions"][0]["schema"]["openAPIV3Schema"] + enum = spec["properties"]["spec"]["properties"]["requires"]["items"]["properties"]["kind"][ + "enum" + ] + assert set(enum) == set(KINDS) | {"Secret"} diff --git a/tests/test_avd_compat.py b/tests/test_avd_compat.py new file mode 100644 index 0000000..face1d7 --- /dev/null +++ b/tests/test_avd_compat.py @@ -0,0 +1,148 @@ +"""The two things `eos_designs-twodc-5stage-clos` needs, proven against golden. + +That scenario is the only fabric in AVD's corpus that turns on `pool_manager`, +and it also pins two `.j2` addressing templates. Both are unreachable through +pyavd's public API as written — Jinja is not implemented, and nothing composes a +pool — so it renders as a failure and the reason it reports is whichever one it +meets first. + +This is the offline oracle for both. With the pool supplied and the templates +replaced by the class AVD's own schema offers instead, the render must reproduce +AVD's checked-in golden **exactly**. Without that, a later failure in a cluster +could be the pool, the class or the render, and nothing would say which. + +⚠ It does not claim the *migrated* design renders. It pins `.j2` and always +will; the substitution below is what a fabric would carry instead. +""" + +from __future__ import annotations + +import shutil +import tempfile +from pathlib import Path + +import pytest +import yaml + +from function.kinds import resolve +from function.migrate import migrate + +pytestmark = pytest.mark.corpus + +SCENARIO = Path( + "avd/ansible_collections/arista/avd/extensions/molecule/eos_designs-twodc-5stage-clos" +) +COLLECTIONS = Path("avd").resolve() + +#: what the scenario's own group_vars name, relative to the working directory +POOL = Path("intended/data/test-ids.yml") + + +def _differences(path: str, ours: object, golden: object, out: list[str]) -> None: + if type(ours) is not type(golden) and not ( + isinstance(ours, (int, float)) and isinstance(golden, (int, float)) + ): + out.append(f"{path}: type {type(ours).__name__} != {type(golden).__name__}") + elif isinstance(golden, dict): + assert isinstance(ours, dict) + for key in sorted(set(ours) | set(golden)): + if key not in ours: + out.append(f"{path}.{key}: only in golden") + elif key not in golden: + out.append(f"{path}.{key}: only in ours") + else: + _differences(f"{path}.{key}", ours[key], golden[key], out) + elif isinstance(golden, list): + assert isinstance(ours, list) + if len(ours) != len(golden): + out.append(f"{path}: list len {len(ours)} != {len(golden)}") + for index, (a, b) in enumerate(zip(ours, golden)): + _differences(f"{path}[{index}]", a, b, out) + elif ours != golden: + out.append(f"{path}: {ours!r} != {golden!r}") + + +def _use_compat_class(design: dict) -> int: + """Point the node-type block at the class instead of the templates.""" + swapped = 0 + for entry in design.get("node_type_keys") or []: + block = entry.get("ip_addressing") if isinstance(entry, dict) else None + if isinstance(block, dict) and any(str(v).endswith(".j2") for v in block.values()): + entry["ip_addressing"] = { + "python_module": "function.avd_compat", + "python_class_name": "AvdIpAddressingV2Spine", + } + swapped += 1 + return swapped + + +def _point_at(design: dict, pool_file: Path) -> bool: + """Redirect `pools_file` at a copy of the pool. + + ⚠ When `pools_file` is set it is used **verbatim**, relative to the current + working directory — it is *not* joined to the PoolManager's output_dir, which + only supplies the default path. Getting that wrong silently assigns fresh IDs + and every address moves. + """ + node_id = (design.get("fabric_numbering") or {}).get("node_id") + if isinstance(node_id, dict) and node_id.get("pools_file"): + node_id["pools_file"] = str(pool_file) + return True + return False + + +def test_the_pool_and_the_compat_class_reproduce_avds_golden() -> None: + if not SCENARIO.is_dir(): + pytest.skip("AVD submodule not initialised") + + from pyavd.api.pool_manager import PoolManager + + from function.engine import render_structured_configs + + fabric = migrate(SCENARIO, collections=COLLECTIONS)[0] + assert sum(_use_compat_class(i.design) for i in fabric.inputs) == 1, ( + "expected exactly one node-type block pinning .j2 addressing" + ) + + with tempfile.TemporaryDirectory() as workdir: + # A copy, so a run can never write into AVD's own tree. + pool_file = Path(workdir) / POOL.name + shutil.copy(SCENARIO / POOL, pool_file) + assert any(_point_at(i.design, pool_file) for i in fabric.inputs), ( + "the scenario should still name a pools_file" + ) + + rendered = render_structured_configs( + resolve(fabric.inputs), pool_manager=PoolManager(Path(workdir)) + ) + + golden_dir = SCENARIO / "intended" / "structured_configs" + differences: list[str] = [] + compared = 0 + for hostname, structured in sorted(rendered.items()): + target = golden_dir / f"{hostname}.yml" + if not target.is_file(): + continue + compared += 1 + _differences(hostname, structured, + yaml.safe_load(target.read_text()) or {}, differences) + + assert compared == 26, f"expected AVD's 26 devices, compared {compared}" + assert not differences, ( + f"{len(differences)} differences: {differences[:5]}" + ) + + +def test_the_compat_class_only_changes_p2p_uplinks() -> None: + """Everything else falls through to AVD's own implementation, so selecting + the class cannot quietly move an address it was not written for.""" + from pyavd.api.ip_addressing import AvdIpAddressing + + from function.avd_compat import AvdIpAddressingV2Spine + + overridden = { + name for name in vars(AvdIpAddressingV2Spine) + if not name.startswith("_") and callable(getattr(AvdIpAddressingV2Spine, name)) + } + assert overridden == {"p2p_uplinks_ip", "p2p_uplinks_peer_ip"}, overridden + assert issubclass(AvdIpAddressingV2Spine, AvdIpAddressing) diff --git a/tests/test_categories.py b/tests/test_categories.py new file mode 100644 index 0000000..704804a --- /dev/null +++ b/tests/test_categories.py @@ -0,0 +1,126 @@ +"""The kind a key belongs to is AVD's statement, not ours. + +`function.kinds` classifies a top-level eos_designs key two ways, and this +guards both against upstream moving under us: + +* the **generated** part -- which key names exist at all -- comes from pyavd's + public schema at runtime, so a family gaining a member (upstream added + `cameras` to `connected_endpoints_keys`) needs no edit here. Nothing to guard; + it cannot drift. +* the **named** part -- the handful of keys that are not settings -- is a + literal, because AVD publishes that categorisation only as documentation + metadata. This test reads `documentation_options.table` out of AVD's own + schema and requires the two to agree in both directions. + +Offline, and reads the `avd` submodule the same way the golden tests do. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest +import yaml + +from function.kinds import Vocabulary, kind_of + +SCHEMA = Path("avd/python-avd/pyavd/_eos_designs/schema/eos_designs.schema.yml") + +#: AVD's documentation table -> the kind that owns it. The one editorial step, +#: and it is made on AVD's names rather than on any key's content. +TABLE_KIND = { + "node-type-structure": "NodeSet", + "type-setting": "NodeSet", + "node-type-keys": "NodeSet", + "node-type-l3-interfaces-configuration": "NodeSet", + "network-services": "NetworkServiceSet", + "network-services-l2vlans-settings": "NetworkServiceSet", + "network-services-vrfs-settings": "NetworkServiceSet", + "evpn-vlan-bundles": "NetworkServiceSet", + "connected-endpoints": "ConnectedEndpointSet", + "connected-endpoints-keys": "ConnectedEndpointSet", + "default-connected-endpoints-description": "ConnectedEndpointSet", + "default-network-ports-description": "ConnectedEndpointSet", +} + +#: Keys AVD tags with no table at all, placed here by this repo. Listed so the +#: test states them rather than silently tolerating them. +UNTAGGED = { + "port_profiles": "ConnectedEndpointSet", + "network_ports": "ConnectedEndpointSet", + "network_services_keys": "NetworkServiceSet", +} + + +def _schema() -> dict: + if not SCHEMA.is_file(): + pytest.skip(f"{SCHEMA} missing -- run `git submodule update --init`") + return yaml.safe_load(SCHEMA.read_text()) + + +def _tables() -> dict[str, str]: + """Top-level key -> AVD's documentation table.""" + return { + key: (body.get("documentation_options") or {}).get("table") or "" + for key, body in (_schema().get("keys") or {}).items() + } + + +def test_every_key_avd_categorises_lands_in_that_kind() -> None: + """AVD tags a key; we must agree. This is the direction that catches a + *new* upstream key we have never seen.""" + vocabulary = Vocabulary.default() + wrong = { + key: (kind_of(key, None, vocabulary), TABLE_KIND[table]) + for key, table in _tables().items() + if table in TABLE_KIND + } + wrong = {k: v for k, v in wrong.items() if v[0] != v[1]} + assert not wrong, f"kind_of disagrees with AVD's own table: {wrong}" + + +def test_no_key_is_promoted_out_of_settings_without_avd_saying_so() -> None: + """The other direction: nothing is quietly special-cased. A key we do not + call a setting must be one AVD categorises, a dynamic key name, or listed in + UNTAGGED with a reason.""" + vocabulary = Vocabulary.default() + dynamic = vocabulary.node_types | vocabulary.network_services | vocabulary.connected_endpoints + tables = _tables() + unexplained = { + key: kind_of(key, None, vocabulary) + for key in tables + if kind_of(key, None, vocabulary) != "SettingSet" + and key not in dynamic + and tables[key] not in TABLE_KIND + and key not in UNTAGGED + } + assert not unexplained, f"promoted out of SettingSet with nothing backing it: {unexplained}" + + +def test_untagged_placements_are_still_untagged_upstream() -> None: + """If AVD starts tagging one of these, the entry moves to TABLE_KIND and + stops being this repo's opinion.""" + tables = _tables() + now_tagged = {k: tables[k] for k in UNTAGGED if tables.get(k)} + assert not now_tagged, f"AVD now categorises these; move them to TABLE_KIND: {now_tagged}" + + +def test_the_dynamic_families_come_from_pyavd_not_from_a_literal() -> None: + """The defaults are read, so they cannot be short. `cameras` is the case + that proves it: it exists upstream and the literal this replaced lacked it.""" + vocabulary = Vocabulary.default() + schema_defaults = { + source: { + str(entry[field]) + for entry in ((_schema()["keys"].get(source) or {}).get("default") or []) + if isinstance(entry, dict) and entry.get(field) + } + for source, field in ( + ("node_type_keys", "key"), + ("network_services_keys", "name"), + ("connected_endpoints_keys", "key"), + ) + } + assert schema_defaults["node_type_keys"] <= vocabulary.node_types + assert schema_defaults["network_services_keys"] <= vocabulary.network_services + assert schema_defaults["connected_endpoints_keys"] <= vocabulary.connected_endpoints diff --git a/tests/test_e2e_migrated_corpus.py b/tests/test_e2e_migrated_corpus.py new file mode 100644 index 0000000..553bce2 --- /dev/null +++ b/tests/test_e2e_migrated_corpus.py @@ -0,0 +1,160 @@ +"""What `avd-migrate` emits applies to a cluster and renders AVD's own output. + +Requires a cluster from `scripts/kind-up.sh`: + + uv run pytest -m e2e tests/test_e2e_migrated_corpus.py -s + +The offline suite proves the migration reproduces Ansible's hostvars and that +pyavd turns them into AVD's golden. Neither of those meets the API server. This +does the whole chain -- AVD inventory -> XRs -> Crossplane -> rendered config -- +and compares the result against AVD's checked-in golden. + +⚠ It exists because a hand-written fabric is not the same shape as a migrated +one. The first e2e here carried `spec.design`, passed, and hid that the Fabric +XRD marked `design` **required** -- so every Fabric the migration emits, which +names its inputs in `spec.requires` and has no `design` at all, was rejected by +the API server. No offline test could see it: they drive RunFunction directly. + +No devices are pushed to: these fabrics have no `spec.push`, so nothing boots. +""" + +from __future__ import annotations + +import json +import os +import subprocess +import tempfile +import time +from pathlib import Path + +import pytest +import yaml + +pytestmark = pytest.mark.e2e + +CTX = os.getenv("AVD_KUBE_CONTEXT", "kind-avd") +# ⚠ One namespace per scenario, and it is not tidiness. `single-dc-l3ls`, +# `single-dc-l3ls-ipv6` and `single-dc-multipod-l3ls` all name their devices +# `dc1-leaf1a`, `dc1-spine1` and so on. Applied side by side, a lookup by +# `avd.netclab.dev/device=` label matched whichever came first and compared one +# scenario's render against another's golden -- 420 differences that said +# nothing about the code. +NS_PREFIX = "migrated" +EXAMPLES = Path("avd/ansible_collections/arista/avd/examples") +COLLECTIONS = Path("avd").resolve() +TIMEOUT = 300 + +# Every bundled example. They render clean offline, so anything that fails here +# is about the cluster -- which is the whole point of running them here. +# `campus-fabric` and `l2ls-fabric` write their hostnames in capitals, which is +# what found the RFC 1123 bug; `cv-pathfinder` carries ansible-vault and Jinja, +# resolved by the migration before the XRs are written. +SCENARIOS = [ + "single-dc-l3ls", + "single-dc-l3ls-ipv6", + "single-dc-multipod-l3ls", + "dual-dc-l3ls", + "l2ls-fabric", + "campus-fabric", + "isis-ldp-ipvpn", + "cv-pathfinder", +] + + +def _kubectl(namespace: str, *args: str, check: bool = True) -> str: + proc = subprocess.run( + ["kubectl", "--context", CTX, "-n", namespace, *args], + capture_output=True, text=True, check=check, + ) + return proc.stdout.strip() + + +def _differences(path: str, ours: object, golden: object, out: list[str]) -> None: + if type(ours) is not type(golden) and not ( + isinstance(ours, (int, float)) and isinstance(golden, (int, float)) + ): + out.append(f"{path}: type {type(ours).__name__} != {type(golden).__name__}") + elif isinstance(golden, dict): + assert isinstance(ours, dict) + for key in sorted(set(ours) | set(golden)): + if key not in ours: + out.append(f"{path}.{key}: only in golden") + elif key not in golden: + out.append(f"{path}.{key}: only in ours") + else: + _differences(f"{path}.{key}", ours[key], golden[key], out) + elif isinstance(golden, list): + assert isinstance(ours, list) + if len(ours) != len(golden): + out.append(f"{path}: list len {len(ours)} != {len(golden)}") + for index, (a, b) in enumerate(zip(ours, golden)): + _differences(f"{path}[{index}]", a, b, out) + elif ours != golden: + out.append(f"{path}: {ours!r} != {golden!r}") + + +def _wait_ready(namespace: str, fabric: str) -> str: + deadline = time.monotonic() + TIMEOUT + last = "" + while time.monotonic() < deadline: + last = _kubectl( + namespace, "get", "fabric", fabric, + "-o", 'jsonpath={.status.conditions[?(@.type=="Ready")].status}', + check=False, + ) + if last == "True": + return last + time.sleep(5) + pytest.fail(f"{fabric} never went Ready within {TIMEOUT}s; last saw {last!r}") + + +@pytest.mark.parametrize("scenario", SCENARIOS) +def test_migrated_xrs_render_the_golden_on_a_cluster(scenario: str) -> None: + root = EXAMPLES / scenario + if not root.is_dir(): + pytest.skip("AVD submodule not initialised") + + namespace = f"{NS_PREFIX}-{scenario}" + subprocess.run( + ["kubectl", "--context", CTX, "apply", "-f", "-"], + input=yaml.safe_dump({"apiVersion": "v1", "kind": "Namespace", + "metadata": {"name": namespace}}), + capture_output=True, text=True, check=True, + ) + + with tempfile.TemporaryDirectory() as out: + subprocess.run( + ["uv", "run", "avd-migrate", str(root), "--emit", out, + "--namespace", namespace, "--collections", str(COLLECTIONS)], + capture_output=True, text=True, check=True, + ) + manifests = sorted(Path(out).glob("*.yaml")) + assert len(manifests) == 1, f"expected one fabric, emitted {manifests}" + subprocess.run( + ["kubectl", "--context", CTX, "apply", "-f", str(manifests[0])], + capture_output=True, text=True, check=True, + ) + fabric = yaml.safe_load(manifests[0].read_text().split("---")[-1])["metadata"]["name"] + + _wait_ready(namespace, fabric) + + golden_dir = root / "intended" / "structured_configs" + differences: list[str] = [] + compared = 0 + for golden_file in sorted(golden_dir.glob("*.yml")): + hostname = golden_file.stem + raw = _kubectl( + namespace, "get", "device", "-l", f"avd.netclab.dev/device={hostname}", + "-o", "jsonpath={.items[0].spec.structuredConfig}", check=False, + ) + if not raw: + differences.append(f"{hostname}: no Device composed") + continue + compared += 1 + _differences(hostname, json.loads(raw), + yaml.safe_load(golden_file.read_text()) or {}, differences) + + assert compared == len(list(golden_dir.glob("*.yml"))), ( + f"{scenario}: rendered {compared} of {len(list(golden_dir.glob('*.yml')))} devices" + ) + assert not differences, f"{scenario}: {len(differences)} differences: {differences[:5]}" diff --git a/tests/test_e2e_node_id_pool.py b/tests/test_e2e_node_id_pool.py new file mode 100644 index 0000000..8302f01 --- /dev/null +++ b/tests/test_e2e_node_id_pool.py @@ -0,0 +1,209 @@ +"""A Fabric keeps its node-ID pool on a cluster, proven against AVD's own output. + +Requires a cluster from `scripts/kind-up.sh`: + + uv run pytest -m e2e tests/test_e2e_node_id_pool.py -s + +The scenario is **AVD's own**, not one written for the test. +`eos_designs-twodc-5stage-clos` is the only fabric in AVD's corpus that turns on +`pool_manager`, and it ships the assignments AVD generated -- 26 of them, in +`intended/data/test-ids.yml`, beside the golden configs those assignments +produced. That makes the strongest possible assertion available: **seed the pool +and the render must reproduce the golden exactly**. If the seed is ignored, or +the pool is not read back, ids move and every derived address moves with them. + +Two substitutions the migration deliberately does not make, both stated by +`avd-migrate` in its own output rather than done silently: + +* the design pins two `.j2` addressing templates, and pyavd implements no Jinja + templating -- swapped here for `function.avd_compat`, which is the route AVD's + schema offers instead (`python_module`) and which this image ships; +* the assignments live in a file that does not travel with a migration -- carried + in as a seed ConfigMap. + +No devices are pushed to: the migrated fabric has no `spec.push`, so 26 devices +cost no cEOS at all. +""" + +from __future__ import annotations + +import json +import os +import subprocess +import time +from pathlib import Path + +import pytest +import yaml + +pytestmark = [pytest.mark.e2e] + +CTX = os.getenv("AVD_KUBE_CONTEXT", "kind-avd") +NS = "twodc" +SCENARIO = Path( + "avd/ansible_collections/arista/avd/extensions/molecule/eos_designs-twodc-5stage-clos" +) +COLLECTIONS = Path("avd").resolve() +SEED_NAME = "twodc-seed-ids" +TIMEOUT = 300 + + +def _kubectl(*args: str, check: bool = True) -> str: + proc = subprocess.run( + ["kubectl", "--context", CTX, "-n", NS, *args], + capture_output=True, text=True, check=check, + ) + return proc.stdout.strip() + + +def _apply(document: object) -> None: + subprocess.run( + ["kubectl", "--context", CTX, "apply", "-f", "-"], + input=yaml.safe_dump_all(document if isinstance(document, list) else [document]), + capture_output=True, text=True, check=True, + ) + + +def _differences(path: str, ours: object, golden: object, out: list[str]) -> None: + if type(ours) is not type(golden) and not ( + isinstance(ours, (int, float)) and isinstance(golden, (int, float)) + ): + out.append(f"{path}: type {type(ours).__name__} != {type(golden).__name__}") + elif isinstance(golden, dict): + assert isinstance(ours, dict) + for key in sorted(set(ours) | set(golden)): + if key not in ours: + out.append(f"{path}.{key}: only in golden") + elif key not in golden: + out.append(f"{path}.{key}: only in ours") + else: + _differences(f"{path}.{key}", ours[key], golden[key], out) + elif isinstance(golden, list): + assert isinstance(ours, list) + if len(ours) != len(golden): + out.append(f"{path}: list len {len(ours)} != {len(golden)}") + for index, (a, b) in enumerate(zip(ours, golden)): + _differences(f"{path}[{index}]", a, b, out) + elif ours != golden: + out.append(f"{path}: {ours!r} != {golden!r}") + + +def _use_compat_class(document: dict) -> int: + swapped = 0 + for entry in (document.get("spec", {}).get("design", {}).get("node_type_keys") or []): + block = entry.get("ip_addressing") if isinstance(entry, dict) else None + if isinstance(block, dict) and any(str(v).endswith(".j2") for v in block.values()): + entry["ip_addressing"] = { + "python_module": "function.avd_compat", + "python_class_name": "AvdIpAddressingV2Spine", + } + swapped += 1 + return swapped + + +@pytest.fixture(scope="module") +def fabric() -> str: + if not SCENARIO.is_dir(): + pytest.skip("AVD submodule not initialised") + + from function import pools + + # A namespace still Terminating from a previous run refuses new objects, and + # re-running a test right after cleaning up is the normal case. + deadline = time.monotonic() + 60 + while True: + try: + _apply({"apiVersion": "v1", "kind": "Namespace", "metadata": {"name": NS}}) + break + except subprocess.CalledProcessError: + if time.monotonic() > deadline: + raise + time.sleep(3) + _apply({ + "apiVersion": "v1", "kind": "ConfigMap", + "metadata": {"name": SEED_NAME, "namespace": NS}, + "data": {pools.DATA_KEY: (SCENARIO / "intended/data/test-ids.yml").read_text()}, + }) + + # ⚠ The first play, by index, not the first file by name. This scenario runs + # eos_designs twice over the same hosts and the **second** play sets + # `avd_digital_twin_mode: true`, rendering a different config into a + # different golden directory. Sorting the emitted filenames picked that one + # and compared it against the other's golden -- 12 differences that were + # entirely the test's fault. + from function.migrate import migrate, to_manifests + + fabrics = migrate(SCENARIO, collections=COLLECTIONS) + first = min(fabrics, key=lambda f: (f.play.playbook, f.play.index)) + documents = to_manifests(first, namespace=NS) + + swapped = sum(_use_compat_class(d) for d in documents if d["kind"] == "NodeSet") + assert swapped == 1, f"expected one node-type block pinning .j2 addressing, got {swapped}" + + name = "" + for document in documents: + if document["kind"] == "Fabric": + document["spec"]["nodeIdPool"] = {"seedConfigMapName": SEED_NAME} + name = document["metadata"]["name"] + assert name, "no Fabric emitted" + + _apply(documents) + deadline = time.monotonic() + TIMEOUT + while time.monotonic() < deadline: + if _kubectl("get", "fabric", name, "-o", + 'jsonpath={.status.conditions[?(@.type=="Ready")].status}', + check=False) == "True": + return name + time.sleep(5) + pytest.fail(f"{name} never went Ready; status: {_kubectl('get', 'fabric', name, '-o', 'jsonpath={.status}', check=False)[:400]}") + + +def test_the_seeded_fabric_renders_avds_golden(fabric: str) -> None: + """⚠ The strongest assertion available, and it needs every piece at once. + + The golden was produced with the assignments in the seed. Reproducing it + means the seed was read, the pool was composed and read back, the compat + class computed the v2.x addresses, and the migration carried the rest -- + any one of those failing moves an address and fails this. + """ + golden_dir = SCENARIO / "intended" / "structured_configs" + goldens = sorted(golden_dir.glob("*.yml")) + differences: list[str] = [] + compared = 0 + for golden_file in goldens: + hostname = golden_file.stem + raw = _kubectl( + "get", "device", "-l", f"avd.netclab.dev/device={hostname}", + "-o", "jsonpath={.items[0].spec.structuredConfig}", check=False, + ) + if not raw: + differences.append(f"{hostname}: no Device composed") + continue + compared += 1 + _differences(hostname, json.loads(raw), + yaml.safe_load(golden_file.read_text()) or {}, differences) + + assert compared == len(goldens), f"rendered {compared} of {len(goldens)} devices" + assert not differences, f"{len(differences)} differences: {differences[:5]}" + + +def test_the_fabric_kept_the_seeded_assignments(fabric: str) -> None: + """The pool is composed from the seed, not started over beside it.""" + from function import pools + + # ⚠ Not `avd.netclab.dev/fabric`: that label is on all 27 ConfigMaps this + # fabric composes -- 26 device renders and the pool -- so selecting on it and + # taking the first match reads an `eos.cfg`. The pool says what it is. + name = _kubectl( + "get", "cm", "-l", "avd.netclab.dev/artifact=node-id-pool", + "-o", "jsonpath={.items[*].metadata.name}", + ).split() + assert len(name) == 1, f"expected one pool, found {len(name)}" + + body = yaml.safe_load( + _kubectl("get", "cm", name[0], "-o", rf"jsonpath={{.data.{pools.DATA_KEY.replace('.', chr(92) + '.')}}}") + ) or {} + seeded = yaml.safe_load((SCENARIO / "intended/data/test-ids.yml").read_text()) or {} + assert body.get("node_id_pools") == seeded.get("node_id_pools"), ( + "the composed pool differs from the seed it was given" + ) diff --git a/tests/test_engine_fidelity.py b/tests/test_engine_fidelity.py deleted file mode 100644 index f6804ea..0000000 --- a/tests/test_engine_fidelity.py +++ /dev/null @@ -1,48 +0,0 @@ -"""Milestone 1: the pyavd engine reproduces AVD's golden structured config. - -Offline -- no cluster. This is `avd-verify` as assertions instead of a report. -""" - -from __future__ import annotations - -from pathlib import Path - -import pytest -import yaml - -from function.ansible_inputs import build_all_inputs -from function.engine import render_structured_configs -from function.verify_example import _diff - -EXAMPLES_ROOT = Path("avd/ansible_collections/arista/avd/examples") - -# Every bundled example that reproduces golden from full Ansible inputs. -# cv-pathfinder is deferred (ansible-vault secrets); `common` holds shared vars -# and is not a runnable fabric. -EXAMPLES = [ - "single-dc-l3ls", - "single-dc-l3ls-ipv6", - "single-dc-multipod-l3ls", - "dual-dc-l3ls", - "campus-fabric", - "l2ls-fabric", - "isis-ldp-ipvpn", -] - - -@pytest.mark.parametrize("example", EXAMPLES) -def test_reproduces_golden_structured_config(example: str) -> None: - example_dir = EXAMPLES_ROOT / example - golden_dir = example_dir / "intended" / "structured_configs" - - rendered = render_structured_configs(build_all_inputs(example_dir)) - - # The device sets must match first: "zero diffs" over an empty render would - # otherwise pass vacuously. - assert set(rendered) == {p.stem for p in golden_dir.glob("*.yml")} - - diffs: list[str] = [] - for hostname in sorted(rendered): - gold = yaml.safe_load((golden_dir / f"{hostname}.yml").read_text()) or {} - _diff(hostname, rendered[hostname], gold, diffs) - assert diffs == [], "\n".join(diffs[:20]) diff --git a/tests/test_fabric_collect.py b/tests/test_fabric_collect.py new file mode 100644 index 0000000..5a2eca3 --- /dev/null +++ b/tests/test_fabric_collect.py @@ -0,0 +1,490 @@ +"""A Fabric collects the inputs it names, and refuses to render without them. + +Offline -- drives RunFunction directly, with no cluster and no Crossplane. The +gate is the most safety-critical piece in the collect path: requirements are +answered on the *next* reconcile, so the first one always arrives with nothing, +and a fabric rendered short of its inputs would be pushed to devices as a full +config replacement. +""" + +from __future__ import annotations + +import asyncio + +import pytest +from crossplane.function import resource +from crossplane.function.proto.v1 import run_function_pb2 as fnv1 + +from function.fn import FunctionRunner + +API = "avd.netclab.dev/v1alpha1" + + +def _run(req: fnv1.RunFunctionRequest) -> fnv1.RunFunctionResponse: + return asyncio.run(FunctionRunner().RunFunction(req, None)) + + +def _fabric(requires: list[dict], design: dict | None = None) -> dict: + return { + "apiVersion": API, + "kind": "Fabric", + "metadata": {"name": "fabric", "namespace": "avd"}, + "spec": {"fabricName": "FABRIC", "design": design or {}, "requires": requires}, + } + + +def _input_xr(kind: str, name: str, spec: dict) -> dict: + return { + "apiVersion": API, + "kind": kind, + "metadata": {"name": name, "namespace": "avd"}, + "spec": spec, + } + + +def _request(xr: dict, required: dict[str, list[dict]] | None = None) -> fnv1.RunFunctionRequest: + req = fnv1.RunFunctionRequest() + req.observed.composite.resource.CopyFrom(resource.dict_to_struct(xr)) + for key, objects in (required or {}).items(): + # An empty list is Crossplane saying "I looked and found nothing", which + # the proto distinguishes from a key that is absent entirely. + entry = req.required_resources[key] + for obj in objects: + entry.items.add().resource.CopyFrom(resource.dict_to_struct(obj)) + return req + + +def _condition(rsp: fnv1.RunFunctionResponse, typ: str): + return next((c for c in rsp.conditions if c.type == typ), None) + + +# A spine rather than a leaf, only because a leaf defaults to being a VTEP and +# would drag in the VXLAN pools -- this fixture is about the collect path, not +# about exercising AVD. +SPINES = _input_xr( + "NodeSet", + "spines", + { + # `type` rides in the same input: AVD needs it (or default_node_types) + # to know what the device is, and it applies to whoever sees this input. + "design": { + "type": "spine", + "spine": { + "defaults": {"loopback_ipv4_pool": "10.255.0.0/27"}, + "nodes": [{"name": "spine1", "id": 1, "bgp_as": 65100}], + }, + } + }, +) + + +def test_first_reconcile_asks_and_renders_nothing() -> None: + """Requirements are answered next time round, so the first pass is empty. + + This is the case the gate exists for: not a slow operator, but the protocol. + """ + rsp = _run(_request(_fabric([{"kind": "NodeSet", "name": "spines"}]))) + + assert set(rsp.requirements.resources) == {"000-nodeset-spines"} + selector = rsp.requirements.resources["000-nodeset-spines"] + assert (selector.kind, selector.match_name, selector.namespace) == ("NodeSet", "spines", "avd") + + assert not rsp.desired.resources, "nothing may be composed before the inputs arrive" + condition = _condition(rsp, "InputsResolved") + assert condition.reason == "WaitingForInputs" + + +def test_missing_input_is_distinguished_from_not_yet_fetched() -> None: + """An empty Resources means Crossplane looked and found nothing.""" + rsp = _run( + _request( + _fabric([{"kind": "NodeSet", "name": "spines"}]), + required={"000-nodeset-spines": []}, + ) + ) + + assert not rsp.desired.resources + assert _condition(rsp, "InputsResolved").reason == "InputsMissing" + assert "not found" in _condition(rsp, "InputsResolved").message + + +def test_resolved_inputs_compose_devices() -> None: + rsp = _run( + _request( + _fabric([{"kind": "NodeSet", "name": "spines"}]), + required={"000-nodeset-spines": [SPINES]}, + ) + ) + + assert _condition(rsp, "InputsResolved").status == fnv1.STATUS_CONDITION_TRUE + assert set(rsp.desired.resources) == {"spine1"} + + +def test_design_without_requires_still_composes() -> None: + """The released path is untouched: one document, handed to every device. + + Guards the refactor that put both paths through render_structured_configs -- + v0.1.6 is published and netclab-xp pins it, so this must keep working with no + inputs in sight. + """ + rsp = _run(_request(_fabric(requires=[], design=SPINES["spec"]["design"]))) + + assert not any(r.severity == fnv1.SEVERITY_FATAL for r in rsp.results) + assert set(rsp.desired.resources) == {"spine1"} + assert not rsp.requirements.resources, "a fabric with no requires asks for nothing" + + +def test_secret_input_is_refused_until_implemented() -> None: + """It is in the enum so the mechanism can land without a schema change. + + Rendering a fabric whose credentials are silently absent would push a config + without them, so refusing is the only safe placeholder. + """ + rsp = _run(_request(_fabric([{"kind": "Secret", "name": "creds"}]))) + + assert any(r.severity == fnv1.SEVERITY_FATAL for r in rsp.results) + assert not rsp.desired.resources + + +def test_pattern_matching_no_device_is_refused() -> None: + """A pattern is silent about matching nothing, so it cannot be allowed to.""" + settings = _input_xr( + "SettingSet", + "typo", + # The fabric holds only spine1, so this pattern matches nothing. + {"design": {"ntp_settings": {}}, "appliesTo": {"matchHostnames": ["leaf.*"]}}, + ) + rsp = _run( + _request( + _fabric( + [ + {"kind": "NodeSet", "name": "spines"}, + {"kind": "SettingSet", "name": "typo"}, + ] + ), + required={"000-nodeset-spines": [SPINES], "001-settingset-typo": [settings]}, + ) + ) + + fatal = [r for r in rsp.results if r.severity == fnv1.SEVERITY_FATAL] + assert fatal and "matched no device" in fatal[0].message + assert not rsp.desired.resources + + +@pytest.mark.parametrize("kind", ["NodeSet", "NetworkServiceSet", "ConnectedEndpointSet", "SettingSet"]) +def test_input_kinds_reconcile_and_report_their_keys(kind: str) -> None: + """Each input reports its own shape on its own object, composing nothing.""" + rsp = _run(_request(_input_xr(kind, "an-input", {"design": {"ntp_settings": {}, "type": "x"}}))) + + assert not any(r.severity == fnv1.SEVERITY_FATAL for r in rsp.results) + status = resource.struct_to_dict(rsp.desired.composite.resource).get("status", {}) + assert status["keys"] == ["ntp_settings", "type"] + + +# --- node-ID pools ---------------------------------------------------------- + +def _pooled(*, static_id: int | None) -> dict: + """The spine design with node IDs coming from a pool. + + ⚠ `static_id` is the whole point of having two variants. AVD *reserves* an + id written on the node, so a pool holding a different number for that device + is refused rather than applied. Only a device with no id of its own takes + what the pool holds. + """ + node = {"name": "spine1", "bgp_as": 65100} + if static_id is not None: + node["id"] = static_id + return { + "type": "spine", + "spine": { + "defaults": {"loopback_ipv4_pool": "10.255.0.0/27"}, + "nodes": [node], + }, + "fabric_numbering": { + "node_id": {"algorithm": "pool_manager", "pools_file": "intended/data/x-ids.yml"} + }, + } + + +POOLED = _pooled(static_id=None) + + +def _pool_configmap(pool: str) -> dict: + from function import pools + + return { + "apiVersion": "v1", + "kind": "ConfigMap", + "metadata": {"name": "fabric-id-pool", "namespace": "avd"}, + "data": {pools.DATA_KEY: pool}, + } + + +def _with_observed(req: fnv1.RunFunctionRequest, name: str, obj: dict) -> fnv1.RunFunctionRequest: + req.observed.resources[name].resource.CopyFrom(resource.dict_to_struct(obj)) + return req + + +def test_a_fabric_asking_for_a_pool_composes_one() -> None: + """`pool_manager` keeps its assignments in a file. There is no file in a + cluster, so the Fabric composes a ConfigMap and reads it back next time.""" + from function import pools + + rsp = _run(_request(_fabric(requires=[], design=POOLED))) + + assert not any(r.severity == fnv1.SEVERITY_FATAL for r in rsp.results), rsp.results + assert pools.RESOURCE_NAME in rsp.desired.resources, "no pool was composed" + + composed = resource.struct_to_dict(rsp.desired.resources[pools.RESOURCE_NAME].resource) + body = composed["data"][pools.DATA_KEY] + assert "spine1" in body, body + # `child_name` appends a hash, as it does for every composed resource. + assert composed["metadata"]["name"].startswith("fabric-id-pool") + + +def test_a_fabric_without_a_pool_composes_none() -> None: + """Nothing is created for a fabric that never asked -- an empty object with + a warning about renumbering would be worse than no object.""" + from function import pools + + rsp = _run(_request(_fabric(requires=[], design=SPINES["spec"]["design"]))) + assert pools.RESOURCE_NAME not in rsp.desired.resources + + +def test_assignments_survive_the_next_reconcile() -> None: + """Two identical reconciles produce the same pool and the same config. + + ⚠ **Weak on its own, deliberately kept.** Assignment is deterministic from + the device set, so this passes even when the observed pool is ignored + entirely -- verified by making `observed_pool` return nothing. It guards + idempotency; `test_the_pool_decides_the_ids` is what proves the ConfigMap is + read at all. + """ + from function import pools + + first = _run(_request(_fabric(requires=[], design=POOLED))) + pool = resource.struct_to_dict( + first.desired.resources[pools.RESOURCE_NAME].resource + )["data"][pools.DATA_KEY] + + second = _run( + _with_observed( + _request(_fabric(requires=[], design=POOLED)), + pools.RESOURCE_NAME, + _pool_configmap(pool), + ) + ) + again = resource.struct_to_dict( + second.desired.resources[pools.RESOURCE_NAME].resource + )["data"][pools.DATA_KEY] + + assert again == pool, "the pool moved between two identical reconciles" + + def rendered(rsp: fnv1.RunFunctionResponse) -> str: + cm = resource.struct_to_dict(rsp.desired.resources["spine1"].resource) + return str(cm) + + assert rendered(second) == rendered(first), "the device changed although its ID did not" + + +def test_the_designs_own_pools_file_is_not_followed() -> None: + """It names a path relative to somebody's working directory. Honouring it in + a cluster would read nothing and quietly assign a fresh set of IDs.""" + from function import pools + + rsp = _run(_request(_fabric(requires=[], design=POOLED))) + body = resource.struct_to_dict( + rsp.desired.resources[pools.RESOURCE_NAME].resource + )["data"][pools.DATA_KEY] + assert body.strip(), "the pool came back empty -- the file was read from the wrong place" + + +def test_the_pool_decides_the_ids() -> None: + """The pool is read, not merely written. + + A fabric handed a pool that assigns `spine1` the id 7 must render the device + with id 7 -- AVD reserves an existing assignment rather than handing out the + next free number. Without this, every test here passes on a function that + throws the ConfigMap away and reassigns from scratch, because a fresh pool + over an unchanged device set produces the same numbers. + """ + from function import pools + + fresh = _run(_request(_fabric(requires=[], design=POOLED))) + assert _loopback(fresh) == "10.255.0.1/32", "unexpected baseline" + + + moved = _pool_configmap( + "node_id_pools:\n" + " fabric_name=FABRIC/type=spine:\n" + " hostname=spine1: 7\n" + ) + rsp = _run( + _with_observed( + _request(_fabric(requires=[], design=POOLED)), pools.RESOURCE_NAME, moved + ) + ) + + assert _loopback(rsp) == "10.255.0.7/32", ( + "the device did not take the id the pool holds -- the pool was not read" + ) + kept = resource.struct_to_dict( + rsp.desired.resources[pools.RESOURCE_NAME].resource + )["data"][pools.DATA_KEY] + assert "hostname=spine1: 7" in kept, kept + + +def _loopback(rsp: fnv1.RunFunctionResponse) -> str: + device = resource.struct_to_dict(rsp.desired.resources["spine1"].resource) + loopbacks = device["spec"]["structuredConfig"].get("loopback_interfaces") or [] + return next(i["ip_address"] for i in loopbacks if i["name"] == "Loopback0") + + +def test_an_id_written_on_the_node_outranks_the_pool() -> None: + """AVD reserves a statically set id; the pool does not get to move it. + + Worth a test because the opposite is the natural expectation -- "the pool + assigns ids" -- and getting it backwards would mean quietly renumbering a + device whose id somebody wrote down on purpose. + """ + from function import pools + + design = _pooled(static_id=1) + rsp = _run( + _with_observed( + _request(_fabric(requires=[], design=design)), + pools.RESOURCE_NAME, + _pool_configmap( + "node_id_pools:\n" + " fabric_name=FABRIC/type=spine:\n" + " hostname=spine1: 7\n" + ), + ) + ) + + assert _loopback(rsp) == "10.255.0.1/32", "the pool overrode an id set on the node" + + +def _seeded(design: dict, name: str = "old-ids") -> dict: + xr = _fabric(requires=[], design=design) + xr["spec"]["nodeIdPool"] = {"seedConfigMapName": name} + return xr + + +def test_a_named_seed_gates_the_first_reconcile() -> None: + """⚠ Without this gate the seed could never work. + + Requirements are answered on the *next* reconcile, so the first one arrives + with nothing. Rendering then would assign a fresh set of IDs and compose + them as the pool, and the seed would be read into a fabric that had already + renumbered itself. + """ + from function import pools + + rsp = _run(_request(_seeded(POOLED))) + + assert pools.RESOURCE_NAME not in rsp.desired.resources, "a pool was composed anyway" + assert not rsp.desired.resources, "nothing may be composed before the seed arrives" + assert _condition(rsp, "InputsResolved").reason == "WaitingForSeed" + assert pools.SEED_NAME in rsp.requirements.resources + + +def test_a_seed_that_does_not_exist_is_fatal() -> None: + """Named and absent is an error, not an empty pool: carrying on renumbers + every device, which is the one thing the field exists to prevent.""" + rsp = _run(_request(_seeded(POOLED), required={"id-pool-seed": []})) + + assert any(r.severity == fnv1.SEVERITY_FATAL for r in rsp.results) + assert "renumber" in " ".join(r.message for r in rsp.results) + + +def test_a_seed_supplies_the_first_pool() -> None: + """The migration case: a fabric that was already running elsewhere keeps the + IDs AVD gave it, instead of starting over.""" + from function import pools + + seed = { + "apiVersion": "v1", + "kind": "ConfigMap", + "metadata": {"name": "old-ids", "namespace": "avd"}, + "data": { + pools.DATA_KEY: "node_id_pools:\n" + " fabric_name=FABRIC/type=spine:\n" + " hostname=spine1: 9\n" + }, + } + rsp = _run(_request(_seeded(POOLED), required={"id-pool-seed": [seed]})) + + assert _loopback(rsp) == "10.255.0.9/32", "the seeded assignment was not used" + kept = resource.struct_to_dict( + rsp.desired.resources[pools.RESOURCE_NAME].resource + )["data"][pools.DATA_KEY] + assert "hostname=spine1: 9" in kept + + +def test_a_seed_never_overrides_a_pool_the_fabric_already_has() -> None: + """It seeds, it does not steer. Once the fabric keeps its own assignments, + an old ConfigMap left lying around must not pull them back.""" + from function import pools + + seed = { + "apiVersion": "v1", + "kind": "ConfigMap", + "metadata": {"name": "old-ids", "namespace": "avd"}, + "data": { + pools.DATA_KEY: "node_id_pools:\n" + " fabric_name=FABRIC/type=spine:\n" + " hostname=spine1: 9\n" + }, + } + req = _request(_seeded(POOLED), required={"id-pool-seed": [seed]}) + _with_observed( + req, + pools.RESOURCE_NAME, + _pool_configmap( + "node_id_pools:\n fabric_name=FABRIC/type=spine:\n hostname=spine1: 3\n" + ), + ) + rsp = _run(req) + + assert _loopback(rsp) == "10.255.0.3/32", "the seed overrode the fabric's own pool" + + +# --- hostnames a Kubernetes name cannot spell ------------------------------- + +def _spines(*names: str) -> dict: + return { + "type": "spine", + "spine": { + "defaults": {"loopback_ipv4_pool": "10.255.0.0/27"}, + "nodes": [{"name": n, "id": i + 1, "bgp_as": 65100 + i} + for i, n in enumerate(names)], + }, + } + + +def test_an_uppercase_hostname_still_composes() -> None: + """AVD hostnames are free text, and two of its eight bundled examples -- + `campus-fabric` and `l2ls-fabric` -- write them entirely in capitals. + + ⚠ Found on a cluster, not here: a composed resource named after one is + rejected outright (*"invalid name ... Must be a valid RFC 1123 subdomain + name"*) and the whole fabric fails to compose. Offline tests never met it + because they never reach an API server. + """ + rsp = _run(_request(_fabric(requires=[], design=_spines("SPINE2")))) + + assert not any(r.severity == fnv1.SEVERITY_FATAL for r in rsp.results), rsp.results + device = resource.struct_to_dict(rsp.desired.resources["SPINE2"].resource) + assert device["metadata"]["name"].startswith("fabric-spine2-"), device["metadata"]["name"] + # The hostname itself is untouched -- only the object's name is spelled + # differently, or the render would be for a device that does not exist. + assert device["spec"]["hostname"] == "SPINE2" + + +def test_two_hostnames_needing_one_object_name_are_refused() -> None: + """One Device standing in for two switches would push one config to both.""" + rsp = _run(_request(_fabric(requires=[], design=_spines("SPINE1", "spine1")))) + + assert any(r.severity == fnv1.SEVERITY_FATAL for r in rsp.results), rsp.results + assert "rename one" in " ".join(r.message for r in rsp.results) diff --git a/tests/test_kinds_equivalence.py b/tests/test_kinds_equivalence.py new file mode 100644 index 0000000..6265fb0 --- /dev/null +++ b/tests/test_kinds_equivalence.py @@ -0,0 +1,478 @@ +"""The input-kind model resolves exactly as Ansible does. + +The reference side is **real Ansible** -- `ansible-inventory --list`, run by +:mod:`function.ansible_cli`. That matters more than it sounds. The previous +version of this test compared two of our own readers against each other, so a +source neither read vanished from both sides and the comparison agreed. Four +such gaps sat behind a green result and surfaced only when a render disagreed +with AVD's own golden: `host_vars/*.yaml`, `host_vars//` directories, +inline group `vars:` blocks, and ansible-vault. + +Byte equality is the assertion. It is stricter than necessary -- a hostvar AVD +never reads cannot change a rendered config -- and that is deliberate: it fails +before a render can hide a difference. + +The 8 bundled examples run by default. The molecule corpus is behind +``-m corpus``: it reaches 501 devices in one play and 71 plays in one scenario, +and every one of them costs an ansible subprocess. +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +from function.ansible_cli import read_inventory +from function.kinds import Input, Vocabulary, by_kind, classify, resolve +from function.migrate import ( + Fabric, + MigrationError, + _discover, + drop_description_templates, + migrate, + to_manifests, + unsupported, +) + +AVD = Path("avd/ansible_collections/arista/avd") +EXAMPLES_ROOT = AVD / "examples" +MOLECULE_ROOT = AVD / "extensions" / "molecule" +COLLECTIONS = Path("avd").resolve() + +EXAMPLES = _discover(EXAMPLES_ROOT) if EXAMPLES_ROOT.is_dir() else [] +MOLECULE = _discover(MOLECULE_ROOT) if MOLECULE_ROOT.is_dir() else [] + +# Scenarios with no eos_designs play at all. Not deferrals -- correct answers: +# the three `eos_cli_config_gen` scenarios carry structured config directly and +# render at the device layer, and cv_deploy/cv_workflow push what was built. +NO_DESIGN_PLAY = { + "cv_deploy", "cv_workflow", "eos_cli_config_gen", + "eos_cli_config_gen_deprecated_vars", "eos_cli_config_gen_negative_unit_tests", +} + +# Renders that still meet an AVD feature this path does not carry. Expected- +# failure semantics: one that starts passing is reported, so a deferral cannot +# rot silently. **All eight examples render clean**, so this covers the molecule +# corpus only -- and every entry is one of three causes, not eight. +DEFERRED_RENDER: dict[str, str] = {} + +DEFERRED_RENDER_CORPUS = { + # pyavd implements no Jinja templating: `get_device_structured_config` passes + # `templar=None` and the call raises `NotImplementedError`. Neither public + # entry point accepts a templar, so carrying the .j2 files on an XR would + # change nothing -- nothing would read them. Upstream, not ours. + "ansible_only": "custom ip_addressing template -- pyavd implements no Jinja templating", + "evpn_underlay_ebgp_overlay_ebgp": "custom interface_descriptions templates -- " + "pyavd implements no Jinja templating", + # Two blockers, not one: `pool_manager` needs a pool that survives + # reconciliation, and the design also pins .j2 addressing. The *migrated* + # design always fails here, because it faithfully carries both. + # ⚠ Both are solved and proven in `tests/test_avd_compat.py`, which renders + # this scenario clean against AVD's golden with the pool supplied and the + # templates replaced by the class AVD's schema offers instead. What is left + # is somewhere for a Fabric to *keep* the pool. + "eos_designs-twodc-5stage-clos": "pool_manager + .j2 addressing -- see test_avd_compat", + # Code, not data: a function image is immutable and must not load arbitrary + # Python. ⚠ And this scenario is not a fabric -- 59 unrelated feature groups + # under one play, with six different `fabric_name` values. + "eos_designs_unit_tests": "templates.*.python_module -- code, not data", +} + + +def _differences(path: str, ours: object, golden: object, out: list[str]) -> None: + """Collect readable differences between a rendered config and its golden.""" + if type(ours) is not type(golden) and not ( + isinstance(ours, (int, float)) and isinstance(golden, (int, float)) + ): + out.append(f"{path}: type {type(ours).__name__} != {type(golden).__name__}") + elif isinstance(golden, dict): + assert isinstance(ours, dict) + for key in sorted(set(ours) | set(golden)): + if key not in ours: + out.append(f"{path}.{key}: only in golden") + elif key not in golden: + out.append(f"{path}.{key}: only in ours") + else: + _differences(f"{path}.{key}", ours[key], golden[key], out) + elif isinstance(golden, list): + assert isinstance(ours, list) + if len(ours) != len(golden): + out.append(f"{path}: list len {len(ours)} != {len(golden)}") + for index, (a, b) in enumerate(zip(ours, golden)): + _differences(f"{path}[{index}]", a, b, out) + elif ours != golden: + out.append(f"{path}: {ours!r} != {golden!r}") + + +def _fabrics(root: Path) -> tuple[list[Fabric], dict[str, dict]]: + inv = read_inventory(root, collections=COLLECTIONS) + return migrate(root, collections=COLLECTIONS, inv=inv), inv.hostvars + + +def _assert_equivalent(root: Path) -> None: + if root.name in NO_DESIGN_PLAY: + with pytest.raises(MigrationError): + migrate(root, collections=COLLECTIONS) + return + + fabrics, hostvars = _fabrics(root) + assert fabrics, f"{root.name}: no fabric produced" + for fabric in fabrics: + got = resolve(fabric.inputs) + want = {device: hostvars.get(device, {}) for device in fabric.devices} + differing = sorted( + f"{host}.{key}" + for host in set(got) | set(want) + for key in set(got.get(host, {})) | set(want.get(host, {})) + if got.get(host, {}).get(key) != want.get(host, {}).get(key) + ) + assert set(got) == set(want), ( + f"{fabric.name}: device sets differ " + f"(missing {sorted(set(want) - set(got))[:5]}, " + f"extra {sorted(set(got) - set(want))[:5]})" + ) + assert not differing, f"{fabric.name}: {len(differing)} differ: {differing[:5]}" + + +@pytest.mark.parametrize("root", EXAMPLES, ids=lambda p: p.name) +def test_resolves_identically_to_ansible(root: Path) -> None: + _assert_equivalent(root) + + +@pytest.mark.corpus +@pytest.mark.parametrize("root", MOLECULE, ids=lambda p: p.name) +def test_resolves_identically_to_ansible_over_the_molecule_corpus(root: Path) -> None: + _assert_equivalent(root) + + +def _assert_renders(root: Path, deferrals: dict[str, str]) -> None: + import yaml + + from function.engine import render_structured_configs + + golden = root / "intended" / "structured_configs" + if not golden.is_dir(): + pytest.skip(f"{root.name} ships no golden") + + fabrics, _ = _fabrics(root) + total: list[str] = [] + try: + for fabric in fabrics: + rendered = render_structured_configs(resolve(fabric.inputs)) + for hostname, structured in sorted(rendered.items()): + target = golden / f"{hostname}.yml" + if target.is_file(): + _differences(hostname, structured, + yaml.safe_load(target.read_text()) or {}, total) + except Exception as err: # noqa: BLE001 - a render failure is a difference too + total.append(f"{type(err).__name__}: {err}") + + if root.name in deferrals: + assert total, ( + f"{root.name} renders clean now -- drop its deferral " + f"(was: {deferrals[root.name]})" + ) + return + assert not total, f"{root.name}: {len(total)} differences: {total[:5]}" + + +@pytest.mark.parametrize("root", EXAMPLES, ids=lambda p: p.name) +def test_render_reproduces_golden(root: Path) -> None: + """Rendered configs still match the checked-in golden. + + Redundant as a check on the model -- matching hostvars render identically -- + and that is not what it is for. It guards pyavd itself changing: an AVD + upgrade slips past resolution equivalence and fails here. + """ + _assert_renders(root, DEFERRED_RENDER) + + +@pytest.mark.corpus +@pytest.mark.parametrize("root", MOLECULE, ids=lambda p: p.name) +def test_render_reproduces_golden_over_the_molecule_corpus(root: Path) -> None: + if root.name in NO_DESIGN_PLAY: + pytest.skip("no play runs eos_designs") + if root.name == "eos_designs_negative_unit_tests": + # Every fixture is deliberately invalid and every play asserts a specific + # failure message. Rendering it clean would mean AVD's own negative + # suite had stopped working. + pytest.skip("AVD's negative corpus -- these are meant to fail") + _assert_renders(root, DEFERRED_RENDER_CORPUS) + + +def test_corpus_is_not_empty() -> None: + # The submodule is optional in a fresh worktree; an empty parametrisation + # would make this whole file pass while testing nothing. + assert len(EXAMPLES) == 8, f"expected AVD's 8 examples, found {len(EXAMPLES)}" + + +def test_the_migration_refuses_an_order_it_cannot_stand_behind(monkeypatch) -> None: + """The safety property, exercised rather than described. + + Group order (depth, name) is the one rule the Ansible CLI does not print, so + the migration layers the fragments with it and compares the result against + `ansible-inventory --list`. Break the order and nothing is emitted. + + ⚠ `campus-fabric` specifically, and the reason is worth keeping: reversing + the order changes **nothing** in `single-dc-l3ls` or `dual-dc-l3ls`, because + no two fragments there set the same key for the same device. Precedence is + load-bearing in exactly two of AVD's eight examples -- `campus-fabric`'s + role-level `aaa_settings` and `cv-pathfinder`'s `ipv4_acls`. A guard written + against either of the other six would pass while proving nothing. + """ + root = EXAMPLES_ROOT / "campus-fabric" + if not root.is_dir(): + pytest.skip("AVD submodule not initialised") + + import function.migrate as migrate_module + + original = migrate_module._fragments + + def reversed_order(inv, devices): + return list(reversed(original(inv, devices))) + + monkeypatch.setattr(migrate_module, "_fragments", reversed_order) + with pytest.raises(MigrationError) as refused: + migrate(root, collections=COLLECTIONS) + + # Two gates catch a wrong order, and the earlier one is the more useful: + # reading a resolved value back off the devices where a fragment wins finds + # `aaa_settings` disagreeing and names it, before the layering comparison + # gets to count differences. + assert "aaa_settings" in str(refused.value), str(refused.value) + + +def test_a_host_outside_the_play_is_not_a_device() -> None: + """`cv-pathfinder` holds `cloudvision`, which is not a switch. + + Its inventory carries the CloudVision API server so `cv_deploy` can reach + it; `build.yml` runs eos_designs on `hosts: WAN`. Declaring it would compose + a Device for it -- and a declared device with no node type fails the whole + fabric's render with AVD's `No device type found`, not just its own. + """ + root = EXAMPLES_ROOT / "cv-pathfinder" + if not root.is_dir(): + pytest.skip("AVD submodule not initialised") + + fabrics = migrate(root, collections=COLLECTIONS) + devices = {device for fabric in fabrics for device in fabric.devices} + assert "cloudvision" not in devices, "an API server is not a fabric device" + assert "pf1" in devices, "the play's own devices must survive the restriction" + + +def test_manifests_carry_every_input_in_requires_order() -> None: + fabric = Fabric( + name="f", devices=("leaf1",), fabric_name="FABRIC", + inputs=[ + Input("leaves", "NodeSet", {"l3leaf": {"nodes": [{"name": "leaf1"}]}}, + declares=["leaf1"]), + Input("base", "SettingSet", {"ntp_settings": {}}, all_devices=True), + ], + ) + manifests = to_manifests(fabric, namespace="avd") + assert [m["kind"] for m in manifests] == ["NodeSet", "SettingSet", "Fabric"] + assert manifests[-1]["spec"]["requires"] == [ + {"kind": "NodeSet", "name": "leaves", "namespace": "avd"}, + {"kind": "SettingSet", "name": "base", "namespace": "avd"}, + ] + assert manifests[0]["spec"]["declares"] == ["leaf1"] + assert "appliesTo" not in manifests[0]["spec"], "a NodeSet seen by what it declares says nothing" + assert manifests[1]["spec"]["appliesTo"] == {"all": True} + + +def test_later_input_overwrites_earlier() -> None: + """Precedence is list order, and replacement is whole-key.""" + inputs = [ + Input("nodes", "NodeSet", {"l3leaf": {"nodes": [{"name": "leaf1"}]}}, + node_sets=["nodes"], declares=["leaf1"]), + Input("base", "SettingSet", {"ntp_settings": {"servers": ["a"]}}, all_devices=True), + Input("narrow", "SettingSet", {"ntp_settings": {"servers": ["b"]}}, hosts=["leaf1"]), + ] + assert resolve(inputs)["leaf1"]["ntp_settings"] == {"servers": ["b"]} + + +def test_input_applies_only_where_scoped() -> None: + """A device sees an input only if appliesTo names it -- this is what + replaces group membership, and what keeps two DCs' node blocks apart.""" + inputs = [ + Input("dc1", "NodeSet", {"l3leaf": {"defaults": {"loopback_ipv4_pool": "10.0.0.0/24"}}}, + node_sets=["dc1"], declares=["leaf1"]), + Input("dc2", "NodeSet", {"l3leaf": {"defaults": {"loopback_ipv4_pool": "10.1.0.0/24"}}}, + node_sets=["dc2"], declares=["leaf2"]), + ] + out = resolve(inputs) + assert out["leaf1"]["l3leaf"]["defaults"]["loopback_ipv4_pool"] == "10.0.0.0/24" + assert out["leaf2"]["l3leaf"]["defaults"]["loopback_ipv4_pool"] == "10.1.0.0/24" + + +def test_network_services_is_classified_under_either_spelling() -> None: + """AVD 6.x reads two spellings for the same content, and both are services. + + `shared_utils/filtered_tenants.py` reads `inputs.network_services` and then + the dynamic keys named by `network_services_keys` (default `tenants`). + """ + tenants = [{"name": "TENANT_A", "vrfs": [{"name": "VRF10"}]}] + assert classify({"tenants": tenants}) == "NetworkServiceSet" + assert classify({"network_services": tenants}) == "NetworkServiceSet" + + +def test_a_bare_type_is_node_content() -> None: + """`type` is the commonest key in AVD's whole corpus -- 639 fragments -- and + it says which node-type block a device belongs to. AVD gives it a table of + its own (`type-setting`). A group whose only variable is `type` is the + purest NodeSet there is, and calling it a setting is what forced the + migration to invent synthetic `-devices` NodeSets beside it. + """ + assert classify({"type": "spine"}) == "NodeSet" + assert by_kind({"type": "spine", "ntp_settings": {}}) == { + "NodeSet": {"type": "spine"}, + "SettingSet": {"ntp_settings": {}}, + } + + +def test_a_fragment_is_split_by_category_not_won_by_one() -> None: + """Merged as Ansible merges, then split -- so each XR carries one category. + + `cv-pathfinder`'s `group_vars/WAN/` is four files whose author had already + separated settings, interface profiles, management and tenants. Ansible + merges a group_vars directory into one namespace, so the fragment arrives + carrying all four -- and one `tenants` key among 22 others must not decide + what the other 21 are. + """ + parts = by_kind({ + "l3leaf": {"nodes": [{"name": "leaf1"}]}, + "tenants": [{"name": "TENANT_A"}], + "aaa_settings": {"local_users": [{"name": "admin"}]}, + }) + assert list(parts) == ["NodeSet", "NetworkServiceSet", "SettingSet"] + assert [sorted(design) for design in parts.values()] == [ + ["l3leaf"], ["tenants"], ["aaa_settings"] + ] + + +def test_a_custom_dynamic_key_is_classified_when_its_generator_travels_along() -> None: + """The open-kinds limit, and the half of it that is not a limit. + + Top-level key names come from the document's own content, so no static map + can name them all. But a fragment carrying its own `network_services_keys` + says what its keys mean, and then they are classifiable. + """ + design = {"network_services_keys": [{"name": "tenant_a"}], "tenant_a": [{"name": "T"}]} + assert by_kind(design) == {"NetworkServiceSet": design} + # Without the generator there is nothing to read it by. + assert by_kind({"tenant_a": [{"name": "T"}]}) == {"SettingSet": {"tenant_a": [{"name": "T"}]}} + + +def test_the_endpoint_family_is_read_from_pyavd_not_from_a_literal() -> None: + """`cameras` is the case that proves it: AVD ships it in + `connected_endpoints_keys`, and the hand-written list this replaced never + grew it. Nothing here names the members.""" + assert "cameras" in Vocabulary.default().connected_endpoints + assert classify({"cameras": [{"name": "cam1"}]}) == "ConnectedEndpointSet" + + +def test_unscoped_nodeset_reaches_only_what_it_declares() -> None: + """An omitted `appliesTo` means the whole fabric -- except on a NodeSet. + + Ansible has no unscoped group_vars file: a node-type block is read by its + own group. + """ + inputs = [ + Input("spines", "NodeSet", {"spine": {"nodes": [{"name": "spine1"}]}}, + declares=["spine1"]), + Input("leaves", "NodeSet", {"l3leaf": {"nodes": [{"name": "leaf1"}]}}, + declares=["leaf1"]), + Input("base", "SettingSet", {"ntp_settings": {"servers": ["a"]}}), + ] + out = resolve(inputs) + assert "spine" in out["spine1"] and "spine" not in out["leaf1"] + assert "l3leaf" in out["leaf1"] and "l3leaf" not in out["spine1"] + assert out["spine1"]["ntp_settings"] == out["leaf1"]["ntp_settings"] + + +def test_a_nodeset_may_be_seen_wider_than_it_declares() -> None: + """`eos_designs-twodc-5stage-clos` has a DC-level NodeSet declaring 4 + super_spines and visible to all 16 devices of that DC.""" + inputs = [ + Input("dc1", "NodeSet", {"super_spine": {"nodes": [{"name": "ss1"}]}}, + declares=["ss1"], node_sets=["dc1", "dc1-pod1"]), + Input("dc1-pod1", "NodeSet", {"l3leaf": {"nodes": [{"name": "leaf1"}]}}, + declares=["leaf1"]), + ] + out = resolve(inputs) + assert "super_spine" in out["leaf1"], "a widened NodeSet must still reach the pod" + assert "l3leaf" not in out["ss1"], "the pod's own block stays in the pod" + + +def test_undeclared_node_is_not_a_device() -> None: + """A block may name a node the fabric does not declare -- AVD's own + anta_runner does -- and it must not become a device.""" + inputs = [ + Input( + "leaves", "NodeSet", + {"l3leaf": {"nodes": [{"name": "leaf1"}, {"name": "ghost"}]}}, + node_sets=["leaves"], declares=["leaf1"], + ) + ] + assert set(resolve(inputs)) == {"leaf1"} + + +def test_only_description_templates_are_droppable() -> None: + """Both are `.j2` paths pyavd cannot honour; only one is safe to lose. + + An `interface_descriptions` template decides a `description` string -- on + `evpn_underlay_ebgp_overlay_ebgp`, dropping them renders all 16 devices and + differs from AVD's golden in 168 places, every one of them a `description`. + An `ip_addressing` template decides an address: `eos_designs-twodc-5stage-clos` + computes its P2P uplink IPs that way. Dropping that would emit a different + network without saying so, which is why one flag cannot cover both. + """ + design = { + "node_type_keys": [{ + "key": "spine", + "interface_descriptions": {"underlay_ethernet_interfaces": "d/eth.j2"}, + "ip_addressing": {"p2p_uplinks_ip": "a/p2p.j2"}, + }], + } + assert unsupported(design) == { + "interface_descriptions": ["node_type_keys.[0].interface_descriptions." + "underlay_ethernet_interfaces"], + "ip_addressing": ["node_type_keys.[0].ip_addressing.p2p_uplinks_ip"], + } + + removed = drop_description_templates(design) + assert removed == ["node_type_keys.[0].interface_descriptions." + "underlay_ethernet_interfaces"] + entry = design["node_type_keys"][0] + assert "interface_descriptions" not in entry, "an emptied key is removed, not left bare" + assert entry["ip_addressing"] == {"p2p_uplinks_ip": "a/p2p.j2"}, ( + "addressing must survive the flag that drops descriptions" + ) + + +def test_custom_python_modules_are_reported_and_never_dropped() -> None: + """Code, not data. A function image is immutable and loads no arbitrary + Python, so this cannot travel and cannot be quietly discarded either.""" + design = {"templates": {"ip_addressing": {"python_module": "custom_ip_addressing"}}} + assert unsupported(design) == { + "python_module": ["templates.ip_addressing.python_module"] + } + assert drop_description_templates(design) == [] + + +def test_a_migration_says_what_it_could_not_carry() -> None: + """Reported without the flag too -- someone migrating a real inventory has + to learn this from the tool, not from a diff on a device.""" + root = MOLECULE_ROOT / "evpn_underlay_ebgp_overlay_ebgp" + if not root.is_dir(): + pytest.skip("AVD submodule not initialised") + + fabric = migrate(root, collections=COLLECTIONS)[0] + assert any("interface_descriptions" in note for note in fabric.notes), fabric.notes + assert all("dropped" not in note for note in fabric.notes), ( + "nothing may be dropped unless asked" + ) + + dropped = migrate(root, collections=COLLECTIONS, drop_descriptions=True)[0] + assert any(note.startswith("dropped ") for note in dropped.notes), dropped.notes diff --git a/tests/test_nulls.py b/tests/test_nulls.py new file mode 100644 index 0000000..a43dfbd --- /dev/null +++ b/tests/test_nulls.py @@ -0,0 +1,85 @@ +"""An explicit null survives the trip through an object that cannot hold one. + +Offline. The property under test is not "encode then decode returns the input" +-- that passes on an encoder that does nothing at all. It is that the value +survives *the pruning*, so every round trip here prunes in the middle, the way +an API server does. Measured against a real one: a null map value is dropped, a +null list item is not. +""" + +from __future__ import annotations + +import pytest + +from function import nulls +from function.kinds import Input, resolve +from function.migrate import Fabric, MigrationError, to_manifests + + +def pruned(node): + """What an API server stores: map values that are null are gone.""" + if isinstance(node, dict): + return {k: pruned(v) for k, v in node.items() if v is not None} + if isinstance(node, list): + return [None if v is None else pruned(v) for v in node] + return node + + +def through_apiserver(design): + """A design as it comes back out of an object it was applied to.""" + return nulls.restored(pruned(nulls.encoded(design))) + + +def test_a_null_map_value_survives_the_pruning(): + # twodc's own shape: a fabric-wide default, cancelled on one link. + design = { + "p2p_uplinks_qos_profile": "QOS-PROFILE", + "l3_edge": {"p2p_links": [{"id": 1, "qos_profile": None, "mtu": 1499}]}, + } + assert through_apiserver(design) == design + + +def test_the_encoder_is_what_makes_it_survive(): + """The same document without the marker loses the key -- the guard's point.""" + design = {"l3_edge": {"p2p_links": [{"id": 1, "qos_profile": None}]}} + assert pruned(design) == {"l3_edge": {"p2p_links": [{"id": 1}]}} + + +def test_a_null_list_item_is_left_alone(): + """It survives on its own, so marking it would change what AVD reads.""" + design = {"items": ["a", None, "b"]} + assert nulls.encoded(design) == design + assert through_apiserver(design) == design + + +def test_nothing_else_is_touched(): + design = {"none_the_value": "none", "empty": "", "zero": 0, "no": False} + assert nulls.encoded(design) == design + assert through_apiserver(design) == design + + +def test_restoring_happens_after_layering(): + """A later input may overwrite a null with a value, or a value with a null.""" + inputs = [ + Input(name="fabric", kind="SettingSet", design={"qos": "P1"}, all_devices=True, + declares=["s1"]), + Input(name="site", kind="SettingSet", design=nulls.encoded({"qos": None}), + all_devices=True), + ] + assert resolve(inputs) == {"s1": {"qos": None}} + + inputs[0], inputs[1] = inputs[1], inputs[0] + inputs[0].declares, inputs[1].declares = ["s1"], [] + assert resolve(inputs) == {"s1": {"qos": "P1"}} + + +def test_a_document_already_carrying_the_marker_is_refused(): + """Encoding it would make somebody's string indistinguishable from a null.""" + fabric = Fabric( + name="f", + devices=("s1",), + inputs=[Input(name="settings", kind="SettingSet", + design={"description": nulls.MARKER}, all_devices=True)], + ) + with pytest.raises(MigrationError, match="already contains"): + to_manifests(fabric) diff --git a/tests/test_xr_fold.py b/tests/test_xr_fold.py deleted file mode 100644 index 3a9c425..0000000 --- a/tests/test_xr_fold.py +++ /dev/null @@ -1,29 +0,0 @@ -"""Milestone 2: the Ansible -> Fabric-XR fold still reproduces golden. - -Offline -- no cluster. Guards `xr.fabric_design_from_inputs` (block union + -defaults push-down), which is the part most likely to silently lose a value. -""" - -from __future__ import annotations - -from pathlib import Path - -import pytest - -from function.verify_xr import DEFERRED, EXAMPLES_ROOT, _discover, verify_one - - -@pytest.mark.parametrize("example_dir", _discover(EXAMPLES_ROOT), ids=lambda p: p.name) -def test_fold_reproduces_golden(example_dir: Path) -> None: - status, _ = verify_one(example_dir) - - if example_dir.name in DEFERRED: - # Expected failure. Asserting it still fails means a deferral can never - # rot: if the example starts folding, this fails and tells us to drop it. - assert not status.startswith("ok"), ( - f"{example_dir.name} folds now -- remove it from verify_xr.DEFERRED " - f"(was deferred: {DEFERRED[example_dir.name]})" - ) - return - - assert status == "ok", f"{example_dir.name}: {status}" diff --git a/uv.lock b/uv.lock index a39fcc7..07c467c 100644 --- a/uv.lock +++ b/uv.lock @@ -11,6 +11,22 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/78/b6/6307fbef88d9b5ee7421e68d78a9f162e0da4900bc5f5793f6d3d0e34fb8/annotated_types-0.7.0-py3-none-any.whl", hash = "sha256:1f02e8b43a8fbbc3f3e0d4f0f4bfc8131bcb4eebe8849b8e5c773f3a1c582a53", size = 13643, upload-time = "2024-05-20T21:33:24.1Z" }, ] +[[package]] +name = "ansible-core" +version = "2.21.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "cryptography" }, + { name = "jinja2" }, + { name = "packaging" }, + { name = "pyyaml" }, + { name = "resolvelib" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/1c/11/cb53834d320c38d739e756e2458852d6e74a6c7018a9ab9f6d4ab5e5196e/ansible_core-2.21.3.tar.gz", hash = "sha256:4194fbd82273cbacfd06d86d74d2d7168c3c4b8426c03e93562cd7217f811ae1", size = 3397615, upload-time = "2026-08-10T16:46:14.554Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a3/fa/938fe0504372377af2bef42abe0d8e463fdcd011ad803eaf300e4577dd7a/ansible_core-2.21.3-py3-none-any.whl", hash = "sha256:9e7dd367f7dc5d5e9fc5ae1baf8af9c4edc09e916a73a40108a3f32e3ad93f10", size = 2446988, upload-time = "2026-08-10T16:46:12.947Z" }, +] + [[package]] name = "anta" version = "1.8.0" @@ -337,6 +353,7 @@ dependencies = [ [package.dev-dependencies] dev = [ + { name = "ansible-core" }, { name = "pytest" }, ] @@ -348,7 +365,10 @@ requires-dist = [ ] [package.metadata.requires-dev] -dev = [{ name = "pytest", specifier = ">=8.0" }] +dev = [ + { name = "ansible-core", specifier = ">=2.18" }, + { name = "pytest", specifier = ">=8.0" }, +] [[package]] name = "grpcio" @@ -1020,6 +1040,15 @@ socks = [ { name = "pysocks" }, ] +[[package]] +name = "resolvelib" +version = "1.2.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/1d/14/4669927e06631070edb968c78fdb6ce8992e27c9ab2cde4b3993e22ac7af/resolvelib-1.2.1.tar.gz", hash = "sha256:7d08a2022f6e16ce405d60b68c390f054efcfd0477d4b9bd019cc941c28fad1c", size = 24575, upload-time = "2025-10-11T01:07:44.582Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e2/23/c941a0d0353681ca138489983c4309e0f5095dfd902e1357004f2357ddf2/resolvelib-1.2.1-py3-none-any.whl", hash = "sha256:fb06b66c8da04172d9e72a21d7d06186d8919e32ae5ab5cdf5b9d920be805ac2", size = 18737, upload-time = "2025-10-11T01:07:43.081Z" }, +] + [[package]] name = "rich" version = "14.3.4"