ToR Peers

The tor_peers configuration enables you to define ToR (Top of Rack) switch pairings within your VXLAN EVPN fabric (iBGP and eBGP types of fabrics). ToR pairing establishes a parent-child relationship between leaf switches and ToR switches, with ToR switches connecting downstream to servers and upstream to leaf switches.

This feature requires NDFC 12.2.2.241+ and Nexus Dashboard 3.2.1i+ or later.

Diagram

Classes

topology (vxlan)

Name	Type	Constraint	Mandatory	Default Value
tor_peers	List	[tor_peers]	No

tor_peers (vxlan.topology)

Name	Type	Constraint	Mandatory	Default Value
pairing_id	String		No
parent_leaf1	String		Yes
parent_leaf2	String		No
tor1	String		Yes
tor2	String		No

Unsuported Scenarios

You cannot preprovision and pair ToR switch(es) with unavailable leaf switch(es). Both ToR and leaf switch(es) must be discovered by ND and be available.

Supported Scenarios

The feature supports three deployment scenarios:

Scenario	Description	Required Fields
vPC-to-vPC	Two ToR switches in vPC paired with two leaf switches in vPC	tor1, tor2, parent_leaf1, parent_leaf2
vPC-to-Standalone	Single ToR switch paired with two leaf switches in vPC	tor1, parent_leaf1, parent_leaf2
Standalone-to-Standalone	Single ToR switch paired with single leaf switch	tor1, parent_leaf1

Example 1 - vPC-to-vPC Pairing

This configuration pairs two ToR switches (ToR_24 and ToR_23) in a vPC domain with two leaf switches (LEAF_11 and LEAF_21) that are also in a vPC domain.

Note

Both the ToR switches and leaf switches must be defined in vxlan.topology.vpc_peers before creating this pairing.

vxlan:
  topology:
    tor_peers:
      - tor1: ToR_24
        tor2: ToR_23
        parent_leaf1: LEAF_11
        parent_leaf2: LEAF_21

Example 2 - vPC-to-Standalone Pairing

This configuration pairs a single standalone ToR switch (ToR_31) with two leaf switches in a vPC domain.

vxlan:
  topology:
    tor_peers:
      - tor1: ToR_31
        parent_leaf1: LEAF_11
        parent_leaf2: LEAF_21

Example 3 - Standalone-to-Standalone Pairing

This configuration pairs a single ToR switch with a single leaf switch.

vxlan:
  topology:
    tor_peers:
      - tor1: ToR_13
        parent_leaf1: LEAF_14

Example 4 - Multiple ToR Pairings

You can define multiple ToR pairings in a single configuration file.

vxlan:
  topology:
    tor_peers:
      # vPC-to-vPC pairing
      - tor1: ToR_24
        tor2: ToR_23
        parent_leaf1: LEAF_11
        parent_leaf2: LEAF_21

      # vPC-to-Standalone pairing
      - tor1: ToR_31
        parent_leaf1: LEAF_11
        parent_leaf2: LEAF_21

      # Standalone-to-Standalone pairing
      - tor1: ToR_13
        parent_leaf1: LEAF_14

Example 5 - ToR in Multi-Site (MSD) and MCFG Deployments

When a ToR pair is part of a child fabric in a multi-site topology — either Multi-Site Domain (MSD) on a single ND cluster or Multi-Cluster Fabric Group (MCFG) spanning multiple ND clusters — the ToR pairing itself is configured only in the child VXLAN_EVPN fabric. The parent fabric (MSD or MCFG) references the ToR hostnames and ports inside its network_attach_groups and (optionally) vrf_attach_groups.

The example below uses three child fabrics (FAB1 VXLAN_EVPN, FAB2 VXLAN_EVPN, ISN ISN) and one parent fabric. FAB1 has a vPC-paired ToR pair (ToR+ToR2) connected downstream over a vPC port-channel for VLAN extension; the parent fabric stretches networks across FAB1 and FAB2 and attaches the ToR ports.

Step 1 — Child Fabric FAB1 (VXLAN_EVPN with ToR)

The child fabric data model is identical for MSD and MCFG. It uses the same tor_peers block as the previous examples plus the supporting topology, vPC peering, and host port-channel.

host_vars/FAB1/global.nac.yaml

---
vxlan:
  fabric:
    name: FAB1
    type: VXLAN_EVPN
  global:
    ibgp:
      bgp_asn: "65001"
      route_reflectors: 2
      anycast_gateway_mac: 12:34:56:78:90:00

host_vars/FAB1/topology_switches.nac.yaml

---
vxlan:
  topology:
    switches:
      - name: Leaf-101
        serial_number: 9648E3J5P6M
        role: leaf
        management:
          default_gateway_v4: 192.168.1.254
          management_ipv4_address: 192.168.1.101
      # vPC-paired parent leaves
      - name: Leaf-103
        serial_number: 9O2321WXR3E
        role: leaf
        management:
          default_gateway_v4: 192.168.1.254
          management_ipv4_address: 192.168.1.103
      - name: Leaf-104
        serial_number: 9WUI3ZWRYLF
        role: leaf
        management:
          default_gateway_v4: 192.168.1.254
          management_ipv4_address: 192.168.1.104
      # vPC-paired ToRs
      - name: ToR
        serial_number: 99A99521KEC
        role: tor
        management:
          default_gateway_v4: 192.168.1.254
          management_ipv4_address: 192.168.1.107
      - name: ToR2
        serial_number: 9L3VFZKRMQ2
        role: tor
        management:
          default_gateway_v4: 192.168.1.254
          management_ipv4_address: 192.168.1.115
      - name: BGW-1
        serial_number: 9J3FN3XDNGH
        role: border_gateway
        management:
          default_gateway_v4: 192.168.1.254
          management_ipv4_address: 192.168.1.110

host_vars/FAB1/topology_vpc.nac.yaml

---
vxlan:
  topology:
    vpc_peers:
      # Parent leaves vPC pair
      - peer1: Leaf-103
        peer1_peerlink_interfaces:
          - name: Ethernet1/3
          - name: Ethernet1/4
        peer2: Leaf-104
        peer2_peerlink_interfaces:
          - name: Ethernet1/3
          - name: Ethernet1/4
      # ToRs vPC pair
      - peer1: ToR
        peer1_peerlink_interfaces:
          - name: Ethernet1/4
        peer2: ToR2
        peer2_peerlink_interfaces:
          - name: Ethernet1/4

host_vars/FAB1/tor.nac.yaml

---
vxlan:
  topology:
    tor_peers:
      - tor1: ToR
        tor2: ToR2
        parent_leaf1: Leaf-103
        parent_leaf2: Leaf-104

host_vars/FAB1/topology_int_switch_ToR.nac.yaml

---
vxlan:
  topology:
    switches:
      - name: ToR
        interfaces:
          - name: port-channel10
            mode: trunk
            description: "vPC trunk to downstream L2 switch"
            enabled: true
            pc_mode: active
            vpc_id: 10
            mtu: jumbo
            trunk_allowed_vlans:
              - from: 2301
                to: 2302
            enable_lacp_vpc_convergence: true
            lacp_rate: fast
            members:
              - Ethernet1/5

Step 2 — Parent Fabric (MSD or MCFG variant)

In both MSD and MCFG, networks and VRFs are declared under vxlan.multisite.overlay.*. The only differences between the two parent variants are:

Field	MSD	MCFG
vxlan.fabric.type	MSD	MCFG
vxlan.multisite.child_fabrics[*].cluster	omitted (single ND)	required (per-cluster routing)

MSD Parent Example

host_vars/MSD_PARENT/global.nac.yaml

---
vxlan:
  fabric:
    name: MSD_1
    type: MSD
  multisite:
    anycast_gateway_mac: 12:34:56:78:90:00
    child_fabrics:
      - name: FAB1
      - name: FAB2
      - name: ISN
    overlay_dci:
      underlay_autoconfig: true
      deployment_method: Centralized_To_Route_Server
      route_server:
        peers:
          - ip_address: 10.101.101.101
            bgp_asn: "65000"
      enable_bgp_send_community: true

MCFG Parent Example

host_vars/MCFG_PARENT/mcfg.nac.yaml

---
vxlan:
  fabric:
    name: MCFG_1
    type: MCFG
  multisite:
    anycast_gateway_mac: 12:34:56:78:90:00
    child_fabrics:
      - name: FAB1
        cluster: CLUSTER1
      - name: FAB2
        cluster: CLUSTER2
      - name: ISN
        cluster: CLUSTER1
    overlay_dci:
      underlay_autoconfig: true
      deployment_method: Centralized_To_Route_Server
      route_server:
        peers:
          - ip_address: 10.101.101.101
            bgp_asn: "65000"
      enable_bgp_send_community: true

Step 3 — Networks and VRFs (parent attaches ToR ports)

Networks and VRFs are declared in the parent. The network_attach_groups lists the ToRs directly as hostnames with their host-facing ports (Ethernet1/3, port-channel10, etc.). The same YAML applies to both MSD and MCFG parents — only the parent type and child_fabrics[*].cluster change between variants.

host_vars/<MSD_PARENT|MCFG_PARENT>/networks.nac.yaml

---
vxlan:
  multisite:
    overlay:
      networks:
        - name: MCFG-WEB
          vrf_name: MCFG-VRF01
          net_id: 30001
          vlan_id: 2301
          vlan_name: MCFG-WEB_vlan2301
          gw_ip_address: "192.168.1.254/24"
          network_attach_group: all
          child_fabrics:
            - name: FAB1
              multicast_group_address: 239.101.101.1
              l3gw_on_border: true
            - name: FAB2
              multicast_group_address: 239.102.102.1
              l3gw_on_border: true
        - name: MCFG-Net02_l2
          is_l2_only: true
          net_id: 30002
          vlan_id: 2302
          vlan_name: MCFG-Net02_l2_vlan2302
          network_attach_group: sites_only
          child_fabrics:
            - name: FAB1
              multicast_group_address: 239.101.101.2
            - name: FAB2
              multicast_group_address: 239.102.102.2
      network_attach_groups:
        - name: all
          switches:
            # FAB1 nodes — ToRs listed directly with host ports
            - { hostname: Leaf-101, ports: [] }
            - hostname: BGW-1
            - { hostname: ToR,  ports: [Ethernet1/3, port-channel10] }
            - { hostname: ToR2, ports: [port-channel10] }
            # FAB2 nodes
            - { hostname: Leaf-201, ports: [] }
            - hostname: BGW-3
        - name: sites_only
          switches:
            - { hostname: Leaf-101, ports: [] }
            - hostname: BGW-1
            - { hostname: ToR,  ports: [port-channel10] }
            - { hostname: ToR2, ports: [port-channel10] }
            - { hostname: Leaf-201, ports: [] }
            - hostname: BGW-3

host_vars/<MSD_PARENT|MCFG_PARENT>/vrfs.nac.yaml

---
vxlan:
  multisite:
    overlay:
      vrfs:
        - name: MCFG-VRF01
          vrf_id: 50001
          vlan_id: 2001
          vrf_attach_group: all
          child_fabrics:
            - name: FAB1
              adv_host_routes: true
            - name: FAB2
              adv_default_routes: true
      vrf_attach_groups:
        - name: all
          switches:
            # Only leaves and BGWs — ToRs intentionally excluded
            - { hostname: Leaf-101 }
            - { hostname: BGW-1 }
            - { hostname: Leaf-201 }
            - { hostname: BGW-3 }

ToRs excluded from VRF attach group

In multi-site topologies with ToR pairing, the ToRs typically carry only the L2 attachment via their port-channel — the L3 SVI / VRF is hosted on the parent leaves. The ToR hostnames are therefore listed in network_attach_groups (with their ports) but excluded from vrf_attach_groups.

ToR ports represented as parent-leaf subports

NDFC internally represents ToR ports as subports of the parent leaf. After deploy, you may see entries like Leaf-103(port-channel10) / Leaf-104(port-channel10) for a port-channel that physically belongs to ToR/ToR2. This is the expected MSD/MCFG-ToR behaviour and does not require any change to the data model.

Prerequisites

Before configuring ToR pairing, ensure:

1. All ToR switches are defined in vxlan.topology.switches with role: tor
2. All leaf switches are defined in vxlan.topology.switches with role: leaf
3. For vPC scenarios, switches must be defined in vxlan.topology.vpc_peers
4. Switches must have a valid serial_number and be discovered (not in preprovision mode)

Network Attachments for ToR Switches

After pairing ToR switches, you can attach networks using network_attach_groups:

vxlan:
  overlay:
    network_attach_groups:
      - name: tor_networks
        switches:
          - hostname: ToR_23
            ports:
              - Port-channel555
              - Ethernet1/15
          - hostname: ToR_24
            ports:
              - Port-channel555
              - Ethernet1/15
          - hostname: ToR_31
            ports:
              - Ethernet1/13
          - hostname: ToR_13
            ports:
              - Ethernet1/13

Order of Operations

• Adding ToR: First add to tor_peers, then add to network_attach_groups
• Removing ToR: First remove from network_attach_groups, then remove from tor_peers

Enabling Removal Operations

By default, ToR pairing removal is disabled for safety. To enable automatic removal:

tor_pairing_delete_mode: true

Caution

When tor_pairing_delete_mode: true, removing a ToR pairing from your configuration will automatically unpair it in NDFC on the next playbook run.

Reference

The tor_peers configuration enables you to define ToR (Top of Rack) switch pairings within your VXLAN EVPN fabric (iBGP and eBGP types of fabrics). ToR pairing establishes a parent-child relationship between leaf switches and ToR switches, with ToR switches connecting downstream to servers and upstream to leaf switches.

This feature requires NDFC 12.2.2.241+ and Nexus Dashboard 3.2.1i+ or later.

Diagram

Classes

topology (vxlan)

Name	Type	Constraint	Mandatory	Default Value
tor_peers	List	[tor_peers]	No

tor_peers (vxlan.topology)

Name	Type	Constraint	Mandatory	Default Value
pairing_id	String		No
parent_leaf1	String		Yes
parent_leaf2	String		No
tor1	String		Yes
tor2	String		No

Unsuported Scenarios

You cannot preprovision and pair ToR switch(es) with unavailable leaf switch(es). Both ToR and leaf switch(es) must be discovered by ND and be available.

Supported Scenarios

The feature supports three deployment scenarios:

Scenario	Description	Required Fields
vPC-to-vPC	Two ToR switches in vPC paired with two leaf switches in vPC	tor1, tor2, parent_leaf1, parent_leaf2
vPC-to-Standalone	Single ToR switch paired with two leaf switches in vPC	tor1, parent_leaf1, parent_leaf2
Standalone-to-Standalone	Single ToR switch paired with single leaf switch	tor1, parent_leaf1

Example 1 - vPC-to-vPC Pairing

This configuration pairs two ToR switches (ToR_24 and ToR_23) in a vPC domain with two leaf switches (LEAF_11 and LEAF_21) that are also in a vPC domain.

Note

Both the ToR switches and leaf switches must be defined in vxlan.topology.vpc_peers before creating this pairing.

vxlan:
  topology:
    tor_peers:
      - tor1: ToR_24
        tor2: ToR_23
        parent_leaf1: LEAF_11
        parent_leaf2: LEAF_21

Example 2 - vPC-to-Standalone Pairing

This configuration pairs a single standalone ToR switch (ToR_31) with two leaf switches in a vPC domain.

vxlan:
  topology:
    tor_peers:
      - tor1: ToR_31
        parent_leaf1: LEAF_11
        parent_leaf2: LEAF_21

Example 3 - Standalone-to-Standalone Pairing

This configuration pairs a single ToR switch with a single leaf switch.

vxlan:
  topology:
    tor_peers:
      - tor1: ToR_13
        parent_leaf1: LEAF_14

Example 4 - Multiple ToR Pairings

You can define multiple ToR pairings in a single configuration file.

vxlan:
  topology:
    tor_peers:
      # vPC-to-vPC pairing
      - tor1: ToR_24
        tor2: ToR_23
        parent_leaf1: LEAF_11
        parent_leaf2: LEAF_21

      # vPC-to-Standalone pairing
      - tor1: ToR_31
        parent_leaf1: LEAF_11
        parent_leaf2: LEAF_21

      # Standalone-to-Standalone pairing
      - tor1: ToR_13
        parent_leaf1: LEAF_14

Prerequisites

Before configuring ToR pairing, ensure:

1. All ToR switches are defined in vxlan.topology.switches with role: tor
2. All leaf switches are defined in vxlan.topology.switches with role: leaf
3. For vPC scenarios, switches must be defined in vxlan.topology.vpc_peers
4. Switches must have a valid serial_number and be discovered (not in preprovision mode)

Network Attachments for ToR Switches

After pairing ToR switches, you can attach networks using network_attach_groups:

vxlan:
  overlay:
    network_attach_groups:
      - name: tor_networks
        switches:
          - hostname: ToR_23
            ports:
              - Port-channel555
              - Ethernet1/15
          - hostname: ToR_24
            ports:
              - Port-channel555
              - Ethernet1/15
          - hostname: ToR_31
            ports:
              - Ethernet1/13
          - hostname: ToR_13
            ports:
              - Ethernet1/13

Order of Operations

• Adding ToR: First add to tor_peers, then add to network_attach_groups
• Removing ToR: First remove from network_attach_groups, then remove from tor_peers

Enabling Removal Operations

By default, ToR pairing removal is disabled for safety. To enable automatic removal:

tor_pairing_delete_mode: true

Caution

When tor_pairing_delete_mode: true, removing a ToR pairing from your configuration will automatically unpair it in NDFC on the next playbook run.

Reference