Tags

, , , , , , , , , , , , , , 

The other day I received a question on Ether-channel and the Nexus
7000 - based on the question I felt it would be also good to
include the information here.
This will be a 2-part post, first part is the Nexus configuration
for vPC, the second post will be on the mutli-chassis ether-channel
configuration around the 3750 as well as the Nexus 7000 switches.
What are the benefits of Multi-chassis (vPC) ether-channel? 
Basically all your up-links from your switches are in FORWARDING
mode, nothing is in blocking mode in your spanning tree domain. 
What this means is that you have a loop free topology in your
data center and all links can be utilized.
Below is the diagram of the configuration that I will be
showing here.  There will be a Layer 2 Ether-channel vPC between
the Nexus 7010-1 and Nexus 7010-2 (Orangish line), a Layer 3
Ether-channel for vPC keep-alive (Red line), as well as a
mutli-chassis (vPC) ether-channel from a 3750 stack to Nexus 7010-1
and Nexus 7010-2 with all links in a single ether-channel bundle.

Configuration for both of the Nexus switches is the same except where noted.


Configuration for the Nexus switches
First thing to do is enable the vPC feature:
      feature vpc
 
Once you have enabled the vPC feature, you should create your keep-alive links.
Here I create a port-channel via LACP over ports 9/1 and 10/1.  You will also
notice that I have spread the channel over two line cards.  This has been done
to help assure maximum redundancy.  If a card where to go bad, the other card would
still be active in the port-channel. 
      interface Ethernet9/1
       description [----[ vPC KeepAlive to CoreSwitch2 ]----]
       channel-group 101 mode active  ! Assign port to port-channel 101 via LACP
       no shutdown
     interface Ethernet10/1
       description [----[ vPC KeepAlive to CoreSwitch2 ]----]
       channel-group 101 mode active
       no shutdown
Now we can create the VRF for the keep-alive link.  I suggest using a dedicated
VRF for security and sanity purpose.  This VRF will not participate in your
global routing table, thus allowing for more stability and also the prevention
of duplicate IP addresses in the network.
     vrf context VPC100_KA
Now we can create the Layer 3 interface on the port-channel and assign it
to the new VRF, VPC100_KA
     interface port-channel101
       description [----[ vPC Keep-Alive link between CoreSwitches ]----]
       vrf member VPC100_KA ! Assign this interface into the appropriate VRF
       ip address 10.10.10.1/30  ! The other side of the link is .2/30

Now you can configuration the vPC Peer links (Orangish lines).  Since I am using
10G links for this connection, I have set the rate mode to Dedicated.  This prevents
any chance for over subscription on the 10G port.  It also disables the other 3 ports in
group, so you need to keep that in mind when you are designing your deployment.
     interface Ethernet7/1
       description [-[ vPC Connection to Nexus 7010-2 - E7/1 ]-]
       switchport
       switchport mode trunk  ! Set the mode to trunk
       rate-mode dedicated force ! Force the rate-mode
       mtu 9216
       udld enable ! Since this is also fiber, enable UDLD
       channel-group 100 mode active ! Assign to port-channel 100
       no shutdown
     !
     interface Ethernet8/1
       description [-[ vPC Connection to Nexus 7010-2 - E8/1 ]-]
       switchport
       switchport mode trunk
       rate-mode dedicated force
       mtu 9216
       udld enable
       channel-group 100 mode active
       no shutdown
     !

Now to configure the port-channel as a vPC link as well as the vPC
domain information.
     interface port-channel100
       description [-[ vPC Peer-Link between Nexus Switches ]-]
       switchport
       switchport mode trunk
       vpc peer-link ! Assign this port-channel as a vpc peer-link
       spanning-tree port type network
       mtu 9216
     !
vpc domain 100
role priority 16000 ! Here I hard-coded switch 1 to be the vPC master.
                       switch 2 was left as the default
peer-keepalive destination 10.10.10.2 source 10.10.10.1 vrf VPC100_KA
                    ! The other side has the IP addresses reversed
Had to move the formatting above to get the command to fit, sorry.
Let's check the port-channel and make sure it is up with the appropriate members.
As you can see from the output, Eth7/1 and Eth8/1 are members of the channel.

     N7K1# sh int port-channel 100
      port-channel100 is up
      [------ SNIP - Output omitted! ------]
      Members in this channel: Eth7/1, Eth8/1
     N7K1#

Also check the vPC and the vPC keep-alive link
     N7K1# sh vpc
      Legend:
             (*) - local vPC is down, forwarding via vPC peer-link
      vPC domain id                        : 100 
      Peer status                          : peer adjacency formed ok      
      vPC keep-alive status                : peer is alive                 
      Configuration consistency status     : success 
      Type-2 consistency status            : success 
      vPC role                             : primary, operational secondary
      Number of vPCs configured            : 9   
      Peer Gateway                         : Disabled
      Dual-active excluded VLANs           : -
      vPC Peer-link status
      ---------------------------------------------------------------------
      id   Port   Status Active vlans    
      --   ----   ------ --------------------------------------------------
      1    Po100  up     1-224
     N7K1# sh vpc peer-keepalive
      vPC keep-alive status           : peer is alive                
      --Peer is alive for             : (1486816) seconds, (684) msec
      --Send status                   : Success
      --Last send at                  : 2010.09.11 12:38:36 872 ms
      --Sent on interface             : Po101
      --Receive status                : Success
      --Last receive at               : 2010.09.11 12:38:36 872 ms
      --Received on interface         : Po101
      --Last update from peer         : (0) seconds, (161) msec
     vPC Keep-alive parameters
     --Destination                    : 10.10.10.2
      --Keepalive interval            : 1000 msec
      --Keepalive timeout             : 5 seconds
      --Keepalive hold timeout        : 3 seconds
      --Keepalive vrf                 : VPC100_KA
      --Keepalive udp port            : 3200
      --Keepalive tos                 : 192
     N7K1#
As of now, both switches are connected via vPC.
This concludes the first post, the second post will be up shortly and will focus
around the Cisco 3750 configuration as well as the associated configs on the
Nexus 7000 switches.