So let me talk a moment and talk about DMVPN – Dynamic Multipoint VPN technology.
Currently I am working on a project to test the viability of a DMVPN network as well as help bring the team up to speed on this technology. Since I am doing that I figured it would be a good idea to share some of what I have learned with others – plus blogging it like this helps me to review the material at a later date. 🙄
So the first thing that probably should be answered – What is DMVPN? It is a solution for building a scalable IPsec VPN network utilizing the Internet (or other public network – say MPLS) for the WAN backbone. DMVPN takes advantage of another protocol, Next Hop Resolution Protocol (NHRP) and a Multipoint GRE tunnel interface. With NHRP the router will query the Next Hop Server (NHS) to find out a mapping for a network. Once it finds out the remote IP, the Multipoint GRE will build a dynamic tunnel between the two routers. It will then place that network in its NHRP mapping table.
DMVPN can provide a full-mesh topology with a simplified configuration. You only need to have all the spokes communicate with the hub initially, and then from there the spokes can learn the outside IP of the other routers and build dynamic tunnels when they need to communicate. What is nice with this is that if you add a new spoke to the network, there are neither changes on the hub nor any other spoke – only the new spoke needs to be configured.
What are some of the uses for DMVPN? Work at home workers, remote working locations via cellular, backup to a private WAN, and those are just to name a few. I am sure you can come up with some of you own based on your experiences. (continued)
So lets start the fun stuff, the configuration!
Router/Switch Output
Commands
Notes
Our network diagram looks like the diagram below. We have four routers – DMVPN, R3, R4 and R5. Each are connected to a router that is connected to the “internet” and they all have a 0/0 route pointing to the ISP router.
Ok, since this is a SECURE connection we need to configure the crypto polices and keys
Rack1DMVPN#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Rack1DMVPN(config)# crypto isakmp policy 10
Rack1DMVPN(config-isakmp)# authentication pre-share
Rack1DMVPN(config-isakmp)#exit
! Now we can add the dynamic pre-shared key ( Labbing )for all the remote VPN routers. In this lab network they are all on the 150.1.0.0/16 network. In production you can make this 0.0.0.0 0.0.0.0
Rack1DMVPN(config)#crypto isakmp key Labbing address 150.1.0.0 255.255.0.0
! Now we create a Phase 2 policy for the data encryption
Rack1DMVPN(config)#crypto ipsec transform-set Labbing esp-aes esp-sha-hmac
! And finally create the IPSec profile to be applied to the GRE over IPSEC tunnels.
Rack1DMVPN(config)#crypto ipsec profile Labbing
Rack1DMVPN(ipsec-profile)# set transform-set Labbing
! And now we can create the Tunnel interface on the DMVPN Hub, we will use a 192.168.11.x/24 network for all the Tunnel interfaces.
! Notice, they are all on the same subnet, not /31 or /30, but a /24 network.
Rack1DMVPN(config)# interface Tunnel0
Rack1DMVPN(config-if)# ip address 192.168.11.1 255.255.255.0
Rack1DMVPN(config-if)# no ip redirects
! You should bump the MTU down a bit as this is a tunnel. Depending on your upstream routers and such, your values might be different.
Rack1DMVPN(config-if)# ip mtu 1400
! Now we start to configure the NHRP portion of DMVPN. First a Network-id that is unique to this DMVPN configuration as well as the timers.
Rack1DMVPN(config-if)# ip nhrp network-id 1
Rack1DMVPN(config-if)# ip nhrp holdtime 600
! Now just like any tunnel we need to tell it the source, here I am using G0/1.
Rack1DMVPN(config-if)# tunnel source GigabitEthernet0/1
! Now we tell it the tunnel mode type, gre multipoint
Rack1DMVPN(config-if)# tunnel mode gre multipoint
! And now apply the IPSec configuration to the interface to encrypt traffic.
Rack1DMVPN(config-if)# tunnel protection ipsec profile Labbing
Now, lets configure the hubs. I will do a full configuration on R4 with explanation and then just note the differences when it comes to R5 and R5. You will see many commands that are the same as the hub (crypto, gre, etc), but I will display them with explanation anyways.
Rack1R4#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Rack1R4(config)# crypto isakmp policy 10
Rack1R4(config-isakmp)# authentication pre-share
Rack1R4(config-isakmp)#exit
! Now we can add the dynamic pre-shared key ( Labbing )for all the remote VPN routers. In this lab network they are all on the 150.1.0.0/16 network. In production you can make this 0.0.0.0 0.0.0.0
Rack1R4(config)#crypto isakmp key Labbing address 150.1.0.0 255.255.0.0
! Now we create a Phase 2 policy for the data encryption
Rack1R4(config)#crypto ipsec transform-set Labbing esp-aes esp-sha-hmac
! And finally create the IPSec profile to be applied to the GRE over IPSEC tunnels.
Rack1R4(config)#crypto ipsec profile Labbing
Rack1R4(ipsec-profile)# set transform-set Labbing
! And now we can create the Tunnel interface on the DMVPN Hub, we will use a 192.168.11.x/24 network for all the Tunnel interfaces.
Rack1R4(config)# interface Tunnel0
Rack1R4(config-if)# ip address 192.168.11.4 255.255.255.0
Rack1R4(config-if)# no ip redirects
! You should bump the MTU down a bit as this is a tunnel. Depending on your upstream routers and such, your values might be different.
Rack1R4(config-if)# ip mtu 1400
! Now we start to configure the NHRP portion of DMVPN. First things we tell the router is the NHRP of the hub (tunnel IP and outside IP), then a Network-id that is unique to this DMVPN configuration, the NHS Resolver (192.168.11.1) as well as the timers.
Rack1R4(config-if)# ip nhrp map 192.168.11.1 150.1.254.254
Rack1R4(config-if)# ip nhrp network-id 1
Rack1R4(config-if)# ip nhrp nhs 192.168.11.1
Rack1R4(config-if)# ip nhrp holdtime 600
! Now just like any tunnel we need to tell it the source, here I am using G0/1. This is a great idea if you have a dynamic outside IP address, only the HUB needs a static IP.
Rack1R4(config-if)# tunnel source GigabitEthernet0/1
! Now we tell it the tunnel mode type, gre multipoint
Rack1R4(config-if)# tunnel mode gre multipoint
! And now apply the IPSec configuration to the interface to encrypt traffic.
Rack1R4(config-if)# tunnel protection ipsec profile Labbing
Onto R5!
Rack1R5#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Rack1R5(config)# crypto isakmp policy 10
Rack1R5(config-isakmp)# authentication pre-share
Rack1R5(config-isakmp)#exit
Rack1R5(config)#crypto isakmp key Labbing address 150.1.0.0 255.255.0.0
Rack1R5(config)#crypto ipsec transform-set Labbing esp-aes esp-sha-hmac
Rack1R5(config)#crypto ipsec profile Labbing
Rack1R5(ipsec-profile)# set transform-set Labbing
Rack1R5(config)# interface Tunnel0
! Since this is R5, we need to change the Tunnel IP address – thats it!
Rack1R5(config-if)# ip address 192.168.11.5 255.255.255.0
Rack1R5(config-if)# no ip redirects
Rack1R5(config-if)# ip mtu 1400
Rack1R5(config-if)# ip nhrp map 192.168.11.1 150.1.254.254
Rack1R5(config-if)# ip nhrp network-id 1
Rack1R5(config-if)# ip nhrp nhs 192.168.11.1
Rack1R5(config-if)# ip nhrp holdtime 600
Rack1R5(config-if)# tunnel source GigabitEthernet0/1
Rack1R5(config-if)# tunnel mode gre multipoint
Rack1R5(config-if)# tunnel protection ipsec profile Labbing
And finally, R3!
Rack1R3#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Rack1R3(config)# crypto isakmp policy 10
Rack1R3(config-isakmp)# authentication pre-share
Rack1R3(config-isakmp)#exit
Rack1R3(config)#crypto isakmp key Labbing address 150.1.0.0 255.255.0.0
Rack1R3(config)#crypto ipsec transform-set Labbing esp-aes esp-sha-hmac
Rack1R3(config)#crypto ipsec profile Labbing
Rack1R3(ipsec-profile)# set transform-set Labbing
Rack1R3(config)# interface Tunnel0
! Since this is R3, we need to change the Tunnel IP address – thats it!
Rack1R3(config-if)# ip address 192.168.11.3 255.255.255.0
Rack1R3(config-if)# no ip redirects
Rack1R3(config-if)# ip mtu 1400
Rack1R3(config-if)# ip nhrp map 192.168.11.1 150.1.254.254
Rack1R3(config-if)# ip nhrp network-id 1
Rack1R3(config-if)# ip nhrp nhs 192.168.11.1
Rack1R3(config-if)# ip nhrp holdtime 600
Rack1R3(config-if)# tunnel source GigabitEthernet0/1
Rack1R3(config-if)# tunnel mode gre multipoint
Rack1R3(config-if)# tunnel protection ipsec profile Labbing
Ok, so what did we just build? Here is a picture:
So, if all was done correctly, we should now have a DMVPN mapping on the HUB that shows each of the remote spokes. To check we can use the SHOW DMVPN command. You will notice that the hub at the Attribute D (Dynamic) for each of the remote spokes. That is because it was learned and not statically configured on the hub.
Rack1DMVPN#sh dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer NHS Status: E --> Expecting Replies, R --> Responding UpDn Time --> Up or Down Time for a Tunnel ========================================================================== Interface: Tunnel0, IPv4 NHRP Details Type:Hub, NHRP Peers:3, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 150.1.13.3 192.168.11.3 UP 00:00:51 D 1 150.1.9.4 192.168.11.4 UP 00:00:04 D 1 150.1.10.5 192.168.11.5 UP 00:00:48 D Rack1DMVPN#
So now that we have the configuration up and working, lets test a PING from R3 to the DMVPN hub.
Rack1R3#ping 192.168.11.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.11.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms
Rack1R3#
Yup, that worked. Lets take a look at the SHOW DMVPN on R3:
Rack1R3#sh dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer NHS Status: E --> Expecting Replies, R --> Responding UpDn Time --> Up or Down Time for a Tunnel ==========================================================================
Interface: Tunnel0, IPv4 NHRP Details Type:Spoke, NHRP Peers:1,
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 150.1.254.254 192.168.11.1 UP 07:00:07 S
Rack1R3#
Notice that only R1 is listed right now, no R4 or R5 IPs listed? Lets see if we can ping R4
Rack1R3#ping 192.168.11.4
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.11.4, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms
Rack1R3#
Yup! So what does the SHOW DMVPN look like now? Lets see:
Rack1R3#sh dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer NHS Status: E --> Expecting Replies, R --> Responding UpDn Time --> Up or Down Time for a Tunnel ==========================================================================
Interface: Tunnel0, IPv4 NHRP Details Type:Spoke, NHRP Peers:2,
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 150.1.254.254 192.168.11.1 UP 07:01:17 S 1 150.1.9.4 192.168.11.4 UP 00:00:26 D
Rack1R3#
Nice! We can see that the DMVPN attribute is D for Dynamic, and hte peer NBMA address is 150.1.9.4, the outside IP of R4.
Lets ping R5 now – but first lets turn on a debug to see what is going on. The debug I am using is debug nhrp. That will allow us to see what is going on with the NHRP queries and such.
Rack1R3#debug nhrp
NHRP protocol debugging is on
Rack1R3#
Rack1R3#ping 192.168.11.5
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.11.5, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/3/4 ms
Rack1R3#
And the debug looks like this:
First time it tries to communicate with R5 it tells the router to talk to R1
*Aug 3 01:02:45.786: NHRP: MACADDR: if_in null netid-in 0 if_out Tunnel0 netid-out 1
*Aug 3 01:02:45.786: NHRP: Sending packet to NHS 192.168.11.1 on Tunnel0
*Aug 3 01:02:45.786: NHRP: NHRP successfully resolved 192.168.11.1 to NBMA 150.1.254.254
*Aug 3 01:02:45.786: NHRP: Checking for delayed event /192.168.11.5 on list (Tunnel0).
*Aug 3 01:02:45.786: NHRP: No node found.
*Aug 3 01:02:45.786: NHRP: Adding Tunnel Endpoints (VPN: 192.168.11.5, NBMA: 150.1.254.254)
*Aug 3 01:02:45.786: NHRP: Enqueued NHRP Resolution Request for destination: 192.168.11.5
*Aug 3 01:02:45.790: NHRP: NHRP successfully resolved 192.168.11.5 to NBMA 150.1.254.254
*Aug 3 01:02:45.794: NHRP: NHRP successfully resolved 192.168.11.5 to NBMA 150.1.254.254
*Aug 3 01:02:45.798: NHRP: Checking for delayed event /192.168.11.5 on list (Tunnel0).
*Aug 3 01:02:45.798: NHRP: No node found.
*Aug 3 01:02:45.798: NHRP: Sending NHRP Resolution Request for dest: 192.168.11.5 to NHS: 192.168.11.1 using our src: 192.168.11.3
*Aug 3 01:02:45.798: NHRP: Attempting to send packet via DEST 192.168.11.1
*Aug 3 01:02:45.798: NHRP: NHRP successfully resolved 192.168.11.1 to NBMA 150.1.254.254
*Aug 3 01:02:45.798: NHRP: Encapsulation succeeded. Tunnel IP addr 150.1.254.254
*Aug 3 01:02:45.798: NHRP: Send Resolution Request via Tunnel0 vrf 0, packet size: 72
*Aug 3 01:02:45.798: NHRP: 96 bytes out Tunnel0
*Aug 3 01:02:45.798: NHRP: NHRP successfully resolved 192.168.11.5 to NBMA 150.1.254.254
*Aug 3 01:02:45.802: NHRP: NHRP successfully resolved 192.168.11.5 to NBMA 150.1.254.254
*Aug 3 01:02:45.802: NHRP: Receive Resolution Request via Tunnel0 vrf 0, packet size: 92
*Aug 3 01:02:45.802: NHRP: netid_in = 1, to_us = 1
*Aug 3 01:02:45.802: NHRP: nhrp_rtlookup yielded Tunnel0
*Aug 3 01:02:45.802: NHRP: request was to us, responding with ouraddress
*Aug 3 01:02:45.802: NHRP: Checking for delayed event 192.168.11.5/192.168.11.3 on list (Tunnel0).
*Aug 3 01:02:45.802: NHRP: No node found.
It gets its request answer from R1 but needs to build the IPSec session first, but it did get the outside
*Aug 3 01:02:45.802: NHRP: Delaying resolution request nbma src:150.1.13.3 nbma dst:150.1.10.5 reason:IPSEC-IFC: need to wait for IPsec SAs.
*Aug 3 01:02:45.854: NHRP: Enqueueing delayed event to be processed. src:104:0:C0A8:B05:: dst:104:0:C0A8:B03::
*Aug 3 01:02:45.858: NHRP: Process delayed resolution request src:192.168.11.5 dst:192.168.11.3
*Aug 3 01:02:45.858: NHRP: nhrp_rtlookup yielded Tunnel0
*Aug 3 01:02:45.858: NHRP: request was to us, responding with ouraddress
*Aug 3 01:02:45.858: NHRP: Checking for delayed event 192.168.11.5/192.168.11.3 on list (Tunnel0).
*Aug 3 01:02:45.858: NHRP: No node found.
*Aug 3 01:02:45.858: NHRP: No need to delay processing of resolution event nbma src:150.1.13.3 nbma dst:150.1.10.5
*Aug 3 01:02:45.858: NHRP: Attempting to send packet via DEST 192.168.11.5
*Aug 3 01:02:45.858: NHRP: NHRP successfully resolved 192.168.11.5 to NBMA 150.1.254.254
*Aug 3 01:02:45.858: NHRP: Encapsulation succeeded. Tunnel IP addr 150.1.254.254
*Aug 3 01:02:45.858: NHRP: Send Resolution Reply via Tunnel0 vrf 0, packet size: 120
*Aug 3 01:02:45.858: NHRP: 144 bytes out Tunnel0
*Aug 3 01:02:45.866: NHRP: Receive Resolution Reply via Tunnel0 vrf 0, packet size: 120
*Aug 3 01:02:45.866: NHRP: netid_in = 0, to_us = 1
*Aug 3 01:02:45.866: NHRP: Checking for delayed event /192.168.11.5 on list (Tunnel0).
Now that all that is over, we can check the show dmvpn and viola, we now have a tunnel for 150.1.10.5 for 192.168.11.5 Rack1R3#sh dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer NHS Status: E --> Expecting Replies, R --> Responding UpDn Time --> Up or Down Time for a Tunnel ==========================================================================
Interface: Tunnel0, IPv4 NHRP Details Type:Spoke, NHRP Peers:2,
# Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 150.1.254.254 192.168.11.1 UP 07:09:27 S 1 150.1.10.5 192.168.11.5 UP 00:00:35 D
Rack1R3#
One thing to notice is that since we did not talk to R4 in a bit, that network aged out.
Next post will be on adding routing protocols over the DMVPN network.
Pingback: » DMVPN and Routing Protocols – RIP FryGuy's Blog
Pingback: » DMVPN and Routing Protocols – BGP FryGuy's Blog
Pingback: » DMVPN and Routing Protocols – CDP FryGuy's Blog
Wow Jeff, this is awesome….great post.
hi Jeff, can you please elaborate on 150.1.254.254 and 150.1.x.x address assignments? I understand that on spokes you are manually mapping the hub’s tunnel ip address to 150.1.254.254. How does the assignment on spokes happen automagically to 150.1.x.x space? thanks, -igor
Those IP addresses are the outside interfaces on the remote routes. They are the connection to the ISP. I just chose different networks to demonstrate what it would look like on a real Internet connection.
hi Jeff, can you please elaborate on 150.1.254.254 and 150.1.x.x address assignments? I understand that on spokes you are manually mapping the hub’s tunnel ip address to 150.1.254.254. How does the assignment on spokes happen automagically to 150.1.x.x space? thanks, -igor
Those IP addresses are the outside interfaces on the remote routes. They are the connection to the ISP. I just chose different networks to demonstrate what it would look like on a real Internet connection.
hi Jeff, can you please elaborate on 150.1.254.254 and 150.1.x.x address assignments? I understand that on spokes you are manually mapping the hub’s tunnel ip address to 150.1.254.254. How does the assignment on spokes happen automagically to 150.1.x.x space? thanks, -igor
Those IP addresses are the outside interfaces on the remote routes. They are the connection to the ISP. I just chose different networks to demonstrate what it would look like on a real Internet connection.
“Now, lets configure the hubs” you meant to say “Now, lets configure the SPOKES”….
“Now, lets configure the hubs” you meant to say “Now, lets configure the SPOKES”….
“Now, lets configure the hubs” you meant to say “Now, lets configure the SPOKES”….
Could you explain about debug nhrp ? I still don’t get it
Thanks