VASI – VRF Aware Service Interfaces, what the heck are those?!
Cisco has defined a VASI interface as:
VASI is implemented using virtual interfaces that provide the framework necessary to configure a firewall and NAT between VRF instances. Each interface pair is associated with two different VRF instances. The two virtual interfaces, called vasileft and vasiright, in a pair are logically wired back-to-back and are completely symmetrical. Each interface has an index. The association of the pairing is done automatically based on the two interface indexes such that vasileft automatically gets paired to vasiright.
So you could look at a VASI interface as like a back-to-back connection between VRFs – call it a virtual cable of sort. You can also see that the traditional use of a VASI interface is to connect two VRFs, on the same router, together so they can route and NAT between each other. But what I am saying about Global Route Leaking?
Well, recently I had a problem where a client wanted to leak routes, dynamically, from a VRF to the Global table on an ASR1000 with an RP1 installed. We could easily have done it via a physical cable, but I knew their had to be a better way. I did some digging on the VASI interface and possible configurations and came across this – added in IOS XE 3.10S code is the ability to actually BGP peer over the VASI interfaces to exchange routes.
You may click on the image below to view the PDF on this.
So what we did, after much additional research and trial-and error was come up with a solution where we created a VASI interface on a PE and ran BGP over the connection. When I got home, I was able to replicate this configuration on the CSR1000V routers in my ESXi lab for this post. Below is the diagram of the CSR1000V network that I created (oh, and these CSR1000V are running 3.10S code!)
So what we have here is a CSR1 Customer router in VRF Green; a SP network consitsing of CSR2, CSR5, and CSR3; and a Internet router called CSR4.
Each router has a loopback address of X.X.X.X/32 (so CSR1 is 1.1.1.1/32, CSR2 is 2.2.2.2/32, etc)
The SP Core network is running OSPF Area 0 and BGP 22061, with CSR5 acting as a route-reflector. CSR3 has an eBGP peer with CSR4 AS100 and is advertising the SP learned network routes. CSR4 is only advertising the 4.4.4.4/32 network to R3 via eBGP.
The routing protocol between CSR1 and CSR2 is OSPF and all interfaces are in OSPF Area 0 under VRF Green.
Our goal here is to get Loopback 0 on CSR1 to be able to PING the Loopback interface on CSR4.
Ok, high level stuff explained, lets get to the fun!
So, first the easy stuff – CSR4 BGP config:
router bgp 100
bgp log-neighbor-changes
network 4.4.4.4 mask 255.255.255.255
neighbor 10.1.34.3 remote-as 22061
See, straight forward and normal.
Now here is CSR3 config for OSPF and BGP:
router ospf 22061
router-id 3.3.3.3
passive-interface Loopback0
network 3.3.3.3 0.0.0.0 area 0
network 10.1.2.3 0.0.0.0 area 0
router bgp 22061
bgp log-neighbor-changes
redistribute ospf 22061 match internal external 1 external 2
neighbor 5.5.5.5 remote-as 22061
neighbor 5.5.5.5 update-source Loopback0
neighbor 5.5.5.5 next-hop-self
neighbor 10.1.34.4 remote-as 100
And for OSPF and BGP, pretty straight forward. Oh, the reason for the Next-hop-self here on CSR3 to CSR5 is that I am not advertising the link between CSR3 and CSR4 into OSPF.
Ok, CSR5 time
router ospf 22061
router-id 5.5.5.5
passive-interface Loopback0
network 5.5.5.5 0.0.0.0 area 0
network 10.1.1.5 0.0.0.0 area 0
network 10.1.2.5 0.0.0.0 area 0
!
router bgp 22061
bgp log-neighbor-changes
neighbor 2.2.2.2 remote-as 22061
neighbor 2.2.2.2 route-reflector-client
neighbor 3.3.3.3 remote-as 22061
neighbor 3.3.3.3 route-reflector-client
Straight forward BGP adn OSPF config, nothing fancy.
Now we will do CSR1 quick as well:
router ospf 100
no auto-cost
network 1.1.1.1 0.0.0.0 area 0
network 172.16.0.1 0.0.0.0 area 0
Nothing to see their either, now for the fun one – CSR2.
First we will do the normal config, nothing fancy with VASI interfaces – yet.
First thing we can do is configure our VRF Green OSPF process, here 100. I am also configuring CSR2 VRF Green to originate a default-route to CSR1. This is being done as CSR1 only needs a default route advertised to it for outbound connectivity.
router ospf 100 vrf Green
priority 127
no auto-cost
redistribute connected subnets
network 172.16.0.2 0.0.0.0 area 0
default-information originate always
!
Here is the SP Core OSPF and BGP network, normal stuff
router ospf 22061
router-id 2.2.2.2
no auto-cost
redistribute connected subnets
passive-interface Loopback0
network 2.2.2.2 0.0.0.0 area 0
network 10.1.1.2 0.0.0.0 area 0
!
router bgp 22061
bgp log-neighbor-changes
redistribute connected
neighbor 5.5.5.5 remote-as 22061
neighbor 5.5.5.5 update-source Loopback0
Ok, so what does our routing table on CSR1 look like now:
Gateway of last resort is 172.16.0.2 to network 0.0.0.0
O*E2 0.0.0.0/0 [110/1] via 172.16.0.2, 00:00:03, GigabitEthernet1
1.0.0.0/32 is subnetted, 1 subnets
C 1.1.1.1 is directly connected, Loopback0
172.16.0.0/16 is variably subnetted, 2 subnets, 2 masks
C 172.16.0.0/24 is directly connected, GigabitEthernet1
L 172.16.0.1/32 is directly connected, GigabitEthernet1
CSR1#
Good, we have a default route – so we should try to PING CSR4 loopback quick:
CSR1#p 4.4.4.4 so l0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 4.4.4.4, timeout is 2 seconds:
Packet sent with a source address of 1.1.1.1
U.U.U
Success rate is 0 percent (0/5)
As expected, that did not work. Now lets see if CSR2 can ping it, but first lets make sure we have a route:
Gateway of last resort is not set
2.0.0.0/32 is subnetted, 1 subnets
C 2.2.2.2 is directly connected, Loopback0
3.0.0.0/32 is subnetted, 1 subnets
O 3.3.3.3 [110/12] via 10.1.1.5, 10:14:54, GigabitEthernet2
4.0.0.0/32 is subnetted, 1 subnets
B 4.4.4.4 [200/0] via 3.3.3.3, 00:32:11
5.0.0.0/32 is subnetted, 1 subnets
O 5.5.5.5 [110/11] via 10.1.1.5, 10:14:54, GigabitEthernet2
10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
C 10.1.1.0/24 is directly connected, GigabitEthernet2
L 10.1.1.2/32 is directly connected, GigabitEthernet2
O 10.1.2.0/24 [110/11] via 10.1.1.5, 10:14:54, GigabitEthernet2
CSR2#
Yes, we have a route – so the PING should work
CSR2#p 4.4.4.4 so l0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 4.4.4.4, timeout is 2 seconds:
Packet sent with a source address of 2.2.2.2
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/3 ms
CSR2#
And it does., good.
So what we have right now is a lonely VRF (Green) that needs Internet access, and a SP network that can provide this. But the customer does not want to peer with anyone, install any new routing protocols, or use Statics. We (the SP) needs to be able to advertise any network that he customer advertises to us. So what do we do?
Sure, we could do static routes – but if the customer adds a new network, we will need to make changes.
This is where the VASI interface comes in. Here is what we are going to configure:
On CSR2 we are going to create a VASILEFT and VASIRIGHT interface. The pair are mated with a numerical identifier, here we are using 1.
CSR2#sh run int vasileft 1
Building configuration…
Current configuration : 116 bytes
!
interface vasileft1
ip vrf forwarding Green
ip address 198.18.0.42 255.255.255.252
no keepalive
end
CSR2#sh run int vasiright 1
Building configuration…
Current configuration : 82 bytes
!
interface vasiright1
ip address 198.18.0.41 255.255.255.252
no keepalive
end
CSR2#
Now to test VASI connectivity, lets PING each interface
CSR2#ping 198.18.0.42
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 198.18.0.42, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
CSR2#ping vrf Green 198.18.0.41
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 198.18.0.41, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
CSR2#
Good, we can now PING from the global to the VRF and visa-verse.
Now time for the BGP config part – here is where it gets fun!
First we need to configure the VASILeft as an iBGP peer
router bgp 22061
neighbor 198.18.0.42 remote-as 22061
neighbor 198.18.0.42 route-reflector-client
neighbor 198.18.0.42 next-hop-self
neighbor 198.18.0.42 default-originate
But note – we configured a few different options here.
We told BGP that the VASI interface is a RR client since iBGP will not advertise learned iBGP routes. This tells this router that it does need to advertise iBGP routes to the peer. And the other thing that we did was set the next hop to self, ensuring that there is a valid next hop.
Instead of sending (redistributing) BGP into OSPF on the Green VRF, instead we instructed BGP to send a default route to the VRF. This will ensure that anything that CSR1 wants to talk to, it will have a valid route. If this step is not done, the Green VRF on CSR2 will not have a valid route out. (Yeah, I forgot to add that originally and had to figure it out!)
Here, you can see the problem if you do not originate a default to the VASI Peer, no default route.
CSR2#sh ip route vrf Green
Routing Table: Green
Codes: L – local, C – connected, S – static, R – RIP, M – mobile, B – BGP
D – EIGRP, EX – EIGRP external, O – OSPF, IA – OSPF inter area
N1 – OSPF NSSA external type 1, N2 – OSPF NSSA external type 2
E1 – OSPF external type 1, E2 – OSPF external type 2
i – IS-IS, su – IS-IS summary, L1 – IS-IS level-1, L2 – IS-IS level-2
ia – IS-IS inter area, * – candidate default, U – per-user static route
o – ODR, P – periodic downloaded static route, H – NHRP, l – LISP
a – application route
+ – replicated route, % – next hop override
Gateway of last resort is not set
1.0.0.0/32 is subnetted, 1 subnets
O 1.1.1.1 [110/11] via 172.16.0.1, 1d09h, GigabitEthernet1
2.0.0.0/32 is subnetted, 1 subnets
B 2.2.2.2 [200/0] via 198.18.0.41, 22:50:25
10.0.0.0/24 is subnetted, 1 subnets
B 10.1.1.0 [200/0] via 198.18.0.41, 22:50:25
172.16.0.0/16 is variably subnetted, 2 subnets, 2 masks
C 172.16.0.0/24 is directly connected, GigabitEthernet1
L 172.16.0.2/32 is directly connected, GigabitEthernet1
198.18.0.0/24 is variably subnetted, 2 subnets, 2 masks
C 198.18.0.40/30 is directly connected, vasileft1
L 198.18.0.42/32 is directly connected, vasileft1
Now, back to the discussion …
Now for the BGP VRF configuration, here is the fun part 🙂
router bgp 22061
address-family ipv4 vrf Green
bgp router-id 198.18.0.42
redistribute ospf 100 match internal external 1 external 2
neighbor 198.18.0.41 remote-as 22061
neighbor 198.18.0.41 activate
neighbor 198.18.0.41 next-hop-self
exit-address-family
As you can see, we are redistributing the OSPF 100 VRF routes into BGP – thus any route that the client advertises to us via OSPF will be in our BGP advertisements as well.
We also configure the BGP router-id to be the VASI interface. This is done as the BGP router is peering with itself and will throw back an error of same router-id if we do not do this.
From there, everything else is normal BGP stuff.
If we look at our BGP summary, we will see we now have a peer with 198.18.0.42
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
5.5.5.5 4 22061 67 61 35 0 0 00:46:27 4
198.18.0.42 4 22061 14 23 35 0 0 00:07:15 2
CSR2#
So, what does the route-table on CSR2 look like after this (and you can see the 0/0 route is pointing to the VASI interface IP in the global table)
Gateway of last resort is 198.18.0.41 to network 0.0.0.0
B* 0.0.0.0/0 [200/0] via 198.18.0.41, 00:05:32
1.0.0.0/32 is subnetted, 1 subnets
O 1.1.1.1 [110/11] via 172.16.0.1, 1d09h, GigabitEthernet1
2.0.0.0/32 is subnetted, 1 subnets
B 2.2.2.2 [200/0] via 198.18.0.41, 00:05:32
10.0.0.0/24 is subnetted, 1 subnets
B 10.1.1.0 [200/0] via 198.18.0.41, 00:05:32
172.16.0.0/16 is variably subnetted, 2 subnets, 2 masks
C 172.16.0.0/24 is directly connected, GigabitEthernet1
L 172.16.0.2/32 is directly connected, GigabitEthernet1
198.18.0.0/24 is variably subnetted, 2 subnets, 2 masks
C 198.18.0.40/30 is directly connected, vasileft1
L 198.18.0.42/32 is directly connected, vasileft1
CSR2#
Nice, we now have the routes for VRF Green leaked into the global table!
Let us check CSR1 routing table:
O*E2 0.0.0.0/0 [110/1] via 172.16.0.2, 00:04:59, GigabitEthernet1
1.0.0.0/32 is subnetted, 1 subnets
C 1.1.1.1 is directly connected, Loopback0
172.16.0.0/16 is variably subnetted, 2 subnets, 2 masks
C 172.16.0.0/24 is directly connected, GigabitEthernet1
L 172.16.0.1/32 is directly connected, GigabitEthernet1
198.18.0.0/30 is subnetted, 1 subnets
O E2 198.18.0.40 [110/20] via 172.16.0.2, 00:08:52, GigabitEthernet1
CSR1#
Hmm, we now see that the 198.18.0.40 subnet is there – where did that come from?
Well, if you look at the OSPF config for CSR2, I had redistribute connected subnets configured. Since that was there, and we added the VASILeft interface into VRF green, we started to advertise it.
But other then that, all the rest is the same. Wonder if we can ping CSR4 loopback now?
CSR1#p 4.4.4.4 so l0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 4.4.4.4, timeout is 2 seconds:
Packet sent with a source address of 1.1.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms
CSR1#
And if we do a tracert to 4.4.4.4 with a source of 1.1.1.1
CSR1#traceroute 4.4.4.4 source l0
Type escape sequence to abort.
Tracing the route to 4.4.4.4
VRF info: (vrf in name/id, vrf out name/id)
1 172.16.0.2 1 msec 0 msec 1 msec
2 198.18.0.41 1 msec 1 msec 1 msec
3 10.1.1.5 1 msec 1 msec 0 msec
4 10.1.2.3 1 msec 1 msec 1 msec
5 10.1.34.4 2 msec * 2 msec
CSR1#
We see the VASI interface in the path.
YES, we can ping!
All the configs for these routers can be found here https://www.dropbox.com/s/mb0pp23z0wwmq0m/VasliGlobalLeak.zip
!