Carrier Supporting Carrier - The whole story (2)

In this post we are going to discuss both CSC options that we have highlighted in the previous post in details. The two available options are either an ISP customer carrier, or a BGP/MPLS VPN customer carrier.

We are going to conquer the control plane and data plane separately, in order to well understand the complete architecture operation. We are also going to separate the control plane into three phases; the backbone carrier network, the CSC peering and the customer carrier network.

From the control plane prospective, the backbone carrier and the CSC peering parts are identical in both options, the only difference will lie within the customer carrier network configuration, more specifically between its different sites connected over the backbone carrier.

Backbone carrier control plane:

It is a typical MPLS VPN service provider control plane, with MP-iBGP between the PE routers, and IGP + MPLS (LDP) on all the network routers, providing network wide connectivity and label switching. In simple words the backbone carrier handles the customer carrier as if an MPLS VPN customer (with a minor difference of the need do label exchange between the CSC-PE and the CSC-CE routers).

CSC peering control plane:

As discussed in the previous post, we have two options to provide the CSC peering, lets remember that what we want to do is to exchange the customer carrier internal networks with their relative labels, in order to be label switched allover the backbone carrier, with no traditional routing required at any point.

The available two options are:

• Using IGP for route exchange plus LDP for label exchange - Like normal MPLS VPN PE-CE routing (static/RIP/EIGRP/OSPF), but with MPLS enabled under the VRF interface on the CSC-PE side and the CSC-CE side (enabling LDP simply via mpls ip under the interfaces on both sides).

• Using BGP for both route and label exchange - Using eBGP between the CSC-PE and CSC-CE routers, like normal MPLS VPN PE-CE eBGP under the address-family ipv4 vrf on the CSC-PE side but adding the neighbor x.x.x.x send-label command and the mpls bgp forwarding interface command on both sides - When using eBGP for label exchange using the send-label option, mpls bgp forwarding is automatically configured under the interface.

Below is a typical configuration example for the CSC-PE and the CSC-CE routers.

CSC-PE router configuration example:

!
ip vrf csc-test
 rd 100:100
 route-target import 100:100
 route-target export 100:100
!
interface Loopback0
 ip address 1.1.1.1 255.255.255.255
!
interface ATM1/0.1 point-to-point
 description Connection to CSC-P1
 ip address 10.10.12.1 255.255.255.0
 ip router isis
 mpls ip
!
interface ATM2/0.1 point-to-point
 description Connection to CSC-CE1
 ip vrf forwarding csc-test
 ip address 192.168.120.1 255.255.255.0
 mpls bgp forwarding
!
router isis
 net 49.0001.0010.0100.1001.00
 is-type level-2-only
 metric-style wide
 log-adjacency-changes
 passive-interface Loopback0
!
router bgp 1
 no synchronization
 bgp log-neighbor-changes
 neighbor 3.3.3.3 remote-as 1
 neighbor 3.3.3.3 update-source Loopback0
 no auto-summary
 !
 address-family vpnv4
  neighbor 3.3.3.3 activate
  neighbor 3.3.3.3 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf csc-test
  neighbor 192.168.120.20 remote-as 2
  neighbor 192.168.120.20 send-label
  neighbor 192.168.120.20 as-override
  no auto-summary
  no synchronization
 exit-address-family
 !

CSC-PE router configuration example:

!
interface Loopback0
 ip address 20.20.20.20 255.255.255.255
!
interface ATM1/0.1 point-to-point
 description Connection to PE1
 ip address 192.168.10.20 255.255.255.0
 ip router isis
 mpls ip
!
interface ATM2/0.1 point-to-point
 description Connection to CSC-PE1
 ip address 192.168.120.20 255.255.255.0
 mpls bgp forwarding
!
router isis
 net 49.0010.0200.2002.0020.00
 is-type level-2-only
 metric-style wide
 log-adjacency-changes
 passive-interface Loopback0
!
router bgp 2
 no synchronization
 bgp log-neighbor-changes
 redistribute isis
 neighbor 192.168.120.1 remote-as 1
 neighbor 192.168.120.1 send-label
 no auto-summary
!

Now lets examine the two different options for the customer carrier in details.

ISP customer carrier:

Mainly providing Internet Access to its customers, while using the backbone carrier network as a transit network to connect its sites.

Customer carrier control plane:

In this case the customer carrier edge routers located in different sites connected over the backbone carrier backbone need to run iBGP between each other to exchange their customers' IPv4 routes.

Below is an example for the customer carrier edge routers configuration in the case of ISP customer carrier.

Customer carrier edge router configuration example:

!
interface Loopback0
 ip address 10.10.10.10 255.255.255.255
!
interface ATM1/0.1 point-to-point
 description Connection to CSC-CE1
 ip address 192.168.10.10 255.255.255.0
 ip router isis
 mpls ip
!
interface FastEthernet0/0.10
 encapsulation dot1q 10
 ip address x.x.x.1 255.255.255.252
!
interface FastEthernet0/0.10
 encapsulation dot1q 11
 ip address x.x.x.1 255.255.255.252
!
interface FastEthernet0/0.20
 encapsulation dot1q 20
 ip address x.x.x.1 255.255.255.252
!
router isis
 net 49.0010.0100.1001.0010.00
 is-type level-2-only
 metric-style wide
 log-adjacency-changes
 passive-interface Loopback0
!
router bgp 2
 no synchronization
 bgp log-neighbor-changes
 redistribute static
 neighbor 40.40.40.40 remote-as 2
 neighbor 40.40.40.40 update-source Loopback0
 no auto-summary
 !
!
ip route x.x.x.x 255.255.255.0 x.x.x.2
ip route x.x.x.x 255.255.255.0 x.x.x.2
ip route x.x.x.x 255.255.255.0 x.x.x.2
!

Data plane:

Lets assume that we are talking about a packet traveling from site 1 to site 2:

• The packet is either IP forwarded or label switched within the individual customer carrier sites (according to whether MPLS is enabled or not withing the customer carrier sites).

• After the packet reaches CSC-CE1, it exits it towards CSC-PE1, and it is labeled with one label, this is the label which was exchanged either via LDP or eBGP between CSC-PE1 and CSC-CE1.

• When the packet is received by CSC-PE1, the Top label is replaced by the MP-iBGP label learned from CSC-PE2 and then a new LDP label is Pushed for the packet to reach CSC-PE2 over the backbone carrier cloud (the backbone carrier egress PE LDP label).

• The packet is label switched on the backbone carrier LSRs using the Top LDP label until it reaches the CSC-PE2.

• When the packet is received by the PHP of CSC-PE2, the Top label is popped.

• When the packet is received by CSC-PE2, the final Top label is replaced by the label which was exchanged either via LDP or eBGP between CSC-PE2 and CSC-CE2.

• When the packet is received by CSC-CE2, the final Top label is popped and the packet is either forwarded as a pure IP packet or labeled packet according to whether MPLS is enabled within the customer carrier sites or not.

BGP/MPLS VPN customer carrier:

This type of customer carriers provide MPLS VPN services to its customers, while using the backbone carrier network as a transit network to connect its sites.

Customer carrier control plane:

In this case the customer carrier edge routers located in different sites connected over the backbone carrier backbone need to run MP-iBGP between each other to exchange their customers' VPNv4 routes.

Below is an example for the customer carrier edge routers configuration in the case of BGP/MPLS VPN customer carrier.

Customer carrier edge router configuration example:

!
ip vrf test
 rd 1:1
 route-target import 1:1
 route-target export 1:1
!
ip vrf test2
 rd 2:2
 route-target import 2:2
 route-target export 2:2
!
interface Loopback0
 ip address 10.10.10.10 255.255.255.255
!
interface ATM1/0.1 point-to-point
 description Connection to CSC-CE1
 ip address 192.168.10.10 255.255.255.0
 ip router isis
 mpls ip
!
interface FastEthernet0/0.10
 encapsulation dot1q 10
 ip vrf forwarding test
 ip address 10.10.10.1 255.255.255.0
!
interface FastEthernet0/0.10
 encapsulation dot1q 11
 ip vrf forwarding test
 ip address 11.11.11.1 255.255.255.0
!
interface FastEthernet0/0.20
 encapsulation dot1q 20
 ip vrf forwarding test2
 ip address 20.20.20.1 255.255.255.0
!
router isis
 net 49.0010.0100.1001.0010.00
 is-type level-2-only
 metric-style wide
 log-adjacency-changes
 passive-interface Loopback0
!
router bgp 2
 no synchronization
 bgp log-neighbor-changes
 neighbor 40.40.40.40 remote-as 2
 neighbor 40.40.40.40 update-source Loopback0
 no auto-summary
 !
 address-family vpnv4
  neighbor 40.40.40.40 activate
  neighbor 40.40.40.40 send-community extended
 exit-address-family
 !
!
 address-family ipv4 vrf test
  redistribute connected
  redistribute static
  no auto-summary
  no synchronization
 exit-address-family
!
 address-family ipv4 vrf test2
  redistribute connected
  redistribute static
  no auto-summary
  no synchronization
 exit-address-family
!
ip route vrf test 192.168.1.0 255.255.255.0 10.10.10.2
ip route vrf test 192.168.3.0 255.255.255.0 11.11.11.2
ip route vrf test2 192.168.1.0 255.255.255.0 20.20.20.2
!

Data plane:

Again, lets assume we are talking about a packet traveling from site 1 to site 2:

• The packet is label switched within the individual customer carrier site - In this case the customer carrier customers' packets must be label switched within the customer carrier sites since it is MPLS VPN traffic.

• After the packet reaches CSC-CE1, it exits it towards CSC-PE1, and it is labeled with two labels, the Top label is the label which was exchanged either via LDP or eBGP between CSC-PE1 and CSC-CE1, and the bottom label is the label that was exchanged between the customer carrier edge routers in the different sites (this bottom label is only significant to the customer carrier edge routers).

• When the packet is received by CSC-PE1, the Top label is replaced by the MP-iBGP label learned from CSC-PE2 and then a new LDP label is Pushed for the packet to reach CSC-PE2 over the backbone carrier cloud (the backbone carrier egress PE LDP label), while the customer carrier original VPN label is preserved. Accordingly after this step the packet will have 3 labels in its label stack.

• The packet is label switched on the backbone carrier LSRs using the Top LDP label until it reaches the CSC-PE2.

• When the packet is received by the PHP of CSC-PE2, the Top label is popped.

• When the packet is received by CSC-PE2, the now Top label is replaced by the label which was exchanged either via LDP or eBGP between CSC-PE2 and CSC-CE2.

• When the packet is received by CSC-CE2, the now Top label is replaced by the LDP label of the customer carrier site.

• Now the packet has two labels in its stack, the Top label is the IGP/LDP label to take it to the egress edge router within the customer carrier site, and the bottom label is the MPLS VPN label of the customer carrier customer.

I hope that I've covered this topic in a decent manner. It is obvious that I haven't covered the case of using IGP+LDP instead of eBGP between the CSC-PE and CSC-CE, but use your imagination, it is straight forward, all that is required is the use of an IGP between the CSC-PE and the CSC-CE in a PE-CE routing protocol configuration (only the backbone carrier needs to learn the customer carrier internal IPs, nothing more and nothing less), and finally enable MPLS on the links between the CSC-PE and the CSC-CE routers.

I hope that I've been informative.

BR,
Mohammed Mahmoud.