BGP performance tuning - Convergence, Stability, Scalability and NSF (Part 2)

As we agreed in the last post, we are going to discuss the available tools and parameters for BGP performance tuning. In this post we are going to start discussing the different types of timers that BGP uses for its operation.

BGP network timers (Keepalive and Holdtime)

Keepalive messages are sent periodically to ensure the liveness of the connection, RFC4271 suggests 30 seconds for Keepalive and 90 seconds for HoldTime, with a recommendation that with any implementation of BGP the reasonable maximum time between keepalive messages would be one third of the HoldTime interval. If a router didn't receive a keepalive message within the HoldTime interval the software declares the peer dead.

The Cisco IOS default keepalive timer is 60 seconds and the default HoldTime is 180 seconds, while Juniper routers use a default keepalive of 30 seconds and a default HoldTime of 90 seconds.

When a connection is started, BGP will negotiate the HoldTime with its neighbor(s). The smaller of the two HoldTimes will be chosen. The keepalive timer is then set based on the negotiated HoldTime and the locally configured keepalive time, each router may choose a different keepalive timer, since each router will compare the value of the negotiated HoldTime divided by 3 to its locally configured keepalive value, and the lower number will be used.

NOTE Juniper routers uses a default of 30/90 seconds - Accordingly if a Cisco and a Juniper routers are peering they will agree to use 30/90 seconds.

We can change these timers using the below commands:

Router(config-router)#timers bgp <keepalive:0-65535> <HoldTime:0-65535> <min_HoldTime:0-65535>
Router(config-router)#neighbor x.x.x.x timers <keepalive:0-65535> <HoldTime:0-65535> <min_HoldTime:0-65535>

If you tried to set a HoldTime of less than 20 seconds you'll get this warning: "% Warning: A HoldTime of less than 20 seconds increases the chances of peer flapping"

When the minimum acceptable HoldTime parameter is configured on a BGP router, a remote BGP peer session is established only if the remote peer is advertising a HoldTime that is equal to, or greater than, the minimum acceptable HoldTime interval, otherwise the session will fail and the local router will send a notification stating "unacceptable HoldTime."

NOTE After changing the BGP timers the BGP session must be hard cleared in order for the timers to be negotiated again during peer establishment (since the soft clear doesn't force the router to renegotiate a peer establishment, and this timer is negotiated via the OPEN message, while soft clear only requests updates refresh).

Advertisement Interval

By default, BGP sends the updates (either locally generated or newly received from another neighbor) to iBGP neighbors every 5 seconds (0 in some codes) and to eBGP neighbors every 30 seconds.

We can control the minimum interval between sending BGP routing updates using the following command:

Router(config-router)#neighbor x.x.x.x advertisement-interval <0-600>

Tuning this timer affects both advertisement and withdrawal.

Initial delay for sending updates

When BGP is started, it waits a specified period of time for its neighbors to be established themselves and to begin sending their initial updates. Once that period is complete, or when the time expires, the best path is calculated for each route, and the software starts sending advertisements out to its peers.

This behavior improves convergence time because, if the software were to start sending advertisements out immediately, it would have to send extra advertisements if it later received a better path for the prefix from another peer.

We can control the max initial delay for sending update (When the BGP session first comes up) using the following command:

Router(config-router)#bgp update-delay <1-3600>

The bgp update-delay command is used to tune the maximum time the software will wait after the first neighbor is established until it starts calculating best paths and sending out advertisements - The default is 120 seconds.

NOTE This timer controls both the delay to calculate the best paths locally and the sending of the initial advertisement to its neighbors.

Background BGP scanner

It is responsible for BGP housekeeping by scanning both the BGP RIB and the IP RIB and cleaning and sorting things out.

BGP monitors the next hop of the installed routes to verify next-hop reachability and to select, install, and validate the BGP best path. By default, the BGP scanner polls the RIB for this information every 60 seconds. During the 60 second time period between scan cycles, IGP instabilities or other network failures can cause temporarily black holes and routing loops.

NOTE With Cisco IOS the default timer is 60 seconds for the IPv4 address family and 15 seconds for the VPNv4 address family in order to optimize the VPNs routing table convergence.

This timer can be controlled via the following command:

Router(config-router)#bgp scan-time <5-60>

NOTE If the above command is configured under the BGP process it will only affect the IPv4 address family, to control it for the VPNv4 address family it must be configured under the address-family configuration.

There is another feature that changed the BGP scanner logic, called BGP Next-Hop address tracking. The BGP next-hop address tracking feature is enabled by default when a supporting Cisco IOS software image is installed. BGP next-hop address tracking is event driven. BGP prefixes are automatically tracked as peering sessions are established. Next-hop changes are rapidly reported to the BGP routing process as they are updated in the RIB. This optimization improves overall BGP convergence by reducing  the response time to next-hop changes for routes installed in the RIB. When a bestpath calculation is run in between BGP scanner cycles, only next-hop changes are tracked and processed. BGP next-hop address tracking is enabled by default under the IPv4 and VPNv4 address families. Disabling next hop address tracking may be useful if you the network has unstable IGP peers and route dampening is not resolving the stability issues. To control this feature use the bgp next-hop command.

I hope that I've been informative, and as agreed we shall cover the rest of the tools and parameters in the upcoming posts.

BR,
Mohammed Mahmoud.