Observed BGP peering issues when Md5 was enabled

Review in progress for https://review.opencontrail.org/12118
Submitter: Nischal Sheth (<email address hidden>)

Review in progress for https://review.opencontrail.org/12119
Submitter: Nischal Sheth (<email address hidden>)

Review in progress for https://review.opencontrail.org/12120
Submitter: Nischal Sheth (<email address hidden>)

Review in progress for https://review.opencontrail.org/12121
Submitter: Nischal Sheth (<email address hidden>)

I suspect that the issue is orthogonal to MD5 configuration.
Have submitted a fix based on traces from the affected control-node
and examination of code.

Reviewed: https://review.opencontrail.org/12119
Committed: http://github.org/Juniper/contrail-controller/commit/120322ce39882b7dc0cb4a540eb3def3d7ec5101
Submitter: Zuul
Branch: master

commit 120322ce39882b7dc0cb4a540eb3def3d7ec5101
Author: Nischal Sheth <email address hidden>
Date: Tue Jun 30 16:48:53 2015 -0700

Fix corner case in IFMapDependencyTracker

Existing code prevents the same node from getting added to the change
list multiple times. This can cause problems if there's some mutual
dependency between the nodes.

In the case of this bug, a bgp-peering happened to be on the change
list, followed by the bgp-router object for the local control node.
The bgp-peering should have been added to the change list again as
part of the dependency evaluation for the bgp-router. However, it
was not added since it was already on the list. This caused an issue
because the local router id was not available when the bgp-peering
was evaluated. The local router id became available when bgp-router
was evaluated, but the bgp-peering was never evaluated again.

Change-Id: I54404ee8505d31aff49d153afd8b93488611c638
Closes-Bug: 1470256

Verified by taking the diff for this fix and building contrail-control.

Copied the binary on to the setup that had issue and restarted contrail-control so that the fix can kick in.

I can now see the peering established.

Fix holds good for this case.

As a workaround run the below query on each of the contrail-control nodes in the cluster.

curl -s http://localhost:8083/Snh_ShowBgpNeighborConfigReq | xmllint --format - | grep local_identifier | awk '{print $3}' | grep 0.0.0.0 | wc -l

if the above query returns a value greater than 0, then you need to do the following:

1-> Delete the Contrail-Control node from API / webUI
2-> Re-Add using API (/opt/contrail/utils/provision_control.py) or via the webUI

Reviewed: https://review.opencontrail.org/12121
Committed: http://github.org/Juniper/contrail-controller/commit/fcdc384380b825a91ffaa251bd581459ada26a4a
Submitter: Zuul
Branch: R2.0

commit fcdc384380b825a91ffaa251bd581459ada26a4a
Author: Nischal Sheth <email address hidden>
Date: Tue Jun 30 16:48:53 2015 -0700

Fix corner case in IFMapDependencyTracker

Existing code prevents the same node from getting added to the change
list multiple times. This can cause problems if there's some mutual
dependency between the nodes.

In the case of this bug, a bgp-peering happened to be on the change
list, followed by the bgp-router object for the local control node.
The bgp-peering should have been added to the change list again as
part of the dependency evaluation for the bgp-router. However, it
was not added since it was already on the list. This caused an issue
because the local router id was not available when the bgp-peering
was evaluated. The local router id became available when bgp-router
was evaluated, but the bgp-peering was never evaluated again.

Change-Id: I54404ee8505d31aff49d153afd8b93488611c638
Closes-Bug: 1470256

Reviewed: https://review.opencontrail.org/12118
Committed: http://github.org/Juniper/contrail-controller/commit/45ef3d6f3f3cf808ae3af778b96632d92a5fb193
Submitter: Zuul
Branch: R2.20

commit 45ef3d6f3f3cf808ae3af778b96632d92a5fb193
Author: Nischal Sheth <email address hidden>
Date: Tue Jun 30 16:48:53 2015 -0700

Fix corner case in IFMapDependencyTracker

Existing code prevents the same node from getting added to the change
list multiple times. This can cause problems if there's some mutual
dependency between the nodes.

In the case of this bug, a bgp-peering happened to be on the change
list, followed by the bgp-router object for the local control node.
The bgp-peering should have been added to the change list again as
part of the dependency evaluation for the bgp-router. However, it
was not added since it was already on the list. This caused an issue
because the local router id was not available when the bgp-peering
was evaluated. The local router id became available when bgp-router
was evaluated, but the bgp-peering was never evaluated again.

Change-Id: I54404ee8505d31aff49d153afd8b93488611c638
Closes-Bug: 1470256

Reviewed: https://review.opencontrail.org/12120
Committed: http://github.org/Juniper/contrail-controller/commit/ed4053fcc8ad542b8a9ec6e9b42b84a675cf5cc8
Submitter: Zuul
Branch: R2.1

commit ed4053fcc8ad542b8a9ec6e9b42b84a675cf5cc8
Author: Nischal Sheth <email address hidden>
Date: Tue Jun 30 16:48:53 2015 -0700

Fix corner case in IFMapDependencyTracker

Existing code prevents the same node from getting added to the change
list multiple times. This can cause problems if there's some mutual
dependency between the nodes.

In the case of this bug, a bgp-peering happened to be on the change
list, followed by the bgp-router object for the local control node.
The bgp-peering should have been added to the change list again as
part of the dependency evaluation for the bgp-router. However, it
was not added since it was already on the list. This caused an issue
because the local router id was not available when the bgp-peering
was evaluated. The local router id became available when bgp-router
was evaluated, but the bgp-peering was never evaluated again.

Change-Id: I54404ee8505d31aff49d153afd8b93488611c638
Closes-Bug: 1470256