Datapath loss during schema restart

Bug #1588204 reported by eon
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R2.20
Won't Fix
High
Ignatious Johnson Christopher
R2.21.x
Fix Committed
High
Ignatious Johnson Christopher
R2.22.x
Won't Fix
High
Ignatious Johnson Christopher
R3.0
Fix Committed
High
Ignatious Johnson Christopher
R3.0.2.x
Fix Committed
High
Ignatious Johnson Christopher
R3.1
Fix Committed
High
Ignatious Johnson Christopher
R3.2
In Progress
High
Ignatious Johnson Christopher
Trunk
Fix Committed
High
Ignatious Johnson Christopher
OpenContrail
Fix Committed
High
Ignatious Johnson Christopher

Bug Description

On initialization schema will set the default RT for all RI:

https://github.com/Juniper/contrail-controller/blob/R2.21.x/src/config/schema-transformer/to_bgp.py#L640

At this point we loose all data path that involves a router (E/O via router, SNAT)

Later, in the initial IFMAP poll, the RT are added back to RIs and the traffic is restored.

Observed on R2.21.x

Tags: config
Nischal Sheth (nsheth)
Changed in opencontrail:
assignee: nobody → Sachin Bansal (sbansal)
importance: Undecided → High
no longer affects: juniperopenstack/r2.20.x
information type: Public → Public Security
Sachin Bansal (sbansal)
Changed in opencontrail:
assignee: Sachin Bansal (sbansal) → Ignatious Johnson Christopher (ijohnson-x)
Revision history for this message
Sachin Bansal (sbansal) wrote :

The problem is happening because the route targets of the connected logical routers are not added to routing instances when schema transformer starts. They are added later. We are working on the problem.

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/21341
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.21.x

Review in progress for https://review.opencontrail.org/21382
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.22.x

Review in progress for https://review.opencontrail.org/21403
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.21.x

Review in progress for https://review.opencontrail.org/21382
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.20

Review in progress for https://review.opencontrail.org/21430
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.22.x

Review in progress for https://review.opencontrail.org/21403
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/21436
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/21341
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/21430
Committed: http://github.org/Juniper/contrail-controller/commit/2c07117d06dc2bdbef6505a8079ab5a7765a2293
Submitter: Zuul
Branch: R2.20

commit 2c07117d06dc2bdbef6505a8079ab5a7765a2293
Author: Ignatious Johnson Christopher <email address hidden>
Date: Thu Jun 23 16:10:13 2016 +0000

Route targets of the connected logical routers
are not added to routing instances during reinit,
when schema transformer restarts. They are added later
during the Logical Router objects are evaluated.

Fix is to add them during reinit of routing instances.
Closed-Bug: 1588204

Change-Id: Id321a12e6c4a3aacb05f63b6ea0f6a2d5e9793fa

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/21341
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/21436
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.21.x

Review in progress for https://review.opencontrail.org/21382
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.22.x

Review in progress for https://review.opencontrail.org/21403
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/21436
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/21341
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.22.x

Review in progress for https://review.opencontrail.org/21403
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/21341
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/21436
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.21.x

Review in progress for https://review.opencontrail.org/21382
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.22.x

Review in progress for https://review.opencontrail.org/21403
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/21436
Committed: http://github.org/Juniper/contrail-controller/commit/88b614e4e5538e3fab583db82091938f1caf5318
Submitter: Zuul
Branch: master

commit 88b614e4e5538e3fab583db82091938f1caf5318
Author: Ignatious Johnson Christopher <email address hidden>
Date: Fri Jun 24 22:56:25 2016 +0000

Route targets of the connected logical routers
are not added to routing instances during reinit,
when schema transformer restarts. They are added later
during the Logical Router objects are evaluated.

Fix is to update only the new route targets and remove
all stale route targets after reint.

Change-Id: Iefd644379d04a9f6dfbacb36bd1c7fac3db260e5
Closed-Bug: 1588204

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/21382
Committed: http://github.org/Juniper/contrail-controller/commit/9815d55e8e73b241c855623baaaf014ab0245270
Submitter: Zuul
Branch: R2.21.x

commit 9815d55e8e73b241c855623baaaf014ab0245270
Author: Ignatious Johnson Christopher <email address hidden>
Date: Thu Jun 23 16:10:13 2016 +0000

Route targets of the connected logical routers
are not added to routing instances during reinit,
when schema transformer restarts. They are added later
during the Logical Router objects are evaluated.

Fix is to add them during reinit of routing instances.
Closed-Bug: 1588204

Change-Id: Id321a12e6c4a3aacb05f63b6ea0f6a2d5e9793fa

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0

Review in progress for https://review.opencontrail.org/21341
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Sachin Bansal (sbansal)
information type: Public Security → Public
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/21341
Committed: http://github.org/Juniper/contrail-controller/commit/0e019b2cc9e09edbe38e5b8b69c5f385ddc13c05
Submitter: Zuul
Branch: R3.0

commit 0e019b2cc9e09edbe38e5b8b69c5f385ddc13c05
Author: Ignatious Johnson Christopher <email address hidden>
Date: Wed Jun 22 07:50:38 2016 +0000

Route targets of the connected logical routers
are not added to routing instances during reinit,
when schema transformer restarts. They are added later
during the Logical Router objects are evaluated.

Fix is to update only the new route targets and remove
all stale route targets after reint.

Closes-Bug: 1588204

Also contrail python packages are cleared from the pip cache
before creation of test virtual enviroiment to make sure
the virtual enviroinment is installed with the newly built
python package.

Change-Id: Iaf7aa002020db7b7de01c32d44c3c74f2215e90f

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Reviewed: https://review.opencontrail.org/21403
Committed: http://github.org/Juniper/contrail-controller/commit/1cacb1a959b9e7d45401cb9bf2540bdb360d2dfa
Submitter: Zuul
Branch: R2.22.x

commit 1cacb1a959b9e7d45401cb9bf2540bdb360d2dfa
Author: Ignatious Johnson Christopher <email address hidden>
Date: Thu Jun 23 16:10:13 2016 +0000

Route targets of the connected logical routers
are not added to routing instances during reinit,
when schema transformer restarts. They are added later
during the Logical Router objects are evaluated.

Fix is to add them during reinit of routing instances.
Closed-Bug: 1588204

Change-Id: Id321a12e6c4a3aacb05f63b6ea0f6a2d5e9793fa

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.0.2.x

Review in progress for https://review.opencontrail.org/21682
Submitter: Ignatious Johnson Christopher (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/21682
Committed: http://github.org/Juniper/contrail-controller/commit/b2786fd4852e21665519d40347666038c931c3a5
Submitter: Zuul
Branch: R3.0.2.x

commit b2786fd4852e21665519d40347666038c931c3a5
Author: Ignatious Johnson Christopher <email address hidden>
Date: Wed Jun 22 07:50:38 2016 +0000

Route targets of the connected logical routers
are not added to routing instances during reinit,
when schema transformer restarts. They are added later
during the Logical Router objects are evaluated.

Fix is to update only the new route targets and remove
all stale route targets after reint.

Closes-Bug: 1588204

Also contrail python packages are cleared from the pip cache
before creation of test virtual enviroiment to make sure
the virtual enviroinment is installed with the newly built
python package.

Change-Id: Iaf7aa002020db7b7de01c32d44c3c74f2215e90f
(cherry picked from commit 0e019b2cc9e09edbe38e5b8b69c5f385ddc13c05)

Sachin Bansal (sbansal)
Changed in opencontrail:
status: New → Fix Committed
Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R3.2

Review in progress for https://review.opencontrail.org/43353
Submitter: Jean-Philippe Braun (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/43390
Submitter: Édouard Thuleau (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote :

Review in progress for https://review.opencontrail.org/43390
Submitter: Jean-Philippe Braun (<email address hidden>)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.