Juniper Openstack

R1.10-build-2271: collector not responding

Bug #1348563 reported by Saravanan Musuvathi on 2014-07-25

This bug affects 2 people

	Status	Importance	Assigned to	Milestone
Juniper Openstack	Status tracked in Trunk
R1.1	Fix Released	Critical	Sundaresan Rajangam	Juniper Openstack r1.10-fcs
Trunk	Fix Released	Critical	Sundaresan Rajangam

Bug Description

Have collected gcore of vizd process for further analysis...
Core available in bhushana@mayamruga-E71:~$ ~/Documents/technical/bugs/

Tags:

Saravanan Musuvathi (smusuvathi) on 2014-07-25

Changed in juniperopenstack:
importance:	Undecided → Critical

Saravanan Musuvathi (smusuvathi) on 2014-07-25

tags:

added: vizis

Raj Reddy (rajreddy) on 2014-07-29

Changed in juniperopenstack:
status:	New → Incomplete
assignee:	nobody → Saravanan Musuvathi (smusuvathi)

Raj Reddy (rajreddy) on 2014-08-12

Changed in juniperopenstack:
assignee:	Saravanan Musuvathi (smusuvathi) → Sundaresan Rajangam (srajanga)
status:	Incomplete → In Progress

Raj Reddy (rajreddy) on 2014-08-18

tags:

added: blocker

Revision history for this message

Sundaresan Rajangam (srajanga) wrote on 2014-08-20:

fixed in mainline with commit:
https://github.com/Juniper/contrail-controller/commit/cc9327482367c12dd4122bcc6aebee8f67561f08

Revision history for this message

Sundaresan Rajangam (srajanga) wrote on 2014-08-21:

fixed in R1.10 with commit:
https://github.com/Juniper/contrail-controller/commit/1c3385761b4697101be62d03c36b2cfdcdae6216

Revision history for this message

Sundaresan Rajangam (srajanga) wrote on 2014-10-01:

SandeshStateMachine::WorkQueue task acquires cdbq_mutex_ [CdbIf::Db_AddColumn()] and then WorkQueue::mutex_[WorkQueue<CdbIf::CdbIfColList>::MayBeStartRunner()] when sandesh is received from the generator. At the same time, CdbIf::WorkQueue task acquires WorkQueue::mutex_ [WorkQueue<CdbIf::CdbIfColList>::RunnerDone()] and then cdbq_mutex_ [CdbIf::Db_Uninit()] when connection to the Analytics database is disconnected and TTransportException is raised in CdbIf::Db_BatchAddColumn(). This results in deadlock as the locks are acquired in the reverse direction in different tasks that run in parallel.