| 2016-06-28 09:00:35 |
Sandip Dey |
bug |
|
|
added bug |
| 2016-06-28 09:01:57 |
Sandip Dey |
description |
Build:R3.02 52 Kilo Ubuntu14.04
Logs saved at :http://10.204.216.50/Docs/bugs/<bug-id>
1.Had 3 database node - 'nodei27', 'nodei28' , 'nodei35'
In all the 3 nodes , contrail-database-nodemgr exited with the below traceback.
From the code in /usr/lib/python2.7/dist-packages/nodemgr/common/event_manager.py, looks like it tries to calculate the cpu/memory utilization of a list of analytics processes.
During this calculation , if one of them dies, contrail-database-nodemgr also dies.
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/gevent/greenlet.py", line 327, in run
result = self._run(*self.args, **self.kwargs)
File "/usr/lib/python2.7/dist-packages/nodemgr/database_nodemgr/database_event_manager.py", line 343, in runforever
prev_current_time = self.event_tick_60(prev_current_time)
File "/usr/lib/python2.7/dist-packages/nodemgr/common/event_manager.py", line 511, in event_tick_60
process_mem_cpu_usage = self.get_all_processes_mem_cpu_usage()
File "/usr/lib/python2.7/dist-packages/nodemgr/common/event_manager.py", line 396, in get_all_processes_mem_cpu_usage
process_mem_cpu = mem_cpu_usage_data.get_process_mem_cpu_info()
File "/usr/lib/python2.7/dist-packages/nodemgr/common/cpuinfo.py", line 86, in get_process_mem_cpu_info
process_mem_cpu.cpu_share = self._process.get_cpu_percent(interval=0.1)/psutil.NUM_CPUS
File "/usr/lib/python2.7/dist-packages/psutil/__init__.py", line 709, in get_cpu_percent
pt2 = self._platform_impl.get_cpu_times()
File "/usr/lib/python2.7/dist-packages/psutil/_pslinux.py", line 470, in wrapper
raise NoSuchProcess(self.pid, self._process_name)
NoSuchProcess: process no longer exists (pid=3876)
<Greenlet at 0x7ffbc3e39870: <bound method DatabaseEventManager.runforever of <nodemgr.database_nodemgr.database_event_manager.DatabaseEventManager object at 0x7ffbc40f0e50>>> failed with NoSuchProcess |
Build:R3.02 52 Kilo Ubuntu14.04
Logs saved at :http://10.204.216.50/Docs/bugs/<bug-id>
1.Had 3 database node - 'nodei27', 'nodei28' , 'nodei35'
In all the 3 nodes , contrail-database-nodemgr exited with the below traceback.
From the code in /usr/lib/python2.7/dist-packages/nodemgr/common/event_manager.py, looks like it tries to calculate the cpu/memory utilization of a list of analytics processes.
During this calculation , if one of them dies, contrail-database-nodemgr also dies.
In this setup,cassandra died due to
https://bugs.launchpad.net/juniperopenstack/+bug/1596803
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/gevent/greenlet.py", line 327, in run
result = self._run(*self.args, **self.kwargs)
File "/usr/lib/python2.7/dist-packages/nodemgr/database_nodemgr/database_event_manager.py", line 343, in runforever
prev_current_time = self.event_tick_60(prev_current_time)
File "/usr/lib/python2.7/dist-packages/nodemgr/common/event_manager.py", line 511, in event_tick_60
process_mem_cpu_usage = self.get_all_processes_mem_cpu_usage()
File "/usr/lib/python2.7/dist-packages/nodemgr/common/event_manager.py", line 396, in get_all_processes_mem_cpu_usage
process_mem_cpu = mem_cpu_usage_data.get_process_mem_cpu_info()
File "/usr/lib/python2.7/dist-packages/nodemgr/common/cpuinfo.py", line 86, in get_process_mem_cpu_info
process_mem_cpu.cpu_share = self._process.get_cpu_percent(interval=0.1)/psutil.NUM_CPUS
File "/usr/lib/python2.7/dist-packages/psutil/__init__.py", line 709, in get_cpu_percent
pt2 = self._platform_impl.get_cpu_times()
File "/usr/lib/python2.7/dist-packages/psutil/_pslinux.py", line 470, in wrapper
raise NoSuchProcess(self.pid, self._process_name)
NoSuchProcess: process no longer exists (pid=3876)
<Greenlet at 0x7ffbc3e39870: <bound method DatabaseEventManager.runforever of <nodemgr.database_nodemgr.database_event_manager.DatabaseEventManager object at 0x7ffbc40f0e50>>> failed with NoSuchProcess |
|
| 2016-06-28 15:46:51 |
Raj Reddy |
juniperopenstack: assignee |
Raj Reddy (rajreddy) |
Santosh Gupta (sangupta) |
|
| 2016-06-28 15:47:03 |
Raj Reddy |
juniperopenstack: importance |
Undecided |
Medium |
|
| 2016-07-01 20:36:31 |
OpenContrail Admin |
nominated for series |
|
juniperopenstack/trunk |
|
| 2016-07-01 20:36:31 |
OpenContrail Admin |
bug task added |
|
juniperopenstack/trunk |
|
| 2016-07-01 20:36:31 |
OpenContrail Admin |
bug task added |
|
juniperopenstack/trunk |
|
| 2016-07-02 02:29:33 |
OpenContrail Admin |
juniperopenstack/trunk: status |
In Progress |
Fix Committed |
|
| 2016-07-02 02:29:34 |
OpenContrail Admin |
juniperopenstack/trunk: milestone |
|
r3.1.0.0-fcs |
|
| 2016-08-11 18:22:34 |
Sundaresan Rajangam |
information type |
Proprietary |
Public |
|
| 2016-08-11 18:23:34 |
Sundaresan Rajangam |
nominated for series |
|
juniperopenstack/r3.0 |
|
| 2016-08-11 18:23:34 |
Sundaresan Rajangam |
bug task added |
|
juniperopenstack/r3.0 |
|
| 2016-08-11 18:23:42 |
Sundaresan Rajangam |
juniperopenstack/r3.0: importance |
Undecided |
Medium |
|
| 2016-08-11 18:23:45 |
Sundaresan Rajangam |
juniperopenstack/r3.0: importance |
Medium |
High |
|
| 2016-08-11 18:23:54 |
Sundaresan Rajangam |
juniperopenstack/r3.0: assignee |
|
Santosh Gupta (sangupta) |
|
| 2016-08-11 19:33:20 |
OpenContrail Admin |
juniperopenstack/r3.0: importance |
High |
Medium |
|
| 2016-08-11 19:33:20 |
OpenContrail Admin |
juniperopenstack/r3.0: status |
New |
In Progress |
|
| 2016-08-12 00:38:55 |
OpenContrail Admin |
juniperopenstack/r3.0: status |
In Progress |
Fix Committed |
|
| 2016-08-12 00:38:57 |
OpenContrail Admin |
juniperopenstack/r3.0: milestone |
|
r3.0.3.0 |
|
| 2016-10-31 20:48:27 |
OpenContrail Admin |
nominated for series |
|
juniperopenstack/r3.0.2.x |
|
| 2016-10-31 20:48:27 |
OpenContrail Admin |
bug task added |
|
juniperopenstack/r3.0.2.x |
|
| 2016-10-31 20:48:27 |
OpenContrail Admin |
bug task added |
|
juniperopenstack/r3.0.2.x |
|
| 2016-10-31 21:17:40 |
Jim Reilly |
bug |
|
|
added subscriber Jim Reilly |
| 2016-10-31 21:18:07 |
Jim Reilly |
tags |
analytics |
analytics att-aic-contrail |
|
| 2016-11-01 02:51:16 |
Raj Reddy |
nominated for series |
|
juniperopenstack/r3.0.3.x |
|
| 2016-11-01 02:51:16 |
Raj Reddy |
bug task added |
|
juniperopenstack/r3.0.3.x |
|
| 2016-11-01 02:51:16 |
Raj Reddy |
nominated for series |
|
juniperopenstack/r3.1 |
|
| 2016-11-01 02:51:16 |
Raj Reddy |
bug task added |
|
juniperopenstack/r3.1 |
|
| 2016-11-01 02:51:29 |
Raj Reddy |
juniperopenstack/r3.0.3.x: assignee |
|
Santosh Gupta (sangupta) |
|
| 2016-11-01 02:51:37 |
Raj Reddy |
juniperopenstack/r3.1: assignee |
|
Santosh Gupta (sangupta) |
|
| 2016-11-01 02:51:42 |
Raj Reddy |
juniperopenstack/r3.0.3.x: importance |
Undecided |
Medium |
|
| 2016-11-01 02:51:44 |
Raj Reddy |
juniperopenstack/r3.1: importance |
Undecided |
Medium |
|
| 2016-11-01 04:31:48 |
Santosh Gupta |
juniperopenstack/r3.1: status |
New |
Confirmed |
|
| 2016-11-01 04:32:28 |
Santosh Gupta |
juniperopenstack/r3.0.3.x: status |
New |
Confirmed |
|
| 2016-11-02 21:58:45 |
OpenContrail Admin |
juniperopenstack/r3.0.2.x: status |
In Progress |
Fix Committed |
|
| 2016-11-02 21:58:46 |
OpenContrail Admin |
juniperopenstack/r3.0.2.x: milestone |
|
r3.0.2.1 |
|
| 2016-12-06 17:39:05 |
Raj Reddy |
juniperopenstack/r3.0.3.x: status |
Confirmed |
Invalid |
|
| 2016-12-06 17:39:12 |
Raj Reddy |
juniperopenstack/r3.1: status |
Confirmed |
Invalid |
|
| 2017-01-06 18:13:15 |
Jim Reilly |
information type |
Public |
Private |
|
| 2017-01-11 18:19:38 |
Jim Reilly |
bug |
|
|
added subscriber att-aic-contrail |
| 2017-01-25 21:02:10 |
Umamaheshwar |
tags |
analytics att-aic-contrail |
analytics att-aic-contrail sscc-contrail |
|
| 2017-03-20 21:02:59 |
Sundaresan Rajangam |
information type |
Private |
Public |
|
| 2017-03-20 21:59:29 |
Jim Reilly |
information type |
Public |
Private |
|
| 2017-03-20 22:00:52 |
Jim Reilly |
information type |
Private |
Public |
|
