OpenContrail

contrail-api eating 100% CPU and not responding to requests

Bug #1416982 reported by Martin Gerhard Loschwitz
14
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OpenContrail
Fix Committed
Undecided
Unassigned

Bug Description

The problem I have described in http://lists.opencontrail.org/pipermail/dev_lists.opencontrail.org/2015-January/001882.html happens several times a day. This renders opencontrail unusable, as neutron requests time out regularly. What would be a good way to debug this?

TLDR: Contrail will eat 100% cpu time after a while and not respond in time until it gets restarted, which resets the cycle.

Tags: config
Nagabhushana R (bhushana)
tags: added: config
Revision history for this message
Prakash Bailkeri (prakashmb) wrote :

Looked at strace of contrail-api from Martin. Saw GET on virtual-machine-interfaces quite often

Suggested two changes:
1. patch fix from https://review.opencontrail.org/#/c/6786/. Should optimize the list operation
2. heal_instance_info_cache_interval on nova compute to 0(or to higher value than default).

Revision history for this message
Prakash Bailkeri (prakashmb) wrote :

Several config fixes for optimization are committed.

https://review.opencontrail.org/#/c/6605/
https://review.opencontrail.org/#/c/7703/
https://review.opencontrail.org/#/c/7062/
https://review.opencontrail.org/#/c/7192/
https://github.com/Juniper/contrail-controller/commit/78aa4a45822db32e201ddbdff846194fbb9cca41
https://github.com/Juniper/contrail-generateDS/commit/28137d4a15cc921b3300bfb150617bff5cc7c0be
https://github.com/Juniper/contrail-generateDS/commit/a96a28280d1010f472a21b5f90e7aa7933b5d74f
https://github.com/Juniper/contrail-generateDS/commit/48550eff29ec91d3645ee678276dc712c340e7a3

Changed in opencontrail:
status: New → Fix Committed
Roman Rufanov (rrufanov)
tags: added: customer-found support
Revision history for this message
Andrew Woodward (xarses) wrote :

What version is this fix in?

tags: removed: customer-found support
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.