NSX plugin request retry failure with "[Errno 104] Connection reset by peer"
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
neutron |
Fix Released
|
Medium
|
Han Zhou |
Bug Description
Http connections in connection pool to NSX controller can be reset due to reasons such as when LB is in the middle, and connection idle timeout.
In such case, while recreating the connection, NSX plugin would retry the request with "next" connection, which could also be reset already because of idle timeout. This would leads to continuous retry failure and the request fails finally, with a misleading error log: "Request timeout ...".
Error log example:
2014-10-12 11:07:18,163 43793136 DEBUG [neutron.
2014-10-12 11:07:18,163 43793136 DEBUG [neutron.
2014-10-12 11:07:18,163 43793136 DEBUG [neutron.
2014-10-12 11:07:18,164 43793136 WARNING [neutron.
2014-10-12 11:07:18,164 43793136 WARNING [neutron.
2014-10-12 11:07:18,164 43793136 WARNING [neutron.
2014-10-12 11:07:18,164 43793136 DEBUG [neutron.
2014-10-12 11:07:18,165 43793136 INFO [neutron.
2014-10-12 11:07:18,165 43793136 DEBUG [neutron.
2014-10-12 11:07:18,165 43793136 DEBUG [neutron.
2014-10-12 11:07:18,165 43793136 DEBUG [neutron.
2014-10-12 11:07:18,166 43793136 WARNING [neutron.
2014-10-12 11:07:18,166 43793136 WARNING [neutron.
2014-10-12 11:07:18,166 43793136 WARNING [neutron.
2014-10-12 11:07:18,166 43793136 DEBUG [neutron.
2014-10-12 11:07:18,166 43793136 INFO [neutron.
2014-10-12 11:07:18,167 43793136 DEBUG [neutron.
2014-10-12 11:07:18,167 43793136 DEBUG [neutron.
2014-10-12 11:07:18,167 43793136 DEBUG [neutron.
2014-10-12 11:07:18,167 43793136 WARNING [neutron.
2014-10-12 11:07:18,167 43793136 WARNING [neutron.
2014-10-12 11:07:18,168 43793136 WARNING [neutron.
2014-10-12 11:07:18,168 43793136 DEBUG [neutron.
2014-10-12 11:07:18,168 43793136 INFO [neutron.
2014-10-12 11:07:18,168 133917104 ERROR [NVPApiHelper] Request timed out: GET to /ws.v1/
2014-10-12 11:07:18,169 133917104 ERROR [NeutronPlugin] An exception occured while selecting logical switch for the port
...
Suggestion to the fix issue by using the newly created connection when retry instead of using the "next" connection.
summary: |
- NSX plugin request retry should delay in case of "[Errno 104] Connection - reset by peer" + NSX plugin request retry failure with "[Errno 104] Connection reset by + peer" |
description: | updated |
Changed in neutron: | |
importance: | Undecided → Medium |
milestone: | none → kilo-1 |
Changed in neutron: | |
status: | Fix Committed → Fix Released |
Changed in neutron: | |
milestone: | kilo-1 → 2015.1.0 |
Fix proposed to branch: master /review. openstack. org/128880
Review: https:/