"stick on src" haproxy option is a cause of slow keystone
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Fuel for OpenStack |
Fix Committed
|
Critical
|
Alex Schultz | ||
Mitaka |
Fix Released
|
Critical
|
Alex Schultz |
Bug Description
Detailed bug description:
stick on src is a cause of slow performance of kystone. We performed rally scenario which boots an instance with secgroup with 10 rules and a cinder volume, then migrate the instance. (http://
When the option persists in /etc/haproxy/
openstack user list - real 0m27.563s
nova list --all-tenant - real 0m47.577s
root@node-179:~# for i in 77 96 179; do grep keystone-2 /var/log/
1124
313
2283
When I commented out two lines in the files (during rally test):
#stick on src
#stick-table type ip size 200k expire 2m
openstack user list - real 0m1.704s
nova list --all-tenant - real 0m6.011s
Round Robin works well
Steps to reproduce:
1. Fuel 9.0-308 has been deployed
2. curl -s 'https:/
rpm -Uvh epel-release-
sed -i s/^enabled=
yum --enablerepo=epel install uwsgi uwsgi-plugin-python python-
service receiverd restart && service nailgun restart
due https:/
fuel-agent and fuel-library have been updated: http://
3. patch has been applied to keep rotated logs http://
4. patch has been applied to increase rsyslog chunks to 4к http://
5. LMA, ElasticSearch, Grafana plugins have been installed:
yum -y install createrepo rpm rpm-build dpkg-devel git
easy_install pip
pip install fuel-plugin-builder
git clone https:/
fpb --check ./fuel-
fpb --build ./fuel-
fuel plugins --install ./fuel-
git clone https:/
fpb --check ./fuel-
fpb --build ./fuel-
fuel plugins --install ./fuel-
git clone https:/
fpb --check ./fuel-
fpb --build ./fuel-
fuel plugins --install ./fuel-
patch -b -d /var/www/
fuel plugins --sync
6. deploy cluster 3 controllers, 20 computes+Ceph, 172 computes, vxlan+DVR, Ceph for all
7. Perform rally tests and execute openstack user list command during the testing
Expected results:
The command shows results in 5 seconds
Actual result:
The command shows results in ~50 seconds (timeout 60)
Reproducibility:
Each time
Workaround:
Comment out lines if /etc/haproxy/
#stick on src
#stick-table type ip size 200k expire 2m
Impact:
OpenStack cluster performance
Description of the environment:
Operation system: ubuntu
Versions of components: MOS 9.0
Reference architecture: 3 controllers, 20 computes+Ceph, 172 computes, vxlan+DVR, Ceph for all
Network model: vxlan+DVR
Related projects installed: LMA
Additional information:
-
description: | updated |
tags: | added: area-library |
no longer affects: | fuel/newton |
Changed in fuel: | |
importance: | High → Critical |
Fix proposed to branch: master /review. openstack. org/316947
Review: https:/