tripleo

memcached firewall rules are not being created since merging " Add non-tls listener to Memcached"

Bug #1918891 reported by Michele Baldessari on 2021-03-12

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	tripleo	Fix Committed	Critical	Grzegorz Grasza	tripleo wallaby-3

Bug Description

So I have also been observing keystone timeouts on RDO on some of my patches. The timeouts are because some of the memcached patches that have been merged lately are broken in regards to opening up the firewall rules.

Let's take this one for example:
https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master/6b98df3/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz

We see that nova is configured to use memcached:
https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master/6b98df3/logs/overcloud-controller-0/var/lib/config-data/puppet-generated/nova/etc/nova/nova.conf.txt.gz

memcache_servers=172.17.0.139:11211,172.17.0.15:11211,172.17.0.220:11211

The problem is there are no iptables rules for memcached:
https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master/6b98df3/logs/overcloud-controller-0/var/log/extra/network.txt.gz

In there you will see no rules opening up 11211. I think the logic I7a38a01f498d350d065a7c312a6654832fe24e6a is broken in regards to firewall rules, hence the timeouts. Damien and I have observed this locally as well while deploying master.

Tags:

Michele Baldessari (michele) on 2021-03-12

tags:

added: alert ci

Bogdan Dobrelya (bogdando) on 2021-03-12

Changed in tripleo:
importance:	High → Critical

Bhagyashri Shewale (bhagyashri-shewale) on 2021-03-12

tags:

added: promotion-blocker

Revision history for this message

yatin (yatinkarel) wrote on 2021-03-12:

Should be fixed with https://review.opendev.org/c/openstack/tripleo-heat-templates/+/780268

Changed in tripleo:
status:	Triaged → In Progress
assignee:	nobody → Grzegorz Grasza (xek)

Grzegorz Grasza (xek) on 2021-03-15

Changed in tripleo:
status:	In Progress → Fix Committed

wes hayutin (weshayutin) on 2021-03-16

tags:

removed: ci promotion-blocker

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2021-05-05: Fix included in openstack/tripleo-heat-templates 14.1.0

This issue was fixed in the openstack/tripleo-heat-templates 14.1.0 release.

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.