Juniper Openstack

Redis server service is not running after provisioning in R3.2

Bug #1711366 reported by musharani
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Won't Fix
High
Sundaresan Rajangam
R3.2
Won't Fix
High
Sundaresan Rajangam
Juniper Openstack r3.2.5.0

Bug Description

After provisioning R3.2-47 mitaka build redis-server service is not running. Because of this webui page is not opening.

root@nodec41:~# service redis-server status
redis-server stop/waiting

root@nodec41:~# contrail-status
== Contrail vRouter ==
supervisor-vrouter: active
contrail-vrouter-agent active (Collector connection down)
contrail-vrouter-nodemgr initializing (Collector connection down)

== Contrail Control ==
supervisor-control: active
contrail-control initializing (Collector connection down)
contrail-control-nodemgr initializing (Collector connection down)
contrail-dns active
contrail-named active

== Contrail Analytics ==
supervisor-analytics: active
contrail-alarm-gen initializing (Collector, Redis-UVE:AggregateRedis[None] connection down)
contrail-analytics-api initializing (Collector, UvePartitions:UVE-Aggregation[Partitions:0], Redis-UVE:LOCAL[None] connection down)
contrail-analytics-nodemgr initializing (Collector connection down)
contrail-collector initializing (Number of connections:6, Expected:7 Missing: Discovery:Collector)
contrail-query-engine initializing (Number of connections:3, Expected:4 Missing: Redis-Query:Query)
contrail-snmp-collector initializing (Collector connection down)
contrail-topology initializing (Collector connection down)

== Contrail Config ==
supervisor-config: active
contrail-api:0 initializing (Collector connection down)
contrail-config-nodemgr initializing (Collector connection down)
contrail-device-manager initializing (Collector connection down)
contrail-discovery:0 initializing (Collector connection down)
contrail-schema initializing (Collector connection down)
contrail-svc-monitor initializing (Collector connection down)
ifmap active

== Contrail Web UI ==
supervisor-webui: active
contrail-webui active
contrail-webui-middleware active

== Contrail Database ==
contrail-database: active

== Contrail Supervisor Database ==
supervisor-database: active
contrail-database-nodemgr active
kafka active

== Contrail Support Services ==
supervisor-support-service: active
rabbitmq-server active

redis-server log:
=================
[876] 17 Aug 18:54:03.863 # Creating Server TCP listening socket 10.204.217.81:6379: bind: Cannot assign requested address
[879] 17 Aug 18:54:03.866 # Unable to set the max number of files limit to 10032 (Operation not permitted), setting the max clients configuration to 3984.
[879] 17 Aug 18:54:03.866 # Creating Server TCP listening socket 10.204.217.81:6379: bind: Cannot assign requested address

Tags: analytics
Jeba Paulaiyan (jebap)
Changed in juniperopenstack:
milestone: r3.2.5.0 → none
Anish Mehta (amehta00)
Changed in juniperopenstack:
assignee: Biswajit Mandal (bmandal) → Naga Kiran (nagakiran)
Revision history for this message
Naga Kiran (nagakiran) wrote :
Download full text (3.3 KiB)

From: Sundaresan Rajangam
Sent: Wednesday, August 23, 2017 12:07 PM
To: Usha Rani Mani
Cc: Naga Kiran K Y S; Anish Mehta; Manoj Naik; Biswajit Mandal; Megh Bhatt
Subject: Re: Reg: https://bugs.launchpad.net/juniperopenstack/+bug/1711366

+Megh

From the fab log, I see that redis was started twice during provisioning time

setup_all_2017_08_22_17_04_57_009210.log:2017-08-22 17:11:44:340936: [root@10.204.217.81] sudo: service redis start
setup_all_2017_08_22_17_04_57_009210.log:2017-08-22 17:12:12:420994: [root@10.204.217.81] sudo: service redis start

The corresponding logs from redis.log:

[27217] 22 Aug 17:11:44.512 # Server started, Redis version 2.8.19
[28824] 22 Aug 17:12:12.483 # Server started, Redis version 2.8.19

However, the system was rebooted @ 17:17

[root@nodec41 fabric-utils]# last reboot
reboot system boot 3.10.0-229.el7.x Tue Aug 22 17:17 - 10:45 (17:27)
reboot system boot 3.10.0-229.el7.x Tue Aug 22 17:04 - 10:45 (17:40)
reboot system boot 3.10.0-229.el7.x Tue Aug 22 16:49 - 10:45 (17:55)

From the /var/log/messages, it seems that redis started before the network was up and hence failed to start.
redis service was not started later after the network was up.

Aug 22 17:17:39 nodec41 kernel: IPv6: ADDRCONF(NETDEV_UP): p1p0p0: link is not ready
Aug 22 17:17:40 nodec41 network: Bringing up interface p1p0p0: [ OK ]
Aug 22 17:17:40 nodec41 network: Bringing up interface vhost0: ERROR : [/etc/sysconfig/network-scripts/ifup-eth] Device vhost0 does not seem to be present, delaying initialization.
Aug 22 17:17:41 nodec41 /etc/sysconfig/network-scripts/ifup-eth: Device vhost0 does not seem to be present, delaying initialization. <<<<<<<
Aug 22 17:17:41 nodec41 network: [FAILED] <<<<<<<<
Aug 22 17:17:41 nodec41 systemd-sysctl: Overwriting earlier assignment of net/ipv4/ip_local_reserved_ports in file '/usr/lib/sysctl.d/openstack-keystone.conf'.
Aug 22 17:17:41 nodec41 systemd-sysctl: Overwriting earlier assignment of net/ipv4/ip_local_reserved_ports in file '/etc/sysctl.conf'.
Aug 22 17:17:41 nodec41 systemd: network.service: control process exited, code=exited status=1 <<<<<<<<<<<<
Aug 22 17:17:41 nodec41 systemd: Failed to start LSB: Bring up/down networking.
Aug 22 17:17:41 nodec41 systemd: Starting Redis persistent key-value database… <<<<<<<<<<<
Aug 22 17:17:41 nodec41 systemd: Started Redis persistent key-value database. <<<<<
Aug 22 17:17:43 nodec41 redis-shutdown: Could not connect to Redis at 10.204.217.81:6379: Network is unreachable <<<<<<<
Aug 22 17:17:43 nodec41 systemd: redis.service: control process exited, code=exited status=1
Aug 22 17:17:43 nodec41 systemd: Unit redis.service entered failed state.
...
Aug 22 17:18:06 nodec41 systemd-sysctl: Overwriting earlier assignment of net/ipv4/ip_local_reserved_ports in file '/etc/sysctl.conf'.
Aug 22 17:18:06 nodec41 NetworkManager[704]: <info> (vhost0): carrier is OFF
Aug 22 17:18:06 nodec41 NetworkManager[704]: <info> (vhost0): new Generic device (driver: 'vhost' ifindex: 7)
Aug 22 17:18:06 nodec41 NetworkManager[704]: <info> (vhost0): exported as /org/freedesktop/NetworkManager/Devices/6
Aug 22 17:18:06 nodec41 NetworkManager[704]: <info> (v...

Read more...

Changed in juniperopenstack:
assignee: Naga Kiran (nagakiran) → Sundaresan Rajangam (srajanga)
Revision history for this message
Sundaresan Rajangam (srajanga) wrote :

This is a corner case and not a typical use case, where compute and analytics run on the same server. Hence marking it as Won't Fix.

Changed in juniperopenstack:
status: New → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.