ntpd on undercloud is not configured

Bug #1777140 reported by Sergii Golovatiuk
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Medium
Alex Schultz

Bug Description

Description
===========
ntpd is not configured on undercloud

ntpd.conf on undercloud looks like
---%<---
[stack@undercloud ~]$ cat /etc/ntp.conf
# ntp.conf: Managed by puppet.
#
# Enable next tinker options:
# panic - keep ntpd from panicking in the event of a large clock skew
# when a VM guest is suspended and resumed;
# stepout - allow ntpd change offset faster
tinker panic 0

disable monitor

# Permit time synchronization with our time source, but do not
# permit the source to query or modify the service on this system.
restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery
restrict 127.0.0.1
restrict -6 ::1

# Set up servers for ntpd with next options:
# server - IP address or DNS name of upstream NTP server
# iburst - allow send sync packages faster if upstream unavailable
# prefer - select preferrable server
# minpoll - set minimal update frequency
# maxpoll - set maximal update frequency

# Driftfile.
driftfile /var/lib/ntp/drift

---%<---

ntpd is not working.

[root@undercloud ~]# ntpq
ntpq> peers
ntpq: read: Connection refused
ntpq>

Steps to reproduce
==================
Install Undercloud, check ntpd

Environment
===========
Just undercloud

CVE References

Changed in tripleo:
status: New → Triaged
importance: Undecided → Medium
milestone: none → rocky-3
Revision history for this message
Emilien Macchi (emilienm) wrote :

The NTP configuration on the containerized undercloud and non containerized undercloud is the same:

http://logs.openstack.org/07/571307/1/gate/tripleo-ci-centos-7-undercloud-oooq/adc122a/logs/undercloud/etc/ntp.conf.txt.gz
http://logs.openstack.org/12/575712/2/check/tripleo-ci-centos-7-undercloud-containers/031191f/logs/undercloud/etc/ntp.conf.txt.gz

So if there is a problem, it's on both containerized and non-containerized undercloud.

Revision history for this message
Emilien Macchi (emilienm) wrote :

I just deployed a fresh containerized undercloud and NTP works fine:

https://asciinema.org/a/k3AkDOTneOhz5uhr6qwksLJFH

Changed in tripleo:
status: Triaged → Incomplete
Revision history for this message
Emilien Macchi (emilienm) wrote :

Closing the bug as I couldn't reproduce. I'm happy to re-open if you provide a way to reproduce.

Changed in tripleo:
status: Incomplete → Invalid
Changed in tripleo:
status: Invalid → Confirmed
Revision history for this message
Sergii Golovatiuk (sgolovatiuk) wrote :

That's not correct. I've just installed fresh master by quickstart where I have

extra_args: >-
  --libvirt-type qemu
  --ntp-server pool.ntp.org

I looked at ntp.conf and it was created as shown in #1. However, I found that undercloud was not containerized. I am trying to reproduce the same behaviour on containerized undercloud.

Revision history for this message
Sergii Golovatiuk (sgolovatiuk) wrote :

Here is a /etc/ntp.conf. when we install containerized undercloud

---%<---
# For more information about this file, see the man pages
# ntp.conf(5), ntp_acc(5), ntp_auth(5), ntp_clock(5), ntp_misc(5), ntp_mon(5).

driftfile /var/lib/ntp/drift

# Permit time synchronization with our time source, but do not
# permit the source to query or modify the service on this system.
restrict default nomodify notrap nopeer noquery

# Permit all access over the loopback interface. This could
# be tightened as well, but to do so would effect some of
# the administrative functions.
restrict 127.0.0.1
restrict ::1

# Hosts on local network are less restricted.
#restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap

# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
server 0.centos.pool.ntp.org iburst
server 1.centos.pool.ntp.org iburst
server 2.centos.pool.ntp.org iburst
server 3.centos.pool.ntp.org iburst

#broadcast 192.168.1.255 autokey # broadcast server
#broadcastclient # broadcast client
#broadcast 224.0.1.1 autokey # multicast server
#multicastclient 224.0.1.1 # multicast client
#manycastserver 239.255.254.254 # manycast server
#manycastclient 239.255.254.254 autokey # manycast client

# Enable public key cryptography.
#crypto

includefile /etc/ntp/crypto/pw

# Key file containing the keys and key identifiers used when operating
# with symmetric key cryptography.
keys /etc/ntp/keys

# Specify the key identifiers which are trusted.
#trustedkey 4 8 42

# Specify the key identifier to use with the ntpdc utility.
#requestkey 8

# Specify the key identifier to use with the ntpq utility.
#controlkey 8

# Enable writing of statistics records.
#statistics clockstats cryptostats loopstats peerstats

# Disable the monitoring facility to prevent amplification attacks using ntpdc
# monlist command when default restrict does not include the noquery flag. See
# CVE-2013-5211 for more details.
# Note: Monitoring will not be disabled with the limited restriction flag.
disable monitor
---%<---

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to instack-undercloud (master)

Fix proposed to branch: master
Review: https://review.openstack.org/576181

Changed in tripleo:
assignee: nobody → Alex Schultz (alex-schultz)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to instack-undercloud (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/576450

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on instack-undercloud (master)

Change abandoned by wes hayutin (<email address hidden>) on branch: master
Review: https://review.openstack.org/576181
Reason: failed in the gate, clearing

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on instack-undercloud (stable/queens)

Change abandoned by Sergii Golovatiuk (<email address hidden>) on branch: stable/queens
Review: https://review.openstack.org/576450

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to instack-undercloud (master)

Reviewed: https://review.openstack.org/576181
Committed: https://git.openstack.org/cgit/openstack/instack-undercloud/commit/?id=d49442152a3318e5afcc86bb8af6e58a74b23878
Submitter: Zuul
Branch: master

commit d49442152a3318e5afcc86bb8af6e58a74b23878
Author: Alex Schultz <email address hidden>
Date: Mon Jun 18 09:40:50 2018 -0600

    Fall back to puppet-ntp defaults

    If the ntp server is not configured in the undercloud.conf, we should
    fall back to the defaults provided by puppet-ntp otherwise we end up
    with an invalid ntp configuration.

    Change-Id: I0000e1cf736b513dbc58c0d39f7e1c0137b660dd
    Closes-BUg: #1777140

Changed in tripleo:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to instack-undercloud (stable/queens)

Reviewed: https://review.openstack.org/576450
Committed: https://git.openstack.org/cgit/openstack/instack-undercloud/commit/?id=cc05b49e5e471f2a1c8647507ecfd71e12c24ad2
Submitter: Zuul
Branch: stable/queens

commit cc05b49e5e471f2a1c8647507ecfd71e12c24ad2
Author: Alex Schultz <email address hidden>
Date: Mon Jun 18 09:40:50 2018 -0600

    Fall back to puppet-ntp defaults

    If the ntp server is not configured in the undercloud.conf, we should
    fall back to the defaults provided by puppet-ntp otherwise we end up
    with an invalid ntp configuration.

    Change-Id: I0000e1cf736b513dbc58c0d39f7e1c0137b660dd
    Closes-BUg: #1777140
    (cherry picked from commit d49442152a3318e5afcc86bb8af6e58a74b23878)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to instack-undercloud (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/577254

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to instack-undercloud (stable/pike)

Reviewed: https://review.openstack.org/577254
Committed: https://git.openstack.org/cgit/openstack/instack-undercloud/commit/?id=cdd28f8c2ed60beb8780d97b3fb38a89836d27ba
Submitter: Zuul
Branch: stable/pike

commit cdd28f8c2ed60beb8780d97b3fb38a89836d27ba
Author: Alex Schultz <email address hidden>
Date: Mon Jun 18 09:40:50 2018 -0600

    Fall back to puppet-ntp defaults

    If the ntp server is not configured in the undercloud.conf, we should
    fall back to the defaults provided by puppet-ntp otherwise we end up
    with an invalid ntp configuration.

    Change-Id: I0000e1cf736b513dbc58c0d39f7e1c0137b660dd
    Closes-BUg: #1777140
    (cherry picked from commit d49442152a3318e5afcc86bb8af6e58a74b23878)

tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/instack-undercloud 8.4.3

This issue was fixed in the openstack/instack-undercloud 8.4.3 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/instack-undercloud 7.4.13

This issue was fixed in the openstack/instack-undercloud 7.4.13 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/instack-undercloud 9.2.0

This issue was fixed in the openstack/instack-undercloud 9.2.0 release.

Revision history for this message
David Hill (david-hill-ubisoft) wrote :

If we manually configured a ntpd server for the overcloud nodes on the undercloud, this revert the manual changes . I've spent some time trying to figureout what was happening in my undercloud because I use it as a ntp server.

tags: added: ux
Revision history for this message
Sudeep Batra (sudeep.batra) wrote :

I tried installing the Rocky release ,followed the below steps :
    1 export VIRTHOST=127.0.0.2
    2 git clone https://github.com/openstack/tripleo-quickstart.git
    3 cd tripleo-quickstart/
    4 bash quickstart.sh --install-deps
    5 ssh-keygen
    6 ssh-copy-id root@$VIRTHOST # otherwise installation fails #
    7 bash quickstart.sh -R rocky --no-clone --tags all --nodes config/nodes/1ctlr_1comp_1ceph.yml -I $VIRTHOST
Error still for NTP in the undercloud_install.log
TASK [Ensure system is NTP time synced] ****************************************
fatal: [undercloud]: FAILED! ntpdate[9153]: no server suitable for synchronization found",etc..

so the manual steps are still needed.

Revision history for this message
Alex Schultz (alex-schultz) wrote :

Sudeep, that means the ntp servers provided (they can be configured in undercloud.conf) are not reachable. You must have available NTP servers int order to install. By default it'll use pool.ntp.org (or related hosts). If NTP is blocked on your network, you need to provide an internal time source.

Revision history for this message
Sudeep Batra (sudeep.batra) wrote :

Thanks, so when I run ntpdate on undercloud it works.
[stack@undercloud ~]$ sudo ntpdate USMIIWEGAD00001.ericsson.se
 9 Jan 23:09:09 ntpdate[9560]: step time server 147.117.251.28 offset -92.814406 sec

But the install undercloud still fails.. log attached.

Revision history for this message
Sudeep Batra (sudeep.batra) wrote :

Thanks, so when I run ntpdate on undercloud it works.
[stack@undercloud ~]$ sudo ntpdate 147.117.251.28
 9 Jan 23:17:57 ntpdate[9584]: adjust time server 147.117.251.28 offset 0.030118 sec

But the install undercloud still fails.. log attached.

Revision history for this message
Alex Schultz (alex-schultz) wrote :

Yes, the undercloud_ntp_servers is still not set correctly. From the logs: ["ntpdate", "-u", "pool.ntp.org"]

Revision history for this message
Sudeep Batra (sudeep.batra) wrote :

Thanks Alex,
used the below steps to deploy TripleO using Rocky >
export RELEASE=rocky
export VIRTHOST=127.0.0.2
time bash quickstart.sh \
      --teardown all \
      --release $RELEASE \
      --nodes /home/user/tripleo-quickstart/nodes.yaml \
      --config /home/user/tripleo-quickstart/config.yaml \
      -e undercloud_undercloud_ntp_servers=147.117.251.28 \
      --clean \
      $VIRTHOST

But it does not install the overcloud as well.
How to install the overcloud also automated using quickstart.

Revision history for this message
Bogdan Dobrelya (bogdando) wrote :

AFAIK, quickstart can only install an undercloud for you.
The overcloud installation requires some manual preparations, and there is also a flavors matching issue https://bugs.launchpad.net/tripleo/+bug/1788875 ... Or you can try installing a standalone all in one cloud once you have your undercloud operational, see https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/standalone.html.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.