Cinder

Cannot change volume type using ceph with alternative (non-default) cluster name

Bug #1563889 reported by Eric
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Cinder
New
Medium
Unassigned

Bug Description

Steps to reproduce:

1. Configure a ceph cluster using an alternative cluster name (e.g. 'my-ceph') instead of the default (ceph)
2. Configure two ceph backends using this cluster to use in cinder
3. Add the appropriate volume type(s)
4. Create a volume
5. Change the volume type of the volume

Expected result:

Migration of the volume

Actual result:

Exception in cinder-volume: Error: error calling conf_read_file: errno EINVAL

Cause:

The os-brick RBDConnector initiates a RBDClient only specifying user and pool. This causes the client to default to configuration file "/etc/ceph/ceph.conf"

The cinder rbd volume driver specifies all required information for a connection (while to me it seems to be tailored to nova/libvirt) but it is not used by os-brick. If the data specified by the volume driver cannot be used by the os-brick RBDClient, either the ceph cluster name or full path to the appropriate ceph configfile should be added to the connection data which in turn should be used by the os-brick connector.

PS.

I wouldn't mind writing a patch, but since I'm relatively new to cinder I'm not aware of the bigger picture. Because of that I would need to know what the desired fix should be. Either adapting os-brick to use the already available connection info or appending either the cluster name or config full path to the connection info.

See original description
Tags: drivers rdb
Sean McGinnis (sean-mcginnis)
tags: added: drivers rdb
Eric (ericvanblokland)
description: updated
Danny Al-Gaaf (danny-al-gaaf)
Changed in cinder:
status: New → Confirmed
assignee: nobody → Danny Al-Gaaf (danny-al-gaaf)
status: Confirmed → In Progress
Revision history for this message
Danny Al-Gaaf (danny-al-gaaf) wrote :

If I have not overseen something cinder doesn't use os_brick connector for rbd but uses it own implementation. Can you please provide your cinder configuration and commands to reproduce?

Revision history for this message
Eric (ericvanblokland) wrote :

I'm using Horizon to issue the commands required for migration.

I've attached the stacktrace (sorry for not including it initially) which clearly shows the volume manager using os-brick. Are you sure this could be caused by configuration?

If so let me know and I'll post the configuration, as I would need to remove some potentially sensitive information.

Revision history for this message
Eric (ericvanblokland) wrote :

I've just noticed the cinder-backup process has the same issue.

Sean McGinnis (sean-mcginnis)
Changed in cinder:
importance: Undecided → Medium
Revision history for this message
jcat (jcat) wrote :

We have the same issue for cinder-backup on mitaka.

We have two different ceph clusters, one for cinder-volume, and one for cinder-backup, each with a different ${cluster_name}.conf.

We get a different error than "conf_read_file", but only because we still have a ceph.conf in place with some default client settings.

Trace attached.

Revision history for this message
Sean McGinnis (sean-mcginnis) wrote : Bug Assignee Expired

Unassigning due to no activity for > 6 months.

Changed in cinder:
assignee: Danny Al-Gaaf (danny-al-gaaf) → nobody
Sean McGinnis (sean-mcginnis)
Changed in cinder:
status: In Progress → New
Revision history for this message
Xiaojun Liao (wwba) wrote :

Hi, Eric:

I have followed the steps in my newton envirenment, but didn't reproduce it. I configure two ceph backends like this:
[rbd1]
volume_backend_name=rbd1
volume_driver=cinder.volume.drivers.rbd.RBDDriver
rbd_pool=rbd1
rbd_max_clone_depth=5
rbd_user=cinder
rbd_flatten_volume_from_snapshot=False
rbd_ceph_conf=/etc/ceph/ceph.conf

[rbd2]
volume_backend_name=rbd2
volume_driver=cinder.volume.drivers.rbd.RBDDriver
rbd_pool=rbd2
rbd_max_clone_depth=5
rbd_user=cinder
rbd_flatten_volume_from_snapshot=False
rbd_cluster_name = mycluster
rbd_ceph_conf=/etc/ceph/mycluster.conf

did you configure the 'rbd_cluster_name' option for you non-default cluster?

Revision history for this message
Xiaojun Liao (wwba) wrote :

For two ceph backends, it seems retype works well in newton.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.