Storage support for vSphere provider

Storage customization support would be beneficial for the on premises deployments as it would allow to utilize local storage of ESXi hosts via software SAN solution like Ceph.

At the moment Ceph silently installs itself onto loopback devices and this causes two issues:
1. performance is not optimal
2. loopback OSD devices can fill up / filesystem and crash the node.

BTW Issue No.2 I think also applicable to container runtime nodes. For the long running or runaway containers their logs can grow large in size and fill up the disk. If container runtime location is not separated from / this results in node crash (sometimes even root login does not work).

subscribe ~field-high

Workaround:
Add machines manually with juju add-machine then attach new disks with the govc tool using this script:

#!/bin/bash

export GOVC_USERNAME=***
export GOVC_PASSWORD="***"
export GOVC_TLS_CA_CERTS=***
export GOVC_URL=***

vm=$1
size="1TB"

# Get the VM Path from the DNS Name
vm_path=$(./govc vm.info -vm.dns=$vm | grep Path | awk '{print $2}')
echo $vm_path
# Get the VM JSON Parameters
vm_json=$(./govc vm.info -vm.dns=$vm -json=true)

# Get the number of attached disks
disk_num=$(echo $vm_json | jq .VirtualMachines[0].Layout.Disk[].DiskFile[0] | awk '{print $2}' | wc -l)

# Get the last disk number
next_disk_num=$((disk_num - 1))

# Get the base disk path/name
base_disk=$(echo $vm_json | jq -r .VirtualMachines[0].Layout.Disk[].DiskFile[0] | awk '{print $2}' | head -1 | sed 's/.vmdk//')

# Get the next disk name
next_disk_name=$base_disk\_$next_disk_num
echo $next_disk_name

#### DS
# Get the Datastore
ds=$(echo $vm_json | jq -r .VirtualMachines[0].Layout.Disk[].DiskFile[0] | head -1 | awk '{print $1}' | tr -d '[]')
echo $ds

# Get the Datastore Path
ds_path=$(./govc ls -t datastore '*/*' | grep $ds)
echo $ds_path

# Add disk
./govc vm.disk.create -vm.dns=${vm} -name={$next_disk_name} -ds=${ds_path} -size ${size}