[toc]
指令
usage: cinder extend <volume> <new_size>
Attempts to extend size of an existing volume.
Positional arguments:
<volume> Name or ID of volume to extend.
<new_size> New size of volume, in GiBs.
功能說明
最新的openstack(3.42)代碼支持了熱擴(kuò)容功能：
Add ability to extend 'in-use' volume. User should be aware of the
whole environment before using this feature because it's dependent
on several external factors below:
1. nova-compute version - needs to be the latest for Pike.
2. only the libvirt compute driver supports this currently.
3. only iscsi and fibre channel volume types are supported
on the nova side currently.
Administrator can disable this ability by updating the
'volume:extend_attached_volume' policy rule. Extend in reserved
state is intentionally NOT allowed.
注意：
- 目前只有 libvirt 驅(qū)動能支持，且只有 iscsi 和 fibre channel 協(xié)議的卷類型能支持！
- cinder逗噩、nova做完熱擴(kuò)容后，fdisk -l查看可以看到磁盤大小變大楷兽，但是之前做的文件系統(tǒng)的大小是不會變化的，df -h 的結(jié)果跟擴(kuò)容前一樣华临。用戶還必須用resize2fs 指令自行resize文件系統(tǒng)芯杀！
《openstack 官網(wǎng)對于熱擴(kuò)容在 CINDER 方面的說明》
《openstack 官網(wǎng)對于熱擴(kuò)容在 NOVA 方面的說明》
Cinder方面代碼
Cinder 方面其實沒有什么改動，跟冷擴(kuò)容做的事一樣雅潭，擴(kuò)容后的size必須是整數(shù)GB且大于擴(kuò)容前size揭厚。核心實現(xiàn)是調(diào)用卷驅(qū)動里的方法，如netapp扶供，調(diào)用的是cinder.volume.drivers.netapp.dataontap.client.client_base.Client#do_direct_resize實現(xiàn)筛圆。唯一的區(qū)別是，擴(kuò)容后椿浓，會調(diào)用nova api太援。
- cinder.api.contrib.volume_actions.VolumeActionsController#_extend：
# 如果版本是3.42 且 卷狀態(tài)是in-use，則走熱擴(kuò)容
if req_version.matches("3.42") and volume.status in ['in-use']:
self.volume_api.extend_attached_volume(context, volume, size)
else: # 否則扳碍，冷擴(kuò)展
self.volume_api.extend(context, volume, size)
- cinder/volume/manager.py里調(diào)用nova api提岔，通知nova要擴(kuò)展卷：
if orig_volume_status == 'in-use':
nova_api = compute.API()
instance_uuids = [attachment.instance_uuid
for attachment in attachments]
nova_api.extend_volume(context, instance_uuids, volume.id)
這里調(diào)用nova client 里的：
novaclient.v2.server_external_events.ServerExternalEventManager#create:
class ServerExternalEventManager(base.Manager):
resource_class = Event
def create(self, events):
"""Create one or more server events.
:param:events: A list of dictionaries containing 'server_uuid', 'name',
'status', and 'tag' (which may be absent)
"""
body = {'events': events}
return self._create('/os-server-external-events', body, 'events',
return_raw=True)
'events'：
{'name': 'volume-extended',
'server_uuid': server_id,
'tag': volume_id}
Nova方面代碼
Nova將使用現(xiàn)有的外部事件API端點(diǎn)監(jiān)聽來自Cinder的附加卷擴(kuò)展通知。收到通知后笋敞，Nova將使用os-brick觸發(fā)主機(jī)上的設(shè)備重新掃描碱蒙，以發(fā)現(xiàn)卷大小的變化。
nova api '/os-server-external-events'：
- request：
{
"events": [
{
"name": "volume-extended",
"server_uuid": "3df201cf-2451-44f2-8d25-a4ca826fc1f3",
"tag": "0e63d806-6fe4-4ffc-99bf-f3dd056574c0"
}
]
}
- response：
{
"events": [
{
"name": "volume-extended",
"status": "completed",
"code": 200,
"server_uuid": "3df201cf-2451-44f2-8d25-a4ca826fc1f3",
"tag": "0e63d806-6fe4-4ffc-99bf-f3dd056574c0"
}
]
}
- 檢查nova驅(qū)動
compute_driver = libvirt.LibvirtDriver是否能支持卷擴(kuò)展"supports_extend_volume": True
驅(qū)動的支持功能定義在nova.virt.libvirt.driver.LibvirtDriver：
class LibvirtDriver(driver.ComputeDriver):
capabilities = {
"has_imagecache": True,
"supports_recreate": True,
"supports_migrate_to_same_host": False,
"supports_attach_interface": True,
"supports_device_tagging": True,
"supports_tagged_attach_interface": True,
"supports_tagged_attach_volume": True,
"supports_extend_volume": True,
}
- 根據(jù)卷connection_info找對應(yīng)的驅(qū)動，然后調(diào)用驅(qū)動的extend_volume方法赛惩。
def _extend_volume(self, connection_info, instance):
vol_driver = self._get_volume_driver(connection_info)
return vol_driver.extend_volume(connection_info, instance)
- 比如Iscsi哀墓，就會找nova.virt.libvirt.volume.iscsi.LibvirtISCSIVolumeDriver#extend_volume：
new_size = self.connector.extend_volume(connection_info['data'])
- 在調(diào)用ISCSIConnector的extend_volume
os_brick.initiator.connectors.iscsi.ISCSIConnector#extend_volume：
volume_paths = self.get_volume_paths(connection_properties)
if volume_paths:
return self._linuxscsi.extend_volume(volume_paths)
- 調(diào)用linuxscsi的extend_volume
os_brick.initiator.linuxscsi.LinuxSCSI#extend_volume:
def extend_volume(self, volume_paths):
"""Signal the SCSI subsystem to test for volume resize.
This function tries to signal the local system's kernel
that an already attached volume might have been resized.
"""
LOG.debug("extend volume %s", volume_paths)
for volume_path in volume_paths:
device = self.get_device_info(volume_path)
LOG.debug("Volume device info = %s", device)
device_id = ("%(host)s:%(channel)s:%(id)s:%(lun)s" %
{'host': device['host'],
'channel': device['channel'],
'id': device['id'],
'lun': device['lun']})
scsi_path = ("/sys/bus/scsi/drivers/sd/%(device_id)s" %
{'device_id': device_id})
# 如：scsi_path = u'/sys/bus/scsi/drivers/sd/3:0:0:3'
# 獲取設(shè)備大小
size = self.get_device_size(volume_path)
LOG.debug("Starting size: %s", size)
# now issue the device rescan
rescan_path = "%(scsi_path)s/rescan" % {'scsi_path': scsi_path}
# 如：rescan_path = u'/sys/bus/scsi/drivers/sd/3:0:0:3/rescan'
# 在rescan_path文件里寫入1。
# 對于SCSI設(shè)備喷兼，對 rescan 文件設(shè)置為1 可以使SCSI設(shè)備重新掃描篮绰。
self.echo_scsi_command(rescan_path, "1")
new_size = self.get_device_size(volume_path)
LOG.debug("volume size after scsi device rescan %s", new_size)
# 通過指令`/lib/udev/scsi_id --page 0x83 --whitelisted /dev/disk/by-path/ip-172.24.3.180:3260-iscsi-iqn.1992-08.com.netapp:sn.2d72abb030d511e7875800a098ac0ce9:vs.24-lun-3 `拿到wwnid
scsi_wwn = self.get_scsi_wwn(volume_paths[0])
# 如：scsi_wwn = u'3600a09803830387461244a62344f6b52'
mpath_device = self.find_multipath_device_path(scsi_wwn)
# 如果mpath_device非空，說明是多路徑磁盤褒搔。
if mpath_device:
# Force a reconfigure so that resize works
# 執(zhí)行 `multipathd reconfigure` 重新獲取 multipath.conf 配置中變化阶牍。
self.multipath_reconfigure()
# 獲取設(shè)備大小
size = self.get_device_size(mpath_device)
LOG.info("mpath(%(device)s) current size %(size)s",
{'device': mpath_device, 'size': size})
# 調(diào)用指令 multipathd resize map multipath_device 重置多路徑磁盤的大小
result = self.multipath_resize_map(scsi_wwn)
if 'fail' in result:
LOG.error("Multipathd failed to update the size mapping of "
"multipath device %(scsi_wwn)s volume %(volume)s",
{'scsi_wwn': scsi_wwn, 'volume': volume_paths})
return None
# 獲取設(shè)備大小
new_size = self.get_device_size(mpath_device)
LOG.info("mpath(%(device)s) new size %(size)s",
{'device': mpath_device, 'size': new_size})
return new_size
流程簡要：
向scsi掃描文件寫入 1：
tee -a 1 "%(scsi_path)s/rescan"喷面，對于SCSI設(shè)備星瘾，對 rescan 文件設(shè)置為1 可以使SCSI設(shè)備重新掃描。/lib/udev/scsi_id --page 0x83 --whitelisted得到scsi_wwn檢查多路徑設(shè)備惧辈，先檢查
/dev/disk/by-id/dm-uuid-mpath-%(wwn)s是否存在琳状，存在則返回路徑；如果不存在盒齿，則檢查/dev/mapper/%(wwn)s念逞，存在則返回路徑，不存在返回none边翁。如果第3部結(jié)果為none翎承，流程結(jié)束。
如果第3步查到路徑符匾，則需要執(zhí)行
multipathd reconfigure重新獲取 multipath.conf 配置中變化叨咖。執(zhí)行
multipathd resize map scsi_wwn重新設(shè)置設(shè)備大小。多路徑磁盤流程結(jié)束啊胶。
FC和Iscsi驅(qū)動都是這樣流程甸各。
附：
centos 如何使用多路徑磁盤
1.安裝multipath工具
# 安裝multipath工具
yum -y install device-mapper device-mapper-multipath
#加載multipath內(nèi)核模塊
modprobe dm-multipath
modprobe dm-round-robin
# 拷貝默認(rèn)配置文件
cat /usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf > /etc/multipath.conf
#啟動服務(wù)
service multipathd start 或者 /bin/systemctl start multipathd.service
2.配置nova_compute的配置文件
[libvirt]
# iscsi磁盤是否開啟多路徑,True為開啟
volume_use_multipath=True
# 建立連接時的rescan次數(shù)
num_iscsi_scan_tries=5
# 創(chuàng)建iscsi連接使用的iface名稱
iscsi_iface=default
修改在線多路徑設(shè)備容量的方法
redhat官網(wǎng)上介紹了修改在線多路徑設(shè)備容量的方法：
RESIZING AN ONLINE MULTIPATH DEVICE
If you need to resize an online multipath device, use the following procedure.
- Resize your physical device.
- Execute the following command to find the paths to the LUN:
# multipath -l
- Resize your paths. For SCSI devices, writing a 1 to the rescan file for the device causes the SCSI driver to rescan, as in the following command:
# echo 1 > /sys/block/path_device/device/rescan
Ensure that you run this command for each of the path devices. For example, if your path devices are sda, sdb, sde, and sdf, you would run the following commands:
# echo 1 > /sys/block/sda/device/rescan
# echo 1 > /sys/block/sdb/device/rescan
# echo 1 > /sys/block/sde/device/rescan
# echo 1 > /sys/block/sdf/device/rescan
- Resize your multipath device by executing the multipathd resize command:
# multipathd resize map multipath_device
- Resize the file system (assuming no LVM or DOS partitions are used):
# resize2fs /dev/mapper/mpatha
看的出，跟nova的處理流程差不多焰坪，主要少了resize2fs一步趣倾。
測試過程遇到問題
-
問題： 創(chuàng)建虛機(jī)時提示"Host 'localhost.localdomain' is not mapped to any cell"
解決： 執(zhí)行 nova-manage cell_v2 simple_cell_setup
-
問題： n-cpu日志提示libvirt 版本過低
解決： 按照以下步驟升級
$ rpm -qa|grep kvm
$ rpm -qa|grep qemu-system
$ sudo virsh -c qemu:///system version --daemon
$ sudo yum remove qemu-system-x86
$ sudo service libvirtd restart
$ sudo virsh -c qemu:///system version --daemon -
問題： c-vol 提示：ERROR oslo_messaging.rpc.server ConnectionFailedError: HTTPConnectionPool(host='172.24.2.218', port=2379):
Max retries exceeded with url: /v3alpha/lease/grant (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x718cc10>:
Failed to establish a new connection: [Errno 111] ECONNREFUSED',))
ERROR oslo_messaging.rpc.server解決： 配錯了coordination。在cinder.conf里吧[coordination]的backend_url注釋掉即可某饰。
