Nova AZ與Cinder AZ同步的問題

1.背景

在使用OpenStack雲平臺時，有時因業務架構須要拆分不一樣的Availability Zone可用域，包括計算Nova和存儲Cinder，並指望能夠將兩個AZ對應起來以便起到隔離的效果。目前多數OpenStack平臺或多或少採用了Ceph做爲Cinder存儲後端（以及Glance、Swift），並在建立虛擬機時選擇從鏡像建立塊存儲。咱們但願能夠經過建立不一樣的Cinder AZ配置不一樣的Ceph pool，每一個pool關聯不一樣的osd，已便達到不一樣用戶建立的虛擬機操做系統磁盤在Ceph集羣中是徹底隔離的。

2.問題

首先咱們啓動多個cinder-volume實例，配置不一樣的storage_availalibility_zone，指望效果是選擇虛擬機的AZ時能夠匹配Cinder裏的AZ，例如：

storage_availability_zone=AZ1

然而實際狀況並不理想，在建立虛擬機時，選擇了Nova的AZ（好比AZ1），建立出來的卷卻在Cinder的nova可用域裏，由於Cinder的默認域是nova，最終結果並無匹配上。實際上，nova在調用cinder的時候並未把虛擬機實例的availalibility_zone的值傳過去。

3.解決

查看源碼/usr/lib/python2.7/site-packages/nova/conf/cinder.py中的設定，發現一個關鍵參數cross_az_attach，默認值爲True，這意味着虛擬機的磁盤能夠跨域綁定。

cfg.BoolOpt('cross_az_attach',
            default=True,
            help="""
Allow attach between instance and volume in different availability zones.

If False, volumes attached to an instance must be in the same availability
zone in Cinder as the instance availability zone in Nova.
This also means care should be taken when booting an instance from a volume
where source is not "volume" because Nova will attempt to create a volume using
the same availability zone as what is assigned to the instance.
If that AZ is not in Cinder (or allow_availability_zone_fallback=False in
cinder.conf), the volume create request will fail and the instance will fail
the build request.
By default there is no availability zone restriction on volume attach.
"""),
]

順藤摸瓜，繼續查看/usr/lib/python2.7/site-packages/nova/virt/block_device.py，發現若是cross_az_attach爲True，則傳給Cinder的availability_zone爲空！！！若是cross_az_attach爲False，那麼nova會給Cinder傳遞實例的availability_zone。

def _get_volume_create_az_value(instance):
    """Determine az to use when creating a volume

    Uses the cinder.cross_az_attach config option to determine the availability
    zone value to use when creating a volume.

    :param nova.objects.Instance instance: The instance for which the volume
        will be created and attached.
    :returns: The availability_zone value to pass to volume_api.create
    """
    # If we're allowed to attach a volume in any AZ to an instance in any AZ,
    # then we don't care what AZ the volume is in so don't specify anything.
    if CONF.cinder.cross_az_attach:
        return None
    # Else the volume has to be in the same AZ as the instance otherwise we
    # fail. If the AZ is not in Cinder the volume create will fail. But on the
    # other hand if the volume AZ and instance AZ don't match and
    # cross_az_attach is False, then volume_api.check_attach will fail too, so
    # we can't really win. :)
    # TODO(mriedem): It would be better from a UX perspective if we could do
    # some validation in the API layer such that if we know we're going to
    # specify the AZ when creating the volume and that AZ is not in Cinder, we
    # could fail the boot from volume request early with a 400 rather than
    # fail to build the instance on the compute node which results in a
    # NoValidHost error.
    return instance.availability_zone

一切都清楚了，接下來配置nova.conf文件，在Cinder部分添加參數

cross_az_attach = False

再次建立虛擬機，發現Nova的AZ和Cinder的AZ對應上啦，解決！

4.More..

在Cinder中還有個參數是allow_availability_zone_fallback，目的是爲了防止建立虛擬機時的Nova AZ在Cinder中不存在時不報錯，而是使用Cinder中default_availability_zone或者storage_availability_zone進行建立。代碼在/usr/lib/python2.7/site-packages/cinder/volume/flows/api/create_volume.py中

if availability_zone not in self.availability_zones:
            if CONF.allow_availability_zone_fallback:
                original_az = availability_zone
                availability_zone = (
                    CONF.default_availability_zone or
                    CONF.storage_availability_zone)
                LOG.warning(_LW("Availability zone '%(s_az)s' "
                                "not found, falling back to "
                                "'%(s_fallback_az)s'."),
                            {'s_az': original_az,
                             's_fallback_az': availability_zone})
            else:
                msg = _("Availability zone '%(s_az)s' is invalid.")
                msg = msg % {'s_az': availability_zone}
                raise exception.InvalidInput(reason=msg)

可是目測這個參數和cross_az_attach有衝突，由於使用cross_az_attach的狀況下選擇一個Cinder裏沒有的可是Nova中含有的AZ，建立結果仍是失敗的，最終會以cross_az_attach做爲判斷。

5.參考文獻

【1】https://www.mirantis.com/blog...
【2】https://wenku.baidu.com/view/...