OpenStack 實現技術分解 (5) 應用開發 — 使用 OpenStackClients 進行二次開發

時間 2019-11-18

標籤 openstack 實現技術分解應用開發使用 openstackclients 進行二次開發简体版

原文原文鏈接

前文列表

OpenStack 實現技術分解 (1) 開發環境 — Devstack 部署案例詳解
 OpenStack 實現技術分解 (2) 虛擬機初始化工具 — Cloud-Init & metadata & userdata
OpenStack 實現技術分解 (3) 開發工具 — VIM & dotfiles
OpenStack 實現技術分解 (4) 通用技術 — TaskFlowhtml

參考閱讀

Openstack API 類型 & REST 風格
 OpenStackClients
Python bindings to the OpenStack Identity API
Python Bindings for the OpenStack Images API
Python bindings to the OpenStack Nova API
Cinder Python API
Python bindings to the OpenStack Networking APIpython

前言

OpenStack 爲用戶提供了三種操做方式, Web界面/CLI/RESTAPI, 實際上前二者是對 RESTAPI 作了兩種不一樣形式的包裝, 使用戶能夠經過網頁或者指令行的方式來調用 RESTAPI 接口. mysql

本篇博文主要記錄了使用 OpenStackClients (OSC 命令行客戶端) 項目所提供了Python Bindings API 來進行二次開發的技巧, 以及實現一個啓動虛擬機並部署 Workpass+MySQL 自動化腳本的 Demo. 源碼詳見 GitHub: openstackclient-api-demogit

在介紹 OpenStackClients 以前, 咱們能夠嘗試直接使用 curl 指令來查看一個 tenant 所含有的虛擬機列表.github

Step 1: 獲取帳戶 admin 的身份驗證 Temporary token

curl -k -X 'POST' -v http://200.21.18.2:5000/v2.0/tokens -d '{"auth":{"passwordCredentials":{"username": "admin", "password":"fanguiju"}}}' -H 'Content-type: application/json' | python -mjson.tool

Response:web

{
    "access": { "metadata": { "is_admin": 0, "roles": [] }, "serviceCatalog": [], "token": { "audit_ids": [ "AOMhHXq_Qx2Nz41RVoUy7g" ], "expires": "2017-03-19T05:41:20Z", "id": "16ae22b6c36f4ebc97938f51b7d0631b", "issued_at": "2017-03-19T04:41:20.039145" }, "user": { "id": "135b2cb86962401c82044fd4ca9daae4", "name": "admin", "roles": [], "roles_links": [], "username": "admin" } } }

獲取到 Temporary token: 16ae22b6c36f4ebc97938f51b7d0631b, 表示咱們的帳戶信息經過了驗證流程. sql

Step 2: 使用 Temporary token 來獲取 tenants list

curl -X 'GET' -H  "X-Auth-Token:16ae22b6c36f4ebc97938f51b7d0631b" -v http://200.21.18.2:5000/v2.0/tenants | python -mjson.tool

Response:json

{
    "tenants": [ { "description": "", "enabled": true, "id": "6c4e4d58cb9d4451b36e774b348e8813", "name": "admin" }, { "description": "", "enabled": true, "id": "ad9a69f3da8f4aa280389fcdf855aeb5", "name": "demo" } ],
    "tenants_links": [] }

能夠看出 admin 帳戶含有 admin tenant 和 demo tenant. ubuntu

Step 3: 獲取 admin tenant 的 token 信息

curl -k -X 'POST' -v http://200.21.18.2:5000/v2.0/tokens -d '{"auth":{"passwordCredentials":{"username": "admin", "password":"fanguiju"},"tenantId":"6c4e4d58cb9d4451b36e774b348e8813"}}' -H 'Content-type: application/json' | python -mjson.tool

Response:vim

{
    "access": { "metadata": { "is_admin": 0, "roles": [ "14a6da35e3ef4e47a540c6608aa00ca7" ] }, "serviceCatalog": [ { "endpoints": [ { "adminURL": "http://200.21.18.2:8774/v2.1/6c4e4d58cb9d4451b36e774b348e8813", "id": "705f599f3bae42ceb4a70616d9663ad8", "internalURL": "http://200.21.18.2:8774/v2.1/6c4e4d58cb9d4451b36e774b348e8813", "publicURL": "http://200.21.18.2:8774/v2.1/6c4e4d58cb9d4451b36e774b348e8813", "region": "RegionOne" } ], "endpoints_links": [], "name": "nova", "type": "compute" }, { "endpoints": [ { "adminURL": "http://200.21.18.2:8760/v1.1/6c4e4d58cb9d4451b36e774b348e8813", "id": "39ceecd18b754c9495834d0155fe91bf", "internalURL": "http://200.21.18.2:8760/v1.1/6c4e4d58cb9d4451b36e774b348e8813", "publicURL": "http://200.21.18.2:8760/v1.1/6c4e4d58cb9d4451b36e774b348e8813", "region": "RegionOne" } ], "endpoints_links": [], "name": "egis", "type": "recovery" }, { "endpoints": [ { "adminURL": "http://200.21.18.2:8776/v2/6c4e4d58cb9d4451b36e774b348e8813", "id": "218769a91d0943ff8db44887645ec0ff", "internalURL": "http://200.21.18.2:8776/v2/6c4e4d58cb9d4451b36e774b348e8813", "publicURL": "http://200.21.18.2:8776/v2/6c4e4d58cb9d4451b36e774b348e8813", "region": "RegionOne" } ], "endpoints_links": [], "name": "cinderv2", "type": "volumev2" }, { "endpoints": [ { "adminURL": "http://200.21.18.2:9292", "id": "7f2f8036b0194ea0bd5231710b2cddf4", "internalURL": "http://200.21.18.2:9292", "publicURL": "http://200.21.18.2:9292", "region": "RegionOne" } ], "endpoints_links": [], "name": "glance", "type": "image" }, { "endpoints": [ { "adminURL": "http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813", "id": "054567bc62ce4b4fbdbdcd7c3a23748e", "internalURL": "http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813", "publicURL": "http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813", "region": "RegionOne" } ], "endpoints_links": [], "name": "nova_legacy", "type": "compute_legacy" }, { "endpoints": [ { "adminURL": "http://200.21.18.2:8776/v1/6c4e4d58cb9d4451b36e774b348e8813", "id": "2eefe27748774693b635bf48f486f225", "internalURL": "http://200.21.18.2:8776/v1/6c4e4d58cb9d4451b36e774b348e8813", "publicURL": "http://200.21.18.2:8776/v1/6c4e4d58cb9d4451b36e774b348e8813", "region": "RegionOne" } ], "endpoints_links": [], "name": "cinder", "type": "volume" }, { "endpoints": [ { "adminURL": "http://200.21.18.2:8773/", "id": "4d8f727748924cdf9d23591bad2bbd19", "internalURL": "http://200.21.18.2:8773/", "publicURL": "http://200.21.18.2:8773/", "region": "RegionOne" } ], "endpoints_links": [], "name": "ec2", "type": "ec2" }, { "endpoints": [ { "adminURL": "http://200.21.18.2:35357/v2.0", "id": "16e2a0df7fa64c8cbcdb5936e23b19cc", "internalURL": "http://200.21.18.2:5000/v2.0", "publicURL": "http://200.21.18.2:5000/v2.0", "region": "RegionOne" } ], "endpoints_links": [], "name": "keystone", "type": "identity" } ], "token": { "audit_ids": [ "4zrwvCd7TySk7jJKuO4G1Q" ], "expires": "2017-03-19T05:48:41Z", "id": "74e396f8202b481a9cbd95b319a4314b", "issued_at": "2017-03-19T04:48:42.002243", "tenant": { "description": "", "enabled": true, "id": "6c4e4d58cb9d4451b36e774b348e8813", "name": "admin" } }, "user": { "id": "135b2cb86962401c82044fd4ca9daae4", "name": "admin", "roles": [ { "name": "admin" } ], "roles_links": [], "username": "admin" } } }

須要注意的是, 這一步驟所獲取的 Tenant token 是區別於 Temporary token 的, Temporary token 做爲臨時 token 是爲了實現多租戶的場景所提供的鑑權條件(外部鑑權). 而 Tenant token 纔是聯繫不一樣 OpenStack Project 間的認證通行證(內部鑑權). 從這一步驟能夠看出想要獲取 Tenant token 就須要同時向 Keystone 提供帳戶信息和 tenant_id, 此時用戶不只獲得了 Tenant token 還獲取了相應的 endpoints list. 而且用戶可以經過 endpints list 進一步的去訪問註冊在 Keystone 中的其餘 OpenStack 組件.

Step 4: 使用 Tenant token 和 tenant_id 獲取 admin tenant 的虛擬機列表

curl -v -H "X-Auth-Token:74e396f8202b481a9cbd95b319a4314b" http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813/servers

Response:

{
    "servers": [ { "id": "138ecea2-1656-46bd-aefd-39449e11c356", "links": [ { "href": "http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813/servers/138ecea2-1656-46bd-aefd-39449e11c356", "rel": "self" }, { "href": "http://200.21.18.2:8774/6c4e4d58cb9d4451b36e774b348e8813/servers/138ecea2-1656-46bd-aefd-39449e11c356", "rel": "bookmark" } ], "name": "aju_test_dvs" }, { "id": "42da5d12-a470-4193-8410-0209c04f333a", "links": [ { "href": "http://200.21.18.2:8774/v2/6c4e4d58cb9d4451b36e774b348e8813/servers/42da5d12-a470-4193-8410-0209c04f333a", "rel": "self" }, { "href": "http://200.21.18.2:8774/6c4e4d58cb9d4451b36e774b348e8813/servers/42da5d12-a470-4193-8410-0209c04f333a", "rel": "bookmark" } ], "name": "TestVMwareInterface" } ] }

最終, 咱們從 Response 中獲得了 admin tenant 所具備的兩臺虛擬機的信息.

完整的 RESTAPI 請求流程圖片以下:

顯然, 使用 curl 請求 RESTAPI 的方式過於繁複, 不能知足用戶對 OpenStack 多方位的應用需求(e.g. 實現 OpenStack 的自動化操做腳本). 對此, OpenStack 爲用戶提供了更高級別的 RESTAPI 調用封裝 — OpenStackClients

OpenStackClients

(摘自 OpenStackClients 官方文檔)Each OpenStack project has a related client project that includes Python API bindings and a CLI.
每個 OpenStack 項目都具備一個包含了 Python API bindings 和 CLI 相關的 client 項目. 如圖:

有圖可見, OpenStackClients 項目主要實現了將 OpenStack 計算(Compute)、身份識別(Keystone)、鏡像(Glance)、網絡(Neutron)、對象存儲(Swift)和卷存儲(Cinder) 等核心組件所提供出來的 REST API 整合封裝爲具備統一指令結構的 CLI. 簡而言之, 就是 OpenStackClients 項目使得用戶可以經過 CLI 的形式調用以上組件提供的 REST API, 從而實現操做. 而且咱們也能夠從代碼的層面直接導入 OpenStackClients, 更加便於開發者對 OpenStack 功能模塊的調用.

使用 OpenStackClients 獲取 project_client object 的 demo

vim openstack_clients.py

#!/usr/bin/env python
#encoding=utf8

from openstackclient.identity.client import identity_client_v2
from keystoneclient import session as identity_session
import glanceclient
import novaclient.client as novaclient
import cinderclient.client as cinderclient

# 定義 project_client version
NOVA_CLI_VER = 2
GLANCE_CLI_VER = 2
CINDER_CLI_VER = 2


class OpenstackClients(object):
    """Clients generator of openstack."""

    def __init__(self, auth_url, username, password, tenant_name):

        ### Identity authentication via keystone v2
        # An authentication plugin to authenticate the session with.
        # 經過身份驗證信息獲取 keystone 的 auth object
        # Keystoneclient v2 的詳細使用介紹請瀏覽 https://docs.openstack.org/developer/python-keystoneclient/using-api-v2.html
        auth = identity_client_v2.v2_auth.Password(
            auth_url=auth_url,          # http://200.21.18.3:35357/v2.0/
            username=username,          # admin
            password=password,          # fanguiju
            tenant_name=tenant_name)    # admin
        try:
            # 經過 auth object 獲取 Keystone 的 session object
            self.session = identity_session.Session(auth=auth)
        except Exception as err:
            raise

        # Return a token as provided by the auth plugin.
        # 經過 session object 獲取 Tenant token
        self.token = self.session.get_token()

    def get_glance_client(self, interface='public'):
        """Get the glance-client object."""

        # Get an endpoint as provided by the auth plugin.
        # 默認獲取 glance project 的 public endpoint
        glance_endpoint = self.session.get_endpoint(service_type="image",
                                                    interface=interface)
        # Client for the OpenStack Images API.
        # 經過 glance endpoint 和 token 獲取 glance_client object
        # 而後就可使用 glance_client 調用其實例方法來實現對 glance project 的操做了
        # glanceclient v2 所提供的實例方法列表請瀏覽 https://docs.openstack.org/developer/python-glanceclient/ref/v2/images.html
        glance_client = glanceclient.Client(GLANCE_CLI_VER,
                                            endpoint=glance_endpoint,
                                            token=self.token)
        return glance_client

    def get_nova_client(self):
        """Get the nova-client object."""

        # Initialize client object based on given version. Don't need endpoint.
        # 也能夠 不指定 endpoint 的類型, 僅使用 session object 來獲取 nove_client
        # novaclient v2 的實例方法列表請瀏覽 https://docs.openstack.org/developer/python-novaclient/api.html#usage 
        nova_client = novaclient.Client(NOVA_CLI_VER, session=self.session)
        return nova_client

    def get_cinder_client(self, interface='public'):
        """Get the cinder-client object."""

        cinder_endpoint = self.session.get_endpoint(service_type='volume',
                                                    interface=interface)
        # cinder_client v2 的實例方法列表請查看 https://docs.openstack.org/developer/python-cinderclient/
        cinder_client = cinderclient.Client(CINDER_CLI_VER, session=self.session)
        return cinder_client

調用 project_client object 實例方法實現對 project 操做的 demo

vim auto_dep.py

#!/usr/bin/env python
#encoding=utf8


import os
from os import path
import time

import openstack_clients as os_cli


# FIXME(Fan Guiju): Using oslo_config and logging
AUTH_URL = 'http://200.21.18.3:35357/v2.0/'
USERNAME = 'admin'
PASSWORD = 'fanguiju'
PROJECT_NAME = 'admin'

DISK_FORMAT = 'qcow2'
IMAGE_NAME = 'ubuntu_server_1404_x64'
IMAGE_PATH = path.join(path.curdir, 'images',
                       '.'.join([IMAGE_NAME, DISK_FORMAT]))

MIN_DISK_SIZE_GB = 20

KEYPAIR_NAME = 'jmilkfan-keypair'
KEYPAIT_PUB_PATH = '/home/stack/.ssh/id_rsa.pub'

DB_NAME = 'blog'
DB_USER = 'wordpress'
DB_PASS = 'fanguiju'
DB_BACKUP_SIZE = 5
DB_VOL_NAME = 'mysql-volume'
DB_INSTANCE_NAME = 'AUTO-DEP-DB'
MOUNT_POINT = '/dev/vdb'

BLOG_INSTANCE_NAME = 'AUTO-DEP-BLOG'

TIMEOUT = 60


class AutoDep(object):
    def __init__(self, auth_url, username, password, tenant_name):
        # 實例化上述的 openstack_client.OpenstackClients 的對象
        openstack_clients = os_cli.OpenstackClients(
            auth_url,
            username,
            password,
            tenant_name)

        # 經過 openstack_clients 的實例方法獲取 project_client 對象
        self._glance = openstack_clients.get_glance_client()
        self._nova = openstack_clients.get_nova_client()
        self._cinder = openstack_clients.get_cinder_client()

    def _wait_for_done(self, objs, target_obj_name):
        """Wait for action done."""
        count = 0
        while count <= TIMEOUT:
            for obj in objs.list():
                if obj.name == target_obj_name:
                    return
            time.sleep(3)
            count += 3
        raise

    def upload_image_to_glance(self):
        images = self._glance.images.list()
        for image in images:
            if image.name == IMAGE_NAME:
                return image
        # 調用 glanceclient.images.create method 建立一個 image object.
        new_image = self._glance.images.create(name=IMAGE_NAME,
                                               disk_format=DISK_FORMAT,
                                               container_format='bare',
                                               min_disk=MIN_DISK_SIZE_GB,
                                               visibility='public')
        # Open image file with read+binary.
        # 調用 glanceclient.images.upload method 上傳一個 image
        self._glance.images.upload(new_image.id, open(IMAGE_PATH, 'rb'))
        self._wait_for_done(objs=self._glance.images,
                            target_obj_name=IMAGE_NAME)
        image = self._glance.images.get(new_image.id)
        return image

    def create_volume(self):
        # 調用 cinderclient.volumes.list method 獲取 volumes 的列表
        volumes = self._cinder.volumes.list()
        for volume in volumes:
            if volume.name == DB_VOL_NAME:
                return volume
        # cinderclient.v2.volumes:VolumeManager
        # 調用 minderclient.volumes.create method 建立一個 volume
        new_volume = self._cinder.volumes.create(
            size=DB_BACKUP_SIZE,
            name=DB_VOL_NAME,
            volume_type='lvmdriver-1',
            availability_zone='nova',
            description='backup volume of mysql server.')
        if new_volume:
            return new_volume
        else:
            raise

    def get_flavor_id(self):
        # 調用 novaclient.flavors.list method 獲取全部 flavors 的列表
        flavors = self._nova.flavors.list()
        for flavor in flavors:
            if flavor.disk == MIN_DISK_SIZE_GB:
                return flavor.id

    def _get_ssh_pub_key(self):
        if not path.exists(KEYPAIT_PUB_PATH):
            raise
        return open(KEYPAIT_PUB_PATH, 'rb').read()

    def import_keypair_to_nova(self):
        # 調用 novaclient.keypairs.list method 獲取 keypairs 的列表
        keypairs = self._nova.keypairs.list()
        for keypair in keypairs:
            if keypair.name == KEYPAIR_NAME:
                return None
        keypair_pub = self._get_ssh_pub_key()
        # 調用 nova client.keypairs.create method 建立 keypair
        self._nova.keypairs.create(KEYPAIR_NAME, public_key=keypair_pub)

    def nova_boot(self, image, volume):
        flavor_id = self.get_flavor_id()
        self.import_keypair_to_nova()
        db_instance = False

        # 調用 novaclient.servers.list method 獲取 servers 的列表
        servers = self._nova.servers.list()
        server_names = []
        for server in servers:
            server_names.append(server.name)
            if server.name == DB_INSTANCE_NAME:
                db_instance = server

        if not db_instance:
            # Create the mysql server
            db_script_path = path.join(path.curdir, 'scripts/db_server.txt')
            db_script = open(db_script_path, 'r').read()
            db_script = db_script.format(DB_NAME, DB_USER, DB_PASS)
            # 經過 nova client.servers.create method 建立一個 server
            # 這裏由於但願建立 server 並對其進行預設置, 因此使用了 userdata 參數
            # userdata 參數會接收一個 script 文件, 並在 server 第一次啓動的時候執行
            db_instance = self._nova.servers.create(
                # FIXME(Fan Guiju): Using the params `block_device_mapping` to attach the volume.
                DB_INSTANCE_NAME,
                image.id,
                flavor_id,
                key_name=KEYPAIR_NAME,
                userdata=db_script)
            # 經過 novaclient.server.get method 和 server_id 來獲取單個 server 的詳細信息
            if not self._nova.server.get(db_instance.id):
                self._wait_for_done(objs=self._nova.servers,
                                    target_obj_name=DB_INSTANCE_NAME)
        # Attach the mysql-vol to mysql server, device type is `vd`.
        # 經過 cinderclient.volumes.attach method 掛在一個 volume 到 server 上
        # mountpoint 參數執行了掛載到 server 的設備路徑, e.g. /dev/vdb
        self._cinder.volumes.attach(volume=volume,
                                    instance_uuid=db_instance.id,
                                    mountpoint=MOUNT_POINT)
        time.sleep(5)

        if BLOG_INSTANCE_NAME not in server_names:
            # Create the wordpress blog server
            # Nova-Network
            db_instance_ip = self._nova.servers.\
                get(db_instance.id).networks['private'][0]
            blog_script_path = path.join(path.curdir, 'scripts/blog_server.txt')
            blog_script = open(blog_script_path, 'r').read()
            blog_script = blog_script.format(DB_NAME,
                                             DB_USER,
                                             DB_PASS,
                                             db_instance_ip)
            self._nova.servers.create(BLOG_INSTANCE_NAME,
                                      image.id,
                                      flavor_id,
                                      key_name=KEYPAIR_NAME,
                                      userdata=blog_script)
            self._wait_for_done(objs=self._nova.servers,
                                target_obj_name=BLOG_INSTANCE_NAME)

        servers = self._nova.servers.list(search_opts={'all_tenants': True})
        return servers


def main():
    """FIXME(Fan Guiju): Operation manual."""
    os.environ['LANG'] = 'en_US.UTF8'

    deploy = AutoDep(auth_url=AUTH_URL,
                     username=USERNAME,
                     password=PASSWORD,
                     tenant_name=PROJECT_NAME)
    image = deploy.upload_image_to_glance()
    volume = deploy.create_volume()
    deploy.nova_boot(image, volume)

if __name__ == '__main__':
    main()

最後

上面給出了一個自動化運行 OpenStack Project 功能模塊的腳本, 但實際上, 咱們可以使用 OpenStackClients 進行更加複雜的工做, 例如: 自定義一個新的 OpenStack Project, 並使之與 OpenStack 的原生 Project 進行互動, 這纔是真正意義上的二次開發.