openstack運維實戰系列(四)之刪除computes節點

1. 前言

    由於虛擬機業務的撤銷，compute上面的instance愈來愈少，隨着虛擬機的遷移和下線，compute節點可能處於資源空閒的狀態，爲了進一步利用系統資源，節約成本，須要將空閒的計算節點從openstack中刪除。在刪除compute節點以前，首先須要確保該計算節點上沒有業務，能夠經過virsh list --all查看是否還有虛擬機業務，同時保守起見，建議將compute節點的服務關閉關閉，經過service openstack-nova-compute stop && chkconfig openstack-nova-compute off。

2. 刪除nova計算節點

    openstack建議在刪除nova節點的時候，建議經過操做nova.instances中的deleted字段來實現，而不是直接將compute節點的條目從數據庫中刪除，一來是爲了安全考慮，而是可以實現快速的回滾操做，具體的操做以下：

1. 校驗nova-compute的狀態

[root@controller ~]# nova service-list
+------------------+-----------------------+---------------+---------+-------+----------------------------+-----------------+
| Binary           | Host                  | Zone          | Status  | State | Updated_at                 | Disabled Reason |
+------------------+-----------------------+---------------+---------+-------+----------------------------+-----------------+
| nova-scheduler   | controller            | internal      | enabled | up    | 2016-01-20T04:12:33.000000 | -               |
| nova-cert        | controller            | internal      | enabled | up    | 2016-01-20T04:12:31.000000 | -               |
| nova-conductor   | controller            | internal      | enabled | up    | 2016-01-20T04:12:27.000000 | -               |
| nova-consoleauth | controller            | internal      | enabled | up    | 2016-01-20T04:12:33.000000 | -               |
| nova-compute     | YiZhuang_10_1_112_105 | YiZhuangZone2 | enabled | down  | 2016-01-16T09:11:57.000000 | -               |    #已經處於down的狀態
| nova-compute     | YiZhuang_10_1_112_106 | YiZhuangZone2 | enabled | up    | 2016-01-20T04:12:24.000000 | -               |

2. 刪除compute service節點

mysql> select * from nova.services where host='YiZhuang_10_1_112_105';  
+---------------------+---------------------+------------+----+-----------------------+--------------+---------+--------------+----------+---------+-----------------+
| created_at          | updated_at          | deleted_at | id | host                  | binary       | topic   | report_count | disabled | deleted | disabled_reason |
+---------------------+---------------------+------------+----+-----------------------+--------------+---------+--------------+----------+---------+-----------------+
| 2015-09-21 03:21:40 | 2016-01-16 09:11:57 | NULL       | 37 | YiZhuang_10_1_112_105 | nova-compute | compute |      1011705 |        0 |       0 | NULL            |
+---------------------+---------------------+------------+----+-----------------------+--------------+---------+--------------+----------+---------+-----------------+
1 row in set (0.00 sec)
mysql> update nova.services set deleted=1 where host='YiZhuang_10_1_112_105';             #設置deleted字段，即標誌位            
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0
mysql> select * from nova.services where host='YiZhuang_10_1_112_105';                   #檢查deleted字段設置狀況
+---------------------+---------------------+------------+----+-----------------------+--------------+---------+--------------+----------+---------+-----------------+
| created_at          | updated_at          | deleted_at | id | host                  | binary       | topic   | report_count | disabled | deleted | disabled_reason |
+---------------------+---------------------+------------+----+-----------------------+--------------+---------+--------------+----------+---------+-----------------+
| 2015-09-21 03:21:40 | 2016-01-16 09:11:57 | NULL       | 37 | YiZhuang_10_1_112_105 | nova-compute | compute |      1011705 |        0 |       1 | NULL            |   
+---------------------+---------------------+------------+----+-----------------------+--------------+---------+--------------+----------+---------+-----------------+
1 row in set (0.00 sec)

3. 確認節點的service狀態

[root@controller ~]# nova service-list
+------------------+-----------------------+---------------+---------+-------+----------------------------+-----------------+
| Binary           | Host                  | Zone          | Status  | State | Updated_at                 | Disabled Reason |
+------------------+-----------------------+---------------+---------+-------+----------------------------+-----------------+
| nova-scheduler   | controller            | internal      | enabled | up    | 2016-01-20T04:12:33.000000 | -               |
| nova-cert        | controller            | internal      | enabled | up    | 2016-01-20T04:12:31.000000 | -               |
| nova-conductor   | controller            | internal      | enabled | up    | 2016-01-20T04:12:27.000000 | -               |
| nova-consoleauth | controller            | internal      | enabled | up    | 2016-01-20T04:12:33.000000 | -               |
| nova-compute     | YiZhuang_10_1_112_106 | YiZhuangZone2 | enabled | up    | 2016-01-20T04:12:24.000000 | -               |    #service已經刪除

4. 查看hypervisor狀況

[root@controller ~]# nova hypervisor-list
+----+-----------------------+
| ID | Hypervisor hostname   |
+----+-----------------------+
| 25 | YiZhuang_10_1_112_105 |          #services已經刪除，但hypervisor依舊在，相似的方法，將nova.compute_nodes中的deleted字段修改，查看步驟5    
| 27 | YiZhuang_10_1_112_106 |

5. 刪除compute_nodes節點

mysql> update nova.compute_nodes set deleted=1 where hypervisor_hostname='YiZhuang_10_1_112_105'\G;       #設置deleted標誌位           
Query OK, 1 row affected (0.01 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> select * from nova.compute_nodes where hypervisor_hostname='YiZhuang_10_1_112_105'\G;               #確認設置狀況    
*************************** 1. row ***************************
          created_at: 2015-09-21 03:21:40
          updated_at: 2016-01-16 09:11:36
          deleted_at: NULL
                  id: 25
          service_id: 37
               vcpus: 24
           memory_mb: 64396
            local_gb: 3062
          vcpus_used: 0
      memory_mb_used: 1024
       local_gb_used: 20
     hypervisor_type: QEMU
  hypervisor_version: 12001
            cpu_info: {"vendor": "Intel", "model": "SandyBridge", "arch": "x86_64", "features": ["vme", "dtes64", "vmx", "erms", "xtpr", "smep", "pcid", "est", "monitor", "smx", "tm", "acpi", "osxsave", "ht", "dca", "pdcm", "pdpe1gb", "fsgsbase", "f16c", "ds", "tm2", "ss", "pbe", "ds_cpl", "rdrand"], "topology": {"cores": 6, "threads": 2, "sockets": 1}}
disk_available_least: 2671
         free_ram_mb: 63372
        free_disk_gb: 3042
    current_workload: 0
         running_vms: 0
 hypervisor_hostname: YiZhuang_10_1_112_105
             deleted: 1                        #設置成功
             host_ip: 0.0.0.0
 supported_instances: [["i686", "qemu", "hvm"], ["i686", "kvm", "hvm"], ["x86_64", "qemu", "hvm"], ["x86_64", "kvm", "hvm"]]
           pci_stats: []
             metrics: []
     extra_resources: NULL
               stats: {"num_task_None": 2, "io_workload": 0, "num_instances": 2, "num_vm_stopped": 2, "num_proj_a49b16d5324a4d20bde2217b17200485": 2, "num_vcpus_used": 8, "num_os_type_None": 2}
1 row in set (0.00 sec)

6. 確認hypervisor node是否刪除

[root@controller ~]# nova hypervisor-list        #YiZhuang_10_1_112_105已被刪除
+----+-----------------------+
| ID | Hypervisor hostname   |
+----+-----------------------+  
| 27 | YiZhuang_10_1_112_106 |

3. 刪除neutron agent服務

    以上將nova的service和hypervisor從數據庫中刪除，可是compute節點在安裝過程當中，neutron-openvswitch-agent會創建一個OVS的agent，該agent也記錄在數據庫中，經過neutron agent-list能夠查看到agent的狀況，當compute node刪除以後，neutron也應該刪除，方法和上面相相似，具體以下：

1. 確認agent的狀況

[root@controller ~]# neutron agent-list
+--------------------------------------+--------------------+-----------------------+-------+----------------+
| id                                   | agent_type         | host                  | alive | admin_state_up |
+--------------------------------------+--------------------+-----------------------+-------+----------------+
| 0160e0b7-22fb-42d9-8e43-0d13d98db594 | L3 agent           | LuGu_10_1_81_209      | :-)   | True           |
| 4e1c9957-98e5-4516-af2d-6c67a00ecd77 | Open vSwitch agent | YiZhuang_10_1_112_105 | xxx   | True           |    #計算節點已經關閉，因此agent狀態變爲xxx，即不可用,:-）可用
| 6a9e2647-6eed-459d-8f60-e9ea40ae04df | Open vSwitch agent | YiZhuang_10_1_112_106 | :-)   | True           |

2. 刪除OVS agent

mysql> select * from neutron.agents where host='YiZhuang_10_1_112_105'\G;        #沒有deleted標誌位，因此只能從數據庫中刪除
*************************** 1. row ***************************
                 id: 4e1c9957-98e5-4516-af2d-6c67a00ecd77
         agent_type: Open vSwitch agent
             binary: neutron-openvswitch-agent
              topic: N/A
               host: YiZhuang_10_1_112_105
     admin_state_up: 1
         created_at: 2015-09-21 03:22:01
         started_at: 2016-01-19 09:15:45
heartbeat_timestamp: 2016-01-19 09:15:45
        description: NULL
     configurations: {"tunnel_types": [], "tunneling_ip": "", "bridge_mappings": {"physnet1": "br-eth1", "physnet0": "br-eth0"}, "l2_population": false, "devices": 0}
1 row in set (0.00 sec)


mysql> delete from neutron.agents where host='YiZhuang_10_1_112_105'\G;         #刪除         
Query OK, 1 row affected (0.00 sec)

校驗：
[root@controller ~]# neutron agent-list
+--------------------------------------+--------------------+-----------------------+-------+----------------+
| id                                   | agent_type         | host                  | alive | admin_state_up |
+--------------------------------------+--------------------+-----------------------+-------+----------------+
| 0160e0b7-22fb-42d9-8e43-0d13d98db594 | L3 agent           | LuGu_10_1_81_209      | :-)   | True           |
| 6a9e2647-6eed-459d-8f60-e9ea40ae04df | Open vSwitch agent | YiZhuang_10_1_112_106 | :-)   | True           |

4. 總結

   以上是將compute_nodes從opentack中刪除的方法，在Juno以上的版本，能夠經過nova service-delete <hostname> 的方式將compute nodes刪除，在Icehouse如下的版本，暫時未能提供，解決的方法能夠經過調用nova db的api，將其刪除，後續再補充。或者經過編寫的腳本的方式，也能夠實現，建議經過調用API的方式，直接修改數據庫，而不建議直接修改數據庫的方法。