中止節點上全部的osd服務node
systemctl stop ceph-osd@20 systemctl stop ceph-osd@21 systemctl stop ceph-osd@22 systemctl stop ceph-osd@23
在mon節點上移除osd數據庫
ceph osd out 23 ceph osd out 22 ceph osd out 21 ceph osd out 20 ceph osd crush remove osd.23 ceph osd crush remove osd.22 ceph osd crush remove osd.21 ceph osd crush remove osd.20 ceph auth del osd.23 ceph auth del osd.22 ceph auth del osd.21 ceph auth del osd.20 ceph osd rm 23 ceph osd rm 22 ceph osd rm 21 ceph osd rm 20 ceph osd crush remove bdc8
等待集羣自動恢復便可 若是配置文件中有對應osd在某臺節點上的配置,須要刪掉網絡
systemctl stop neutron-openvswitch-agent.service systemctl stop openstack-nova-compute.service systemctl disable neutron-openvswitch-agent.service systemctl disable openstack-nova-compute.service
查看計算節點服務3d
. admin-openrc.sh nova service-list +----+------------------+------+----------+---------+-------+----------------------------+-----------------+ | Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason | +----+------------------+------+----------+---------+-------+----------------------------+-----------------+ | 1 | nova-cert | bdc2 | internal | enabled | up | 2018-03-22T02:54:42.000000 | - | | 2 | nova-conductor | bdc2 | internal | enabled | up | 2018-03-22T02:54:43.000000 | - | | 3 | nova-consoleauth | bdc2 | internal | enabled | up | 2018-03-22T02:54:42.000000 | - | | 4 | nova-scheduler | bdc2 | internal | enabled | up | 2018-03-22T02:54:40.000000 | - | | 5 | nova-compute | bdc3 | nova | enabled | up | 2018-03-22T02:54:41.000000 | - | | 6 | nova-compute | bdc2 | nova | enabled | up | 2018-03-22T02:54:45.000000 | - | | 7 | nova-compute | bdc4 | nova | enabled | up | 2018-03-22T02:54:42.000000 | - | | 8 | nova-compute | bdc5 | nova | enabled | up | 2018-03-22T02:54:44.000000 | - | | 9 | nova-compute | bdc6 | nova | enabled | up | 2018-03-22T02:54:44.000000 | - | | 10 | nova-compute | bdc7 | nova | enabled | up | 2018-03-22T02:54:46.000000 | - | | 11 | nova-compute | bdc8 | nova | enabled | up | 2018-03-22T02:54:39.000000 | - | +----+------------------+------+----------+---------+-------+----------------------------+-----------------+
刪除bdc8上的計算服務code
nova service-disable 11 nova service-delete 11
元數據庫中刪除ip
use nova; select * from compute_nodes where host='bdc2'\G delete from compute_nodes where host='bdc2'; select * from services where host='bdc2'\G delete from services where host='bdc2';
查看網絡服務rem
# neutron agent-list +--------------------------------------+--------------------+------+-------+----------------+---------------------------+ | id | agent_type | host | alive | admin_state_up | binary | +--------------------------------------+--------------------+------+-------+----------------+---------------------------+ | 08bc9d33-3e67-4a06-9223-74be1c969b8a | Open vSwitch agent | bdc2 | :-) | True | neutron-openvswitch-agent | | 19a31ea2-280b-42b4-9f08-c1f99182e826 | Open vSwitch agent | bdc5 | :-) | True | neutron-openvswitch-agent | | 286336ae-22fd-4391-8470-aae559ee5ccb | DHCP agent | bdc3 | :-) | True | neutron-dhcp-agent | | 42ab605e-3d66-402f-b86a-4ef246df2ecf | Open vSwitch agent | bdc8 | :-) | True | neutron-openvswitch-agent | | 4791844d-07f7-43be-8b27-bcfe8d8a7791 | Open vSwitch agent | bdc6 | :-) | True | neutron-openvswitch-agent | | 63c99510-205a-4f94-ac18-61082c6edb44 | L3 agent | bdc3 | :-) | True | neutron-l3-agent | | 830a75fb-c516-4f6f-ae74-e09bfd558291 | Open vSwitch agent | bdc7 | :-) | True | neutron-openvswitch-agent | | 91df9210-0a27-4933-9f03-3ccce6192011 | Open vSwitch agent | bdc4 | :-) | True | neutron-openvswitch-agent | | a9f7bc30-beda-437d-92a8-be64296c57d0 | Open vSwitch agent | bdc3 | :-) | True | neutron-openvswitch-agent | | cf6443f7-ef8e-4738-bfc5-e491e4f89aeb | Metadata agent | bdc3 | :-) | True | neutron-metadata-agent | +--------------------------------------+--------------------+------+-------+----------------+---------------------------+
刪除bdc8上的網絡服務it
# neutron agent-delete 42ab605e-3d66-402f-b86a-4ef246df2ecf Deleted agent: 42ab605e-3d66-402f-b86a-4ef246df2ecf
nova庫io
MariaDB [nova]> select hypervisor_hostname,deleted_at,deleted from compute_nodes; +---------------------+---------------------+---------+ | hypervisor_hostname | deleted_at | deleted | +---------------------+---------------------+---------+ | bdc3 | NULL | 0 | | bdc2 | NULL | 0 | | bdc4 | NULL | 0 | | bdc5 | NULL | 0 | | bdc6 | NULL | 0 | | bdc7 | NULL | 0 | | bdc8 | 2018-03-22 02:58:53 | 7 | +---------------------+---------------------+---------+ MariaDB [nova]> select * from services where host = 'bdc8'; +---------------------+---------------------+---------------------+----+------+--------------+---------+--------------+----------+---------+-----------------+ | created_at | updated_at | deleted_at | id | host | binary | topic | report_count | disabled | deleted | disabled_reason | +---------------------+---------------------+---------------------+----+------+--------------+---------+--------------+----------+---------+-----------------+ | 2016-10-13 09:20:41 | 2018-03-22 02:58:49 | 2018-03-22 02:58:53 | 11 | bdc8 | nova-compute | compute | 4442289 | 0 | 11 | NULL | +---------------------+---------------------+---------------------+----+------+--------------+---------+--------------+----------+---------+-----------------+
先在元數據庫中刪除bdc8的gre信息console
MariaDB [neutron]> select * from ml2_gre_endpoints; +-------------+------+ | ip_address | host | +-------------+------+ | 172.16.13.2 | bdc2 | | 172.16.13.3 | bdc3 | | 172.16.13.4 | bdc4 | | 172.16.13.5 | bdc5 | | 172.16.13.6 | bdc6 | | 172.16.13.7 | bdc7 | | 172.16.13.8 | bdc8 | +-------------+------+ MariaDB [neutron]> delete from ml2_gre_endpoints where ip_address='172.16.13.8';
各個節點中刪除鏈接bdc8的gre隧道
# ovs-vsctl show 8d6ea1e1-9b90-4e6b-94ae-d5a04601daf9 Bridge br-tun fail_mode: secure Port "gre-ac100d03" Interface "gre-ac100d03" type: gre options: {df_default="true", in_key=flow, local_ip="172.16.13.2", out_key=flow, remote_ip="172.16.13.3"} Port "gre-ac100d07" Interface "gre-ac100d07" type: gre options: {df_default="true", in_key=flow, local_ip="172.16.13.2", out_key=flow, remote_ip="172.16.13.7"} Port "gre-ac100d08" Interface "gre-ac100d08" type: gre options: {df_default="true", in_key=flow, local_ip="172.16.13.2", out_key=flow, remote_ip="172.16.13.8"} Port "gre-ac100d05" Interface "gre-ac100d05" type: gre options: {df_default="true", in_key=flow, local_ip="172.16.13.2", out_key=flow, remote_ip="172.16.13.5"} Port "gre-ac100d06" Interface "gre-ac100d06" type: gre options: {df_default="true", in_key=flow, local_ip="172.16.13.2", out_key=flow, remote_ip="172.16.13.6"} Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port "gre-ac100d04" Interface "gre-ac100d04" type: gre options: {df_default="true", in_key=flow, local_ip="172.16.13.2", out_key=flow, remote_ip="172.16.13.4"} Port br-tun Interface br-tun type: internal Bridge br-int fail_mode: secure Port "qvo86bf0e09-3f" tag: 2 Interface "qvo86bf0e09-3f" Port "qvo73d3f4d9-72" tag: 2 Interface "qvo73d3f4d9-72" Port br-int Interface br-int type: internal Port "qvo7c5fbeca-ab" tag: 2 Interface "qvo7c5fbeca-ab" Port "qvo2364aa3b-98" tag: 4 Interface "qvo2364aa3b-98" Port "qvo0194c615-74" tag: 5 Interface "qvo0194c615-74" Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port "qvo4a30d4d8-e3" tag: 3 Interface "qvo4a30d4d8-e3" Port "qvo1d912409-ed" tag: 2 Interface "qvo1d912409-ed" Port "qvoe79c563d-50" tag: 1 Interface "qvoe79c563d-50" Port "qvo4b146bb0-52" tag: 3 Interface "qvo4b146bb0-52" ovs_version: "2.5.0"
全部節點上鍊接bdc8的port同樣,全都須要刪除
# ovs-vsctl del-port br-tun gre-ac100d08