install ceph by ceph-deploy

使用阿里雲源安裝ceph Luminous   

https://liuxu.co/2017/09/19/install-ceph-Luminous-on-centos7-with-ceph-deploy/

 

0、若是曾經在節點上安裝過ceph，那麼須要先清理一下環境

ceph-deploy uninstall ceph-node1 ceph-node2 ceph-node3

ceph-deploy purge ceph-node1 ceph-node2 ceph-node3

ceph-deploy purgedata ceph-node1 ceph-node2 ceph-node3

ceph-deploy forgetkeys

 

一、在每臺ceph節點上都生成一個ceph-deploy用戶，密碼也爲ceph-deply

sudo useradd -d /home/ceph-deploy -m ceph-deploy

passwd ceph-deploy   

(ceph-deploy)

 

echo "ceph-deploy ALL = (root) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/ceph-deploy

sudo chmod 0440 /etc/sudoers.d/ceph-deploy

sudo setenforce 0

 

二、在其中一臺節點上生成ssh key，並將ssh key 拷貝至其他ceph節點

su ceph-deploy

ssh-keygen 一路回車

ssh-copy-id ceph-deploy@ceph-node1

ssh-copy-id ceph-deploy@ceph-node2

ssh-copy-id ceph-deploy@ceph-node3

安裝ceph-deploy

sudo yum install ceph-deploy -y

 

三、安裝ceph

cd /home/ceph-deploy

ceph-deploy install ceph-node1 ceph-node2 ceph-node3 --repo-url=http://mirrors.aliyun.com/ceph/rpm-luminous/el7/ --gpg-url=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7

 

四、新建ceph集羣

ceph-deploy new ceph-node1 ceph-node2 ceph-node3

vim ceph.conf

[global]
fsid = 69e24e55-e1f2-4451-8ad9-5125500daac5
mon_initial_members = ceph-node1,ceph-node2,ceph-node3
mon_host = 192.168.56.131,192.168.56.132,192.168.56.133
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
public network = 192.168.56.130/27

 

五、新建並初始化monitor

ceph-deploy --overwrite-conf mon create-initial

 

六、查看各節點上的disk

ceph-deploy disk list ceph-node1

ceph-deploy disk list ceph-node2

ceph-deploy disk list ceph-node3

 

vim cleanOsd.sh

#!/bin/sh

hosts="ceph-node1 ceph-node2 ceph-node3"
dev="b c"
for hostN in $hosts
do
    for i in $dev
    do
        ceph-deploy disk zap ${hostN}:sd${i}
    done
done

sh cleanOsd.sh

 

vim createOsd.sh

#!/bin/sh

hosts="ceph-node1 ceph-node2 ceph-node3"
dev="b c"
for hostN in $hosts
do
    for i in $dev
    do
        ceph-deploy osd create ${hostN}:sd${i}
    done
done

sh createOsd.sh

 

七、設置admin key

ceph-deploy admin ceph-node1 ceph-node2 ceph-node3

sudo chmod +r /etc/ceph/ceph.client.admin.keyring

 

八、檢驗ceph安裝是否成功

ceph -s

 

Q&A:

1. ceph create mon時出現ip地址爲0.0.0.0的狀況

是由於使用了hostname xxx修改過主機名致使的。重啓一下節點。

2. nohup python /usr/bin/ceph-rest-api -n client.admin > /var/log/ceph/ceph-rest-api.log &

3. curl -H "Accept: application/json" '192.168.56.131:5000/api/v0.1/status'| python -m json.tool

4. minikube start --vm-driver=none --iso-url https://raw.githubusercontent.com/cilium/minikube-iso/master/minikube.iso

5. ceph osd lspools

ceph osd pool create testpool 256

6. centos build nbd

http://www.zphj1987.com/2016/05/19/ceph%E7%9A%84jewel%E6%96%B0%E6%94%AF%E6%8C%81%E7%9A%84rbd-nbd/

http://www.hl10502.com/2017/09/14/centos-build-nbd/

 7. set ceph replica size to 1

ceph osd pool set pool-name size 1