05 RGW對象存儲

參考鏈接：<https://ceph.readthedocs.io/en/latest/radosgw/>;

​ 做爲文件系統的磁盤，操做系統不能直接訪問對象存儲。相反，它只能經過應用程序級別的API訪問。Ceph是一種分佈式對象存儲系統，經過Ceph對象網關提供對象存儲接口，也稱爲RADOS網關(RGW)接口，它構建在Ceph RADOS層之上。RGW使用librgw (RADOS Gateway Library)和librados，容許應用程序與Ceph對象存儲創建鏈接。RGW爲應用程序提 供了一個RESTful S3 / swift兼容的API接口，用於在Ceph集羣中以對象的形式存儲數據。Ceph還支持多租戶對象存儲，能夠經過RESTful API訪問。此外，RGW還支持Ceph管理API，能夠使用本機API調用來管理Ceph存儲集羣。

​ 對於生產環境，建議您在物理專用機器上配置RGW。可是，若是您的對象存儲工做負載不太大，您能夠考慮將任何監視器機器做爲RGW節點使用。RGW是一個獨立的服務，它從外部鏈接到Ceph集羣，並向客戶端提供對象存儲訪問。在生產環境中，建議您運行多個RGW實例，由負載均衡器屏蔽。

1 RGW的配置

1.1 配置RGW

​ [本例以cephmanager01爲例]

​ 1）安裝ceph-radosgw(須要安裝的節點均須要安裝)

[root@cephmanager01 ~]# yum -y install ceph-radosgw

​ 2）部署服務

​ 進入cephuser的工做目錄執行

[cephuser@cephmanager01 cephcluster]$ ceph-deploy rgw create cephmanager01

​ 3)確認部署成功

# 確認是否有rgw是active
[cephuser@cephmanager01 cephcluster]$ sudo ceph -s
# 確實7480端口是否開通 
[cephuser@cephmanager01 cephcluster]$ sudo netstat -antulp |grep 7480
# 訪問看是否有輸出
[cephuser@cephmanager01 ~]$ sudo curl http://cephmanager01:7480
# 確認是否生成了rgw相關的pools
[cephuser@cephmanager01 ~]$ sudo ceph osd lspools

1.2 修改默認端口(7480->80)

1)修改cephuser工做目錄的配置文件ceph.conf

在 [global] 節後添加的內容以下：

#cephmanager01爲gateway-node的節點名稱,80爲須要修改的端口
[client.rgw.cephmanager01]
rgw_frontends = "civetweb port=80"

2）同步配置文件

將該配置文件推送到其餘節點

[cephuser@cephmanager01 cephcluster]$ ceph-deploy --overwrite-conf config push cephmanager01 cephmanager02 cephmanager03

3)重啓服務生效

[cephuser@cephmanager01 cephcluster]$ sudo systemctl restart ceph-radosgw.target

4)確認是否成功

# 確實80端口是否開通 
[cephuser@cephmanager01 cephcluster]$ sudo netstat -antulp |grep rados
# 訪問看是否有輸出
[cephuser@cephmanager01 ~]$ sudo curl http://cephmanager01:80

注：若是開啓了防火牆，須要添加80端口的例外

# firewalld示例
[cephuser@cephmanager01 cephcluster]$ sudo firewall-cmd --list-all
[cephuser@cephmanager01 cephcluster]$ sudo firewall-cmd --zone=public --add-port 80/tcp --permanent
[cephuser@cephmanager01 cephcluster]$ sudo firewall-cmd --reload

2 RGW 使用

2.1 使用 S3 API 訪問 Ceph 對象存儲

2.1.1 服務端配置

#建立 radosgw 用戶
[root@cephmanager01 ~]# radosgw-admin user create --uid="radosgwuser" --display-name="radosgwuser demo"
#注意：請把 access_key 和 secret_key 保存下來,也能夠經過命令獲取
[root@cephmanager01 ~]# radosgw-admin user info --uid="radosgwuser"

2.1.2 客戶端配置

（1）官方給出了一個 Python腳本訪問的案例

​ 1）安裝對應的軟件包python-boto

# yum install python-boto -y

​ 2)修改腳本信息

# 須要修改access_key、secret_key、host、port、bucket名稱等信息
# cat s3test.py 
import boto.s3.connection

access_key = 'B4TZNYMJ2VMUTF97BFS9'
secret_key = 'TpxrusS0f42yyYZNuvUni5Shoc9yJH83myWO5gCc'
conn = boto.connect_s3(
        aws_access_key_id=access_key,
        aws_secret_access_key=secret_key,
        host='192.168.10.51', port=80,
        is_secure=False, calling_format=boto.s3.connection.OrdinaryCallingFormat(),
       )

bucket = conn.create_bucket('my-new-bucket')
for bucket in conn.get_all_buckets():
    print "{name} {created}".format(
        name=bucket.name,
        created=bucket.creation_date,
    )

​ 3）執行腳本並確認

# python s3test.py
my-new-bucket 2020-07-03T02:45:31.174Z

若是第一次建立bucket會生成一個default.rgw.buckets.index的pool，第一次向bucket裏面防入數據，會生成一個default.rgw.buckets.data的pool

（2）s3cmd訪問

​ 1)安裝 s3cmd 客戶端

# yum install s3cmd -y

​ 2)生成一個配置文件，在當前目錄下，輸入生成的key

# s3cmd --configure

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key [UTCFB8H59AF611EYFYDY]: B4TZNYMJ2VMUTF97BFS9 #須要輸入radosgwuser的Access Key
Secret Key [R7DdzYzn7llNlwiLC8FtZEs2vfNlEBBQFBFaxRUb]: TpxrusS0f42yyYZNuvUni5Shoc9yJH83myWO5gCc #須要輸入radosgwuser的Secret Key
Default Region [US]: #保持默認，注：國家只支持US

Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.
S3 Endpoint [cephnode01]: 192.168.10.51:80 #輸入rgw的ip和端口

Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used
if the target S3 system supports dns based buckets.
DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.cephnode01.donghai.com]: 192.168.10.51:80/%(bucket)s #輸入rgw的ip和端口，注意格式

Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password:  #啓用加密
Path to GPG program [/usr/bin/gpg]: 

When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol [No]: no  #是否啓用https

On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can't connect to S3 directly
HTTP Proxy server name:   #是否設置代理

New settings:
  Access Key: B4TZNYMJ2VMUTF97BFS9
  Secret Key: TpxrusS0f42yyYZNuvUni5Shoc9yJH83myWO5gCc
  Default Region: US
  S3 Endpoint: 192.168.10.51:80
  DNS-style bucket+hostname:port template for accessing a bucket: 192.168.10.51:80/%(bucket)s
  Encryption password: 
  Path to GPG program: /usr/bin/gpg
  Use HTTPS protocol: False
  HTTP Proxy server name: 
  HTTP Proxy server port: 0

Test access with supplied credentials? [Y/n] y #測試訪問
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)

Now verifying that encryption works...
Not configured. Never mind.

Save settings? [y/N] y #保持配置
Configuration saved to '/root/.s3cfg #配置保持路徑

​ 3）bucket相關操做

#查看bucket
# s3cmd ls
# 建立test-bucket
# s3cmd mb s3://test-bucket
ERROR: S3 error: 403 (SignatureDoesNotMatch)
#正對找個錯誤，須要在配置文件/root/.s3cfg啓用signature_v2 = True
#上傳單個文件，上傳目錄須要用遞歸方式--recurisve
# s3cmd put /etc/hosts s3://test-bucket/57_hosts
#查看test-bucket裏面存放的文件
# s3cmd ls s3://test-bucket
#從test-bucket下載文件，並確認當前目錄是否有57_hosts_57文件
# s3cmd get s3://test-bucket/57_hosts 57_hosts_57
#刪除文件
# s3cmd rm s3://test-bucket/57_hosts

2.2 使用swift API訪問ceph對象存儲

2.2.1 服務端配置

swift用戶是在現有用戶基礎上建立子用戶

# 查看現有用戶
[root@cephmanager01 ~]# radosgw-admin user list
# 建立swift api子用戶
[root@cephmanager01 ~]# radosgw-admin subuser create --uid=radosgwuser --subuser=radosgwuser:swift --access=full
# --access賦予權限，避免權限問題
#結果注意swift_keys的secret_key，部分版本須要新建生成

2.2.2 客戶端配置

1）安裝python-setuptools和pip

# yum install python-setuptools
# yum install python-pip -y
# 升級python-swiftclient
# pip install --upgrade python-swiftclient
# 若是源有問題，能夠選擇阿里雲的源
# pip install --upgrade python-swiftclient -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
# 確認swift版本
# swift --version

2)查看驗證

# 查看，注意ip、port、用戶名和密鑰，確認輸出了其餘bucket
# swift -A http://192.168.10.51:80/auth/1.0 -U radosgwuser:swift -K '7pqD1mRdOBCm19WO5t7njgKGezEhF2tmepSY8YoZ' list

3）配置環境變量

每次輸入用戶名密鑰ip等信息比較麻煩，建議設置環境變量

# swift list  #會提示你須要定義那些變量
# cat swift_openrc 
export ST_AUTH=http://192.168.10.51:80/auth
export ST_USER=radosgwuser:swift
export ST_KEY=7pqD1mRdOBCm19WO5t7njgKGezEhF2tmepSY8YoZ
# source swift_openrc

4）其餘相關操做

# 新建
# swift post test-swift
# 上傳文件或目錄
# swift upload test-swift /etc/host
# 查看上傳文件內容
# swift list test-swift
# 下載
# swift download test-swift etc/hosts
# 刪除
# swift delete test-swift etc/hosts