iscsi 構建 Oracle12C RAC 共享存儲

關閉防火牆

[root@ol7-122-rac1 ~]# systemctl stop firewalld
[root@ol7-122-rac1 ~]# systemctl disable firewalld

#防火牆若是沒有關掉,必須打開 iSCSI 所使用的 Port
[root@ol7-122-rac1 ~]# firewall-cmd --permanent --add-port=3260/tcp
[root@ol7-122-rac1 ~]# firewall-cmd --reload

修改 /etc/selinux/config 配置

[root@ehs-rac-01 ~]# sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config
[root@ehs-rac-01 ~]# setenforce 0
[root@ehs-rac-01 ~]# cat /etc/selinux/config | grep -i SELINUX= | grep -v "^#"
SELINUX=disabled

環境信息

server:10.0.7.135
rac1:10.0.7.110
rac2:10.0.7.120
這裏咱們選 server 爲 iscsi 服務器,而後把其上的磁盤共享給 rac1 和 rac2 使用;
:這裏咱們能夠選用這個2個節點任何一臺或者其餘機器做爲 iscsi 服務器;
首先咱們在rac1上添加須要共享的磁盤,也就是rac的磁盤組。這裏我已經在 server 上分配了6塊盤,說明以下:
Ocrvotedisk 3塊 4G
Data 1塊 40G 
FRA_ARC 1 塊 20G
:Data數據盤必須大於或等於 40G
 

server 端安裝 iSCSI

首先安裝管理工具
[root@ehs-server-01 ~]# yum install -y targetcli

啓動 iSCSI 服務node

[root@ehs-server-01 ~]# systemctl start target.service
[root@ehs-server-01 ~]# systemctl enable target.service
Created symlink from /etc/systemd/system/multi-user.target.wants/target.service to /usr/lib/systemd/system/target.service.

在 server 節點上查看linux

[root@ehs-server-01 ~]# fdisk -l

Disk /dev/vda: 42.9 GB, 42949672960 bytes, 83886080 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000ae6a1

   Device Boot      Start         End      Blocks   Id  System
/dev/vda1   *        2048     2099199     1048576   83  Linux
/dev/vda2         2099200    83886079    40893440   8e  Linux LVM

Disk /dev/mapper/rhel-root: 37.6 GB, 37576769536 bytes, 73392128 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mapper/rhel-swap: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/vdb: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/vdc: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/vdd: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/vde: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/vdf: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
以上說明磁盤已經在 server 節點上掛載,下面就利用 iscsi 共享給 rac1 和 rac2 節點
 

配置 iscsi server服務端

a)安裝 scsi-target-utils 已用 yum 安裝完成,不須要操做
[root@ehs-server-01 ~]# rpm -q epel-release
package epel-release is not installed
[root@ehs-server-01 ~]# yum install -y epel-release
[root@ehs-server-01 ~]# rpm -q epel-release
epel-release-7-11.noarch
[root@ehs-server-01 ~]# yum --enablerepo=epel -y install scsi-target-utils libxslt

b)配置 targets.conf ,在文件末尾添加以下內容vim

[root@ehs-server-01 ~]# cat >> /etc/tgt/targets.conf << EOF
<target iqn.2019-09.com.oracle:rac>
    backing-store /dev/vdb
    backing-store /dev/vdc
    backing-store /dev/vdd
    backing-store /dev/vde
    backing-store /dev/vdf
    initiator-address 10.0.7.0/24
    write-cache off
</target>
EOF
iqn 名字可任意
initiator-address 限定 容許訪問的客戶端地址段或具體IP
write-cache off 是否開啓或關閉快取
:裏面添加的就是想要共享給 rac1 和 rac2 的磁盤
 
c)啓動 tgtd
[root@ehs-server-01 ~]# /bin/systemctl restart tgtd.service
[root@ehs-server-01 ~]# systemctl restart target.service
[root@ehs-server-01 ~]# systemctl enable tgtd
Created symlink from /etc/systemd/system/multi-user.target.wants/tgtd.service to /usr/lib/systemd/system/tgtd.service.
[root@ehs-server-01 ~]# tgtadm --lld iscsi --mode target --op show
Target 1: iqn.2019-09.com.oracle:rac
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags: 
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 4295 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /dev/vdb
            Backing store flags: 
        LUN: 2
            Type: disk
            SCSI ID: IET     00010002
            SCSI SN: beaf12
            Size: 4295 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /dev/vdc
            Backing store flags: 
        LUN: 3
            Type: disk
            SCSI ID: IET     00010003
            SCSI SN: beaf13
            Size: 4295 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /dev/vdd
            Backing store flags: 
        LUN: 4
            Type: disk
            SCSI ID: IET     00010004
            SCSI SN: beaf14
            Size: 21475 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /dev/vde
            Backing store flags: 
        LUN: 5
            Type: disk
            SCSI ID: IET     00010005
            SCSI SN: beaf15
            Size: 21475 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /dev/vdf
            Backing store flags: 
    Account information:
    ACL information:
        10.0.7.0/24

3.配置 iscsi 客戶端(全部rac節點)

a)安裝 iscsi-initiator-utils,安裝 iSCSI Client 軟件
[root@ehs-rac-01 ~]# yum install -y iscsi-initiator-utils
[root@ehs-rac-01 ~]# rpm -qa | grep iscsi
iscsi-initiator-utils-iscsiuio-6.2.0.874-10.el7.x86_64
libiscsi-1.9.0-7.el7.x86_64
iscsi-initiator-utils-6.2.0.874-10.el7.x86_64
##重啓客戶端
[root@ehs-rac-01 ~]# systemctl restart iscsid.service

b)配置 initiatorname.iscsi服務器

[root@ehs-rac-01 ~]# vim /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.2019-09.com.oracle:rac
:InitiatorName 值就是 server 上 /etc/tgt/targets.conf 裏配置的值
c)啓動iscsi
[root@ehs-rac-01 ~]# systemctl restart iscsi
[root@ehs-rac-01 ~]# systemctl enable iscsi.service
:若是我安裝操做系統時已經安裝了 iscsi-initiator-utils 軟件包因此不用再另外安裝,直接配置就行。
 

4.共享存儲(全部rac節點)

 
a)經過3260端口查看開放了哪些共享存儲:
[root@ehs-rac-01 ~]# iscsiadm -m discovery -tsendtargets -p 10.0.7.110:3260
10.0.7.110:3260,1 iqn.2019-09.com.oracle:rac
[root@ehs-rac-01 ~]# iscsiadm -m node -T discovery -T iqn.2019-09.com.oracle:rac -p 10.0.7.110:3260
# BEGIN RECORD 6.2.0.874-10
node.name = iqn.2019-09.com.oracle:rac
node.tpgt = 1
node.startup = automatic
node.leading_login = No
iface.hwaddress = <empty>
iface.ipaddress = <empty>
iface.iscsi_ifacename = default
...

b)登陸共享存儲:session

[root@ehs-rac-01 ~]# iscsiadm -m node -T iqn.2019-09.com.oracle:rac -p 10.0.7.110:3260 -l
Logging in to [iface: default, target: iqn.2019-09.com.oracle:rac, portal: 10.0.7.110,3260] (multiple)
Login to [iface: default, target: iqn.2019-09.com.oracle:rac, portal: 10.0.7.110,3260] successful.

c)探測下共享存儲的目錄:oracle

[root@ehs-rac-01 ~]# partprobe
[root@ehs-rac-01 ~]# fdisk -l

Disk /dev/vda: 42.9 GB, 42949672960 bytes, 83886080 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000ae6a1

   Device Boot      Start         End      Blocks   Id  System
/dev/vda1   *        2048     2099199     1048576   83  Linux
/dev/vda2         2099200    83886079    40893440   8e  Linux LVM

Disk /dev/mapper/rhel-root: 37.6 GB, 37576769536 bytes, 73392128 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mapper/rhel-swap: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/sdb: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/sda: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/sdc: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/sdd: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/sde: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

d)退出共享存儲目錄:app

[root@ehs-rac-01 ~]# iscsiadm -m node -T iqn.2019-09.com.oracle:rac -p 10.0.7.110:3260 --logout
Logging out of session [sid: 1, target: iqn.2019-09.com.oracle:rac, portal: 10.0.7.110,3260]
Logout of [sid: 1, target: iqn.2019-09.com.oracle:rac, portal: 10.0.7.110,3260] successful.

# fdisk -l (將查看不到共享目錄)
[root@ehs-rac-01 ~]# fdisk -l

Disk /dev/vda: 42.9 GB, 42949672960 bytes, 83886080 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000ae6a1

   Device Boot      Start         End      Blocks   Id  System
/dev/vda1   *        2048     2099199     1048576   83  Linux
/dev/vda2         2099200    83886079    40893440   8e  Linux LVM

Disk /dev/mapper/rhel-root: 37.6 GB, 37576769536 bytes, 73392128 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mapper/rhel-swap: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

e)刪除共享存儲目錄:tcp

[root@ehs-rac-01 ~]# systemctl restart iscsi.service
[root@ehs-rac-01 ~]# iscsiadm -m node -T iqn.2019-09.com.oracle:rac -p 10.0.7.110:3260 --logout
[root@ehs-rac-01 ~]# iscsiadm -m node -T iqn.2019-09.com.oracle:rac -p 10.0.7.110:3260 -o delete

f)從新服務並從新建立登陸:ide

[root@ehs-rac-01 ~]# systemctl restart iscsi.service
[root@ehs-rac-01 ~]# iscsiadm -m discovery -t sendtargets -p 10.0.7.110:3260
[root@ehs-rac-01 ~]# iscsiadm -m node -T iqn.2019-09.com.oracle:rac -p 10.0.7.110:3260 -l

g)爲共享存儲分區建立目錄工具

# fdisk -l 咱們發現多了一個sdb