Oracle RAC 更換存儲實驗
實驗環境準備：
RHEL 6.5 + Oracle 11.2.0.4 RAC (2nodes)css
- OCR和Voting Disk使用的是OCR1磁盤組，底層對應3個1G大小的共享LUN，通常冗餘；
- DATA使用的是DATA1磁盤組，底層對應的是3個5G大小的共享LUN，外部冗餘；
- FRA使用的是FRA1磁盤組，底層對應的是1個5G大小的共享LUN，外部冗餘；
- 額外添加7個共享LUN(3個1G大小，4個5G大小)，用來模擬新的存儲映射出來的共享LUN；
我這裏是使用Oracle VM VirtualBox 4.3虛擬機軟件模擬這個環境，
默認初始的RAC環境已經安裝完畢，且已經存有業務數據。
對於RAC環境的安裝可參考：html
- Linux平臺 Oracle 11gR2 RAC安裝Part1：準備工做
- Linux平臺 Oracle 11gR2 RAC安裝Part2：GI安裝
- Linux平臺 Oracle 11gR2 RAC安裝Part3：DB安裝
模擬的業務數據可參考 模擬業務最小測試用例01 第1～5步進行初始化。node
初始的RAC環境配置信息：sql
#public ip 192.168.56.150 jyrac1 192.168.56.152 jyrac2 #private ip 10.10.10.11 jyrac1-priv 10.10.10.12 jyrac2-priv #virtual ip 192.168.56.151 jyrac1-vip 192.168.56.153 jyrac2-vip #scan ip 192.168.56.160 jyrac-scan
本次需求：
更換RAC的共享存儲，即把以前的存儲設備劃出的共享LUN上的全部數據所有遷移新的存儲設備劃分的LUN上。
具體就涉及到OCR，voting disk遷移，DATA遷移。
在我這裏實驗具體就是指OCR1，DATA1，FRA1磁盤組的遷移。oracle
1、準備工做ide
2、更換存儲測試
1、準備工做
1.1查看asm磁盤組
select name, TOTAL_MB, FREE_MB, state, type, COMPATIBILITY from v$asm_diskgroup;ui
SQL> select name, TOTAL_MB, FREE_MB, state, type, COMPATIBILITY from v$asm_diskgroup; NAME TOTAL_MB FREE_MB STATE TYPE COMPATIBILITY DATA1 15360 13502 MOUNTED EXTERN 11.2.0.0.0 FRA1 5120 4790 MOUNTED EXTERN 11.2.0.0.0 OCR1 3072 2146 MOUNTED NORMAL 11.2.0.0.0
1.2查看asm的磁盤信息
select GROUP_NUMBER, DISK_NUMBER, TOTAL_MB, FREE_MB, NAME, PATH from v$asm_disk order by 1,2;spa
SQL> set linesize 1000
SQL> r
1* select GROUP_NUMBER, DISK_NUMBER, TOTAL_MB, FREE_MB, NAME, PATH from v$asm_disk order by 1,2
GROUP_NUMBER DISK_NUMBER TOTAL_MB FREE_MB NAME PATH
1 0 1024 714 OCR1_0000 /dev/asm-diskb
1 1 1024 716 OCR1_0001 /dev/asm-diskc
1 2 1024 716 OCR1_0002 /dev/asm-diskd
2 0 5120 4552 DATA1_0000 /dev/asm-diske
2 1 5120 4549 DATA1_0001 /dev/asm-diskf
2 2 5120 4549 DATA1_0002 /dev/asm-diskg
3 0 5120 4790 FRA1_0000 /dev/asm-diskh
7 rows selected.
1.3查看存儲設備信息，肯定新的存儲
這裏實驗環境是使用virtualbox模擬添加7個大小和以前同樣的共享存儲；
即分別爲 1G大小的3個（OCR2），5G大小的3個（DATA1），5G大小的1個（FRA1）。3d
NewStorage1
NewStorage2
NewStorage3
NewStorage4
NewStorage5
NewStorage6
NewStorage7
在系統中就是/dev/sdi j k l m n o
udev綁定（兩個節點root執行）
--New Shared Disks for i in i j k l m n o; do echo "KERNEL==\"sd*\", BUS==\"scsi\", PROGRAM==\"/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/\$name\", RESULT==\"`/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/sd$i`\", NAME=\"asm-disk$i\", OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\"" >> /etc/udev/rules.d/99-oracle-asmdevices.rules done
udev綁定後對應就是asm-diski,j,k,l,m,n,o
start_udev （兩個節點root執行）
查看某個磁盤大小：
[root@jyrac1 ~]# fdisk -l /dev/asm-diski Disk /dev/asm-diski: 1073 MB, 1073741824 bytes 255 heads, 63 sectors/track, 130 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000
肯定/dev/asm-diski，/dev/asm-diskj，/dev/asm-diskk對應的是新加的1G大小的3個盤
用於建立新的OCR2
2、更換存儲
2.1遷移OCR1到OCR2
2.1.1 建立新的OCR2磁盤組
su – grid
sqlplus / as sysasm
CREATE DISKGROUP OCR2 NORMAL REDUNDANCY DISK '/dev/asm-diski', '/dev/asm-diskj', '/dev/asm-diskk' ATTRIBUTE 'compatible.asm'='11.2';
SQL> CREATE DISKGROUP OCR2 NORMAL REDUNDANCY DISK '/dev/asm-diski', '/dev/asm-diskj', '/dev/asm-diskk' ATTRIBUTE 'compatible.asm'='11.2'; Diskgroup created.
查看兩個節點是否添加了OCR2：
--節點1： SQL> r 1* select name, TOTAL_MB, FREE_MB, state, type, COMPATIBILITY from v$asm_diskgroup NAME TOTAL_MB FREE_MB STATE TYPE COMPATIBILITY ------------------------------------------------------------ ---------- ---------- ---------------------- ------------ ------------------------------------------------------------------------------------------------------------------------ DATA1 15360 13502 MOUNTED EXTERN 11.2.0.0.0 FRA1 5120 4790 MOUNTED EXTERN 11.2.0.0.0 OCR1 3072 2146 MOUNTED NORMAL 11.2.0.0.0 OCR2 3072 2913 MOUNTED NORMAL 11.2.0.0.0 --節點2： SQL> select name, TOTAL_MB, FREE_MB, state, type, COMPATIBILITY from v$asm_diskgroup 2 ; NAME TOTAL_MB FREE_MB STATE TYPE COMPATIBILITY ------------------------------------------------------------ ---------- ---------- ---------------------- ------------ ------------------------------------------------------------------------------------------------------------------------ DATA1 15360 13502 MOUNTED EXTERN 11.2.0.0.0 FRA1 5120 4790 MOUNTED EXTERN 11.2.0.0.0 OCR1 3072 2146 MOUNTED NORMAL 11.2.0.0.0 OCR2 0 0 DISMOUNTED 0.0.0.0.0
發現節點2須要手動mount新加的ASM磁盤組
sqlplus / as sysasm
SQL> alter diskgroup OCR2 mount; Diskgroup altered.
2.1.2 添加OCR信息到OCR2
more /etc/oracle/ocr.loc
ocrconfig -add +OCR2
ocrcheck -config
more /etc/oracle/ocr.loc
[root@jyrac1 ~]# more /etc/oracle/ocr.loc
ocrconfig_loc=+OCR1
local_only=FALSE
[root@jyrac1 ~]# ocrconfig -add +OCR2
[root@jyrac1 ~]#
[root@jyrac1 ~]# ocrcheck -config
Oracle Cluster Registry configuration is :
Device/File Name : +OCR1
Device/File Name : +OCR2
[root@jyrac1 ~]# more /etc/oracle/ocr.loc
#Device/file getting replaced by device +OCR2
ocrconfig_loc=+OCR1
ocrmirrorconfig_loc=+OCR2
local_only=false
[root@jyrac1 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3116
Available space (kbytes) : 259004
ID : 1718040627
Device/File Name : +OCR1
Device/File integrity check succeeded
Device/File Name : +OCR2
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE ab583d47c34a4f0ebfd46c8b04d7332b (/dev/asm-diskb) [OCR1]
2. ONLINE 853e95652d994fe3bfdf21ca23e65dfc (/dev/asm-diskc) [OCR1]
3. ONLINE 9f712a096a7b4f9bbfd811bff6462157 (/dev/asm-diskd) [OCR1]
Located 3 voting disk(s).
2.1.3遷移VOTE DISK到新建立的ASM磁盤組OCR2上
crsctl replace votedisk +OCR2 （一個節點root執行）
先查詢：
SQL> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
GROUP_NUMBER NAME STATE TYPE TOTAL_MB FREE_MB VO COMPATIBILITY
------------ ------------------------------------------------------------ ---------------------- ------------ ---------- ---------- -- ------------------------------------------------------------------------------------------------------------------------
1 DATA1 MOUNTED EXTERN 15360 13502 N 11.2.0.0.0
2 FRA1 MOUNTED EXTERN 5120 4790 N 11.2.0.0.0
3 OCR1 MOUNTED NORMAL 3072 2146 Y 11.2.0.0.0
4 OCR2 MOUNTED NORMAL 3072 2244 N 11.2.0.0.0
替換爲新添加的OCR2：
[root@jyrac1 ~]# crsctl replace votedisk +OCR2 Successful addition of voting disk 83f3dea9f4c24fccbf05eaa163c5c087. Successful addition of voting disk ad21c1f0452e4f4abff906127ff9fcc3. Successful addition of voting disk 2771819c65984f85bf366e74ee85781a. Successful deletion of voting disk ab583d47c34a4f0ebfd46c8b04d7332b. Successful deletion of voting disk 853e95652d994fe3bfdf21ca23e65dfc. Successful deletion of voting disk 9f712a096a7b4f9bbfd811bff6462157. Successfully replaced voting disk group with +OCR2. CRS-4266: Voting file(s) successfully replaced
再查詢：
SQL> r
1* select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup
GROUP_NUMBER NAME STATE TYPE TOTAL_MB FREE_MB VO COMPATIBILITY
------------ ------------------------------------------------------------ ---------------------- ------------ ---------- ---------- -- ------------------------------------------------------------------------------------------------------------------------
1 DATA1 MOUNTED EXTERN 15360 13502 N 11.2.0.0.0
2 FRA1 MOUNTED EXTERN 5120 4790 N 11.2.0.0.0
3 OCR1 MOUNTED NORMAL 3072 2242 N 11.2.0.0.0
4 OCR2 MOUNTED NORMAL 3072 2148 Y 11.2.0.0.0
發現已經voting file已經在OCR2上，而後在兩個節點root查詢：
ocrcheck && crsctl query css votedisk
[root@jyrac1 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3116
Available space (kbytes) : 259004
ID : 1718040627
Device/File Name : +OCR1
Device/File integrity check succeeded
Device/File Name : +OCR2
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 83f3dea9f4c24fccbf05eaa163c5c087 (/dev/asm-diski) [OCR2]
2. ONLINE ad21c1f0452e4f4abff906127ff9fcc3 (/dev/asm-diskj) [OCR2]
3. ONLINE 2771819c65984f85bf366e74ee85781a (/dev/asm-diskk) [OCR2]
Located 3 voting disk(s).
[root@jyrac2 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3116
Available space (kbytes) : 259004
ID : 1718040627
Device/File Name : +OCR1
Device/File integrity check succeeded
Device/File Name : +OCR2
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 83f3dea9f4c24fccbf05eaa163c5c087 (/dev/asm-diski) [OCR2]
2. ONLINE ad21c1f0452e4f4abff906127ff9fcc3 (/dev/asm-diskj) [OCR2]
3. ONLINE 2771819c65984f85bf366e74ee85781a (/dev/asm-diskk) [OCR2]
Located 3 voting disk(s).
2.1.4 建立ASM實例spfile到新建立的OCR2 ASM磁盤組上
一個節點grid用戶登陸ASM實例執行：
create pfile='/tmp/asmpfile.ora' from spfile;
create spfile='+OCR2' from pfile='/tmp/asmpfile.ora';
SQL> show parameter pfile
NAME TYPE VALUE
------------------------------------ ---------------------- ------------------------------
spfile string +OCR1/jyrac-scan/asmparameterf
ile/registry.253.919993739
SQL> create pfile='/tmp/asmpfile.ora' from spfile;
File created.
SQL> create spfile='+OCR2' from pfile='/tmp/asmpfile.ora';
File created.
SQL> show parameter pfile
NAME TYPE VALUE
------------------------------------ ---------------------- ------------------------------
spfile string +OCR1/jyrac-scan/asmparameterf
ile/registry.253.919993739
2.1.5 刪除OCR原位置ASM磁盤組OCR1
一個節點root執行：
[root@jyrac1 ~]# ocrconfig -delete +OCR1
兩個節點root執行：
查看OCR和VOTE新狀態與位置
[root@jyrac1 ~]# ocrcheck && crsctl query css votedisk
[root@jyrac1 ~]# ocrconfig -delete +OCR1
[root@jyrac1 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3116
Available space (kbytes) : 259004
ID : 1718040627
Device/File Name : +OCR2
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 83f3dea9f4c24fccbf05eaa163c5c087 (/dev/asm-diski) [OCR2]
2. ONLINE ad21c1f0452e4f4abff906127ff9fcc3 (/dev/asm-diskj) [OCR2]
3. ONLINE 2771819c65984f85bf366e74ee85781a (/dev/asm-diskk) [OCR2]
Located 3 voting disk(s).
節點2的ocrcheck輸出也沒問題，略。
2.1.6 重啓整個集羣
兩個節點root執行：
crsctl stop crs && crsctl start crs
2.1.7 將舊的磁盤組drop掉
檢查參數文件：
SQL> show parameter pfile
NAME TYPE VALUE
------------------------------------ ---------------------- ------------------------------
spfile string +OCR2/jyrac-scan/asmparameterf
ile/registry.253.920008067
檢查磁盤組狀態：
Select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup; select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
最後將舊的磁盤組mount並drop掉：
alter diskgroup OCR1 mount;
drop diskgroup OCR1 including contents;
SQL> alter diskgroup OCR1 mount; Diskgroup altered. SQL> drop diskgroup OCR1 including contents; Diskgroup dropped.
注：舊的磁盤組只容許在一個節點mount，若是發現多個節點mount，須要在其餘節點dismount，不然會刪除不掉舊的磁盤組。
2.2遷移DATA1
2.2.1 數據磁盤組DATA1添加
肯定/dev/asm-diskl，/dev/asm-diskm，/dev/asm-diskn對應的是新加的5G大小的3個盤
用於數據磁盤組DATA1添加
alter diskgroup DATA1 add disk '/dev/asm-diskl' rebalance power 7;
alter diskgroup DATA1 add disk '/dev/asm-diskm', '/dev/asm-diskn' rebalance power 11;
SQL> alter diskgroup DATA1 add disk '/dev/asm-diskl' rebalance power 7; Diskgroup altered. SQL> alter diskgroup DATA1 add disk '/dev/asm-diskm', '/dev/asm-diskn' rebalance power 11; Diskgroup altered.
這裏說明能夠單個添加，也能夠多個添加。
2.2.2 查詢同步狀態
set linesize 200 col NAME for a10 select name,allocation_unit_size,state,type,free_mb,required_mirror_free_mb req_mi_fr_mb,usable_file_mb from v$asm_diskgroup;
2.2.3 刪除原存儲的LUN
全部磁盤添加進ASM磁盤組後，刪除原存儲的LUN。
查詢LUN path與asm_disk_name的對應關係
select name,path from v$asm_disk;
SQL> select name,path from v$asm_disk
NAME PATH
---------- ------------------------------------------------------------
/dev/asm-diskc
/dev/asm-diskd
/dev/asm-disko
/dev/asm-diskb
OCR2_0001 /dev/asm-diskj
OCR2_0000 /dev/asm-diski
DATA1_0003 /dev/asm-diskl
DATA1_0004 /dev/asm-diskm
DATA1_0005 /dev/asm-diskn
OCR2_0002 /dev/asm-diskk
FRA1_0000 /dev/asm-diskh
DATA1_0001 /dev/asm-diskf
DATA1_0002 /dev/asm-diskg
DATA1_0000 /dev/asm-diske
14 rows selected.
肯定是DATA1_0000，DATA1_0001，DATA1_0002
刪除磁盤組DATA1中舊存儲對應的LUN：
alter diskgroup DATA1 drop disk DATA1_0000 rebalance power 7; alter diskgroup DATA1 drop disk DATA1_0001,DATA1_0002 rebalance power 7;
2.2.4 確認存儲置換完成
select disk_number,path,total_mb,free_mb from v$asm_disk;
DISK_NUMBER PATH TOTAL_MB FREE_MB
----------- ------------------------------------------------------------ ---------- ----------
0 /dev/asm-diskc 0 0
1 /dev/asm-diskd 0 0
2 /dev/asm-disko 0 0
3 /dev/asm-diske 0 0
4 /dev/asm-diskf 0 0
5 /dev/asm-diskg 0 0
6 /dev/asm-diskb 0 0
1 /dev/asm-diskj 1024 715
0 /dev/asm-diski 1024 717
3 /dev/asm-diskl 5120 4502
4 /dev/asm-diskm 5120 4499
5 /dev/asm-diskn 5120 4501
2 /dev/asm-diskk 1024 714
0 /dev/asm-diskh 5120 4790
14 rows selected.
2.3遷移FRA1
FRA1磁盤組和DATA1磁盤組的添加同樣，
alter diskgroup FRA1 add disk '/dev/asm-disko' rebalance power 7; alter diskgroup FRA1 drop disk FRA1_0000 rebalance power 7; select disk_number,path,total_mb,free_mb from v$asm_disk;
最後肯定全部盤都置換完成：
SQL> select disk_number,path,total_mb,free_mb from v$asm_disk;
DISK_NUMBER PATH TOTAL_MB FREE_MB
----------- ------------------------------------------------------------ ---------- ----------
0 /dev/asm-diskc 0 0
1 /dev/asm-diskd 0 0
3 /dev/asm-diske 0 0
4 /dev/asm-diskf 0 0
5 /dev/asm-diskg 0 0
6 /dev/asm-diskb 0 0
1 /dev/asm-diskj 1024 715
0 /dev/asm-diski 1024 717
1 /dev/asm-disko 5120 4943
3 /dev/asm-diskl 5120 4502
4 /dev/asm-diskm 5120 4499
5 /dev/asm-diskn 5120 4501
2 /dev/asm-diskk 1024 714
0 /dev/asm-diskh 5120 4965
14 rows selected.
SQL> r
1* select disk_number,path,total_mb,free_mb from v$asm_disk
DISK_NUMBER PATH TOTAL_MB FREE_MB
----------- ------------------------------------------------------------ ---------- ----------
0 /dev/asm-diskc 0 0
1 /dev/asm-diskd 0 0
2 /dev/asm-diskh 0 0
3 /dev/asm-diske 0 0
4 /dev/asm-diskf 0 0
5 /dev/asm-diskg 0 0
6 /dev/asm-diskb 0 0
1 /dev/asm-diskj 1024 715
0 /dev/asm-diski 1024 717
1 /dev/asm-disko 5120 4790
3 /dev/asm-diskl 5120 4502
4 /dev/asm-diskm 5120 4499
5 /dev/asm-diskn 5120 4501
2 /dev/asm-diskk 1024 714
14 rows selected.
最終肯定要替換的LUN都沒有數據了，主機側就能夠操做了。
2.4測試
查看集羣各資源狀態：
crsctl stat res -t
查看OCR信息：
ocrcheck && crsctl query css votedisk
查詢ASM實例的spfile文件：
sqlplus / as sysasm <<EOF show parameter pfile EOF
查詢以前的業務數據：
-- 業務查詢SQL 1 select * from t1, t2 where t1.id = t2.t1_id and t1.n = 19; -- 業務查詢SQL 2 select * from t1, t2 where t1.id = t2.t1_id;
至此，整個RAC環境更換存儲的實驗就所有完成了。
相關文章
- 1. 一步一步教你更換 Oracle 11g RAC 共享存儲
- 2. Oracle rac 基於 NFS存儲的實現
- 3. Oracle RAC 11GR2更換主機不換存儲--ASM磁盤組異機掛載
- 4. oracle RAC 更換存儲遷移數據(在線遷移ASM磁盤組）測試
- 5. Oracle 實驗7 存儲過程
- 6. [Oracle] 實驗二 存儲結構管理
- 7. 【RAC】oracle 10gRAC安裝---oracle RAC
- 8. Exchange2003更換存儲設備
- 9. oracle 存儲函數,更新庫存
- 10. oracle RAC
- 更多相關文章...
- • MySQL存儲過程簡介 - MySQL教程
- • MySQL存儲引擎精講（附帶各種存儲引擎的對比） - MySQL教程
- • 三篇文章瞭解 TiDB 技術內幕——說存儲
- • ☆基於Java Instrument的Agent實現
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
最新文章
歡迎關注本站公眾號,獲取更多信息
