Oracle 11g RAC環境下Private IP修改方法及異常處理

Oracle 11g RAC環境下Private IP修改方法及異常處理

Oracle 11g RAC環境下Private IP修改方法及異常處理
1、 修改方法

1. 確認全部節點CRS服務以啓動
# olsnodes -s -n –i
host1 1 host1-vip Active
host2 2 host2-vip Active
2. 修改Private IP配置信息

若是以前只有一個私有網卡，則直接刪除時會報錯，如：PRIF-31: Failed to delete the
specified network interface because it is the last private interface.因此若是隻是修改Private IP
而不用更改網卡時，能夠參考如下操做命令：
oifcfg getif //查看Private IP配置
oifcfg setif -global en1/10.10.12.0:cluster_interconnect //添加en1新的Private IP
oifcfg delif -global en1/10.10.11.0 //刪除en1舊的Private IP
oifcfg getif //查看Private IP配置

若是須要更改Private IP網卡時，能夠參考以下命令：
oifcfg getif //查看Private IP配置
oifcfg setif -global en2/10.10.12.0:cluster_interconnect //增長新的en2 上的Private IP
oifcfg delif -global en1 //刪除舊的en1上的Private IP
oifcfg getif //查看Private IP配

3. 關閉CRS服務（全部節點都要執行）：
參考命令：
#crsctl stop crs -f

4. 修改私有網絡物理網卡IP地址和/etc/hosts文件（全部節點都要執行）：
參考命令：
# smit tcpip-> Minimum Configuration & Startup->en1
#vi /etc/hosts

5. 啓動和查看CRS服務
參考命令：
#crsctl start crs
#crs_stat -t

2、異常處理
1. 問題描述
若是使用oifcfg setif命令修改Private IP配置信息時，subnet與實際配置的物理IP不致，
則重啓CRS服務時會致使CRS沒法啓動，以下所示：
oifcfg setif命令配置：
oifcfg setif -global en1/10.10.12.0:cluster_interconnect //subnet爲10.10.12.0
物理IP配置：
10.10.11.61/255.255.255.0 //subnet爲10.10.11.0

日誌錯誤信息：
Alert*.log
2013-12-08 12:38:41.795
[ohasd(8388676)]CRS-2765:Resource 'ora.crsd' has failed on server 'host1'.
2013-12-08 12:38:43.189
[crsd(11141158)]CRS-0804:Cluster Ready Service aborted due to Oracle Cluster Registry error
[PROC-44: Error in network address and interface operations Network address and interface
operations error [7]]. Details at (:CRSD00111:) in
/u01/app/grid/product/11.2.0/grid/log/host1/crsd/crsd.log.
2013-12-08 12:38:43.878
[ohasd(8388676)]CRS-2765:Resource 'ora.crsd' has failed on server 'host1'.
2013-12-08 12:38:43.879
[ohasd(8388676)]CRS-2771:Maximum restart attempts reached for resource 'ora.crsd'; will not
restart.

Crsd.log
2013-12-08 12:38:42.985: [ OCRAPI][1]clsu_get_private_ip_addresses: no ip addresses found.
[ OCRAPI][1]a_init_clsss: failed to call clsu_get_private_ip_addr (7)
2013-12-08 12:38:43.188: [ OCRAPI][1]a_init:13!: Clusterware init unsuccessful : [44]
2013-12-08 12:38:43.188: [ CRSOCR][1] OCR context init failure. Error: PROC-44: Error in
network address and interface oper
ations Network address and interface operations error [7]
2013-12-08 12:38:43.189: [ CRSMAIN][1] Created alert : (:CRSD00111:) : Could not init OCR,
error: PROC-44: Error in network address and interface operations Network address and interface operations error [7]
2013-12-08 12:38:43.189: [ CRSD][1][PANIC] CRSD exiting: Could not init OCR, code: 44
2013-12-08 12:38:43.189: [ CRSD][1] Done.

2. 處理方法
可使用gpnptool命令處理以上問題。GPNPD（Greater Pittsburgh Nonprofit Partnership）
進程的主要的做用是在集羣各個節點中同步GPnP profile文件，在Clusterware中，CSS、GPnP
等服務的啓動都須要依賴於GPnP profile文件。而GPnP profile文件是一個xml文件，它存
儲 的 位 置 是 在 ： $GRID_HOME/gpnp//profile/peer/profile.xml ，
$GRID_HOME/gpnp/profile/peer/profile.xml（全局備份，最原始的配置信息）。GPnPD進程寫
的trace文件會存放在$GRID_HOME/log//gpnpd/gpnpd.log。
有幾個重要的信息存儲在GPnP profile文件:
? 網絡接口和IP地址(公網和私網)
? ASM diskstring和spfile信息
GPnP profile保存的是RAC的配置信息，包括集羣名稱、網絡類型信息（public/private）、
ASM和CSS的存儲信息、安全的數字簽名，以及ASM實例的SPFILE文件位置。當集羣配置
發生變化時，全部節點的該文件會被自動更新。在安裝、系統引導或者當使用標準的集羣工
具更新期間，這些活動包括：oifcfg 改變網絡信息、crsctl 改變css設備、ASM額外的存儲
等，會經過gpdpd進程複製GPnP profile到全部的其餘節點。
當集羣啓動的時候，Cluster Synchronization Services (CSS)將掃描全部的ASM disks，它利
用的是GPnP profile文件中的ASM discovery string。以下所示:

egistry.253.790450611"/>
這裏能夠看到spfile文件是存放在ASM卷組+DATA中的。可是有一個須要注意到事情是
咱們啓動ASM的時候，須要spfile文件，Oracle從GPnP profile中知道spfile的路徑，而後
它就會從底層磁盤中直接讀取spfile標識，啓動asm實例。
若是GPnP出現問題，可使用cluvfy comp gpnp組件驗證命令檢查在集羣中全部節點
網格即插即用的完整性：cluvfy comp gpnp [-n node_list] [-verbose]
最後，注意Oracle不支持手動修改profile.xml文件，直接對它的修改可能致使沒法修復
的問題，最終致使在全部節點重建Clusterware。

2.1 以排他模式和不啓動crsd進程的方式啓動crs
# crsctl start crs -excl –nocrs

2.2 備份crs配置信息
#mkdir /u01/gpnp
#gpnptool get -o=/u01/gpnp/profile.xml
# more /u01/gpnp/profile.xml

xmlns="http://www.grid-pnp.org/2005/11/gpnp-profile" x
mlns:gpnp="http://www.grid-pnp.org/2005/11/gpnp-profile"
xmlns:orcl="http://www.oracle.com/gpnp/2005/11/gpnp-profile" xmlns:x
si="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.grid-pnp.org/2005/11/gpnp-profile gpnp-profile.
xsd" ProfileSequence="22" ClusterUId="8c32638bd2727ff8bfba9eec919fb4e4"
ClusterName="host-scan" PALocation="">
rofile>
IP="192.168.101.0" Use="public"/>
etwork id="net3" Adapter="en1" IP="10.10.12.0"
Use="cluster_interconnect"/>
S-Profile id="css" DiscoveryString="+asm" LeaseDuration="400"/>
DiscoveryString="" SPFile="+DATA/ho
st-scan/asmparameterfile/registry.253.832710839"/>
xmlns:ds="http://www.w3.org/2000/09/xmldsig#">>

Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
Algorithm="http://www.w3.
org/2000/09/xmldsig#rsa-sha1"/>
Algorithm="http://www.w3.org/2000/09/xmldsig
#enveloped-signature"/>http://www.w3.org/2001/10/xml-exc-c14n#">
www.w3.org/2001/10/xml-exc-c14n#" PrefixList="gpnp orcl
xsi"/>://www.w3.org/2000/09/xmldsig#sha1"/>h2DxPI2dIQbGBji8BAAdimWLkkA=
s:DigestValue>
nfo>O1haZSJ3N6gC7B5blNK/0PeljlfjBoWODejCLCOOsqmb46eJM2TvhIoD7gV
SVF2P0JKzXblsGlqkPpbfgzfasVpS/3nyowy/oMlfXI
KloO445gzImd/lL1FCsvsZpe+GdqtvKgKaSKNJENHF/Ht4w9YCz6Gz6hhXi8u5n3jGU5E=
eValue>

查看CRS的配置信息：
# gpnptool get

Warning: some command line parameters were defaulted. Resulting command line:
/u01/app/grid/product/11.2.0/grid/bin/gpnptool.bin get -o-

xmlns="http://www.grid-pnp.org/2005/11/gpnp-profile"
xmlns:gpnp="http://www.grid-pnp.org/2005/11/gpnp-profile"
xmlns:orcl="http://www.oracle.com/gpnp/2005/11/gpnp-profile"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.grid-pnp.org/2005/11/gpnp-profile gpnp-profile.xsd"
ProfileSequence="22" ClusterUId="8c32638bd2727ff8bfba9eec919fb4e4"
ClusterName="host-scan" PALocation="">
HostName="*">
Use="public"/>
Use="cluster_interconnect"/>
id="css" DiscoveryString="+asm" LeaseDuration="400"/>
DiscoveryString=""
SPFile="+DATA/host-scan/asmparameterfile/registry.253.832710839"/>
xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/>
URI="">
Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">
xmlns="http://www.w3.org/2001/10/xml-exc-c14n#" PrefixList="gpnp orcl
xsi"/>
Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>h2DxPI2dIQbGBji8BA
AdimWLkkA=O1haZSJ3N
6gC7B5blNK/0PeljlfjBoWODejCLCOOsqmb46eJM2TvhIoD7gVSVF2P0JKzXblsGlqkPpbfgzfasVpS/3n
yowy/oMlfXIKloO445gzImd/lL1FCsvsZpe+GdqtvKgKaSKNJENHF/Ht4w9YCz6Gz6hhXi8u5n3jGU5E=

Success.

備註：備份的profile.xml文件配置信息與‘gpnptool get’顯示配置信息徹底相同，同時也
與$ORACLE_HOME /gpnp//profiles/peer下的profile.xml文件信息相同。

2.3 修改備份的CRS配置信息
2.3.1 備份配置文件：
# cp /u01/gpnp/profile.xml /u01/gpnp/p.xml

2.3.2 獲取當前的序列號（每次修改並寫回crs中會有一個序列號做爲標識）：
# gpnptool getpval -p=/u01/gpnp/p.xml -prf_sq -o-
22

2.3.3 獲取共有網絡和私有網絡標識id （與實際網卡名稱不一致，能夠在配置文件中找到）：
# gpnptool getpval -p=/u01/gpnp/p.xml -net -o-
net1 net3

2.3.4 修改配置文件中的序列號（原序列號值加1）和私網的實際網段（subnet）信息：
# gpnptool edit -p=/u01/gpnp/p.xml -o=/u01/gpnp/p.xml -ovr -prf_sq=23 -net3:net_ip=10.10.11.0
Resulting profile written to "/u01/gpnp/p.xml".
Success.
修改網卡名稱
#gpnptool edit -p=/home/app/11.2.0.3/grid/gpnp/profiles/peer/profile.xml -o=/home/app/11.2.0.3/grid/gpnp/profiles/peer/profile.xml -ovr -prf_sq=5 net3:net_ada=em3

2.3.5 查看配置文件是否修改爲功：
# more /u01/gpnp/p.xml

xmlns="http://www.grid-pnp.org/2005/11/gpnp-profile" x
mlns:gpnp="http://www.grid-pnp.org/2005/11/gpnp-profile"
xmlns:orcl="http://www.oracle.com/gpnp/2005/11/gpnp-profile" xmlns:x
si="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.grid-pnp.org/2005/11/gpnp-profile gpnp-profile.
xsd" ProfileSequence="23" ClusterUId="8c32638bd2727ff8bfba9eec919fb4e4"
ClusterName="host-scan" PALocation="">
rofile>
IP="192.168.101.0" Use="public"/>
etwork id="net3" Adapter="en1" IP="10.10.11.0"
Use="cluster_interconnect"/>
S-Profile id="css" DiscoveryString="+asm" LeaseDuration="400"/>
DiscoveryString="" SPFile="+DATA/ho
st-scan/asmparameterfile/registry.253.832710839"/>
xmlns:ds="http://www.w3.org/2000/09/xmldsig#">>

Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
Algorithm="http://www.w3.
org/2000/09/xmldsig#rsa-sha1"/>
Algorithm="http://www.w3.org/2000/09/xmldsig
#enveloped-signature"/>http://www.w3.org/2001/10/xml-exc-c14n#">
www.w3.org/2001/10/xml-exc-c14n#" PrefixList="gpnp orcl
xsi"/>://www.w3.org/2000/09/xmldsig#sha1"/>h2DxPI2dIQbGBji8BAAdimWLkkA=
s:DigestValue>
nfo>O1haZSJ3N6gC7B5blNK/0PeljlfjBoWODejCLCOOsqmb46eJM2TvhIoD7gV
SVF2P0JKzXblsGlqkPpbfgzfasVpS/3nyowy/oMlfXI
KloO445gzImd/lL1FCsvsZpe+GdqtvKgKaSKNJENHF/Ht4w9YCz6Gz6hhXi8u5n3jGU5E=
eValue>
備註：紅色標識改動信息。

2.3.6 驗證序列號是否修改完成：
# gpnptool getpval -p=/u01/gpnp/p.xml -prf_sq -o-
23

2.3.7 用私鑰從新標識配置文件：
# gpnptool sign -p=/u01/gpnp/p.xml -o=/u01/gpnp/p.xml -ovr -w=cw-fs:peer
Resulting profile written to "/u01/gpnp/p.xml".
Success.

2.3.8 查看配置文件改動：
# more /u01/gpnp/p.xml

xmlns="http://www.grid-pnp.org/2005/11/gpnp-profile" x
mlns:gpnp="http://www.grid-pnp.org/2005/11/gpnp-profile"
xmlns:orcl="http://www.oracle.com/gpnp/2005/11/gpnp-profile" xmlns:x
si="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.grid-pnp.org/2005/11/gpnp-profile gpnp-profile.
xsd" ProfileSequence="23" ClusterUId="8c32638bd2727ff8bfba9eec919fb4e4"
ClusterName="host-scan" PALocation="">
rofile>
IP="192.168.101.0" Use="public"/>
etwork id="net3" Adapter="en1" IP="10.10.11.0"
Use="cluster_interconnect"/>
S-Profile id="css" DiscoveryString="+asm" LeaseDuration="400"/>
DiscoveryString="" SPFile="+DATA/ho
st-scan/asmparameterfile/registry.253.832710839"/>
xmlns:ds="http://www.w3.org/2000/09/xmldsig#">>

Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
Algorithm="http://www.w3.
org/2000/09/xmldsig#rsa-sha1"/>
Algorithm="http://www.w3.org/2000/09/xmldsig
#enveloped-signature"/>http://www.w3.org/2001/10/xml-exc-c14n#">
www.w3.org/2001/10/xml-exc-c14n#" PrefixList="gpnp orcl
xsi"/>://www.w3.org/2000/09/xmldsig#sha1"/>6p7GJFTV5ZoPtIqkUEudEvYnHB4=
s:DigestValue>
nfo>QNcNKDq1nbXDh1Htp8DYklSba6jzoYnSruNeJdLi6f9TALAQrLsrSAd6dYW
af4V7f2hGCb9qBGvr1pibl4JUeAnN7BBhgWOKyfwYKE
Bx2FdP2RX5tPkLZC+k2X/PO3SpFfIH7NKbvyqmz5xiso2i5C134ZG0RF9s752ZU0e2jVk=
eValue>
備註：紅色標識改動信息。

2.3.9 回寫配置文件信息到crs中：
# gpnptool put -p=/u01/gpnp/p.xml

2.3.10 查看crs中配置信息：
# gpnptool find -c=host-scan //host-scan爲RAC的scan name;
Found 1 instances of service 'gpnp'.
mdns:service:gpnp._tcp.local.://host1:30391/agent=gpnpd,cname=host-scan,host=host1,pid=12
648872/gpnpd h:host1 c:host-scan

# gpnptool rget -h=host1 //host1爲主機名
Warning: some command line parameters were defaulted. Resulting command line:
/u01/app/grid/product/11.2.0/grid/bin/gpnptool.bin rget -h=host1 -o-
Found 1 gpnp service instance(s) to rget profile from.
RGET from tcp://host1:30391
(mdns:service:gpnp._tcp.local.://host1:30391/agent=gpnpd,cname=host-scan,host=host1,pid=12
648872/gpnpd h:host1 c:host-scan):

xmlns="http://www.grid-pnp.org/2005/11/gpnp-profile" xmlns:gpnp="http://www.grid-pnp.org/2005/11/gpnp-profile" xmlns:orcl="http://www.oracle.com/gpnp/2005/11/gpnp-profile" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.grid-pnp.org/2005/11/gpnp-profile gpnp-profile.xsd" ProfileSequence="23" ClusterUId="8c32638bd2727ff8bfba9eec919fb4e4" ClusterName="host-scan" PALocation="">HostName="*">Use="public"/>Use="cluster_interconnect"/>id="css" DiscoveryString="+asm" LeaseDuration="400"/>DiscoveryString="" SPFile="+DATA/host-scan/asmparameterfile/registry.253.832710839"/>xmlns:ds="http://www.w3.org/2000/09/xmldsig#"> Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/> Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/> URI="">Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/> Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"> xmlns="http://www.w3.org/2001/10/xml-exc-c14n#" PrefixList="gpnp orcl xsi"/>Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>6p7GJFTV5ZoPtIqkUEudEvYnHB4=QNcNKDq1nbXDh1Htp8DYklSba6jzoYnSruNeJdLi6f9TALAQrLsrSAd6dYWaf4V7f2hGCb9qBGvr1pibl4JUeAnN7BBhgWOKyfwYKEBx2FdP2RX5tPkLZC+k2X/PO3SpFfIH7NKbvyqmz5xiso2i5C134ZG0RF9s752ZU0e2jVk= Success. # 2.3.11 啓動crsd進程： # crsctl start res ora.crsd -init CRS-2672: Attempting to start 'ora.crsd' on 'host1' CRS-2676: Start of 'ora.crsd' on 'host1' succeeded # crs_stat -t CRS-0184: Cannot communicate with the CRS daemon. 2.3.12 查看和修改私有網絡配置： # oifcfg getif en0 192.168.101.0 global public en1 10.10.11.0 global cluster_interconnect 如不正確使用下面命令從新修改： # oifcfg setif -global en1/10.10.11.0:cluster_interconnect # oifcfg getif en0 192.168.101.0 global public en1 10.10.11.0 global cluster_interconnect 2.3.13 重啓crs服務： # crsctl stop crs -f # crsctl start crs CRS-4123: Oracle High Availability Services has been started. # crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.DATA.dg ora....up.type ONLINE ONLINE host1 ora....ER.lsnr ora....er.type ONLINE ONLINE host1 ora....N1.lsnr ora....er.type ONLINE ONLINE host1 ora.asm ora.asm.type ONLINE ONLINE host1 ora.cvu ora.cvu.type ONLINE ONLINE host1 ora.gsd ora.gsd.type OFFLINE OFFLINE ora....SM1.asm application ONLINE ONLINE host1 ora....T1.lsnr application ONLINE ONLINE host1 ora.host1.gsd application OFFLINE OFFLINE ora.host1.ons application ONLINE ONLINE host1 ora.host1.vip ora....t1.type ONLINE ONLINE host1 ora.host2.vip ora....t1.type ONLINE ONLINE host1 ora....network ora....rk.type ONLINE ONLINE host1 ora.oc4j ora.oc4j.type ONLINE OFFLINE ora.ons ora.ons.type ONLINE ONLINE host1 ora....ry.acfs ora....fs.type ONLINE ONLINE host1 ora.scan1.vip ora....ip.type ONLINE ONLINE host1 ora.yxdb.db ora....se.type ONLINE ONLINE host1 # # oifcfg getif en0 192.168.101.0 global public en1 10.10.11.0 global cluster_interconnect 2.4 使用ocr自動備份不能恢復以上問題 如下爲使用ocrconfig –restore方法進行恢復過程記錄： # ocrconfig -showbackupp PROT-26: Oracle Cluster Registry backup locations were retrieved from a local copy host1 2013/12/08 10:00:35 /u01/app/grid/product/11.2.0/grid/cdata/host-scan/backup00.ocr host1 2013/12/08 06:00:34 /u01/app/grid/product/11.2.0/grid/cdata/host-scan/backup01.ocr host1 2013/12/05 15:59:07 /u01/app/grid/product/11.2.0/grid/cdata/host-scan/backup02.ocr host1 2013/12/08 06:00:34 /u01/app/grid/product/11.2.0/grid/cdata/host-scan/day.ocr host1 2013/11/29 00:51:41 /u01/app/grid/product/11.2.0/grid/cdata/host-scan/week.ocr PROT-25: Manual backups for the Oracle Cluster Registry are not available # ocrconfig -restore /u01/app/grid/product/11.2.0/grid/cdata/host-scan/backup00.ocr PROT-35: The configured Oracle Cluster Registry locations are not accessible # crsctl start crs CRS-4123: Oracle High Availability Services has been started. # ocrconfig -restore /u01/app/grid/product/11.2.0/grid/cdata/host-scan/backup00.ocr # crsctl stop crs -f # crsctl start crs CRS-4123: Oracle High Availability Services has been started. # crs_stat -t CRS-0184: Cannot communicate with the CRS daemon. # crsctl start crs CRS-4123: Oracle High Availability Services has been started. # crs_stat -t CRS-0184: Cannot communicate with the CRS daemon.