從頭搭建drbd+openfiler+corosync

1. 建立系統要求:

• hostname: filer01
• eth0: 192.168.10.221
• eth1:  192.168.11.221
• 500MB Meta partition
• 4GB+ Data partition

 

• hostname: filer02
• eth0:  192.168.10.223
• eth1:  192.168.11.223
• 500MB Meta partition
• 4GB+ Data partition

virtualip:  192.168.10.248 ( don't use on any adapter, we will make this later with corosync )

1.1 添加到host文件裏面去（filer01 filer02都執行）

192.168.10.221 filer01

192.168.10.223 filer02

 

1.2 創建ssh認證

root@filer01 ~# ssh-keygen -t dsa

Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:

Do the same on filer02.

root@filer02 ~# ssh-keygen -t dsa

Then exchange the files:

root@filer01 ~# scp ~/.ssh/id_dsa.pub root@filer02:~/.ssh/authorized_keys

root@filer02 ~# scp ~/.ssh/id_dsa.pub root@filer01:~/.ssh/authorized_keys

 

2. Create meta/data Partition on both filers

fdisk /dev/sda  建立分區，第一次在擴展分區上面創建，輸入e 而後回車，所有加到擴展分區裏面，而後建立/dev/sda5   dev/sda6

 

2.1 Create DRBD Setup

Edit  /etc/drbd.conf on filer01 and filer02:(配置drbd的主配置文件)

[root@filer02 ~]# cat /etc/drbd.conf 
# You can find an example in  /usr/share/doc/drbd.../drbd.conf.example
include "drbd.d/global_common.conf";
include "drbd.d/*.res";
resource meta {
 on filer01 {
  device /dev/drbd0;
  disk /dev/sda5;
  address 192.168.11.221:7788;
  meta-disk internal;
 }
 on filer02 {
  device /dev/drbd0;
  disk /dev/sda5;
  address 192.168.11.223:7788;
  meta-disk internal;
 }
}
resource data {
 on filer01 {
  device /dev/drbd1;
  disk /dev/sda6;
  address 192.168.11.221:7789;
  meta-disk internal;
 }
 on filer02 {
  device /dev/drbd1;
  disk /dev/sda6;
  address 192.168.11.223:7789;
  meta-disk internal;
 }
}

而後用drbdadm建立meta和data分區，若是第一次建立有錯誤提示的話，能夠先經過下面的方法操做。而後再建立（注意，這兩個分區不能再/etc/fstab分區表中出現。它們是由drbd控制）

dd if=/dev/zero of=/dev/drbdX

root@filer01 ~# drbdadm create-md meta
root@filer01 ~# drbdadm create-md data

root@filer02 ~# drbdadm create-md meta
root@filer02 ~# drbdadm create-md data

Now you can start up drbd with:

service drbd start

on both nodes.

Make one node primary:（建立一個主分區）

root@filer01 ~# drbdsetup /dev/drbd0 primary -o
root@filer01 ~# drbdsetup /dev/drbd1 primary -o

 

2.2 建立文件系統

root@filer01 ~# mkfs.ext3 /dev/drbd0

 

2.2.1 Openfiler to meta-Partition

在filer01上面執行如下腳本

root@filer01 ~# service openfiler stop

==============開始===================== 

#!/bin/bash

 mkdir /meta

 mount /dev/drbd0 /meta

 mv /opt/openfiler/ /opt/openfiler.local

 mkdir /meta/opt

 cp -a /opt/openfiler.local /meta/opt/openfiler

 ln -s /meta/opt/openfiler /opt/openfiler

 rm /meta/opt/openfiler/sbin/openfiler

 ln -s /usr/sbin/httpd /meta/opt/openfiler/sbin/openfiler

 rm /meta/opt/openfiler/etc/rsync.xml

 ln -s /opt/openfiler.local/etc/rsync.xml /meta/opt/openfiler/etc/

 mkdir -p /meta/etc/httpd/conf.d

 service nfslock stop

 umount -a -t rpc-pipefs

 mv /etc/samba/ /meta/etc/

 ln -s /meta/etc/samba/ /etc/samba

 mkdir -p /meta/var/spool

 mv /var/spool/samba/ /meta/var/spool/

 ln -s /meta/var/spool/samba/ /var/spool/samba

 mkdir -p /meta/var/lib

 mv /var/lib/nfs/ /meta/var/lib/

 ln -s /meta/var/lib/nfs/ /var/lib/nfs

 mv /etc/exports /meta/etc/

 ln -s /meta/etc/exports /etc/exports

 mv /etc/ietd.conf /meta/etc/

 ln -s /meta/etc/ietd.conf /etc/ietd.conf

 mv /etc/initiators.allow /meta/etc/

 ln -s /meta/etc/initiators.allow /etc/initiators.allow

 mv /etc/initiators.deny /meta/etc/

 ln -s /meta/etc/initiators.deny /etc/initiators.deny

 mv /etc/proftpd /meta/etc/

 ln -s /meta/etc/proftpd/ /etc/proftpd

 rm /opt/openfiler/etc/httpd/modules

 ln -s /usr/lib64/httpd/modules /opt/openfiler/etc/httpd/modules

 service openfiler start

==============結束===================== 

2.2.4 filer02 Openfiler Configuration

在filer02上面執行如下腳本

[root@filer02 ~]# cat start.sh 

#!/bin/bash

 service openfiler stop

 mkdir /meta

 mv /opt/openfiler/ /opt/openfiler.local

 ln -s /meta/opt/openfiler /opt/openfiler

 service nfslock stop

 umount -a -t rpc-pipefs

 rm -rf /etc/samba/

 ln -s /meta/etc/samba/ /etc/samba

 rm -rf /var/spool/samba/

 ln -s /meta/var/spool/samba/ /var/spool/samba

 rm -rf /var/lib/nfs/

 ln -s /meta/var/lib/nfs/ /var/lib/nfs

 rm -rf /etc/exports

 ln -s /meta/etc/exports /etc/exports

 rm /etc/ietd.conf

 ln -s /meta/etc/ietd.conf /etc/ietd.conf

 rm /etc/initiators.allow

 ln -s /meta/etc/initiators.allow /etc/initiators.allow

 rm /etc/initiators.deny

 ln -s /meta/etc/initiators.deny /etc/initiators.deny

 rm -rf /etc/proftpd

 ln -s /meta/etc/proftpd/ /etc/proftpd

2.3 建立lvm分區

Change the lvm filter in the

/etc/lvm/lvm.conf

file from:

filter = [ "a/.*/" ]

to更改成

filter = [ "a|drbd[0-9]|", "r|.*|" ]

而後傳輸到filer02上面

root@filer01 ~# scp /etc/lvm/lvm.conf root@filer02:/etc/lvm/lvm.conf

After that we can create the actual used stuff:（而後建立lvm分區）

root@filer01 ~# pvcreate /dev/drbd1 root@filer01 ~# vgcreate data /dev/drbd1 root@filer01 ~# lvcreate -L 400M -n filer data