Prerequesits

Server with RHEL5 U2 or CentOS5 U2 and ocfs2 from source (/root/src/ocfs2-1.3.9).

Kernelmodule sources in /root/src/ocfs2-1.3.9 from http://oss.oracle.com/projects/ocfs2/files/source/v1.3/ . Build with:

    # configure
    # make
    # make install

RPMS:

• ocfs2-tools-debuginfo-1.3.9-0.1.el5
• ocfs2-tools-1.3.9-0.1.el5
• ocfs2-tools-devel-1.3.9-0.1.el5
• ocfs2console-1.3.9-0.1.el5
• comoonics-pythonosfix-py-0.1-1
• comoonics-bootimage-listfiles-1.3-6.el5
• SysVinit-comoonics-2.86-14.atix.1
• comoonics-cluster-py-0.1-12
• comoonics-cdsl-py-0.2-11
• comoonics-bootimage-1.3-33
• comoonics-release-0.1-1
• comoonics-cs-py-0.1-54
• comoonics-bootimage-initscripts-1.3-5.el5
• comoonics-bootimage-extras-ocfs2-0.1-1
• comoonics-bootimage-extras-xen-0.1-3 (Only needed for xen Guest)

Clusterconfig:

Redhat Cluster.conf

      <?xml version="1.0"?>
      <cluster config_version="3" name="clurhel5">
         <clusternodes>
            <clusternode name="ocfs2-node1" nodeid="1">
                    <com_info>
                            <syslog name="syslog-server"/>
                            <rootvolume name="/dev/xvdc" fstype="ocfs2"/>
                            <eth name="eth0" ip="dhcp" mac="00:16:3E:00:11:12"/>
                            <fenceackserver user="root" passwd="test123"/>
                    </com_info>
            </clusternode>
            <clusternode name="ocfs2-node2" nodeid="2">
                    <com_info>
                            <syslog name="syslog server"/>
                            <rootvolume name="/dev/xvdc" fstype="ocfs2"/>
                            <eth name="eth0" ip="dhcp" mac="00:16:3E:00:11:13"/>
                            <fenceackserver user="root" passwd="test123"/>
                    </com_info>
            </clusternode>
         </clusternodes>
      </cluster>

OCFS2 Cluster.conf

The OCFS2 Cluster.conf will be automatically generated by the initrd from /etc/cluster/cluster.conf. To validate use com-queryclusterconf convert ocfs2 and you'll see the output. It should be somehow like as follows.

If so redirect the output to /etc/ocfs2/cluster.conf in order to setup the filesystem.

        node:
          ip_port = 7777
          ip_address = 192.168.3.74
          number = 0
          name = ocfs2-node1
          cluster = ocfs2

        node:
          ip_port = 7777
          ip_address = 192.168.3.75
          number = 1
          name = ocfs2-node2
          cluster = ocfs2

        cluster:
          node_count = 2
          name = ocfs2

Create the shared root filesystem

The shared root filesystem must be a logical volume on a shared storage device. Use mkfs.ocfs2 to create the OCFS2 filesystem:

    mkfs.ocfs2 -Losr /dev/xvdc

Mount the new filesystem to '/mnt/newroot':

    mount /dev/xvdc /mnt/newroot/  

Copy all data from the local installed RHEL5 root filesystem to the shared root filesystem:

    cp -ax / /mnt/newroot/

Create some directories:

    mkdir /mnt/newroot/proc
    mkdir /mnt/newroot/sys

Create a new cdsl infrastructure on the shared root filesystem:

    com-mkcdslinfrastructure -r /mnt/newroot -i

Mount the local cdsl infrastructure:

    mount --bind /mnt/newroot/cluster/cdsl/1/ /mnt/newroot/cdsl.local/

Make /var hostdependent:

    com-mkcdsl -r /mnt/newroot -a /var 

Make /var/lib shared again:

    com-mkcdsl -r /mnt/newroot -s /var/lib

Make /etc/sysconfig/network hostdependent:

    com-mkcdsl -r /mnt/newroot -a /etc/sysconfig/network

Create /etc/mtab link to '/proc/mounts':

    cd /mnt/newroot/etc/
    rm -f mtab
    ln -s /proc/mounts mtab

Remove cluster network configuration:

    rm -f /mnt/newroot/etc/sysconfig/network-scripts/ifcfg-eth0

Disable kudzu:

    chroot /mnt/newroot/ chkconfig kudzu --del
    chroot /mnt/newroot/ chkconfig o2cb --del
    chroot /mnt/newroot/ chkconfig ocfs2 --del

Modify '/mnt/newroot/etc/fstab':

    #/dev/vg_clurhel5_sr/lv_sharedroot  /           gfs     defaults        0 0
    #LABEL=/boot             /boot                   ext3    defaults        1 2
    devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
    tmpfs                   /dev/shm                tmpfs   defaults        0 0
    proc                    /proc                   proc    defaults        0 0
   sysfs                   /sys                    sysfs   defaults        0 0
   /dev/vg_system/lv_swap  swap                    swap    defaults        0 0

Create shared bootdevice

Mount shared boot:

    mount /dev/xvdb1 /mnt/newroot/boot/

Copy boot:

    cp -a /boot/* /mnt/newroot/boot

create '/mnt/newroot/boot/grub/grub.conf':

    default=0
    timeout=5
    splashimage=(hd0,0)/grub/splash.xpm.gz
    hiddenmenu
    title Red Hat Enterprise Linux Server (2.6.18-92.el5xen)
        root (hd0,0)
        kernel /vmlinuz-2.6.18-92.el5xen ro root=/dev/xvdb console=xvc0
        initrd /initrd-2.6.18-92.el5xen.img
    title Red Hat Enterprise Linux Server (2.6.18-53.el5xen)
        root (hd0,0)
        kernel /vmlinuz-2.6.18-53.el5xen ro root=/dev/xvdb console=xvc0
        initrd /initrd-2.6.18-53.el5xen.img

Invoke grub shell:

    grub

Install grub:

    GNU GRUB  version 0.97  (640K lower / 3072K upper memory)

     [ Minimal BASH-like line editing is supported.  For the first word, TAB
       lists possible command completions.  Anywhere else TAB lists the possible
       completions of a device/filename.]

    grub> device (hd0) /dev/xvdb

    grub> root (hd0,0)
     Filesystem type is ext2fs, partition type 0x83

    grub> setup (hd0)
     Checking if "/boot/grub/stage1" exists... no
     Checking if "/grub/stage1" exists... yes
     Checking if "/grub/stage2" exists... yes
     Checking if "/grub/e2fs_stage1_5" exists... yes
     Running "embed /grub/e2fs_stage1_5 (hd0)"...  15 sectors are embedded.
    succeeded
     Running "install /grub/stage1 (hd0) (hd0)1+15 p     (hd0,0)/grub/stage2 /grub/grub.conf"... succeeded
    Done.

Create Shared Root initrd

Create ocfs2-tools depfile /etc/comoonics/bootimage/rpms.initrd.d/ocfs2.list:

     [root@vgenerix5 ~]# cat /etc/comoonics/bootimage/rpms.initrd.d/ocfs2.list
     ocfs2-tools

Create the shared root initrd into the shared boot filesystem:

    /opt/atix/comoonics-bootimage/mkinitrd -f /mnt/newroot/boot/initrd-$(uname -r).img $(uname -r)

Boot Cluster

Have Fun !!