Setting up RAID with Debian on a Netra X1

If you haven't installed Debian on your Netra X1 yet, take a look here.

I have a number of Netra X1 systems with identical hard drives. In all of my other systems, I try to make the RAID1, at a minumum. I've done a great deal of digging, and I have yet to find anyone else publishing a procedure for setting up RAID1 volumes. In fact, a number of people have said it's impossible. I found a way to implement RAID1 on boot/root volumes. This same procedure could be used to convert any single drive system into a RAID1 pair.


(2006-05-09) UPDATE: There may be a problem with the instructions below. You'll likely want to start the /boot partition at cylinder 1 instead of 0. Silo corrupts the ext2/3 partition when writing the bootblock if you are using RAID on the first cylinder. Apparently the ext2/3 filesystem leaves space for the disklabel and boot sector when it's created on a plain disk, but doesn't when it's created on a RAID volume.

To repair an already built system using my partition layout:

1) apt-get install rsync parted
2) mkdir /boot2 ; cd /boot2; rsync -vaP /boot/. .
3) /etc/init.d/sysklogd stop
4) /etc/init.d/klogd stop
5) umount /boot
6) mdadm --stop /dev/md0
7) (fdisk /dev/hda and /dev/hdc) (delete partition 1, re-create it starting at 1, set type to "fd")
8) partprobe
9) mdadm --create -l1 -n2 -x0 /dev/md0 /dev/hd[ac]1
10) mkfs.ext3 /dev/md0
11) mount /dev/md0 /boot
12) cd /boot ; rsync -vaP /boot2/. .
13) cd /dev; mv hda3 hda3.old; ln -s hda hda3; mv hdc3 hdc3.old; ln -s hdc hdc3
14) silo
15) mkinitrd -o /boot/initrd.img-`uname -r` `uname -r`
16) (wait for RAID to rebuild)
    cd /etc/mdadm ; echo DEVICE partitions > mdadm.conf ; mdadm -Ds >> mdadm.conf
17) reboot
18) at OBP: setenv boot-device disk0:3 disk2:3 net
19) boot
Except for the obvious manual bits, I was cutting-n-pasting the commands above, and have repaired my Blade 100, an Ultra 2 and seven Netra X1's using this procedure. The only odd bit is that "boot disk" doesn't work any longer, hence the need for step 18.


What you need is: The procedure:
  1. Power on your system and boot it:
     lom>poweron
     lom>
     LOM event: +6d+0h0m1s host power on
    
     Sun Netra X1 (UltraSPARC-IIe 400MHz), No Keyboard
     OpenBoot 4.0, 1024 MB memory installed, Serial #50674428.
     Ethernet address 0:3:ba:5:3a:fc, Host ID: 83053afc.
    
    
     ok boot
     Boot device: disk:a  File and args:
     SILO Version 1.4.9
     boot: Linux
  2. Verify that you are working with two identical hard drives:
     netra0:~# cat /proc/ide/hd?/model
     ST320413A
     ST320413A
  3. Copy the partition table over to the second drive:
     dd if=/dev/hda of=/dev/hdc bs=1024k count=10
    Edit the partition table of the second drive so that all of the partitions, except #3, have type 'fd'.
     netra0:~# fdisk -l /dev/hda /dev/hdc
    
     Disk /dev/hda (Sun disk label): 16 heads, 63 sectors, 38792 cylinders
     Units = cylinders of 1008 * 512 bytes
    
        Device Flag    Start       End    Blocks   Id  System
     /dev/hda1             0       248    124992   83  Linux native
     /dev/hda2           248      2232    999936   83  Linux native
     /dev/hda3             0     38792  19551168    5  Whole disk
     /dev/hda4          2232     10169   4000248   83  Linux native
     /dev/hda5         10169     12153    999936   83  Linux native
     /dev/hda6         12153     18105   2999808   82  Linux swap
     /dev/hda7         18105     28026   5000184   83  Linux native
     /dev/hda8         28026     38792   5426064   83  Linux native
    
     Disk /dev/hdc (Sun disk label): 16 heads, 63 sectors, 38792 cylinders
     Units = cylinders of 1008 * 512 bytes
    
        Device Flag    Start       End    Blocks   Id  System
     /dev/hdc1             0       248    124992   fd  Linux raid autodetect
     /dev/hdc2           248      2232    999936   fd  Linux raid autodetect
     /dev/hdc3             0     38792  19551168    5  Whole disk
     /dev/hdc4          2232     10169   4000248   fd  Linux raid autodetect
     /dev/hdc5         10169     12153    999936   fd  Linux raid autodetect
     /dev/hdc6         12153     18105   2999808   fd  Linux raid autodetect
     /dev/hdc7         18105     28026   5000184   fd  Linux raid autodetect
     /dev/hdc8         28026     38792   5426064   fd  Linux raid autodetect
  4. Install mdadm to manage the arrays.
     apt-get install mdadm
    It'll ask you a series of questions that are highly dependent on your needs. One key one is: "Yes, automatically start RAID arrays"
  5. Load the RAID1 module:
     modprobe raid1
  6. Create the RAID1 volumes. Note that we're setting one mirror as "missing" here. We'll add the second half of the mirror later because we're using it right now.
     mdadm --create -n 2 -l 1 /dev/md0 /dev/hdc1 missing
     mdadm --create -n 2 -l 1 /dev/md1 /dev/hdc2 missing
     mdadm --create -n 2 -l 1 /dev/md2 /dev/hdc4 missing
     mdadm --create -n 2 -l 1 /dev/md3 /dev/hdc5 missing
     mdadm --create -n 2 -l 1 /dev/md4 /dev/hdc6 missing
     mdadm --create -n 2 -l 1 /dev/md5 /dev/hdc7 missing
     mdadm --create -n 2 -l 1 /dev/md6 /dev/hdc8 missing
  7. Make the filesystems:
     mke2fs -j /dev/md0
     mke2fs -j /dev/md1
     mke2fs -j /dev/md2
     mke2fs -j /dev/md3
     mkswap /dev/md4
     mke2fs -j /dev/md5
     mke2fs -j /dev/md6
  8. Install the dump package:
     apt-get install dump
  9. Mount the new volumes, dump & restore from the running copies:
     mount /dev/md1 /mnt
     cd /mnt
     dump 0f - / | restore rf -
     rm restoresymtable
    
     mount /dev/md0 /mnt/boot
     cd /mnt/boot
     dump 0f - /boot | restore rf -
     rm restoresymtable
    
     mount /dev/md2 /mnt/usr
     cd /mnt/usr
     dump 0f - /usr | restore rf -
     rm restoresymtable
    
     mount /dev/md3 /mnt/tmp
     cd /mnt/tmp
     dump 0f - /tmp | restore rf -
     rm restoresymtable
    
     mount /dev/md5 /mnt/var
     cd /mnt/var
     dump 0f - /var | restore rf -
     rm restoresymtable
    
     mount /dev/md6 /mnt/export
     cd /mnt/export
     dump 0f - /export | restore rf -
     rm restoresymtable
  10. Set up the chroot environment:
     mount -t proc none /mnt/proc
    
     chroot /mnt /bin/bash
  11. Edit /boot/silo.conf, and change the following line:
     root=/dev/md1
  12. Edit /etc/fstab, and point them to the MD devices:
     # /etc/fstab: static file system information.
     #
     #                
     proc            /proc           proc    defaults        0       0
     /dev/md1        /               ext3    defaults,errors=remount-ro 0       1
     /dev/md0        /boot           ext3    defaults        0       2
     /dev/md6        /export         ext3    defaults        0       2
     /dev/md3        /tmp            ext3    defaults        0       2
     /dev/md2        /usr            ext3    defaults        0       2
     /dev/md5        /var            ext3    defaults        0       2
     /dev/md4        none            swap    sw              0       0
     /dev/hdc        /media/cdrom0   iso9660 ro,user,noauto  0       0
     /dev/fd0        /media/floppy0  auto    rw,user,noauto  0       0
  13. Save the MD information to /etc/mdadm/mdadm.conf:
     echo DEVICE partitions >> /etc/mdadm/mdadm.conf
     mdadm -D -s >> /etc/mdadm/mdadm.conf

    2006-03-01: added the first line here, because it appears that Sarge ships with either no mdadm.conf, or a blank one. Without it, your raid volumes won't start correctly.

  14. Rebuild the initrd (to add the RAID modules, and boot/root RAID startup information):
     mkinitrd -o /boot/initrd.img-`uname -r` `uname -r`
  15. Leave the chroot environment:
     exit
  16. Unmount /boot. klogd uses the System.map file, and we need to kill it to unmount /boot.
     pkill klogd
     # wait a few seconds
     umount /boot
  17. Add /dev/hda1 to /dev/md0 -- the /boot mirror
     mdadm --add /dev/md0 /dev/hda1
     watch cat /proc/mdstat
    Wait until the mirror is complete. CTRL-C to exit watch.
  18. Mount the mirrored /boot:
     umount /mnt/boot
     mount /dev/md0 /boot
  19. Stamp the boot loader onto both disks, and reboot:
     silo && reboot
  20. Assuming it booted up correctly, verify that we're running on the mirrored copies:
     df -h
    If so, add the other partitions into their respective mirrors:
     mdadm --add /dev/md0 /dev/hda1
     mdadm --add /dev/md1 /dev/hda2
     mdadm --add /dev/md2 /dev/hda4
     mdadm --add /dev/md3 /dev/hda5
     mdadm --add /dev/md4 /dev/hda6
     mdadm --add /dev/md5 /dev/hda7
     mdadm --add /dev/md6 /dev/hda8
     watch cat /proc/mdstat
    And wait until the the mirrors are done building.
  21. Edit /etc/mdadm/mdadm.conf and remove any references to the RAID volumes. Refresh the mdadm.conf information:
     mdadm -D -s >> /etc/mdadm/mdadm.conf
  22. Rebuild the initrd one more time. The previous time only included one half of each mirror for root and swap.
     mkinitrd -o /boot/initrd.img-`uname -r` `uname -r`
  23. Reboot one more time for good measure. Congratulations! You now have a Netra X1 running RAID1.
If you're experienced enough, you may be able to find a few ways to take shortcuts. The steps above aren't optimized. There are a few redundant steps, but I thought going through them would more clearly illustrate what I was doing. This entire operation could also be scripted, and if I had more systems to build, I probably would have done so.