HOWTO setup freebsd vinum raid1 on amd64
From Chaos
Contents |
Summary
Special thanks to this HowTo for the initial information. I have condensed things down into simple steps you need to take to get Vinum with 2 hard drives working in a RAID-1 setup running FreeBSD 5.3-RELEASE+AMD64.
Hardware Specs
- HP Proliant DL145
- 2 x Opteron 244 Processors (1.8GHz)
- 2GB of pc2700 DDR RAM (ECC capable)
- 2 x 146GB SCSI 10,000 RPM Hard Drives (non-hotplug)
Caveat
There is one major caveat to the way I have things setup. Since I have identical disk drives, it was a lot easier to setup the bsdlabels. I had to just copy/paste things around, and I was up and running in no time. If you are using 2 disks of varying capacity, be warned that you increase the risk of messing something up (due to miscalculations in partition sizes, offsets, and partition names). This how-to also assumes you are quite familiar with FreeBSD, and you are running 5.3-RELEASE. This setup was initially done in an x86-64 environment, but should work x86-32.
Details
Preparation
1. Install FreeBSD with whatever software packages you want on the 'primary' drive. This will be (effectively) the source drive, which I will refer to as da0. When you partition, make sure and do swap first since we will need to shrink it later.
2. Reboot into single user mode & run the following:
| root@localhost:~# mount -u / |
3. Run '/stand/sysinstall' and format/partition the 'secondary drive' (da1) identically to the way da0 is setup. This will save you some headaches later. Make sure to enable booting off of it also.
4. Reboot again into single user mode & run the following:
| root@localhost:~# mount -u / |
Primary Drive Setup
1. Edit /boot/loader.conf, adding the following lines. Save and exit the file.
geom_vinum_load="YES" vinum.autostart="YES"
2. Run 'bsdlabel -e da0s1'. Change 'b' (swap) to 281 sectors less than it currently is, and put 281 in the offset. Take the size value of 'c', and create another partition, 'h', subtract 16 sectors from it, and make the offset 16 and fstype vinum.
# /dev/da0s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 4194304 12582912 4.2BSD 2048 16384 28528 b: 12582631 281 swap c: 286744122 0 unused 0 0 # "raw" part, don't edit d: 12582912 16777216 4.2BSD 2048 16384 28528 e: 12582912 29360128 4.2BSD 2048 16384 28528 f: 244801082 41943040 4.2BSD 2048 16384 28528 h: 286744106 16 vinum
3. Save and exit the file. Also take note of the size/offset value for each partition, you will need it later.
4. Edit /etc/vinum.conf (this is for the main drive, da0). Note that you must subtract 16 sectors from each of the driveoffset fields. Mine looks something like this:
drive drive0 device /dev/da0s1h
volume root
plex org concat
sd len 4194304s driveoffset 12582896s drive drive0
volume swap
plex org concat
sd len 12582631s driveoffset 265s drive drive0
volume tmp
plex org concat
sd len 12582912s driveoffset 16777200s drive drive0
volume var
plex org concat
sd len 12582912s driveoffset 29360112s drive drive0
volume usr
plex org concat
sd len 244801082s driveoffset 41943024s drive drive0
You can choose any name you wish for 'drive0', just make sure they all match, and make some sense. 'len' is the 'size' field from our bsdlabel above.
5. Run 'gvinum':
| root@localhost:~# gvinum |
6. At the gvinum prompt, type 'list'. You should see something like this:
gvinum -> list 0 drives: 0 volumes: 0 plexes: 0 subdisks:
This is fine and dandy. To load our config, do the following, everything should come up, and the output should be something like this:
gvinum -> create -f /etc/vinum.conf 1 drives: D drive0 State: up /dev/da0s1h A: 0/140011 MB (0%) 5 volumes: V usr State: up Plexes: 2 Size: 116 GB V var State: up Plexes: 2 Size: 6144 MB V tmp State: up Plexes: 2 Size: 6144 MB V swap State: up Plexes: 2 Size: 6143 MB V root State: up Plexes: 2 Size: 2048 MB 5 plexes: P usr.p0 C State: up Subdisks: 1 Size: 116 GB P var.p0 C State: up Subdisks: 1 Size: 6144 MB P tmp.p0 C State: up Subdisks: 1 Size: 6144 MB P swap.p0 C State: up Subdisks: 1 Size: 6143 MB P root.p0 C State: up Subdisks: 1 Size: 2048 MB 5 subdisks: S usr.p0.s0 State: up D: drive0 Size: 116 GB S var.p0.s0 State: up D: drive0 Size: 6144 MB S tmp.p0.s0 State: up D: drive0 Size: 6144 MB S swap.p0.s0 State: up D: drive0 Size: 6143 MB S root.p0.s0 State: up D: drive0 Size: 2048 MB
7. Exit out of gvinum, and fsck all of our new volumes (swap doesn't need to be fsck'd):
| root@localhost:~# fsck -n -t ufs /dev/gvinum/root |
| root@localhost:~# fsck -n -t ufs /dev/gvinum/tmp |
| root@localhost:~# fsck -n -t ufs /dev/gvinum/var |
| root@localhost:~# fsck -n -t ufs /dev/gvinum/usr |
8. Edit /etc/fstab to reflect our new setup:
# Device Mountpoint FStype Options Dump Pass# /dev/gvinum/swap none swap sw 0 0 /dev/gvinum/root / ufs rw 1 1 /dev/gvinum/tmp /tmp ufs rw 2 2 /dev/gvinum/usr /usr ufs rw 2 2 /dev/gvinum/var /var ufs rw 2 2 /dev/cd0 /cdrom cd9660 ro,noauto 0 0
9. Reboot into multi-user mode, and issue the following to make sure all is well:
root@localhost:~# mount root@localhost:~# gvinum list
Secondary Drive Setup
1. Reboot into single-user mode & run the following. It's now time to setup the secondary (da1) drive.
root@localhost:~# mount -u /
2. Run 'bsdlabel -e da1s1'. Make it reflect our previous (da0) setup:
# /dev/da1s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 4194304 12582912 4.2BSD 0 0 0 b: 12582631 281 swap c: 286744122 0 unused 0 0 # "raw" part, don't edit d: 12582912 16777216 4.2BSD 0 0 0 e: 12582912 29360128 4.2BSD 0 0 0 f: 244801082 41943040 4.2BSD 0 0 0 h: 286744106 16 vinum
3. Save and exit the file. Also take note of the size/offset value for each partition, you will need it later.
4. Now, lets configure /etc/vinum-mirror.conf:
drive drive1 device /dev/da1s1h
volume root
plex org concat
sd len 4194304s driveoffset 12582896s drive drive1
volume swap
plex org concat
sd len 12582631s driveoffset 265s drive drive1
volume tmp
plex org concat
sd len 12582912s driveoffset 16777200s drive drive1
volume var
plex org concat
sd len 12582912s driveoffset 29360112s drive drive1
volume usr
plex org concat
sd len 244801082s driveoffset 41943024s drive drive1
Again, you don't need to use drive1, I just picked it because it makes sense.
5. Run 'gvinum'.
6. At the gvinum prompt, type 'create -f /etc/vinum-mirrof.conf'. You should see something like this:
gvinum -> create -f /etc/vinum-mirror.conf 2 drives: D drive1 State: up /dev/da1s1h A: 0/140011 MB (0%) D drive0 State: up /dev/da0s1h A: 0/140011 MB (0%) 5 volumes: V usr State: up Plexes: 2 Size: 116 GB V var State: up Plexes: 2 Size: 6144 MB V tmp State: up Plexes: 2 Size: 6144 MB V swap State: up Plexes: 2 Size: 6143 MB V root State: up Plexes: 2 Size: 2048 MB 10 plexes: P usr.p1 C State: faulty Subdisks: 1 Size: 116 GB P var.p1 C State: faulty Subdisks: 1 Size: 6144 MB P tmp.p1 C State: faulty Subdisks: 1 Size: 6144 MB P swap.p1 C State: faulty Subdisks: 1 Size: 6143 MB P root.p1 C State: faulty Subdisks: 1 Size: 2048 MB P usr.p0 C State: up Subdisks: 1 Size: 116 GB P var.p0 C State: up Subdisks: 1 Size: 6144 MB P tmp.p0 C State: up Subdisks: 1 Size: 6144 MB P swap.p0 C State: up Subdisks: 1 Size: 6143 MB P root.p0 C State: up Subdisks: 1 Size: 2048 MB 10 subdisks: S usr.p1.s0 State: stale D: drive1 Size: 116 GB S var.p1.s0 State: stale D: drive1 Size: 6144 MB S tmp.p1.s0 State: stale D: drive1 Size: 6144 MB S swap.p1.s0 State: stale D: drive1 Size: 6143 MB S root.p1.s0 State: stale D: drive1 Size: 2048 MB S usr.p0.s0 State: up D: drive0 Size: 116 GB S var.p0.s0 State: up D: drive0 Size: 6144 MB S tmp.p0.s0 State: up D: drive0 Size: 6144 MB S swap.p0.s0 State: up D: drive0 Size: 6143 MB S root.p0.s0 State: up D: drive0 Size: 2048 MB
Don't worry about the stale/fault states. To bring up the vinum partitions, run the following:
gvinum -> start root.p1.s0 gvinum -> start swap.p1.s0 gvinum -> start tmp.p1.s0 gvinum -> start var.p1.s0 gvinum -> start usr.p1.s0
The above process will take quite a while. While the rebuild process is going, you can run 'list' to see the progress.
7. Reboot into multi-user mode and enjoy your new software RAID-1!
References
http://devel.reinikainen.net/docs/how-to/Vinum/
