Technical Exhaustion

I’ve previously discussed how to recover from a disk failure in gmirror. It’s probably a good idea to describe how to set up a new mirror from scratch.

Rather handily this can be done on a live system, remotely via ssh in the space of a few minutes. One reboot is required but other than that the process is seamless.

My existing install is on disk ad4 and the ’spare’ disk I want to mirror this existing installation to is ad6.

Both are 160 gig SATA disks.

Login in as root however way you see fit.

Before we can configure the new mirror we need to disable a safety feature of GEOM :

# sysctl kern.geom.debugflags=16
kern.geom.debugflags: 0 -> 16

This allows us to perform operations on a mounted, and in use disk. This is just set for the current session and will be reset to normal when we reboot - which we’ll be doing shortly.

Whilst we’re doing housekeeping, lets enable gmirror at boot :

# echo geom_mirror_load="YES" >> /boot/loader.conf

You should probably check your loader.conf first. On a fresh system its empty and the above command is safe, however you probably want to have a quick glance over it and backup your existing one as a precaution if it contains existing settings.

Lets create our new mirror - with one command :

# gmirror label -v -b round-robin gm0 /dev/ad4
Metadata value stored on /dev/ad4

And we’re done! We now have a gmirror device but with only one member - the existing disk.

There are a number of different balance algorithms to choose from, the gmirror man page steps in :

load, prefer,round-robin,split

Finding out information on how these actually perform under the hood (beyond the superficial explanation in the man page) is quite tricky. And indeed most information leads to the fact that they don’t really behave as hoped and you’re better off sticking with round-robin which does what it says and gives marginally the best performance.

If you want to test with your specific work load / pattern you can change this anytime you like :

gmirror configure -b load gm0

Now we have a mirror device, and have enabled the drivers to be loaded on boot - we need to tell our system to mount from it next boot. So we need to edit /etc/fstab. Take a backup of your existing one now!

What happens next depends on your specific install. In general you need to add /dev/mirror/ to each entry and you need to change your hard disk device to be gm0 (or whatever you chose in the above to call your mirror device).

So I had a previous entry :

/dev/ad4s1a / ufs rw 1 1

Which now becomes :

/dev/mirror/gm0s1a / ufs rw 1 1

Repeat for each mount point that contains reference to your existing disk. Make sure you get this right, any errors here means you won’t be booting back up without difficulty!

Once you’re happy - reboot!

Cross your fingers.

Wait anxiously.

Nervously fidget.

And you’re back! Log in and do a quick df, hopefully you should see something akin to :

/dev/mirror/gm0s1a 507630 141610 325410 30% /

Looking good so far. Lets check the mirror is up :

# gmirror status
Name Status Components
mirror/gm0 COMPLETE ad4

ok, so far so good. We’re up and running on our new mirror device. But it still only has one member. We need to add our second, ad6, disk.

#gmirror insert gm0 /dev/ad6

And we’re done! You’re now mirrored - your mirror is currently degraded :

# gmirror status
Name Status Components
mirror/gm0 DEGRADED ad4
ad6 (70%)

But will soon become complete when the new disk finished sync’ing.

# gmirror status
Name Status Components
mirror/gm0 COMPLETE ad4
ad6

And we’re done.

There are some caveats that should be mentioned.

I’ve run this config in anger in production for years now and never had problems - but there is a more complicated procedure to configure gmirror :

http://people.freebsd.org/~rse/mirror/

This takes into account the differening gemoetries of your two disks and avoids any problem with data going where it shouldn’t.

http://markmail.org/message/6vqfrgufv6jeu3yq#query:freebsd%20set%20up%20geom%20mirror+page:1+mid:wfkmluqgni45jzdo+state:results

gmirror also seems to be a performance dog, especially when compared to Linux software RAID. Which is unfortunate.

http://www.freebsdwiki.net/index.php/RAID,_performance_tests

But it’s ease of use lets me put up with that. Although hopefully we will see improvements over the next few versions of FreeBSD.

I came across a known bug in the FreeBSD installer for the first time today.

During the text network install (using the netboot iso) I had problems resolving the hostname of my chosen FTP mirror. The installer hung for an age trying to connect.

Assuming I had a made a typo in the network config, I ctrl+c’d the installer and chose to restart the install process when prompted.

This time round I connected and started retrieving package sets correctly - maybe I did make that typo after all.

However in the half dozen or so times I rebooted the server and restarted the install completely from scratch - I received various different errors ranging from simple ‘disk full’ messages to ‘unable to create symlink’ errors and even ‘no free inodes’ !

I was working remotely over a KVMoIP at the time and was starting to fret I had hardware issues on the other end of the line.

This belief was also founded on receiving the notorious disk geometry error from the installer during the partitioning stage. This is a long standing ‘issue’ and these days really means very little (in fact I believe it is being supressed as of FreeBSD 8? Finally!) and from past experience usually presents no problem - but there is always a nagging doubt - especially if more things go wrong!

However it turns out I had fallen foul of yet another long standing bug in the installer. This time if ctrl+c’ing out the installer and restarting (without rebooting) the next time round the installation attempts to write to the ramdisk instead of your allocated spinning disk - which obviously fills up very quickly and soon gives a variety of disk full errors!

http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/45565
http://www.freebsd.org/cgi/query-pr.cgi?pr=42162&cat=bin

These reports are dated 2002! and still this issue remains.

The solution is to point the installer to the IP address of the FTP server and bypass the icky host resolution.

FreeBSD has a ‘notorious’ reputation for being very hardware picky which, in actual fact, is quite far from the truth these days. Whilst not matching Linux, support is very good - certainly when compared to OpenSolaris. However I can’t help but wonder if these two very long standing features / bugs which present themselves very quickly during the already outdated (by modern standards) install procedure, cause people to instantly assume there is a support issue with their disk and / or controller and thus to move on (quite speedily I would warrant) from FreeBSD.

There is constant talk of a revamped installer but as of yet no sign. Let’s hope FreeBSD 8 brings some improvement.

Some quick Solaris 10 Zone management commands.

To list your running zones :

# zoneadm list

And with a bit more detail :

#zoneadm list -v

Change the IP address of a Zone

#zonecfg -z
zonecfg:zonename> select net address=oldip
zonecfg:zonename:net> set address=newip
zonecfg:zonename:net> end
zonecfg:zonename> commit

Change the name of a Zone

#zonecfg -z zonename
zonecfg:zonename> set zonename=newzonename
zonecfg:zonename> commit
zonecfg:zonename> exit

Moving a Zone on disk

Stop the zone (let do this cleanly) :

#zoneadm -z zonename halt

Then move :

#zoneadm -z zonename move /your/new/path

You should see a message similar to :

Moving across file systems; copying zonepath /your/new/path...
Cleaning up zonepath /your/new/path...

And bring it back up :

#zoneadm -z zonename boot