Since the release of Windows Server 2008, there has been no way to build a cluster without purchasing dedicated, proprietary storage (such as an array from HP, Dell, EMC or NetApp, to name just a few). There are open source storage platforms like OpenFiler and FreeNAS; there is an iSCSI target in almost any Linux and BSD platform. Until very recently, none of these have passed the cluster storage tests that Windows 2008 executes before forming the cluster.

Without passing those tests, the cluster cannot be formed at all - and the culprit is normally the "SCSI-3 Persistent Reservation" tests, without which the cluster cannot operate (because it cannot guarantee that the disks are owned by only a single node at a time).

The forums for OpenFiler, OpenSolaris, FreeNAS, BSD and Linux distributions seem to be full of requests for the iSCSI targets to support Windows Server 2008 clustering (and also enable Solaris and OpenSolaris clustering). OpenFiler has been developing the feature, but the latest news I found is that they will be making it a premium (ie paid) feature. OpenSolaris has claimed to have support for months, but I was unable to make it work (despite updating the system past the build number in which support was added, snv_115).

And finally, today, I noticed that FreeNAS has released a new beta build, 0.7 RC1; and one of the key features is that it supports SCSI-3 Persistent Reservations.

I had to try it.

I set up a Hyper-V server (a Dell 2900 with 4GB of RAM and a 500GB RAID 10 set for VMs).

Then, I set up 3 VMs. Here's the plan for what it will look like, if it all works:
 
    
 
First I created a FreeNAS VM. 512MB of RAM, two CPUs, two legacy NICs (as FreeBSD does not yet support the Hyper-V extensions), a 1GB VHD for the "system" and a DVD-ROM drive from which to install. The installation was painless - two or three questions and it's done. Setting up the networks was confusing for a moment - the UI changes the text without highlighting the changes, so it took a few goes to sort it all out. I assigned a public IP to the public NIC and a private IP to the NIC on a new private network for iSCSI.

Then I deployed two Windows Server 2008 R2 Enterprise Edition hosts (the RC). Each has two legacy NICs with the same configuration (one public, one iSCSI). I configured IP addresses and ensured I could ping everything on both networks, then installed the File Server role and the Failover Cluster feature.

While they installed I switched back to configuring FreeNAS. I removed the DVD-ROM drive and added three new 128GB disks on the 3 available IDE controller slots. Then I loaded the FreeNAS config page and configured a RAIDZ1 (RAID 5 ZFS) device, a ZFS Pool using the device, a Portal group, an Initiator group and three extents, each of which provides a single iSCSI target (Quorum, File Server 1 and File Server 2). It sounds like a lot of work but it took less than 15 minutes.

Finally I was ready to connect the drives and hopefully build a cluster. On each server I connected the iSCSI targets, one at a time. On node 2 I brought each device online, initialized the disk and created a simple volume, assigning drive letters as needed. Then after taking the drives offline, I replicated the drive letter assignments on node 1.

With all those steps completed, the Failover Cluster Wizard can run. Normally the storage tests fail after a few moments, but in the FreeNAS case they succeeded - I have a working, validated file server cluster using free software for the storage and Windows 2008 Enterprise for the compute power.

The proof, as they say, is in the image:

Now you probably won't want to use this for a critical production system - FreeNAS is after all beta software. But for your lab environments, or running up a virtual cluster; this might just fit the bill perfectly.

No Comments