I currently have 2 servers in the house... One of them is a NexentaStor storage appliance, and the other is a Windows 7 box (also running VMware Server 2.0 with a handful of VMs). I was interested in consolidating the two of them into a single server since neither box was being heavily utilized. After reading this page, I decided to base the single server off of VMware's ESXi product. The first VM would need to be the storage appliance, and then the rest of the VMs would NFS mount the storage appliance and boot from there. This setup should handle everything I need it to, and if there was a catastrophic failure, I could pull the drives and put into another ZFS system and recover the data.
VMware doesn't make it easy to create Raw Device Mappings using SATA disks. That's okay, ESXi has a feature which allows you to pass through hardware on the host machine directly to the guest. This sounds great. I planned on passing a couple of Intel SASMF8I LSI1068-based controllers to the storage VM. I have a Supermicro X8SAX motherboard, Intel i7-930 CPU, and 24GB of RAM (6x4GB DDR3).
In order to enable hardware passthrough, you have to go into the BIOS and enable Intel VT-d. Unfortunately, any time I enabled this feature ESXi refused to install. If I installed ESXi first and then enabled the feature, it refused to complete booting. I went through all sorts of BIOS settings, ACPI, Intel Virtualization technology, etc. and couldn't find a combination that worked. I upgraded the bios from 1.1a to 2.0 and that still didn't work. I tried ESXi 4.1 along with 4.1U1.
What finally worked was downgrading the BIOS all the way back to 1.0c! The only downside so far is that it only recognizes 20GB of RAM instead of 24GB. Other than that, the hardware passthrough works perfectly. 1.0c isn't available on Supermicro's website. Good luck in finding it... I grabbed it from some website in China after some extensive searching at google.
I've got an e-mail into Supermicro asking why. I'll edit this post if I get an explanation from them.
