Offline Physical 2 Virtual conversion (p2v) using free tools

I’m a bit confused about VMware’s Converter and when you can use it. Which offering is free? Which is paid? From the last time I looked, I think you need the enterprise product to do an offline conversion. And call me old school, but online conversion just seems like voodoo to me. I was in the office this weekend, trying to convert an important physical machine to a virtual one and it failed for all sorts of reasons:

1. SSH keys. If you have a heavily locked down machine with ssh key logins, you cannot use the online converter.

2. Network. If you can’t hit the ’source’ machine with ssh, you can’t convert it. Maybe it’s on another network, or has a firewall running on it.

3. So since all that didn’t work, you’re forced to install the converter on the same machine you are converting. Except you don’t want to do that because you will have to fill out lots of forms because you are installing a new piece of software on a production machine.

This is a fail safe method of converting a physical Linux server into a virtual one. I converted my physical machine to an ESX 4.0 host, but this method would probably work for other versions of VMware as well.

1. Power down the server.

2. Boot up the server using a linux live cd . (I like Slax)

3. At the command line, run:

Where server is a (linux) server with about twice as much disk space as you have, and /dev/sda is the address of the disk you would like to convert. You could probably do more that one disk using this howto, but I only did one.

4. Install Qemu on the Linux server. You should be able to find it in your yum repo, or apt-get repo.

5. Run this to convert the raw dd image to a vmdk.

6. Copy the sda.vmdk to your datastore, using scp if you have ssh enabled on your VMware host, or using the Datastore browser if you don’t. Make sure to create a folder for the new machine.

7. Create a new VM in the vSphere Console, using values similar to the physical machine you are replacing. Make sure to select Custom at the first screen, not typical.

8. When it comes time to create a new disk, point vSphere at your newly converted vmdk file.

9. Boot up the server, and you’re done!

Installing RHEL 5 using the VMware Paravirtualized SCSI driver (pvscsi)

VMware ESX 4.0 offers a number of different SCSI disk controllers. One of the newer and more interesting ones is the paravirtualized scsi adapter. As with any type of paravirtualized  driver the paravirtualized scsi driver promises to be faster and use less CPU cycles than the other storage drivers, and since I/O performance is so important to overall system performance in virtualized environments, I thought it was worth a look. I don’t have any solid numbers about how it performs with WebSphere Portal, but once I do you will be the first to know.

The new driver has a number of drawbacks, most notably it is not supported as a boot device. VMware expect you to configure your OS on one ‘normal’ drive and then install a secondary drive with the paravirtualized adaptor. As well as being a pain from a management standpoint , it also ignores the speed increases that you would derive from having the OS on this faster device.

The method below will allow you to install Red Hat Enterprise 5.4 on your primary boot device using the pvscsi driver, and includes steps about how to automate the process with Redhat’s excellent kickstart system. If you are just interested in doing this on one system, I would follow these steps instead . My steps are good if you want to automate this procedure over many machines, but it’s probably overkill if you just want to try pvscsi on one machine. Hopefully these steps will be outdated soon when the pvscsi module makes it into the regular mainline kernel, which looks like it might happen in 2.6.33!

1. Create a new VM from the vSphere Console in the usual way.

2. Click edit settings on the new VM and select the SCSI controller 0 device, then up on the right hand side pane, click Change SCSI Controller Type and select VMware Paravirtual.

Now we must create boot media. I use iso images and the virtual cdrom device in VMware, but you could modify these steps and use them with PXE booting or USB booting quite easily. The process here is to add the pvscsi kernel module to the boot disk, so the RHEL installer can see the VMware disk.

3. Grab the prebuilt boot.iso from the images/ subdirectory and unextract it. You can do this by running:

You should now have an isolinux subdirectory in this directory.

4. Extract initrd.img . This is the initial ram disk which contains the boot files that the installer requires to run. Create a temp directory and extract it into it.

5. Now we need to extract (again!) the modules archive, so we can add pvscsi to it.

This will make a new subdirectory which will have the same name as your kernel version. Mine looks like this :
2.6.18-164.el5/x86_64
If you are running a 32 bit kernel or a different version of Redhat, yours will look slightly different.

6. Extract the VMware tools rpm and copy the precompiled driver into the modules directory. Make another temp directory and copy the Vmware Tools rpm into it. You should be able to get the VMware tools rpm from your ESX host or from the VMware website.

Then delete the temporary VMware tools directory.

7. Now we need to edit a bunch of text files that are in the modules subdirectory so Redhat can find the new driver.

At the bottom of module-info add:

At the bottom of modules.alias add:

At the bottom of modules.dep add:

In pci.ids, search for ‘15ad’ . You will see an entry for a few VMware devices. To this list add:

The whole stanza will look something like this:

8. Now we need to compress everything. First the modules.cgz (and still in the modules directory) :

9. Compress the initrd.img

10. Everything is compressed up again, now to make a boot cd.

11. Copy pvscsi.iso to your ESX datastore and click edit settings on your paravirtualized guest vm. Set the cdrom to boot from pvscsi.iso .

This is enough to get the installation image to boot correctly with the pvscsi driver, but what will happen next is that the server will reboot using the regular kernel, and it won’t have the pvscsi driver available to it, so the machine will not reboot. We need to install and configure the VMware tools rpm after the RHEL install so when the machine reboots it will have access to the pvscsi driver file.

If you don’t want to use kickstart to do this, and the end of the install, drop into a shell by typing ALT-CTRL-F2 and download the VMware tools rpm from a local webserver, or you could copy it to the guest VM using scp or something. For example :

Reboot and you should be good. Read on if you would like to automate this procedure.

12.The easiest way to create a kickstart file is to run through a normal install and then to copy the ~/anaconda-ks.cfg file from the server, which will have all your settings saved in it. Or look at my example. . All you should need to do here is change the url, key, hostname, rootpw parameters and edit the url to the vmware tools rpm to something valid for your environment and you should be good to go.

Stay tuned for performance numbers to show if any of this is even worth it!