View source

{{TRADS}}
__NOTOC__
== Introduction ==
P2V Linux migrations are a combination of science, art and luck.

P2V stands for '''Physical to Virtual'''. In other words, it is the process or procedure of moving a running system (operating system and everything installed) from a physical machine to a virtual machine.

This page describes some of the usual steps necessary to convert a Linux system into a virtual machine running under [http://www.vmware.com/products/vi/esx/ VMware ESX Server]. The same steps should mostly apply to [http://www.vmware.com/products/server/ VMware Server], [http://www.vmware.com/products/ws/ Workstation] or even  [http://fabrice.bellard.free.fr/qemu/ QEMU].

The focus of this P2V explanation is on [http://www.redhat.com/ Red Hat] and [http://www.centos.org/ CentOS] guests as they are not only supported, but benefit from kudzu and rescue disk capability built-in. Other distributions can of course be converted but the exact steps will vary.

So, let's get started.

== What to use (or not) ==

'''The following software products claim to do P2V, but in fact do not support Linux, so don't bother.'''


* [http://www.microsoft.com/windowsserversystem/virtualserver/evaluation/vsmt.mspx Virtual Server Migration Toolkit]
* [http://www.ezp2v.net/ EZP2V]

'''These products do support Linux in some way...'''
* [http://www.vmware.com/products/converter/ VMware Converter]
VMware converter will work, however any options such as resizing the disks and post migration configuration are greyed-out. This is due to the fact that it just does a raw block-by-block copy of the source disk. It is most useful if your target for migration is ESX 3.x
* [http://www.platespin.com/ Platespin] [http://www.platespin.com/products/PowerConvertMain.aspx PowerConvert]?
PowerConvert works (somewhat) with Linux but it does not support LVM and in my experience is an unreliable product with lackluster support.
* [http://liveview.sourceforge.net/ liveview] looks promising, however it only runs on Windows (due to dependency on VMDK disk mounter) and works with vmware-server, not ESX server. It has "limited" Linux support.

'''These products or methods offer full support for Linux'''...
* [http://www.rtfm-ed.co.uk/?page_id=174 UltimateP2V] appears to be worthy of consideration.
* Good ol' dd + netcat, followed by rescue disk of some kind (to fix the modules and make a new initrd).
* [http://www.mondorescue.org MondoRescue] - whilst primarily a backup/restore utility it lends itself extremely well to P2P/P2V migrations

== Preparation ==
You will want to have the necessary tools in place as well as some calculations.
Consider the follow aspects of your system.
* How much physical RAM? Is it over or under-utilized? 

* How much Swap space and where?

* Disk type - IDE or SATA and the disk device will be /dev/hda, SCSI will be /dev/sda. You may also have multiple disks (hdb, sdb, etc).

* Disk size - use a command such as ''sudo sfdisk -s''. Blocks are in 1KB units, do the math to figure out the equivalents in MB by dividing by 1024 and in GB by dividing by 1024 again.
** Example:
 jetson:~> sudo sfdisk -s
 /dev/hda:  39070080
 total: 39070080 blocks

 (39070080/1024) = 38154.375 MB
 (38154.375/1024) = 37.260 GB

* Partition layout -know exactly the disk partitions, sizes and filesystem types. This can be gleaned from the output of `fdisk -l /dev/sda` and the content of /etc/fstab. Understand that a disk partition is not the same as the filesystem installed on it. Some disk partitions don't have a filesystem at all, such as those used for swap, LVM or software raid (md).

* Rescue disk - this may be necessary for the recovery of the system once the disk data has been converted over. Conversion puts the system into a "new" environment of emulated devices, and some cases kudzu will not quite get you there or won't even be available (kudzu is a Red Hat software, not normally found on other Linux distributions).

* Knowledge of destination environment. For instance, see http://www.vmware.com/pdf/GuestOS_guide.pdf for vi3.

* ISOs - you will want to have the following ISOs for easy access and to map to the CDROM device in the guest. 
** Install disc #1 for the Red Hat or CentOS version of the source system
** [http://www.knoppix.org/ Knoppix] (recommended) or [http://www.sysresccd.org/ System Rescue CD]

=== Preparing the source system ===
Take the time to consider and perform the following tasks while the source system is still running in its native state.
* Disable any services you don't think will be necessary after the conversion, such as system-management agents (think Dell OMSA or IBM Director) and ntpd. 
* Purge out old logfiles, scratch files in /tmp and unnecessary software.
* Cleanup old/extraneous kernels. You will likely want to end up with just 2 kernels, the latest Non-SMP kernel and a previous one.
* Building a new initrd with the mptscsih (RHEL4/CentOS4) or BusLogic (RHEL3/CentOS3, RHEL2/CentOS2) SCSI driver loaded, this may save you from having to boot into '''linux rescue''' mode after the conversion.
For RHEL4/CentOS4, add '''--with=mptscsih'''
For RHEL3/CentOS3 and earlier, use '''--with=BusLogic'''
  mkinitrd -v -f --with=BusLogic /boot/initrd-`uname -r`.img `uname -r`

* Zero-out each of the disk partitions... this can speed up the data transfer later on. e.g.
 dd if=/dev/zero of=/usr/bigfile; rm -f /usr/bigfile

Boot your system with '''Knoppix''' or '''System Rescue CD'''. The state we want is an at-rest hard drive(s) and network connectivity. NOTHING should be running/writing to the hard drive(s).

Optional step: run '''md5sum /dev/sda''' and '''record the resulting hash'''. Usually the last 6 characters will suffice.
This can take awhile but gives you a ''fingerprint'' of the hard drive data that you can use later to verify the integrity of the data after transferring to the target system. '

=== Preparing the target system (VM) ===
Using the MUI or vmware-server-console, create a VM with the following parameters:
* Operating System: '''Linux'''. You can be more specific on vmware-server or ESX, '''such as Red Hat Enterprise Linux 4'''.
* Disk: slightly larger than source-system (see below). Create same number of disks as exist on the source system.
* Network (NIC): Use vlance if given the choice, can be upgrade to vmxnet later when vmware-tools is installed.
* CDROM: assign to either Knoppix ISO or System Rescue CD ISO

Boot the target system (I enter '''knoppix 2''' at the boot prompt) and verify the disk(s) are recognized using '''sfdisk -s'''. Also verify the network is up using '''ifconfig eth0'''. You should have an IP address assigned to eth0 via DHCP or static. Now try pinging the source system e.g. '''ping 10.4.1.2'''

== Network Acquisition (Disk Cloning) ==
This is where we transfer the bits from drive A on the source to drive B on the target.
The process is functionally very similar to a network acquisition often used in the field of computer forensics.

For our part, a simple example will show how to clone the bits from a single drive: /dev/sda

You'll need to know the IP address of your target-system, which can be learned from ''ifconfig eth0''.

These commands can be used to clone the blocks to the target-system disk. I assume you have netcat (nc) installed on the source system.

First, run this on the target system
 nc -l -p 9001 | dd of=/dev/sda

Then run this on the source system
 dd if=/dev/sda | nc <target-system-ip> 9001

In the real-world, repeat the above process as necessary for the remaining
disks.

This is the slowest part of the process. Unfortunately, dd does not show a progress meter. dd will, however, display it's progress in blocks if sent the -USR1 signal see dd's info page for details. I have seen a 36GB drive take 40 minutes to transfer over on a gigabit network, where the actual throughput was ~14.3MB/s. Another P2V took just over an hour for the same size drive, albeit on a different gigabit network.

The network transfer speed is an important consideration when planning your scheduled outage. You may want to run some tests before your P2V, with a smaller set of data (1GB?) to get an estimate of your throughput, then run the numbers to figure out how long it will take to do the entire drive(s). This can be done while the system is still online. Also, consider the tip above about zero-filling your disk partitions beforehand.

Once your drive(s) have been bit-copied over the network to the target, shutdown your target system and remove the virtual CDROM or ISO mapping.

Optional step: run '''md5sum /dev/sda''' and verify the result matches what you saw earlier.

== Extras ==
This is a perfect time to make some adjustments if you want to be clever about your disk & paritition sizes, the following may come in handy.
Verify the partitions
 fdisk -l /dev/sda

Check a filesystem.
 e2fsck -f /dev/sda1

Align ext[23] filesystem size with (new) partition size of sda1
 resize2fs -p /dev/sda1


== First-boot ==

Assuming you got this far, the next step is to immediately shutdown the system again. You'll want to re-assign the NIC to vmxnet and assign the vmware-tools ISO to the CDROM (path: /usr/lib/vmware/isoimages/linux.iso). Boot up into single-user mode (at grub prompt hit e, select kernel line, e, append "single" to the line, then hit b). Install the vmware-tools (detailed elsewhere) which should get you the vmxnet driver module. Adjust network settings now! Cleanup and reboot. You should be 98% there. Congratulations!

NOTE: I had to rename /etc/rc3.d/S19vmware-tools to /etc/rc3.d/S09vmware-tools to "fix" my network bootup sequence.

On the second reboot, the kudzu command will run and (may well) deal with the remaining hardware changes.

When kudzu runs, it recognizes that certain devices (Broadcom NICs) are no longer there while others (LSI Logic card, pcnet32 NIC) had been added. Usually it is easiest to just accept what kudzu tells us & fine tune later.

See '''When things go wrong''' below if you don't get back to a login: prompt.

== When things go wrong ==
If ''kudzu'' does not get you back to a login: prompt, the next step is to boot with the rescue disk. This entails mapping the install cd#1 ISO file to your CDROM device using the MUI or vmware-server-console. Make sure the VM BIOS is also set to use your CDROM in the boot order before the hard drive(s).

Once booted, type ''linux rescue'' at the boot prompt and shortly thereafter you will be able to type ''chroot /mnt/sysimage'' to get at your disk partitions, which should automatically be mounted there.

The first thing to look at is /etc/modules.conf (RHEL3/CentOS3) or /etc/modprobe.conf (RHEL4/CentOS4). Make sure the appropriate SCSI driver is listed, either BusLogic or mptscsih (based on what you configured this VM to use and the recommendations above).

 alias scsi_hostadapter mptscsih
 #or
 alias scsi_hostadapter BusLogic

Also, take note of the eth0 setting, which should be either pcnet32 for the vlance device, or vmxnet for the vmxnet device. After you install the vmware-tools with the vmxnet device assigned it should be configured automatically.
 alias eth0 pcnet32
 #or
 alias eth0 vmxnet

If you went the route of cloning individual partitions instead of the entire disk(s), it may be necessary to clone the MBR. This will be evident if you try to boot from the drive and get the message "No operating system found". The process is described [http://www.rajeevnet.com/hacks_hints/os_clone/os_cloning.html here].

== Tying up loose ends ==
Consider the new state of the system...
Disable daemons you don't need like smartd, pcmcia and the like. Do you really need to run NTP anymore? Read VMware's timekeeping whitepaper, set tools.timeSync="TRUE" in the .vmx file and add clock=pit or clocksource=acpi_pm to the grub kernel line. Consider an optimized kernel or adding divider=100 (to newer 2.6 kernels).

== See Also ==
* ESX Server 3 Systems Compatibility Guide ([http://www.vmware.com/pdf/vi3_systems_guide.pdf PDF])
* ESX Server 2.x Systems Compatibility Guide ([http://www.vmware.com/pdf/esx_system_guide.pdf PDF])
* [[Hard Disk Cloning]]
* [[UbuntuonVMware]]
* [http://www.rajeevnet.com/hacks_hints/os_clone/os_cloning.html Wonders of 'dd' and 'netcat': Cloning OS harddrives]
* [http://www.sleuthkit.org/informer/sleuthkit-informer-11.txt The Sleuth Kit Informer]
* [http://www.garloff.de/kurt/linux/ddrescue/ dd_rescue] - looks like a GREAT alternative to dd for P2Ving systems with failing hard drives. (It's called gddrescue on [[Ubuntu]]).  Now available as GNU ddrescue from [ftp://ftp.gnu.org/pub/gnu/ddrescue/]
* [http://dcfldd.sourceforge.net/ dcfldd] - another dd alternative that can update the user of its progress in terms of the amount of data transferred and how much longer operation will take.
* MondoRescue for P2P / P2V migrations ([http://www.mondorescue.org/docs/p2v.pdf PDF])

[[Category:Virtualization]]
[[Category:Linux]]
[[Category:P2V]]
{{BADS}}