Archive

Posts Tagged ‘ESXi’

Comparing Performance of Virtualization Products

Nowadays there are several different virtualization products available on the market. All these products offer the same basic functionality and can be divided into two major categories: software based virtualization and hardware based virtualization.

The first category requires an Operating System like Windows or Linux to be installed on the machine first. The second category doesn’t require an Operating System to be installed, but in fact provides it’s own OS or a hypervisor. Since this hypervisor runs straight on top of the hardware it is often referred to as a bare metal virtualization solution.

As far as performance is concerned, a bare metal solution will without a doubt offer better performance than a software based virtualization solution. But how do these bare metal solution compare to each other? And is there a noticeable performance drop when using a virtualized workstation compared to a “normal” installation?

Just to satisfy my own curiosity, I decided to put this to the test.

Tested products and test environment

There are numerous bare metal virtualization products available on the market, but I limited the tests to several products I have experience with myself. The products I included in the tests are:

  • Microsoft Windows Hyper-V 2008 R2
  • VMWare ESXi 4.1
  • Citrix Xen Server 5.6
  • Citrix Xen Client 1.0

All tests were performed on the same workstation. For this purpose I have used a HP Compaq 6000 Pro SFF. The hardware specifications of this machine are:

  • Intel Core2 Duo CPU E8400 @ 3,00 GHz
  • 2048 Megabyte RAM
  • Intel Q45 / Q43 graphics chipset
  • Western Digital WD3200AAJS-60Z0A0 Hard Disk

For the tests I created a new Virtual Machine with a single CPU / core with 1024 MB of memory and a 64 GB hard disk. In the Virtual Machine I installed Windows 7 Professional (32-bit). After installing the Operating System I also installed the appropriate tools (Xen tools, Integration Services, VMWare tools, etc.) and installed all available updates for Windows 7 that were available through Windows Update.

For comparison I also installed Windows 7 straight onto the workstation. During the tests I limited the number of available CPUs to 1 and the amount of memory to 1024 MB.

To test the performance I used Passmark Performance Test and ran the standard test suite. Because none of the virtualization products supported 3D acceleration I excluded the 3D graphics test from the results. I also excluded the CD performance test, because these will not be very important in everyday situations.

For the final test result I repeated the standard test suite 5 times, dropped the highest and the lowest result and computed the average of the remaining results. These final results were compared to the result of the performance test of a normal Windows 7 installation. The latter received a score of 100%, the others were computed as a score relative to this.

Overall Score

For the overall score Passmark Performance Test calculates a score for each category:

  • CPU
  • 2D graphics
  • Memory
  • Disk

image

As one would expect the performance of the Virtual Machines is slightly worse than the performance of the physical machine. The difference between the Virtual Machines is almost negligible. The only exception is the 2D graphics score where VMWare ESXi score considerably worse compared to the other virtual machines.

Surprisingly, the disk score of most virtual machines is higher than the score of the physical machine. Also here VMWare scores considerably worse.

CPU score

The overall CPU score shows there is not much difference between the virtual machines, neither is there a large performance drop. The only considerable difference compared to the physical machine is during the Find Prime Numbers test.

image

2D graphics score

After installing Windows 7 in the virtual machines, only the Hyper-V virtual machine had a suitable driver for the display adapter installed. The others were using the standard VGA display driver. During the tests the physical machine performed exceptionally poor, compared to the virtual machines. This poor performance was caused by the Intel graphics driver that was installed. After installing the standard VGA driver, the performance was normal. Since most virtual machines were using the standard VGA driver anyway, using this driver also on the physical machine would provide better results for comparing the graphics performance.

image

In several tests (Image Filter, Windows Interface, Complex Vectors) Microsoft Hyper-V clearly outperforms its competitors. But in the Image Rendering test it performs the worst. The overall score of Hyper-V is almost equal to Xen Server, with Xen Client following an a short distance. VMWare ESXi is trailing in almost every 2D graphics test which results in the lowest overall 2D graphics performance.

Memory score

The memory test do not show much different between the physical machine and the virtual machines. Only on the Large RAM test the physical machine performs much better. Of the virtual machines Hyper-V performs much better than its rivals during this test. Because of this, Hyper-V achieves a slightly better overall memory score compared to the other virtualization products.

image

Disk score

During the disk tests almost all products, except VMWare ESXi, perform better than the physical machine. Especially the test scores of Xen Client are consistently high. Hyper-V does exceptionally well on the sequential write test, but scores considerably lower during the random seek + read/write test.

Because of the high score during the sequential write test, Hyper-V achieves the highest overall score, closely followed by Citrix Xen client.

image

So why are most virtual machines performing better during the disk tests than the physical machine? The only reason I can think of is caching. During the standard disk tests, Passmark Performance Test uses Windows API calls to read from and write to the disk. Because of the use of caching, the read/write operation seems to be finished as far as the OS is concerned, while the data has not been written physically. Because the data is still in the cache, the read operation will also finish much quicker.

This also seems to be supported by the fact that especially Hyper-V showed strange results during the tests. The tests are most likely designed in such a way that the sequential write test writes some data and the same data is read again during the read test. Because the data written is still in cache, the read operation can also retrieve the data from the cache. This resulted in scores that were about 100 times higher the the score of the physical machine. I have neglected these scores during the tests and included only “normal” test results.

Besides the standard disk tests, Passmark Performance Test also provides advanced tests. These tests provide a simulation of read/write patterns for workstations, file servers, database servers and web servers. It is also possible to use different methods of accessing the disk, besides the Windows API calls. Perhaps these advanced tests will show different results. Maybe I will do a similar comparison focusing on the advanced disk tests in the future, if time permits it.

Summary

My basic goal for this comparison was to compare the performance of different virtualization products compared to a physical machine. For all products I have used the out-of-the-box configuration settings.Most likely it is possible to tweak the configuration to achieve a better score.

It was also not my intention to compare all features of the different virtualization products and pass judgment on which of these products is the better one. I think this is something that is also a matter of personal taste. The remote console of Citrix Xen server seemed very sluggish to me, which made working in the virtual machine quite cumbersome. Both Hyper-V and VMWare use a similar remote console, but both did not suffer from this sluggish performance.

Citrix Xen client is a fairly new product, which is far from finished at the moment. Especially the full 3D graphics support it is supposed to deliver looks very promising. For now this 3D support is still limited to certain Intel graphics chipsets only, but other graphics chipsets and cards may be supported in the future.

But other products are also still being improved constantly and start offering additional features. Maybe this comparison should be repeated about the same time next year. It will be interesting to see if this will show major differences compared to this test.

Advertisements