Virtualization: an old concept transforming datacenters in

new ways
technology brief

Abstract.............................................................................................................................................. 2
Introduction......................................................................................................................................... 2
Layers of virtualization in the datacenter................................................................................................. 2
Virtualization within software layers ....................................................................................................... 4
Multitasking OS ............................................................................................................................... 4
Virtual machines .............................................................................................................................. 4
Cluster file systems ........................................................................................................................... 6
Virtualization using hardware within the platform .................................................................................... 7
Processor virtualization ..................................................................................................................... 7
Protected ring layers ..................................................................................................................... 7
Newest virtualization extensions and protected ring layers ................................................................ 8
Storage RAID................................................................................................................................... 9
Virtualization in the network layers ...................................................................................................... 10
Network partitioning ...................................................................................................................... 10
Platform collection .......................................................................................................................... 10
HP direction for future ........................................................................................................................ 11
Virtualized resources using industry-standard platforms....................................................................... 11
Automated resource management .................................................................................................... 12
Conclusion........................................................................................................................................ 13
Glossary........................................................................................................................................... 14
For more information.......................................................................................................................... 15
Call to action .................................................................................................................................... 15

Abstract
Virtualizing compute or storage resources in the datacenter is becoming increasingly important for
improved flexibility, availability, and productivity. Even though it is gaining prominence and being
incorporated more often in the datacenter, virtualization is not a new concept. This paper defines
virtualization and provides several examples to show how resources are already being virtualized
within the network layers, in the platform hardware layers, and in the software layers of the
datacenter. HP is working internally and with external working groups to ensure that the next steps in
virtualization will allow servers to be easily pooled, shared, or re-provisioned, based on the needs of
the business.

Introduction
The terms virtual and virtualization are seen throughout the IT industryvirtual machines, virtual
memory, virtual network, virtual storage, CPU virtualization, and so on. Furthermore, other terms such
as utility computing, grid computing, and Adaptive Enterprise refer to computing strategies that
incorporate strategic goals of virtualizing compute resources. While these terms often refer to new
products and may be presented as new technologies, the concept of virtualization is not new.
In this technology brief, the term virtualization refers to abstracting, or masking, a physical resource to
make it appear different logically than it is physically. All forms of virtualization share at least two
common elements: First, the virtualization technology abstracts a physical resource to make it appear
logically different than its physical reality. This abstraction allows resources to be pooled and/or
shared. Second, there is always a management layer or control point for the virtualized resources. As
virtualization technologies multiply in the datacenter, it is important for IT managers to understand
how these various control points might affect security and how each control layer interacts with other
management control layers.

Layers of virtualization in the datacenter

Every IT datacenter is comprised of layers of resources, as illustrated in Figure 1:
Software layersincluding operating systems, business applications, and software applications
such as virtual machines
Server platform hardwareincluding processors, memory, local I/O, and distributed I/O
connectivity hardware
Network or system interconnect layerincluding the network managers and switches for storage
area networks (SANs) and local area networks (LANs)
Power and cooling infrastructure for the datacenter

Figure 1. Layers of resources in the datacenter architecture: most already include virtualization technologies
System software
(applications, OS)

Platform Software Layer

CPU

Platform Hardware
Layer

Memory

BIOS

Platform
Manager

KVM

Local Interconnect (PCI)

I/O
Slots

Distributed Network Connection

Direct-attached
storage

Cluster, LAN, SAN Endpoints

(NIC, HBA, HCA)

LAN (Ethernet)

Cluster (RDMA/IB)

SAN (iSCSI, FC)

Network Layer
Network Manager

Infrastructure Layer

Firmware

Power

Switch

Cooling

The ultimate goal of IT administrators is to build an adaptive datacenter that can share any IT resource
so that utilization is optimized and supply automatically meets demandwithin all the architecture
layers. Many components within these layers already have the capability to be virtualized resources.
As more and more components and layers within the datacenter are virtualized, the IT administrator
gains:
Increased flexibility because resources can be moved as needed
Increased scalability because resources can be scaled up or down based on changing workload
demands
Improved resiliency by simplifying backup, failover, and disaster recovery solutions
Reduced total cost of ownership as physical resources are used to their fullest potential
With those benefits, virtualization also brings challenges, including:
Increased management complexity. Every time a specific resource is virtualized, another control
point or layer of management is added, regardless of the layer in which the abstraction is
occurring. Therefore, it is critical that virtual resources use an efficient management structure.
Furthermore, as the IT administrator establishes more virtualized resources, there becomes a need
for a single, unified management layer throughout the datacenter to reduce overhead in managing
multiple types of virtualized resources.
Unknown performance levels and models for meeting performance levels. Virtualizing a resource
involves management overhead that can reduce raw hardware performance in some cases. In
addition, IT administrators need to make hardware and software decisions based on meeting
business objectives or application performance levels. As more of the datacenter becomes
virtualized, it will be important to understand service-level agreements (SLAs) and to build a service-

oriented architecture that includes robust monitoring and metering practices that ensure SLAs are
being met.
Increased acquisition costs for management and hardware infrastructure. Although the end result
may reduce costs because of increased utilization, there may be large initial acquisition costs to
establish a virtualized solution.
Uncertain software licensing models. As virtualization solutions become more prevalent, vendors
will need to address how their software is licensed and paid for. Todays software is normally
licensed based on the physical hardware on which the software is operating. However, as software
instances are established on virtual hardware, the cost structures will need to comprehend those
changes.

Virtualization within software layers

Numerous examples exist of ways that the platform software layers (applications and operating
systems) provide virtualized solutions in todays datacenter. For example, a preemptive multitasking
OS shares the processor across several different applications, making it appear as if all are running
simultaneously. Virtual machines are an extension of this concept, in which software abstracts a
servers physical resources (processors, memory, and I/O) to share the hardware across multiple
instances of operating systems, making it appear as if operating systems are running simultaneously
on different hardware. Clustered file systems also use a software layer to share a common file system
across multiple platforms, making it appear as if the storage devices are all using a single file system.

Multitasking OS
Multitasking operating systems have been widely available in the PC world since IBM and Microsoft
released OS/2 in 1987. Multitasking operating systems provide one of the earliest examples of how
software layers (in this case, the OS kernel) can manage the hardware so that it is shared across
applications, providing flexibility to the end user.
A preemptive multitasking OS determines how long (based on time or priority levels) any single
application can use the processor. If it is a time-based function, the OS allows one application to run
for a defined time period. Then the OS preempts that application off the processor, saving all its state
and task information in special registers. The OS then allows a second application to use the
processor for this same amount of time. The OS will continue to preempt each application after its
allotted time, saving the state information each time, and returning to that state information when each
application resumes operation. Thus, the OS virtualizes the processor so that multiple applications
share its resources. This increases flexibility by allowing the user to run multiple applications
simultaneously. Although at first, performance levels between standard DOS and multitasking
operating systems were compared and questioned, this type of virtualization has become essential to
most users. The incurred overhead is minimal compared to the flexibility that the end-user gains.

Virtual machines
Today, one of the most common examples of software abstraction is the use of virtual machine
software to consolidate multiple operating systems onto a single server. Virtual machines are also
known as soft or logical partitions of a server. Virtual machine software abstracts the physical
resource and creates one or more virtual instances of that physical resource. In other words, virtual
machine software can create multiple virtual machines, each with its own virtual drives, virtual
NICs, virtual host-bus adapters, virtual processor, etc. Each physical resource is shared among these
multiple virtual machines. Each virtual machine can load its own OS and applications (Figure 2) and
is isolated from the actions of any other OS.
A virtual machine uses a separate software layersometimes referred to as a hypervisor or virtual
machine monitorto abstract the computers hardware. It acts as a supervisor, ensuring security and

cleanliness of the shared resources (such as making sure that all buffers are flushed and state
information is saved). It also manages the access of each guest OS (that runs in the virtual machine
instance) to the physical hardware resources.
Prominent examples of virtual machine software include Microsoft Virtual Server, VMware ESX
Server and GSX Server, and Xensource Xen (open-source) software. Virtual machine software can use
either para-virtualization or full virtualization techniques. Para-virtualization requires the guest OS to
be modified before it can run in a virtual machine instance. Modifying the OS simplifies the work of
the virtualization layer. Xen uses this technique. Full virtualization refers to techniques such as those
that Microsoft-Virtual Server uses, in which an un-modified OS runs in the virtual machine instance.
VMware uses a hybrid technique in which the VMware kernel performs a binary-translation of a
portion of the guest OS code that must operate at the most privileged level of processor instructions.

Figure 2. A software layer abstracts the physical resources so that each instance of the OS and application(s)
appears to have its own NIC, processor, disk, and memory, when in fact these are virtual instances.

HP is adding value to virtual machine solutions by providing a centralized management resource for
virtual machines. The HP ProLiant Essentials Virtual Machine Management (VMM) Pack provides a
single management and control point for virtual machines established using Microsoft Virtual Server,
VMware ESX Server, or VMware GSX Server. VMM Pack integrates with HP Systems Insight Manager
(SIM), providing the ability to manage both physical and virtual resources from a user interface that is
consistent with HP SIM. It allows IT administrators to associate the virtual machine to the physical host
and to identify how resources are being consumed in a virtual or physical machine. For example, IT
administrators can easily identify virtual machines or host servers reaching high CPU, memory, or disk
utilization levels. VMM Pack also simplifies management by providing a way for the IT administrator
to control remotely the virtual machine functions (such as start, stop, copy, move, and backup).
VMM Pack provides the foundation for automating physical to virtual migrations through the HP
ProLiant Essentials Server Migration Pack. The Server Migration Pack enables the functionality to
migrate virtual machines and their operating systems from physical (P) to virtual (V) platforms (P2V),
from virtual to physical (V2P), or from one virtual machine to another (V2V).

Cluster file systems

Cluster file systems are another common example of how application software is used to abstract
physical resources such as hard disk drives. For example, clustered servers are often connected to a
SAN that contains many disk drives and storage arrays (Figure 3). To simplify access to data stored
on the SAN, cluster file system management software is added to each server node. In this way, each
server accesses a common file system that is distributed across multiple servers. The client accessing
the data is unaware of whether the data is spread across multiple arrays or multiple disk drives: It
simply requests the data, and the cluster file system software layer performs any necessary translations
to acquire the data from the correct location in the SAN. Furthermore, any server in the cluster can
write to or read from any file in the SANall while maintaining a consistent view of the data, even if
it is being viewed by multiple servers in the cluster.
Cluster file systems reduce the challenges that system administrators or programmers have in
managing multiple file systems across a cluster and also improve the storage utilization efficiency by
allowing disk capacity to be shared among multiple servers. Examples of cluster file system software
include HP StorageWorks Scalable File Share, PolyServe Matrix Server, and Red Hat Global File
System.

Figure 3. Example of a clustered file system

Virtualization using hardware within the platform

The platform hardware level of the datacenter architecture consists of initiators (such as servers and
clients) and target devices (such as storage arrays and printers). This section reviews some examples
of how the components within server platformssuch as processors, memory, storage, and so on
provide virtualization. For example:
Processor virtualization through the use of protected ring layers in x86 processors and through new
virtualization extensions
Storage RAID and RAID memory technology
Even though the virtualization functions reside in the hardware layers, they are not independent of
software. Each virtualization function also requires some software infrastructure.

Processor virtualization
Because all x86 processors are based on designs originally developed decades ago, there are many
cases in which the processor hardware performs some level of masking or translation to enable
added functionality. These functions have been developed in hardware to:
Provide backward compatibility for operating systems and applications (for example, 16-bit mode
versus 32-bit mode versus 64-bit mode)
Enable multitasking operating systems to preempt applications (for example, protected ring layers)
Enable the hardware resources to more easily run multithreaded code without adding additional
execution units (for example, HyperThreading technology)
Protected ring layers
An x86 processor includes four different ring layers which differentiate the type of instructions that
the processor can execute, depending on what software is calling the instruction (Figure 4). In a
typical multitasking OS, the OS operates in ring 0, in which it has full access to all the processor
and platform resources, such as memory mapping. This is the most privileged level, also referred to as
kernel mode. Applications typically operate in ring 3, sometimes referred to as user mode, in which
functions such as memory mapping are restricted to keep one application from adversely affecting
another application. Originally, ring layers 1 and 2 were designed to house elements such as device
drivers and the OS file system; however, these layers are not typically used today. The device drivers
now typically reside with the OS in ring 0, and the file system resides with the application in ring 3 or
with the OS in ring 0. The protected ring layers in the processor hardware work hand-in-hand with the
OS to enable a multitasking OS to preempt an application off the processor, providing the
appearance that multiple applications are operating simultaneously.

Figure 4. In a typical multitasking OS, the application resides in ring 3, while the OS kernel and device drivers
reside in ring 0.

Typical Multitasking
OS
Application

Ring 3

Software layers
Ring 2 (typically not used)
Ring 1 (typically not used)

OS Kernel/DD

Platform hardware layer

Ring 0

Processor

Newest virtualization extensions and protected ring layers

AMD and Intel have recently announced an expansion of this protected ring layer concept with their
hardware-based AMD Pacifica technology and Intel Virtualization Technology. These virtualization
extensions are designed to accommodate virtual machine monitors more efficiently and to reduce
some of the software overhead that such a virtualization layer requires.
Virtual machine software such as Microsoft Virtual Server and VMWare ESX Server adds an
additional software layerthe virtualization layerinto the software stack. Because the virtualization
layer must be able to control all the physical resources of the system, just as the OS normally does
(with privileged instructions), the virtualization layer normally operates in ring 0 and moves the OS
into ring 1 or ring 3 (Figure 5). However, this can create a conflict and potential excessive system
fault when the OS issues a privileged instruction but it is not operating at the privileged ring 0 level.
The result is that the virtual machine monitor must act as a fault handler, executing many lines of code
to determine what the OS has requested and handling the request as an intermediary between the OS
and the hardware.

Figure 5. Today, a typical virtual machine environment allows the OS to operate in ring 1 or ring 3 of the
processor privileged instructions.

Typical Multitasking
OS
Application

Multitasking OS in a
Virtual server environment
Ring 3

Guest
application

Guest
application

Ring 2
Ring 1
OS Kernel/DD

Processor

Ring 0

Ring 3

Ring 2
Guest OS
Kernel/ DD

Guest OS
Kernel/DD

Virtualization layer

Ring 1
Ring 0

Processor

With virtualization extensions, software such as virtual machine monitors will be able to run in a more
privileged ring layer. This will allow guest operating systems to run in their normal ring 0 layer. In
addition, AMD Pacifica and Intel Virtualization Technology are developing hardware support for new
instructions that are specific to virtual machine monitors. The combination of these two factors (higherprivilege ring level and new hardware instructions) will provide a more efficient way for the virtual
machine monitors to virtualize the hardware: It will simplify the way the virtual machine monitor
preempts an instance of an OS, in the same way that an OS preempts an application today in a
multitasking OS environment.

Storage RAID
For years, the computer industry has used arrays of disk drives to improve read/write performance,
increase availability, and provide fault tolerance for disk drive subsystems. In one of its most basic
forms, an array controller manages the physical disk drives and combines multiple physical disk
drives into a larger, single, logical disk drive or logical unit number (LUN) from the standpoint of the
OS. Storage RAID technology took this basic concept of a logical drive and added exclusive-OR
engines to calculate parity data and then store that data on a portion of the physical drive(s). In the
event of a physical drive failure, the OS is unaware that anything has changed in the logical drive:
The RAID controller can recreate the data from parity information even though a physical drive is no
longer accessible. Thus, storage arrays and RAID technology in disk drives present a virtualized or
abstracted view of the physical drives to the OS. The array controller manages that virtualization
process.

Virtualization in the network layers

Two common ways that virtualization occurs today within the network layers of a datacenter use
partitioning techniques, such as in virtual local area networks (VLANs) and platform collection
techniques.

Network partitioning
Ethernet VLANs allow a network administrator to define logical groups of any network-attached end
devices including printers, storage, or compute nodes (for example, servers, workstations, or clients).
The administrator can provide separate virtual LANs that logically isolate one group of nodes from
anotherfor example, setting up all servers, switch ports, or users related to marketing on one VLAN
and setting up all accounting servers on a different VLAN. Even though both may exist on the same
physical network, the VLAN controller in the switch masks the physical attributes of the network by
keeping the two VLANS logically isolated.
On the other hand, a VLAN can also define virtual domains that communicate as if they were on the
same physical network, even if they are physically located on different Ethernet segments (Figure 6).
The use of virtual segments allows multiple physical segments to behave logically as if they were a
single network domain. These same techniques are used for other virtualized networks such as FibreChannel storage area networks zones and InfiniBand Pkey (partition key) networks.

Figure 6. Example of logical group of network components across physically separate network segments.

Platform collection
More recently, network resources are being virtualized through the use of platform collection
technologies, such as those developed by Troika Networks, Inc., (acquired by QLogic Corporation in
2005), and TopSpin Communications, Inc. In platform collection technologies, a single unified fabric
connects multiple servers to Ethernet networks and/or SANs (Figure 7).

10

Figure 7. Representation of a platform collection fabric

Host

Host

Host

Switch
Switch
fabric
fabric
SAN
Address
Mapping
Virtual -Physical

Fibre Channel
Fibre Channel

Ethernet
Address
Mapping
Virtual-Physical

Ethernet
Ethernet

For example, a platform collection fabric can use a hardware-based switch device to virtualize the
Ethernet (IP) or SAN (Fibre Channel) addresses to the servers. The switch device typically performs all
translations of the actual, physical addresses in the LAN or SAN to the virtual addresses that are
presented to the servers by means of the switch fabric. Thus, the platform collection fabric virtualizes
the LAN or SAN network resources: Even though a server appears to be connecting with a specific
local I/O endpoint such as an individual NIC, in reality the endpoint is determined by the fabric and
could be distributed across any of the physical NICs. In other words, this technique presents a number
of servers as a super endpoint on a traditional Ethernet or SAN fabric.

HP direction for future

The previous examples show that virtualization is applied in many different ways in todays
datacenter. HP is working toward a next-generation datacenter architecture that will encompass even
more virtualization technologies to create datacenters in which resources are optimized and supply
automatically meets demand. Even the infrastructure resourcessuch as power and coolingare
being investigated as resources that can be moved or partitioned in response to changing business
requirements. To meet this goal, HP believes that two key components are needed: virtualized
hardware resources and simplified, automated management of those resources.

Virtualized resources using industry-standard platforms

HP is developing key technologies in both hardware and system software that will simplify virtualizing
industry-standard servers. Thus, a server may be identified by its logical or virtual identity (the
applications and OS it is running and the type of LAN and SAN connections it has) rather than by its
physical identity. HP is evaluating options for virtualizing these I/O connections to make it easier to
move a logical identity from one physical resource to another (Figure 8).

11

Figure 8. HP vision for transforming physical resources to virtual and adaptive resources

vm = virtual machine

vm1 vm2 vm3

Hosted virtualization layer
(VMware GSX
Microsoft Virtual Server)

vm1 vm2 vm3

Bare metal
virtualization layer
(VMware ESX)

OS

software identity

Logical
server

Logical server
I/O identity

Virtualization assistance
Ethernet
Ethernet

Fibre
Fibre
Channel
Channel

Ethernet
Ethernet

Fibre
Fibre
Channel
Channel

As a result of servers having a logical identity that can be moved or re-configured as needed, IT
administrators will be able to improve:
Resource utilization by deploying resources where they are needed
Disaster recovery and failover scenarios by seamlessly transferring to different physical resources
HP will use its engineering expertise gained through developing the ProLiant and Integrity server lines
to provide world-class virtualization solutions. Furthermore, HP plans to develop ProLiant virtualization
solutions that will be based on industry standards and will be consistent with existing ProLiant BL, DL,
and ML architectures.

Automated resource management

As shared and virtualized resources continue to proliferate in the datacenter, it is extremely important
that these layers of complexity can be managed efficiently, for example by using a unified resource
management control model. HP is poised to deliver such a management resource by expanding the
roles of HP Systems Insight Manager and HP Openview so that they can provide coordinated
management functions for all logical resources. It is expected that HP SIM and ProLiant Essentials
software tools will supply the automated management needed for monitoring hardware and for
provisioning or reprovisioning servers. In addition, HP plans to expand the role of HP Openview to
monitor and deliver computing resources, allowing the next-generation datacenter to meet business
objectives such as service level agreements.

12

Conclusion
Virtualization is a concept that is being widely touted now because of the pressing need for
computing resources that are truly flexible and adaptive. However, virtualization is not a new
concept. It is simply taking a resource, masking physical characteristics, and dividing it among
multiple users (scale-in virtualization) or conglomerating multiple resources into a single, larger
resource (scale-out virtualization). Multiple examples exist of ways in which virtualization already
exists in the layers of the datacenterin the platform software, platform hardware, and networking
infrastructure layers.
HP is working with internal development teams and with working groups from industry-standard
bodies to ensure that the next-generation datacenter will be equipped with new levels of
virtualization, especially in the area of virtualizing industry-standard server hardware.

13

Glossary
The following terms are used in the text of this document.
Term

Definition

Adaptive
enterprise

HP term for its initiatives to enable virtualization and automation throughout its
portfolio of products.

CPU

Central processing unit, or microprocessor

Grid computing

A computing architecture that shares the processing and storage resources of many
separate computers to solve large-scale computation problems. It differs from a
cluster in that the compute resources are often connected through the Internet.

Hypervisor

Another name for a virtual machine monitor, or the software layer that manages the
process of creating virtual machines in a physical server. Hypervisor software
ensures that only one virtual resource controls a physical resource at any given time
and ensures security and cleanliness of the shared resources.

Initiator

Initiators, such as servers and clients, initiate requests on the network.

LUN

Logical unit number. Originally applied to an address for an individual disk drive
within a SCSI storage array, the term now commonly refers not to the address of an
individual disk drive but to the address of an individual volume, or virtual drive,
within a storage array.

Target device

Target devices, such as storage arrays and printers, are devices that wait until
contacted by an initiator before performing functions.

Utility computing

Utility computing is a service provisioning model in which a service provider

makes computing resources and infrastructure management available to the
customer as needed, and charges them for specific usage rather than a flat rate.
Definition from WhatIs.com website, at:
http://whatis.techtarget.com/definition/0,,sid9_gci904539,00.html

Virtual LAN

Virtual local area network

Virtual machine

A virtual machine provides a faithful implementation of a physical processors

hardware running in a protected and isolated environment. Virtual machine is a
term that is often used interchangeably with virtual server.

Virtual machine
monitor

Software layer that runs directly on the server hardware to enable virtual machines.
Also known as a virtualization layer or a hypervisor.

14

For more information

For additional information, refer to the resources listed below.
Resource description

Web address

AMD website
Processor virtualization information
(AMD Virtualization AMD-V )

www.amd.com/usen/Processors/ProductInformation/0,,30_118_8796,00.html

HP website
Industry-Standard Server
Technology Papers

www.hp.com/servers/technology

ProLiant servers

www.hp.com/go/proliant

ProLiant Essentials (VMM Pack)

www.hp.com/servers/proliantessentials

Intel website
Processor virtualization information

www.intel.com/technology/computing/vptech/

Call to action
Please send comments about this paper to: TechCom@HP.com.

2006 Hewlett-Packard Development Company, L.P. The information contained

herein is subject to change without notice. The only warranties for HP products and
services are set forth in the express warranty statements accompanying such
products and services. Nothing herein should be construed as constituting an
additional warranty. HP shall not be liable for technical or editorial errors or
omissions contained herein.
AMD, AMD Opteron, and combinations thereof are trademarks of Advanced Micro
Devices, Inc.
Intel and Xeon are trademarks or registered trademarks of Intel Corporation or its
subsidiaries in the United States and other countries.
TC060109TB, 1/2006