SUN Logical Domain Quick Start Guide

SUN™ Logical Domains
Quick Start Guide

Peter A. Wilson, Systems Technical Marketing
Gary Combs, Systems Technical Marketing
Part No 821-0316-11
Revision 1.1 1/13/10
Sun Microsystems, Inc.
Table of Contents
Sun™ Logical Domains Quick Start Guide............................................................ 1

Straightforward use case...................................................................................... 2
How this article is organized................................................................................. 3
Initial Server Setup.......................................................................................... 5

Before you begin.................................................................................................. 5
Gather documentation..................................................................................... 5
Assign IP addresses.............................................................................................. 5
Set up the service processor................................................................................. 6
Obtaining the DHCP-assigned address............................................................... 6
Assigning a static IP address............................................................................. 7
Check firmware revision....................................................................................... 8
Update system firmware...................................................................................... 8
Configure the Solaris OS..................................................................................... 10
Command-line setup...................................................................................... 10
Browser user interface setup.......................................................................... 10
Apply operating system patches.......................................................................... 11
Download the Logical Domains software............................................................. 11
Install Logical Domains Software........................................................................ 12
Control and I/O Domain Setup........................................................................ 13

Initial server state.............................................................................................. 13
Create a virtual disk server................................................................................. 14
Create a virtual switch....................................................................................... 14
Create a virtual console device........................................................................... 15
Free CPU, MAU, and memory resources............................................................... 15
Save the configuration in the service processor................................................ 16
Virtual Disk Backend Setup............................................................................. 17

Virtual disk options............................................................................................ 17
Locate or create a disk partition.......................................................................... 17
Guest Domain Creation.................................................................................. 20

Allocate resources to a guest domain.................................................................. 20
Create the guest and allocate resources.............................................................. 21
Boot from the guest domain’s console................................................................. 22
Use Solaris ZFS snapshots and clones.................................................................. 23
Housekeeping............................................................................................... 25
Save configurations to the service processor........................................................ 25
Use proper shutdown procedures........................................................................ 25
Removing guest domains................................................................................... 25
Removing Logical Domains software................................................................... 26
Sun Microsystems, Inc.
Summary...................................................................................................... 27
About the authors.............................................................................................. 28
Acknowledgments............................................................................................. 28
Ordering Sun documents.................................................................................... 28
Accessing Sun documentation online.................................................................. 29
1 Sun Logical Domains Quick Start Guide Sun Microsystems, Inc.
Chapter 1
Sun™ Logical Domains Quick Start Guide
When this document was prepared, Sun™ Sun™ Logical Domains, or Sun LDoms, is Sun’s built-in, no-cost hardware
Logical Domains 1.2 was the current version virtualization technology for servers and blade systems powered by UltraSPARC®
and the document applies to versions 1.2
T1, T2, and T2 Plus processors. These systems are equipped for massive scalability
and greater. Take care to use the latest
recommended versions of LDoms, the using chip multithreading (CMT) technology in which each of up to eight processor
Solaris™ Operating System, and firmware. cores support up to eight concurrent execution threads each for a total of up to
Recommended sets of software can be 64 threads per processor and up to 256 threads per server. The systems based on
found in the Logical Domains Release Notes
these processors deliver excellent performance for highly threaded applications,
for the current version.
along with the lowest power consumption per thread of the current generation of
processors.
LDoms allows server resources to be partitioned and allocated to virtual machines.

Resources that can be partitioned include individual CPU threads (virtual CPUs),
the processor’s cryptographic acceleration modular arithmetic units (MAUs),
memory, PCI root nexus nodes (or “PCI buses”), and network interface units (NIUs).
Architecturally, LDoms is a Type 1 hypervisor because the technology runs on a bare
machine and provides each domain with the illusion that it is running on its own
dedicated hardware — in this case hardware having the SPARC® sun4v architecture.
With LDoms, the control domain manages the hypervisor. I/O is performed through
virtual devices to one or more I/O domains that actually perform physical I/O on
behalf of a domain (these are sometimes referred to as service domains). Figure
.
1 illustrates a server with a control domain that manages the hypervisor, two
I/O domains that perform physical I/O, and three guest domains. In a typical
installation, a combined control and I/O domain performs both functions .
Control I/O Logical Logical Logical I/O

Domain Domain 1 Domain 1 Domain 2 Domain 3 Domain 2
Logical Solaris™ Guest OS Guest OS Guest OS Solaris

Domain 10 OS Image Image Image 10 OS
Manager
Hypervisor
PCI A PCI B
Sun CMT Server
Figure 1. Logical Domains supports multiple guest domains each having their own
secure partition of server resources. Physical I/O is performed by I/O domains that
‘own’ one or more PCI root nexus nodes, or buses.
Where Logical Domains differs significantly from other Type 1 hypervisors on the
market today is that it allows physical server resources (such as CPUs, memory,
MAUs, and even PCI root nexus nodes) to be partitioned, not time shared, between
domains. This provides each domain with dedicated resources and increases isolation
between domains. Partitioning helps to avoid issues such as one domain depriving
other domains of needed resources. Partitioning potentially increases cache
performance by fixing the domain’s relationship to a CPU in contrast to timesharing.
While separating CPU and memory resources is prudent from a resource allocation
standpoint, LDoms also allows I/O resources to be shared in flexible ways to allow
both sharing and separation of resources to improve performance, reliability, and
availability.
Straightforward use case

The purpose of this Sun BluePrints™ article is to provide a simple set of steps for
setting up and running a server with Logical Domains software. This guide is not
intended to be a comprehensive beginner’s guide to LDoms. Instead, it intends to
cover the use case of using Logical Domains in a laboratory environment for the first
time. Thus, we assume a minimal amount of supporting infrastructure, specifically a
desktop or laptop computer connected via network to the server. We do not require,
for example, the use of a network install server.
This article was originally written to describe Logical Domains 1.2 software. The
procedures shown in this document apply to version 1.2 and later, and they may
show references to Logical Domains 1.2 software. We recommend using the latest
version of Logical Domains software for your platform together with the latest
recommended versions of Solaris, patches, and firmware as described in the
applicable Logical Domains Release Notes.
Documentation for all Logical Domains versions is available from http://docs.

sun.com/app/docs/prod/ldoms.
We assume that your environment contains the following:

• Server. A Sun SPARC Enterprise® server based on UltraSPARC T2 or T2 Plus
processors and at least one internal disk drive.
• Operating system. The server runs the version of the Solaris™ Operating System
recommended in the Logical Domains Release Notes for the current LDoms
version.
• Network environment. Both the server’s lights-out management port and its
primary (NET 0) interfaces are connected to a network. The server and its service
processor ports are assigned static IP address.
• Serial port. The server’s serial port should be accessible from the workstation or
laptop either directly or through a terminal concentrator.
• Workstation or laptop. A workstation or laptop is needed to access the server. It

should have a Web browser with a suitable Java™ software plugin.
Note – These assumptions do not cover UltraSPARC T1 processor-powered servers such as the
Sun SPARC Enterprise T1000 and T2000 servers. This guide focuses on administering servers
through Sun’s Integrated Lights-Out Management (ILOM) software using both its command-
line interface and browser user interface (BUI). For UltraSPARC T1 server platforms that use
Sun’s older lights-out management software, ALOM commands will need to be used instead.
How this article is organized

Even these seven steps can be shortend by As this article will demonstrate, it takes only seven commands to set up a guest
using the Logical Domains Configuration domain using Sun™ Logical Domains software using a given virtual disk image:
Assistant described in Appendix D of the
Logical Domains Administration Guide. # ldm create guest1
In a single command, the ldmconfig # ldm set-vcpu 4 guest1
command can set up a virtual disk backend # ldm set-mem 4g guest1
file, create, and start one or more guest # ldm add-vnet vnet0 primary-vsw0 guest1
domains. # ldm add-vdsdev /ldompool/domain1/diskimagefile vol1@primary-vds0
# ldm add-vdisk vdisk0 vol1@primary-vds0 guest1
# ldm bind guest1
There is, however, some preparation needed to set up a new server, update firmware,
install Logical Domains software, and set up virtual devices and disks. That is the
bulk of this article that takes you through the steps from providing a server’s initial
network identity to booting your first logical domain. Although many of these steps
can be condensed by using the Logical Domains Configuration Assistant mentioned
in the sidebar, the purpose of this document is to convey an understanding of the
process and mechanics of setting up a system to run Logical Domains software and
creating guest domains on the system.
• Chapter 2, “Initial Server Setup,” walks you through the tasks that you’ll need
to perform just once: giving your server a network identity, updating the system
firmware, initializing the Solaris OS, and downloading and installing the Logical
Domains Manager software.
• Chapter 3, “Control and I/O Domain Setup,” describes how to turn the OS
running on the server into a control and I/O domain in the context of Logical
Domains software. The chapter shows how to set up the virtual disk and network
infrastructure used by guest domains.
• Chapter 4, “Virtual Disk Backend Setup,” describes the benefits of using Solaris
ZFS™ software to support virtual disks, and it walks you through the steps involved
in carving out a spare disk partition to use as a Solaris ZFS file system.
• Chapter 5, “Guest Domain Creation,” shows how to create a logical domain and
connect it to the virtual infrastructure created in Chapters 3 and 4.
• Chapter 6, “Housekeeping,” describes how to store your current configuration in

the service processor and to follow proper shutdown procedures. It explains how
to remove the guest domain you’ve created and restore your server to its initial
state.
• Chapter 7, “Summary,” summarizes the seven commands that can be used to
create new logical domains and suggests directions for additional exploration.
Chapter 2
Initial Server Setup
This chapter covers the steps that you need to take in order to get your server
connected to the network with current firmware installed.
Before you begin

If you need to upgrade your operating Before you begin, make sure that your server is set up so that it meets the basic
system image, perform these initial setup assumptions under “Straightforward use case” in the previous chapter. This guide
steps before performing the installation.
assumes that you have a new, out-of-the-box server with current firmware and the
version of the Solaris OS required by the Logical Domains Release Notes is installed.
Gather documentation
Even though this guide is intended to give you a complete, self-contained set of
instructions for setting up a first domain using Logical Domains software, it is still
useful to have the basic LDoms documentation on hand to answer questions and
provide more background. Visit http://docs.sun.com/app/docs/prod/
ldoms to download appropriate versions of the following documents:
• Logical Domains Release Notes. This document will give you any last-minute news
on the release and any unresolved issues. You should not run into any issues
following the steps in this guide.
• Logical Domains Administration Guide. This document explains how to administer
Logical Domains software, illustrating a large number of scenarios that are beyond
the scope of this document.
• Logical Domains Reference Manual. This document provides all of the options for
all Logical Domains-related commands in a concise format.
Assign IP addresses
You will need three IP addresses as you work through this guide, one for each of the
following uses:
• Service processor Network Port. This is the port labeled NET MGT.
• Control domain. This is the address of the primary domain on the server, the one
that boots with no Logical Domains software installed. This is typically the port
labeled NET 0.
• Guest domain. This is the address that the guest domain will use when it connects
to the network, and for the purpose of this example is the same NET 0 port as the
control domain.
Obtain these addresses, along with the appropriate netmasks, and router IP
addresses, from your network administrator. Putting the control domain and guest
domain IP addresses on the same subnet makes it easy to set up communication
between the control domain and the guest domain.
Set up the service processor

For help connecting to the server’s serial The first step is to connect to the service processor serial port and assign an IP
port, please refer to the administration address to the service processor’s network port. This procedure shows you how to
guide for your server available at
accomplish this using ILOM commands. With power applied to the server, connect to
http://docs.sun.com.
the serial port and press Return to obtain a login prompt.
Obtaining the DHCP-assigned address

By default, the service processor attempts to obtain an address from a DHCP
server, if one is available. If your network is supported with DHCP, you will need to
determine the address that was assigned so that you can connect via the browser
user interface later on.
Log in with the user root and the initial password changeme. Then use the show
/SP/network command to view the assigned address. In this example, the
address is 10.6.162.25.
SUNSP00144F4641BD login: root

Password: changeme
Waiting for daemons to initialize...
Daemons ready
Sun(TM) Integrated Lights Out Manager
Version 3.0.2.50
Copyright 2008 Sun Microsystems, Inc. All rights reserved.

Use is subject to license terms.
Warning: password is set to factory default.
-> show /SP/network
/SP/network
Targets:
Properties:
commitpending = (Cannot show property)
dhcp_server_ip = none
ipaddress = 10.6.162.25
ipdiscovery = static
ipgateway = 10.6.163.254
ipnetmask = 255.255.252.0
macaddress = 00:14:4F:46:EE:A1
pendingipaddress = 10.6.162.25
pendingipdiscovery = static
pendingipgateway = 10.6.163.254
.
pendingipnetmask = 255.255.252.0
state = enabled
Commands:
cd
set
show
Assigning a static IP address

If you wish to assign a static IP address to the service processor, or override the
address allocated by DHCP, follow these steps.
Log in with the user root and the initial password changeme. Then type the
following commands to set the service console’s IP address and other parameters:
SUNSP00144F4641BD login: root

Password: changeme
Waiting for daemons to initialize...
Daemons ready
Sun(TM) Integrated Lights Out Manager
Version 3.0.2.50
Copyright 2008 Sun Microsystems, Inc. All rights reserved.

Use is subject to license terms.
Warning: password is set to factory default.
-> set /SP/network state=enabled

-> set /SP/network pendingipdiscovery=static
-> set /SP/network pendingipaddress=your-ip-address
-> set /SP/network pendingipnetmask=your-netmask
-> set /SP/network pendingipgateway=your-router-address
-> set /SP/network commitpending=true
Now that the service processor has been set up to have a network identity, you can
choose to make further configuration steps through the browser user interface or
through the serial port.
Check firmware revision

. At the service processor’s command prompt, type the following command line to
obtain the system software version:
-> show /HOST sysfw_version
You can determine your firmware revision from the command output. Firmware
revision numbers are sequential for each server type, so consult http://www.
sun.com/bigadmin/patches/firmware to determine the most current revision.
Locate your server model and see if the listed system firmware revision is greater
than the one installed on your server. If it is, download the latest firmware onto your
workstation and uncompress it. The result should be a package file whose name
ends in .pkg. If your have the latest firmware installed, skip the next section.
Update system firmware

You can both check firmware revisions and update firmware through the service
console’s browser user interface (BUI) by directing your browser to http://
service-processor-address. Click the System Information tab, then the
Versions tab, and you’ll see a display similar to that shown below.
The server power must be turned off before you update the firmware. To turn off
the power, click the Remote Control tab, then the Remote Power Control tab. Select
Graceful Shutdown and Power Off from the pop-up menu and click Save.
To update the service processor firmware, click the Maintenance tab, then click
Firmware Upgrade. Click the Enter Upgrade Mode button.
Now browse to where you have stored the firmware package on your desktop or
laptop, and then click the Upload button.
The upgrade takes several minutes. Do not interrupt the process.
Configure the Solaris OS

If you haven’t already initialized the Solaris OS that is installed on the server, now
is the time. You will need to go through a sequence of steps to give the operating
system its identity, including its name, time zone, and network settings. A new
Solaris OS instance will ask you for this information the first time that it boots.
You need to first power on the server, and then execute the initialization steps
through a command-line interface (via the service console) or through the BUI.
Command-line setup
To work from the command line, you can use an ssh connection and log in to the
service processor’s network port. Use the ILOM start /SYS command to turn
.
on the system’s power, and then connect to the server console through the service
processor.
-> start /SYS

Are you sure you want to start /SYS (y/n)? y
Starting /SYS
-> start /SP/console
Are you sure you want to start /SP/console (y/n)? y
Serial console started. To stop, type #.
Note that the power-on sequence takes some time due to the power-on self tests
that are conducted. Once the Solaris OS boots for the first time, it will prompt you
for information including your language, time zone, and network settings.
Browser user interface setup

The other way to manage power and connect to the operating system console is
through the BUI. First power on the server by selecting the Remote Control tab and
then the Remote Power Control tab. Power on the server by selecting Power On from
the pop-up menu and clicking Save.
To connect to the server console, select the Remote Control tab and then select
Launch Redirection. You can bring up a serial console by selecting I Want To
See Serial Redirection. If you choose the serial console, you will need to use the
sequence of commands under “Command-line setup” above to turn the server power
on. You can bring up a graphical user interface by leaving the option unchecked.
Either approach will allow you to go through the sequence of setting up the system
identity.
Now that you have the Solaris OS initialized and on the network, you can connect to
the server via ssh, through the service console (serial port or ssh), or through the
BUI (graphical or serial console)
Apply operating system patches

.
Refer to the Logical Domains Release Notes for a list of required operating system
patches. Download and apply the patches.
Download the Logical Domains software

Now that your operating system and system firmware is up to date, download the
Logical Domains software. Visit http://www.sun.com/ldoms, click the Get It tab,
and download the current software. For Logical Domains 1.2 the file downloaded was
LDoms_Manager-1_2.zip.
Install Logical Domains Software

It is good security practice to prevent the Now connect to the server console to get a login prompt from the Solaris OS. You
root user from directly logging into a server can do this by ssh, or by connecting to the service processor (via serial port or ssh)
via ssh, which is the default behavior.
and then typing start /SP/console to connect to the server console.
In a laboratory setting, it is sometimes
Transfer the Logical Domains software to the server, and uncompress the zip file.
expedient to override this default. To do so,
edit /etc/ssh/sshd_config. Change
# unzip LDoms_Manager-1_2.zip
the PermitRootLogin property from no
to yes. Then, restart the ssh daemon with
Navigate to the location of the install-ldm script and run it to install the Logical
the command svcadm restart ssh.
Domains Manager software.
# cd LDoms_Manager-1_2
# cd Install
# ./install-ldm
The Logical Domains installation script will ask if you want to harden the server
operating system using the Solaris Security Toolkit. This is an excellent step to follow
for production systems as security of the control domain is important for protecting
guest domains. For the purpose of this quick install exercise, skip the hardening step
proceed with a standard Solaris OS configuration.
At this point you have a server with current firmware, an initialized Solaris operating
system instance, and running Logical Domains software. The next step is to convert
that OS instance into a control and I/O domain.
.
Chapter 3
Control and I/O Domain Setup
Initial server state

A Sun CMT server supports the sun4v architecture, which is implemented in part
by a thin hypervisor layer that is always interposed between the operating system
and the hardware. The hypervisor is always present regardless of whether Logical
Domains software is used. The initial server state is illustrated in Figure 2.
Primary
Control & I/O Domain
Solaris 10 OS and
Logical Domains
Manager
Hypervisor
128 CPU threads

PCI Buses
1 128
16 MAUs 32 GB Memory
Sun CMT Server

.
Internal PCIe Built-in CPU Threads,
Storage Slots Ethernet MAUs, Memory
Figure 2. In the server’s initial state, a single control domain owns all of the system
resources including CPU threads, MAUs, memory, and PCI buses. The control domain
is also referred to as the primary domain.
The example server has two eight-core UltraSPARC T2 Plus processors. The server
has two PCI buses that connect to built-in Ethernet interfaces, internal disk drives,
and PCI Express slots for additional peripherals. Each CPU has 64 threads and eight
modular arithmetic units (MAUs) that accelerate encryption operations.
Verify the server’s initial state using the ldm list command. You’ll note that a
single domain primary owns 128 virtual CPUs (threads) and 64 GB of memory
# ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-c-- SP 128 65312M 0.2% 3d 15h 41m
The end state of the next several steps is illustrated in Figure 3 and consists of the
following:
• A virtual disk server primary-vds0 is ready to connect virtual disks in guest

domains to a service in the primary domain that serves data to them.
• A virtual switch primary-vsw0 is ready to connect virtual network devices in
guest domains to a virtual switch supported in the control and I/O domain. The
virtual switch is connected to a physical Ethernet port.
• The control and I/O domain has given up most of its CPU, MAU, and memory
resources into an unallocated pool that will provide resources to guest domains.
Primary
Control & I/O Domain

Solaris 10 OS & LDM
primary-vsw0
primary-vds0
Unallocated Resources
Hypervisor
primary-vds0
primary-vsw0
124 CPU Threads
4 CPU Threads 1 124
nxge0 24 GB Memory
4 GB Memory
16 MAUs
Sun CMT Server
Built-in Disk Storage Internal Resources

. Ethernet
Figure 3. The end state of this set of steps frees resources and creates virtual device
infrastructure to support guest domains.
Create a virtual disk server

The virtual disk service primary-vds0 will support disk I/O in the guest domain;
actual sources of data will be attached to it later.
# ldm add-vdiskserver primary-vds0 primary
Create a virtual switch

Use the ifconfig command to determine the name of the physical network device
through which the primary domain communicates to the network. In this example,
the device is nxge0.
# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
nxge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 10.6.162.45 netmask fffffc00 broadcast 10.6.163.255
ether 0:14:4f:46:ee:98
Now use the knowledge of that interface to create a virtual switch primary-vsw0
that is connected to the physical interface.
# ldm add-vswitch net-dev=nxge0 primary-vsw0 primary
Create a virtual console device

A virtual console device allows the control and I/O domain to connect to the console
port of the guest domain. When connecting to the guest’s console, you will use the
command telnet localhost port, where port is in the range of 5000-5100
that the virtual console device will assign.
# ldm add-vconscon port-range=5000-5100 primary-vcc0 primary
Free CPU, MAU, and memory resources

The following commands free CPU, MAU, and memory resources, which are currently
allocated to the primary domain. These resources are released into a free pool from
which the resources can be allocated to guest domains as we create them. The
.
commands leave the primary domain with four virtual CPUs, no MAU, and 4 GB of
main memory. Changing these resources puts the Logical Domains Manager into
delayed reconfiguration mode where the actual resources are changed only after the
primary domain is rebooted.
# ldm set-mau 0 primary

LDom primary is unable to dynamically reconfigure crypto units.
Initiating delayed reconfigure operation on LDom primary. All configuration
changes for other LDoms are disabled until the LDom reboots, at which time
the new configuration for LDom primary will also take effect.
# ldm set-vcpu 4 primary
------------------------------------------------------------------------------
Notice: LDom primary is in the process of a delayed reconfiguration.
Any changes made to this LDom will only take effect after it reboots.
------------------------------------------------------------------------------
# ldm set-mem 4g primary
------------------------------------------------------------------------------
Notice: LDom primary is in the process of a delayed reconfiguration.
Any changes made to this LDom will only take effect after it reboots.
------------------------------------------------------------------------------
Save the configuration in the service processor
Note – When making configuration changes, always remember to save the final state in the
service processor.
For the hypervisor to choose this new hardware configuration when the system
is powered on, the configuration must be stored in the service processor. In this
example, we store the changed state in the named location my‑initial.
# ldm add-spconfig my-initial

# ldm list-spconfig
factory-default
my-initial [current]
Now reboot the primary and I/O domain so that the delayed configuration takes
effect.
# shutdown -i6 -g0 -y
.
Chapter 4
Virtual Disk Backend Setup
Virtual disk options

Depending on the underlying storage, this
configuration may create a single point of A guest domain needs to boot from a virtual disk, and the storage for that disk
failure. Techniques for increasing reliability is most easily allocated from an internal disk drive on the server. The storage for
are discussed in the Sun BluePrints articles a virtual disk is called a virtual disk backend, and it can store data in any of the
“Data Reliability with Logical Domains” and
following locations:
“Data Availability with Logical Domains.”
• A flat file stored on a UFS, Solaris ZFS, or other suitable file system
• A physical disk with or without underlying hardware RAID
• A partition on a physical disk
There are significant advantages to using a flat file stored on a Solaris ZFS file
system. You can use snapshots to create a point-in-time image of a virtual disk for
use in backups or roll back a guest domain to a previous, known state. You can use
clones as a quick and space-efficient way to create multiple guest domains from the
same “golden master” Solaris OS installation. When more than one disk device is
dedicated to a ZFS file system pool, various levels of redundancy can be established
to further protect data.
. This set of steps illustrates how you can create a ZFS pool from a single disk
partition. After creating the guest domain we will demonstrate how easy it is to
make a clone of your first guest domain for use in creating subsequent guests.
Locate or create a disk partition

We have assumed that you have at least one internal disk drive on your server, and
that you have a factory-installed OS that is on a UFS file system. This sequence of
steps locates available disk space on one disk and creates a partition for use as a
Solaris ZFS file system.
Use the format command and choose the disk on which you wish to allocate a
partition as a virtual disk backend.
Note – This example repartitions disk 0, where the Solaris OS is usually installed. Take
care when allocating unused space on this volume not to destroy your running Solaris OS
installation for the control domain.
# format
Searching for disks...done
AVAILABLE DISK SELECTIONS:

0. c1t0d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848>
/pci@400/pci@0/pci@8/scsi@0/sd@0,0
1. c1t1d0 <SEAGATE-ST914602SSUN146G-0603-136.73GB>
/pci@400/pci@0/pci@8/scsi@0/sd@1,0
Specify disk (enter its number): 0
The format command will present a menu. Choose p to edit the partition table.
FORMAT MENU:
disk - select a disk
type - select (define) a disk type
partition - select (define) a partition table
current - describe the current disk
format - format and analyze the disk
repair - repair a defective sector
label - write label to the disk
analyze - surface analysis
defect - defect list management
backup - search for backup labels
verify - read and display labels
save - save new disk/partition definitions
inquiry - show vendor, product and revision
volname - set 8-character volume name
!<cmd> - execute <cmd>, then return
quit
. format> p
The format command presents the partition table for the selected disk. The swap
partition uses disk cylinders 0–402. The root partition uses cylinders 403–1912. The
backup partition represents the whole disk, which is 14087 cylinders. We can create
a partition from any of the unassigned partitions. We choose partition 6, starting the
partition at cylinder 1913 (one after the root partition) and make it 12174 cylinders
in size (cylinders 1913–14087). Select 6 to begin this partitioning process.
Current partition table (original):

Total disk cylinders available: 14087 + 2 (reserved cylinders)
Part Tag Flag Cylinders Size Blocks

0 root wm 403 - 1912 14.65GB (1510/0/0) 30731520
1 swap wu 0 - 402 3.91GB (403/0/0) 8201856
2 backup wm 0 - 14086 136.71GB (14087/0/0) 286698624
3 unassigned wm 0 0 (0/0/0) 0
partition> 6
Allocate the remaining space with a partition that begins after the root partition
and extends to the end of the disk. Then choose l to label the disk and q to quit the
partition command.

Enter partition id tag[unassigned]:

Enter partition permission flags[wm]:
Enter new starting cyl[0]: 1913
Enter partition size[0b, 0c, 1913e, 0.00mb, 0.00gb]: 12174c
partition> p
Current partition table (unnamed):
Total disk cylinders available: 14087 + 2 (reserved cylinders)

0 root wm 403 - 1912 14.65GB (1510/0/0) 30731520
1 swap wu 0 - 402 3.91GB (403/0/0) 8201856
2 backup wm 0 - 14086 136.71GB (14087/0/0) 286698624
6 unassigned wm 1913 - 14086 118.14GB (12174/0/0) 247765248
partition> l
Ready to label disk, continue? y
partition> q
. Create a ZFS file system pool ldompool from the c1t0d0s6 partition, and create
the domain1 file system within the pool. By default, ZFS pools are mounted in the
root directory by the name of the pool, and ZFS file systems appear underneath
as directories. Create an empty file with the mkfile command to hold the guest
domain’s disk image. Using the -n option creates the file but does not allocate all of
its data. This example creates a 20 GB file.
# cd /
# zpool create -f ldompool c1t0d0s6
# zfs create ldompool/domain1
# mkfile -n 20g /ldompool/domain1/diskimagefile
Chapter 5
Guest Domain Creation
Allocate resources to a guest domain

The previous chapters have freed up and created resources that now can be allocated
to a guest domain easily. The end state reached in this chapter is illustrated in
Figure 4, and it has the following characteristics:
• A guest domain guest1 has four virtual CPUs and 4 GB memory allocated to it.
• The domain in which it runs has the following:
• A virtual network device connecting it to the network

• A virtual disk connected to the virtual disk backend we created in
the previous chapter
• A virtual cdrom device connected to a downloaded Solaris OS ISO image
Once the guest domain is started, it boots from the virtual cdrom device and begins
an installation of the Solaris OS.
Primary Guest 1
Control & I/O Domain Control & I/O Domain

Solaris 10 OS & LDM Solaris 10 OS
.
/solaris-s10u6-dvd.iso ldompool/domain1/
diskimagefile
primary-vsw0 vnet0 vdisk0 cdrom
primary-vds0
Unallocated Resources
Hypervisor
primary-vds0
primary-vsw0
120 CPU Threads
4 CPU Threads 4 CPU Threads 1 24
nxge0 24 GB Memory
4 GB Memory 4 GB Memory
16 MAUs
Sun CMT Server
Built-in Disk Storage Internal Resources

Ethernet
Figure 4. The end state of this quick-start guide is a guest domain with both physical
and virtual resources connected to it.
Create the guest and allocate resources

Create the guest domain with the ldm create command and allocate physical
resources including CPU threads (virtual CPUs) and memory. In this example, four
virtual CPUs and 4 GB of memory is allocated to the domain guest1.
# ldm create guest1

# ldm set-vcpu 4 guest1
# ldm set-mem 4g guest1
Create a virtual network device vnet0 in guest1 and connect it to the virtual
switch that was created in Chapter 3.
# ldm add-vnet vnet0 primary-vsw0 guest1
Attach the flat file /ldompool/domain1/diskimagefile to the virtual disk

service primary-vds0 with the name vol1. Then make that volume available to
the guest1 domain with the name vdisk0.
# ldm add-vdsdev /ldompool/domain1/diskimagefile vol1@primary-vds0

Note – The Logical Domains Manager does not check to be sure that a file name you provide
is valid. The only indication that you have typed an invalid file name is a virtual disk that does
not work in the guest domain.
Download a Solaris OS ISO image from http://www.sun.com/solaris/get.

.
Connect the downloaded ISO image to the virtual disk service primary-vds0 with
the name cdrom. Then make the volume available to the guest1 domain with the
name cdrom.
# ldm add-vdsdev /solaris-s10u6-dvd.iso cdrom@primary-vds0

# ldm add-vdisk cdrom cdrom@primary-vds0 guest1
Each domain runs its own instance of OpenBoot™ software. You can set the domain’s
boot device so that it first tries to boot from a disk, and then from the virtual cdrom
device. On the first boot attempt, booting from the disk will fail and the domain
will boot from the cdrom device. Once the Solaris OS is installed, the next attempt
to boot from the disk will be successful. Note that you must boot from the cdrom
device’s f partition as shown in the following command.
# ldm set-variable boot-device="disk cdrom:f" guest1
Commit the resources that have been allocated using the bind command.
# ldm bind guest1

You can verify that the resources have been allocated using the
ldm list-bindings command.
# ldm list-bindings guest1

guest1 bound ------ 5000 4 4G
MAC
00:14:4f:fb:e3:4a
HOSTID
0x84fbe34a
VCPU
VID PID UTIL STRAND
0 4 100%
1 5 100%
2 6 100%
3 7 100%
MEMORY
RA PA SIZE
0xe000000 0x10e000000 4G
VARIABLES
boot-device=cdrom:f
NETWORK
NAME SERVICE DEVICE MAC MODE PVID VID
vnet0 primary-vsw0@primary network@0 00:14:4f:fa:c0:ad 1
PEER MAC MODE PVID VID
.
primary-vsw0@primary 00:14:4f:f9:6a:97 1
DISK
NAME VOLUME TOUT DEVICE SERVER MPGROUP
vdisk0 vol1@primary-vds0 disk@0 primary
cdrom cdrom@primary-vds0 disk@1 primary
VCONS
NAME SERVICE PORT
guest1 primary-vcc0@primary 5000
Boot from the guest domain’s console

Use the ldm list command to determine the port number for guest1’s console.
Observe that the console port is 5000.
# ldm list
primary active -n-cv- SP 4 4G 0.8% 5m
guest1 bound ------ 5000 4 4G
Enable the virtual console service (you only need to do this one time), and then start
the guest domain.
# svcadm enable vntsd

# ldm start guest1
LDom guest1 started
Now telnet to guest1’s console. The domain should be in the process of booting
from the Solaris ISO image, and it should begin an interactive Solaris OS installation.
If you do not have automatic booting enabled, you can use commands such as
show-devs and show-nets at the OpenBoot software’s ok prompt to verify that
the devices you created are available to the guest domain.
Note – A good practice is to use a separate ssh connection to the control domain for each
guest domain console connection you make. When you connect to the control domain
through the service processor, only one connection can own the console, which limits
the number of concurrent telnet sessions you can observe. When you create a new ssh
connection to the control domain for each guest domain console connection, you don’t use a
limited resource. Thus, response time is better because the connection is not limited by the
console serial port speed.
# telnet localhost 5000

Trying 127.0.0.1...
Connected to localhost.
Escape character is ‘^]’.
Connecting to console “guest1” in group “guest1” ....

Press ~? for control options ..
Note that, by default, the control domain

.
cannot send network traffic to guest Once the installation is complete, you have a guest domain that runs using its
domains. Therefore, you cannot ping or
own partition of the server’s physical resources, supported by the virtual resources
ssh from the control domain to the guest.
To enable networking between the control
provided through the control and I/O domain. You can boot the domain, shut it
and guest domains, refer to the instructions down, and interact with the OpenBoot software via the guest’s console.
“Enabling Networking Between the Control/
Service Domain and Other Domains” in the Be sure to read on to the next chapter and save the current configuration to the
Logical Domains Administration Guide. service processor.
Use Solaris ZFS snapshots and clones

Since you’ve created the guest’s virtual disk on a Solaris ZFS file system, you can take
advantage of the ability to take snapshots and make clones of the file system.
Details on using Solaris ZFS technology For example, suppose that you don’t want to go through the Solaris OS installation
with Logical Domains are available in process for every new guest domain that you create. You can create a golden master
Chapter 6 of the Sun BluePrints article
virtual disk image that you can clone as many times as needed for each new logical
“Data Reliability with Logical Domains.”
The Logical Domains System Administration domain that you create. One of the benefits of using snapshots and clones in Solaris
Guide also makes suggestions in the section ZFS file systems is that they are very space efficient. Every disk block in a clone is a
“Using ZFS with Virtual Disks” in Chapter 6, reference to the original golden master, and space is allocated only as needed using
“Using Virtual Disks.”
copy-on-write semantics.
To create a clone, first snapshot a Solaris ZFS file system and then make a clone of it.
The cloning process makes a permanent copy of the snapshot. The clone uses copy-
on-write semantics so that the clone is an independent, read/write copy.
Solaris ZFS file system snapshots operate on file systems, not files, so it is a good
practice to put only one virtual disk backend file in each Solaris ZFS file system.
In the following example, we create a snapshot of guest1’s file system named
initial. Then we clone the snapshot and create a new file system golden at the
same time.
# zfs snapshot ldompool/domain1@initial

# zfs clone ldompool/domain1@initial ldompool/golden
The file /ldompool/golden/diskimagefile is a copy of /ldompool/

domain1/diskimagefile except that it occupies almost no disk space until blocks
in the file are changed. If you attach this golden master virtual disk backend to a
logical domain, however, it will have the identity and network settings of the guest1
logical domain you have already created.
You can remove the golden master’s identity with the sys-unconfig command,
however note that this will not remove many useful changes you can prepare in the
golden image, such as patches, applications, and configuration files. This will cause
the Solaris OS to go through the system identity setting prompts upon first boot so
that each clone of the golden master obtains its own identity. To accomplish this,
perform the following steps:
.
1. Shut down the guest1 logical domain.
2. Prepare a new logical domain guest2 using /ldompool/golden/

diskimagefile as its virtual disk backend.
3. Boot guest2 and run the sys-unconfig command. The command will
unconfigure the system and halt it.
4. Snapshot the unconfigured state of the golden master file system.
# zfs snapshot ldompool/golden@goldenmaster
5. Now you can start guest1 without any identity conflicts on the network. You
can start guest2 and give it an identity. To create new logical domains based
on the unconfigured golden master, use the zfs clone command using the
snapshot ldompool/golden@goldenmaster as the source.
Chapter 6
Housekeeping
Save configurations to the service processor

Whenever you change a Logical Domains configuration, be sure to save the
configuration to the service processor. If you don’t do this, and you reboot the
control domain or cycle the power on the server, the service processor will only
re-instate the last saved configuration. This configuration may be as old as the
factory-default configuration. You can save up to eight named configurations
in the service processor. The last one saved is always the one used at the next boot
unless you explicitly set the desired configuration with the ldm set-spconfig
command. One way to save the configuration is to delete and re-create the
configuration that you have already saved, for example:
# ldm remove-spconfig my-initial

# ldm add-spconfig my-initial
Logical Domains software includes a configuration auto-recovery mechanism in case

the configuration was not explicitly saved to the service processor.
Use proper shutdown procedures

. The Logical Domains Release Notes contain important information about how
to properly shut down a server with multiple logical domains. Note that it is not
sufficient to simply shut down the control domain as guest domains will continue to
run. The procedure is to do the following:
1. Shut down and unbind all non-I/O domains. In this example, you could use the
command ldm stop guest1 to gracefully shutdown the guest domain, then
unbind with ldm unbind guest1.
2. Shut down and unbind any active I/O domains. In this example, there is no
secondary I/O domain supporting redundant I/O paths. If there were, you would
use the two commands from Step 1.
3. Shut down and halt the primary domain. You can use the command shutdown
-i5 -g0 -y. Following the shutdown the server will power off since there are
no longer any active domains running.
Removing guest domains

You can use the following sequence of commands to shut down, unbind, and destroy
a logical domain if you no longer need it.
# ldm stop guest1

LDom guest1 stopped
# ldm unbind-domain guest1
# ldm destroy guest1
Removing Logical Domains software

If you wish to restore your server to its original state, not running any logical
domains, you can execute the following sequence of commands.
# ldm rm-config my-initial

# ldm stop-domain -a
LDom guest1 stopped
# ldm unbind-domain guest1
# ldm destroy guest1
# ldm set-config factory-default
# svcadm disable ldmd
# svcadm disable vntsd
# pkgrm SUNWldm
# shutdown -i5 -g0 -y
If you have enabled networking between the control domain and guests, you
may have to manually restore the previous hostname files (for example /etc/
hostname.nxge0) and remove the hostname files that plumbed virtual switches
into the control domain (for example /etc/hostname.vsw0).
.
Chapter 7
Summary
Once you have installed the correct server firmware and the Logical Domains
Manager software, it is simple and straightforward to create new logical domains
given a virtual disk image. As demonstrated in Chapter 1, starting with a populated
virtual disk image file, only seven commands are needed to create a guest domain.
The ldm create command creates the guest. The ldm set-vcpu and ldm set-
mem commands allocate virtual CPUs and memory to the guest. The ldm add-
vnet command gives the guest a network connection. The ldm-add-vdsdev
command attaches a virtual disk to the virtual disk service, and the ldm add-
vdisk command attaches the service to the guest. The ldm bind command
activates the resource allocation.
# ldm create guest1

# ldm set-vcpu 4 guest1
# ldm set-mem 4g guest1
# ldm add-vnet vnet0 primary-vsw0 guest1
# ldm add-vdsdev /ldompool/domain1/diskimagefile vol1@primary-vds0
# ldm bind guest1
Sun Logical Domains provides an enterprise-class virtualization environment that can

provide many more powerful security, availability, and management tools than are
available on physical machines:
.
• Security features such as virtual local-area networks (VLANs) translate directly into
logical domains. In addition, virtual switches can be created to manage each of
the physical interfaces on a server, increasing security and network throughput.
• Data reliability techniques, such as maintaining multiple copies of business-
critical data, translate easily to Logical Domains environments.
• Availability can be enhanced in the virtual world in a way similar to how it is done
in the physical world: with multiple redundant paths to networks and disk storage
through independent I/O domains.
• I/O performance can be enhanced by allowing guest domains to access I/O
devices directly by assigning ownership of individual PCI buses to them (on servers
with more than one PCI bus). Application performance can be enhanced through
the ability to dynamically change the number of virtual CPUs and amount of
memory assigned to a domain without requiring a reboot.
• Guest domains can be created quickly and easily using Solaris ZFS file system
features including snapshots and clones, while using minimal space for new
virtual disks.
• Guest domains can be moved between servers via warm or cold migration,
simplifying datacenter workload management, and simplifying the process of
taking a physical server down for upgrades or maintenance.
Many of these features are described in Sun BluePrints articles available at

http://www.sun.com/blueprints, or in the LDoms Community Cookbook
available at http://wikis.sun.com/display/SolarisLogicalDomains/
LDoms+Community+Cookbook. With an initial guest domain up and running
on a Sun CMT server, an abundance of resources is available to support further
exploration and enterprise deployment.
About the authors

Peter A. Wilson has more than 16 years of industry experience, 14 of which have
been with Sun, serving in a wide variety of hardware, software, systems and product
marketing roles. Peter moved from the United Kingdom to the US in 2000 to lead
the customer tests of Sun’s Netra™ and fault-tolerant servers. Peter is currently a
technical marketing manager responsible for a team of Product Specialists covering
all of Sun’s hardware product portfolio. Peter holds a M.Eng (Master of Engineering)
degree in Microelectronics and Software Engineering from the University of
Newcastle-upon-Tyne, U.K.
Gary Combs is currently a SPARC Specialist in the Systems Technical Marketing Group.
He specializes in mid-range high-end SPARC servers, which includes the popular
. M-Series as well as the higher end CMT servers, such as the Sun SPARC Enterprise
T5440 server. Gary also covers virtualization technologies that are implemented on
these platforms: Dynamic Domains, Logical Domains, and Solaris Containers. Gary
has over 15 years direct sales support experience as a systems engineer. For the
last 10 years, Gary has held marketing positions in product management, product
definition, and now technical marketing.
Acknowledgments
The author would like to thank Steve Gaede, an independent technical writer and
engineer, for preparing this article. Thanks also to Alexandre Chartre, Cathleen
Reiher, and Pierre Reynes for giving the document a thorough review.
Ordering Sun documents

The SunDocsSM program provides more than 250 manuals from Sun Microsystems,
Inc. If you live in the United States, Canada, Europe, or Japan, you can purchase
documentation sets or individual manuals through this program.
Accessing Sun documentation online

The docs.sun.com web site enables you to access Sun technical documentation
online. You can browse the docs.sun.com archive or search for a specific book title or
subject. The URL is
http://docs.sun.com/
To reference Sun BluePrints OnLine articles, visit the Sun BluePrints OnLine Web site:
http://www.sun.com/blueprints/online.html
.
Logical Domains Quick Start Guide Sun Microsystems, Inc.
Sun Microsystems, Inc. 4150 Network Circle, Santa Clara, CA 95054 USA Phone 1-650-960-1300 or 1-800-555-9SUN (9786) Web sun.com
© 2008-2010 Sun Microsystems, Inc. All rights reserved. Sun, Sun Microsystems, the Sun logo, BluePrints, Java, OpenBoot, Solaris, SunDocs, and ZFS are trademarks or registered trademarks of Sun Microsystems,
Inc. or its subsidiaries in the United States and other countries. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the US and other countries.
Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc. Information subject to change without notice.  Printed in USA  1/10

SUN Logical Domain Quick Start Guide

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

SUN Logical Domain Quick Start Guide

Hochgeladen von

Copyright:

Verfügbare Formate

SUN™ Logical Domains

Quick Start Guide

Sun™ Logical Domains Quick Start Guide............................................................ 1

Initial Server Setup.......................................................................................... 5

Control and I/O Domain Setup........................................................................ 13

Virtual Disk Backend Setup............................................................................. 17

Guest Domain Creation.................................................................................. 20

LDoms allows server resources to be partitioned and allocated to virtual machines.

Control I/O Logical Logical Logical I/O

Logical Solaris™ Guest OS Guest OS Guest OS Solaris

Straightforward use case

Documentation for all Logical Domains versions is available from http://docs.

We assume that your environment contains the following:

• Workstation or laptop. A workstation or laptop is needed to access the server. It

How this article is organized

• Chapter 6, “Housekeeping,” describes how to store your current configuration in

Before you begin

Set up the service processor

Obtaining the DHCP-assigned address

SUNSP00144F4641BD login: root

Sun(TM) Integrated Lights Out Manager

Copyright 2008 Sun Microsystems, Inc. All rights reserved.

Warning: password is set to factory default.

-> show /SP/network

Assigning a static IP address

SUNSP00144F4641BD login: root

Sun(TM) Integrated Lights Out Manager

Copyright 2008 Sun Microsystems, Inc. All rights reserved.

Warning: password is set to factory default.

-> set /SP/network state=enabled

Check firmware revision

-> show /HOST sysfw_version

Update system firmware

The upgrade takes several minutes. Do not interrupt the process.

Configure the Solaris OS

-> start /SYS

Serial console started. To stop, type #.

Browser user interface setup

Apply operating system patches

Download the Logical Domains software

Install Logical Domains Software

Initial server state

Control & I/O Domain

128 CPU threads

Sun CMT Server

• A virtual disk server primary-vds0 is ready to connect virtual disks in guest

Control & I/O Domain

124 CPU Threads

4 CPU Threads 1 124

Sun CMT Server

Built-in Disk Storage Internal Resources

Create a virtual disk server

# ldm add-vdiskserver primary-vds0 primary

Create a virtual switch

# ldm add-vswitch net-dev=nxge0 primary-vsw0 primary

Create a virtual console device

# ldm add-vconscon port-range=5000-5100 primary-vcc0 primary

Free CPU, MAU, and memory resources

# ldm set-mau 0 primary

Save the configuration in the service processor

# ldm add-spconfig my-initial

# shutdown -i6 -g0 -y

Virtual disk options