Sie sind auf Seite 1von 89

LDOM Oracle VM for SPARC Tutorial 1

February 18, 2014 in LDOM


SUN LDOMS aka oracle VM for SPARC is a complete virtual machines that runs an independent operating systems and
contains its own virtual CPUs,Storage,Memory, Cryptographic and console.Unlike Solaris zones,LDOMS can run its own
operating system with different kernel patch levels. You can also install Solaris 8,Solaris 9 ,Solaris 10 and Solaris 11 on
LDOMS. LDOM also provides the OBP for each logical domains but we wont get that in zones. LDOM also allow to move
the resources across the logical domains or virtual hosts exclusively. LDOM also allows you to configure zones under
that.

Tutorials Topics:
Installation and Configuring the Primary domain (You are here)
Creating Guest Domains
Configuring I/O Domain
Dynamically adding/removing Resources
Live Migration for Guest Domains
Adding ISO image to Multiple Guests
Configuration Backup & Restore
Removing the LDOM and Services
LDOM Quick Reference

Prerequisite:
Oracle/Sun T-series Server (T5140 Firmware:7.3.3 )
Solaris 10 or Solaris 11 (To be directly installed on T-series server)
LDOM Packages (Ex: 2.1)
Server Hardware Console Access
Contents:
Basic Concepts of LDOM
Installation of LDOM Software
Configuring the Service Domains
Configuring the Control Domains
Saving the SP-Profile
Basic Concepts of LDOM or Oracle VM for SPARC:
LDOM is a hardware based virtualization which works on top the hypervisor. To understand better, you should know the
type of terms used in LDOM or oracle VM for SPARC.
Control Domain is the place where you are going to install the LDOM software and managing the complete logical
domain environment.It used to configure the resources and guest domains
Service domain provides the various virtual services to guest domains.Virtaul services can be virtual disks,network
switches or virtual consoles.Mostly we will be configuring the control domain as service domain too.
I/O Domains have a direct ownership of PCI Bus.You need to directly map the I/O devices to guest domain to improve
the application performance in some cases. There is an another domain called root domain and this domain has
direct ownership of PCI devcies. So it also called I/O domain.
Guest domains doesnt performing any of the above mentioned roles.It jsut runs the operating system instances.
Control domain and service domain can be run together.But the user application should not be installed on this domain
in a order to protect domain stability and performance.

LDOM Basic concept

Installation of LDOM software:
Download the LDOM packages from the oracle support website. For your information ,you cant download without having
the oracle support login credentials. Once you have downloaded the packages ,just copy the packages to the Solaris host
which you have already installed on T- series server.
1.The below mentioned packages needs to be installed .

-rwxr-xr-x 1 root root 3564851 Jan 16 23:26 OracleVM-Server-for-SPARC-2.0.0_(64-bit).zip
drwxrwxrwx 5 root root 512 Jan 16 23:26 SUNWldomr.v
drwxrwxrwx 5 root root 512 Jan 16 23:26 SUNWldomu.v
2.Install the SUNWldomr and SUNWldomu packages first.
-bash-3.2# pkgadd -d . SUNWldom*

Processing package instance from </>

Solaris Logical Domains (Root)(sparc.sun4v) 11.10.0,REV=2006.10.04.00.26
Copyright (c) 2011, Oracle and/or its affiliates. All rights reserved.

This appears to be an attempt to install the same architecture and
version of a package which is already installed. This installation
will attempt to overwrite this package.

Using </> as the package base directory.
## Processing package information.
## Processing system information.
42 package pathnames are already properly installed.
## Verifying package dependencies.
## Verifying disk space requirements.
## Checking for conflicts with packages already installed.
## Checking for setuid/setgid programs.

This package contains scripts which will be executed with super-user
permission during the process of installing this package.

Do you want to continue with the installation of [y,n,?] y

Installing Solaris Logical Domains (Root) as

## Executing preinstall script.
## Installing part 1 of 1.
[ verifying class ]
## Executing postinstall script.

Installation of was successful.

Processing package instance from </>

Solaris Logical Domains (Usr)(sparc.sun4v) 11.10.0,REV=2006.08.08.12.13
Copyright (c) 2011, Oracle and/or its affiliates. All rights reserved.

This appears to be an attempt to install the same architecture and
version of a package which is already installed. This installation
will attempt to overwrite this package.

Using </> as the package base directory.
## Processing package information.
## Processing system information.
11 package pathnames are already properly installed.
## Verifying package dependencies.
## Verifying disk space requirements.
## Checking for conflicts with packages already installed.
## Checking for setuid/setgid programs.

This package contains scripts which will be executed with super-user
permission during the process of installing this package.

Do you want to continue with the installation of [y,n,?] y

Installing Solaris Logical Domains (Usr) as

## Installing part 1 of 1.

Installation of was successful.
-bash-3.2#
3.Install the LDOM software . Unzip the package and continue the installation like below
-bash-3.2# cd OVM_Server_SPARC-2_0
-bash-3.2# ls -lrt
total 12
-r--r--r-- 1 root root 3368 Sep 17 2010 README
drwxr-xr-x 2 root root 512 Sep 18 2010 Install
drwxr-xr-x 7 root root 512 Dec 16 2010 Product
-bash-3.2# cd Install/
-bash-3.2# ls -lrt
total 56
-rwxr-xr-x 1 root root 28471 Sep 18 2010 install-ldm
-bash-3.2# ./install-ldm

Welcome to the Oracle VM Server for SPARC installer.

You are about to install the LDoms Manager package, SUNWldm, that will enable
you to create, destroy and control other domains on your system. You will
also be given the option of running the Oracle VM Server for SPARC Configuration
Assistant (ldmconfig) to setup the control domain and create guest domains.

If the Solaris Security Toolkit (SST) is installed, you will be prompted to
optionally harden your control domain.

Installing LDoms packages.
pkgadd -n -d "/OVM_Server_SPARC-2_0/Product" -a pkg_admin SUNWldm.v
Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.
Feb 17 10:37:54 sol10-10 sendmail[2631]: [ID 702911 mail.alert] unable to qualify my own domain name (sol10-10)
-- using short name

Installation of was successful.
pkgadd -n -d "/OVM_Server_SPARC-2_0/Product" -a pkg_admin SUNWldmp2v
Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.
Feb 17 10:38:54 sol10-10 sendmail[2705]: [ID 702911 mail.alert] unable to qualify my own domain name (sol10-10)
-- using short name

Installation of was successful.

Oracle VM Server for SPARC Configuration

Once installed, you may configure your system for a basic LDoms
deployment. If you select "y" for the following question, the Oracle
VM Server for SPARC Configuration Assistant (tty) will be launched
following a successful installation of the packages.

(You may launch the Configuration Assistant at a later time with the
command: /usr/sbin/ldmconfig, or use the GUI Configuration Assistant
which is bundled in the Oracle VM Server for SPARC zip file - see
README.GUI for more details)

Select an option for configuration:

y) Yes, launch the Configuration Assistant after install
n) No thanks, I will configure the system manually later

Enter y or n [y]: n

Verifying that all packages are fully installed. OK.
You have new mail in /var/mail//root
-bash-3.2#
Configuring the Service Domains:
Login to the Solaris host where you have installed the LDOM or oracle VM software.
1.List the configured services.By default it will be nothing.

-bash-3.2# ldm list-services
2.Create a new virtual disk service.This service is responsible to provide the virtual disks to guest domains.
-bash-3.2# ldm add-vdiskserver primary-vds0 primary
-bash-3.2# ldm list-services
VDS
NAME LDOM VOLUME OPTIONS MPGROUP DEVICE
primary-vds0 primary
-bash-3.2#
3.Create a new virtual switch. By creating this , we can create N-number of VNIC and share to the guest domains.
-bash-3.2# dladm show-dev
nxge0 link: up speed: 1000 Mbps duplex: full
nxge1 link: up speed: 1000 Mbps duplex: full
nxge2 link: up speed: 1000 Mbps duplex: full
nxge3 link: up speed: 1000 Mbps duplex: full
-bash-3.2# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
nxge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 192.168.2.11 netmask ff000000 broadcast 192.168.2.255
ether 0:21:98:57:cd:16
-bash-3.2# ldm add-vswitch net-dev=nxge1 primary-vsw0 primary
-bash-3.2#
-bash-3.2# ldm list-services
VSW
NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID
PVID VID MTU MODE
primary-vsw0 primary 00:14:4f:f9:05:b1 nxge1 0 switch@0 1 1
1500

VDS
NAME LDOM VOLUME OPTIONS MPGROUP DEVICE
primary-vds0 primary

-bash-3.2#
Here i have configured virtual switch using nxge1 . You can also use nxge0 configure the virtual swtich but you need to
remove the IP from nxge0 and need to configure the IP over virtual switch.
-bash-3.2# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
vsw0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
inet 198.168.2.11 netmask ff000000 broadcast 192.168.2.255
ether 0:14:4f:98:83:f3
-bash-3.2#
4.Create a virtual service for terminal consoles.
-bash-3.2# ldm add-vconscon port-range=5000-5100 primary-vc0 primary
-bash-3.2# ldm list-services
VCC
NAME LDOM PORT-RANGE
primary-vc0 primary 5000-5100

VDS
NAME LDOM VOLUME OPTIONS MPGROUP DEVICE
primary-vds0 primary

-bash-3.2#
VSW
NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID
PVID VID MTU MODE
primary-vsw0 primary 00:14:4f:f9:05:b1 nxge1 0 switch@0 1 1
1500
Now you have successfully configured the default services or service domain.
(Creating the spconfig is must after making any changes on the service.Otherwise you will loose the changes after the
system power cycle.)
Configuring the control Domains:
You need to login to Solaris host where you have installed the ldom software to configure the control domain.
1.List the logical domain devices using ldm list-device -a command.You can see all the resource has been allocated to the
primary domain.
bash-3.2# ldm list-devices -a
CORE
ID %FREE CPUSET
0 0 (0, 1, 2, 3, 4, 5, 6, 7)
1 0 (8, 9, 10, 11, 12, 13, 14, 15)
2 0 (16, 17, 18, 19, 20, 21, 22, 23)
3 0 (24, 25, 26, 27, 28, 29, 30, 31)
4 0 (64, 65, 66, 67, 68, 69, 70, 71)
5 0 (72, 73, 74, 75, 76, 77, 78, 79)
6 0 (80, 81, 82, 83, 84, 85, 86, 87)
7 0 (88, 89, 90, 91, 92, 93, 94, 95)

VCPU
PID %FREE PM
0 0 no
1 0 no
2 0 no
3 0 no
4 0 no
5 0 no
6 0 no
7 0 no
8 0 no
9 0 no
10 0 no
11 0 no
12 0 no
13 0 no
14 0 no
15 0 no
16 0 no
17 0 no
18 0 no
19 0 no
20 0 no
21 0 no
22 0 no
23 0 no
24 0 no
25 0 no
26 0 no
27 0 no
28 0 no
29 0 no
30 0 no
31 0 no
64 0 no
65 0 no
66 0 no
67 0 no
68 0 no
69 0 no
70 0 no
71 0 no
72 0 no
73 0 no
74 0 no
75 0 no
76 0 no
77 0 no
78 0 no
79 0 no
80 0 no
81 0 no
82 0 no
83 0 no
84 0 no
85 0 no
86 0 no
87 0 no
88 0 no
89 0 no
90 0 no
91 0 no
92 0 no
93 0 no
94 0 no
95 0 no

MAU
ID CPUSET BOUND
0 (0, 1, 2, 3, 4, 5, 6, 7) primary
1 (8, 9, 10, 11, 12, 13, 14, 15) primary
2 (16, 17, 18, 19, 20, 21, 22, 23) primary
3 (24, 25, 26, 27, 28, 29, 30, 31) primary
4 (64, 65, 66, 67, 68, 69, 70, 71) primary
5 (72, 73, 74, 75, 76, 77, 78, 79) primary
6 (80, 81, 82, 83, 84, 85, 86, 87) primary
7 (88, 89, 90, 91, 92, 93, 94, 95) primary

MEMORY
PA SIZE BOUND
0x0 512K _sys_
0x80000 1536K _sys_
0x200000 94M _sys_
0x6000000 32M _sys_
0x8000000 96M _sys_
0xe000000 32544M primary

IO
DEVICE PSEUDONYM BOUND OPTIONS
pci@400 pci_0 yes
pci@500 pci_1 yes

bash-3.2#
2.List what are the resources assigned to the primary domain or control domains
.
bash-3.2# ldm list-bindings primary
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-c-- SP 64 32544M 0.1% 9m

UUID
f063b0a8-8fb0-4e61-cd5b-91826d2c1550

MAC
00:21:28:57:cd:16

HOSTID
0x8557cd16

CONTROL
failure-policy=ignore

DEPENDENCY
master=

CORE
CID CPUSET
0 (0, 1, 2, 3, 4, 5, 6, 7)
1 (8, 9, 10, 11, 12, 13, 14, 15)
2 (16, 17, 18, 19, 20, 21, 22, 23)
3 (24, 25, 26, 27, 28, 29, 30, 31)
4 (64, 65, 66, 67, 68, 69, 70, 71)
5 (72, 73, 74, 75, 76, 77, 78, 79)
6 (80, 81, 82, 83, 84, 85, 86, 87)
7 (88, 89, 90, 91, 92, 93, 94, 95)

VCPU
VID PID CID UTIL STRAND
0 0 0 1.5% 100%
1 1 0 0.0% 100%
2 2 0 0.0% 100%
3 3 0 0.0% 100%
4 4 0 0.0% 100%
5 5 0 0.0% 100%
6 6 0 0.0% 100%
7 7 0 0.0% 100%
8 8 1 0.0% 100%
9 9 1 0.1% 100%
10 10 1 0.0% 100%
11 11 1 0.1% 100%
12 12 1 0.1% 100%
13 13 1 3.0% 100%
14 14 1 0.0% 100%
15 15 1 0.1% 100%
16 16 2 0.1% 100%
17 17 2 0.2% 100%
18 18 2 0.1% 100%
19 19 2 0.0% 100%
20 20 2 0.1% 100%
21 21 2 0.0% 100%
22 22 2 0.1% 100%
23 23 2 0.0% 100%
24 24 3 0.0% 100%
25 25 3 0.0% 100%
26 26 3 0.0% 100%
27 27 3 0.0% 100%
28 28 3 0.2% 100%
29 29 3 0.1% 100%
30 30 3 0.1% 100%
31 31 3 0.0% 100%
64 64 4 0.0% 100%
65 65 4 0.0% 100%
66 66 4 0.0% 100%
67 67 4 0.0% 100%
68 68 4 0.0% 100%
69 69 4 0.0% 100%
70 70 4 0.0% 100%
71 71 4 0.0% 100%
72 72 5 0.0% 100%
73 73 5 0.0% 100%
74 74 5 0.0% 100%
75 75 5 0.0% 100%
76 76 5 0.0% 100%
77 77 5 0.0% 100%
78 78 5 0.0% 100%
79 79 5 0.0% 100%
80 80 6 0.0% 100%
81 81 6 0.0% 100%
82 82 6 0.0% 100%
83 83 6 0.0% 100%
84 84 6 0.0% 100%
85 85 6 0.0% 100%
86 86 6 0.0% 100%
87 87 6 0.0% 100%
88 88 7 0.0% 100%
89 89 7 0.0% 100%
90 90 7 0.0% 100%
91 91 7 0.0% 100%
92 92 7 0.0% 100%
93 93 7 0.0% 100%
94 94 7 0.0% 100%
95 95 7 0.0% 100%

MAU
ID CPUSET
0 (0, 1, 2, 3, 4, 5, 6, 7)
1 (8, 9, 10, 11, 12, 13, 14, 15)
2 (16, 17, 18, 19, 20, 21, 22, 23)
3 (24, 25, 26, 27, 28, 29, 30, 31)
4 (64, 65, 66, 67, 68, 69, 70, 71)
5 (72, 73, 74, 75, 76, 77, 78, 79)
6 (80, 81, 82, 83, 84, 85, 86, 87)
7 (88, 89, 90, 91, 92, 93, 94, 95)

MEMORY
RA PA SIZE
0xe000000 0xe000000 32544M

IO
DEVICE PSEUDONYM OPTIONS
pci@400 pci_0
pci@500 pci_1

VCONS
NAME SERVICE PORT
SP

bash-3.2#
3.Set the resource control for the control domain aka primary domain.So the resource will be freed up for the logical
domains / Guest domains.
bash-3.2# ldm set-mau 1 primary
bash-3.2# ldm set-vcpu 8 primary
bash-3.2# ldm set-memory 4G primary
A ldm set-mem 4G command would remove 28448MB, which is not a
multiple of 256MB. Instead, run ldm set-mem 4128M primary
to ensure a 256MB alignment.
bash-3.2# ldm set-memory 3872M primary
mau Security Encryption (LDM commands will be sent in encrypted manner.)
vpcu Virtual CPU cores
memory Physical memory
4.Now list the devices again to verify the new settings.
bash-3.2# ldm ls-devices -a
CORE
ID %FREE CPUSET
0 0 (0, 1, 2, 3, 4, 5, 6, 7)
1 100 (8, 9, 10, 11, 12, 13, 14, 15) ----------->(1-7) Free CPU cores
2 100 (16, 17, 18, 19, 20, 21, 22, 23)
3 100 (24, 25, 26, 27, 28, 29, 30, 31)
4 100 (64, 65, 66, 67, 68, 69, 70, 71)
5 100 (72, 73, 74, 75, 76, 77, 78, 79)
6 100 (80, 81, 82, 83, 84, 85, 86, 87)
7 100 (88, 89, 90, 91, 92, 93, 94, 95)

VCPU
PID %FREE PM
0 0 no ----------These (0-7) CPU is not available since its assigned to primary domain
1 0 no
2 0 no
3 0 no
4 0 no
5 0 no
6 0 no
7 0 no
8 100 ---
9 100 ---
10 100 ---
11 100 ---
12 100 ---
13 100 ---
14 100 ---
15 100 ---
16 100 ---
17 100 ---
18 100 ---
19 100 ---
20 100 ---
21 100 ---
22 100 ---
23 100 ---
24 100 ---
25 100 ---
26 100 ---
27 100 ---
28 100 ---
29 100 ---
30 100 ---
31 100 ---
64 100 ---
65 100 ---
66 100 ---
67 100 ---
68 100 ---
69 100 ---
70 100 ---
71 100 ---
72 100 ---
73 100 ---
74 100 ---
75 100 ---
76 100 ---
77 100 ---
78 100 ---
79 100 ---
80 100 ---
81 100 ---
82 100 ---
83 100 ---
84 100 ---
85 100 ---
86 100 ---
87 100 ---
88 100 ---
89 100 ---
90 100 ---
91 100 ---
92 100 ---
93 100 ---
94 100 ---
95 100 ---

MAU
ID CPUSET BOUND
0 (0, 1, 2, 3, 4, 5, 6, 7) primary
1 (8, 9, 10, 11, 12, 13, 14, 15)
2 (16, 17, 18, 19, 20, 21, 22, 23)
3 (24, 25, 26, 27, 28, 29, 30, 31)
4 (64, 65, 66, 67, 68, 69, 70, 71)
5 (72, 73, 74, 75, 76, 77, 78, 79)
6 (80, 81, 82, 83, 84, 85, 86, 87)
7 (88, 89, 90, 91, 92, 93, 94, 95)

MEMORY
PA SIZE BOUND
0x0 512K _sys_
0x80000 1536K _sys_
0x200000 94M _sys_
0x6000000 32M _sys_
0x8000000 96M _sys_
0xe000000 32M primary
0x10000000 28G -----------------------------Free Memory
0x710000000 3840M primary -----------Allocated to Primary or control Domain

IO
DEVICE PSEUDONYM BOUND OPTIONS
pci@400 pci_0 yes
pci@500 pci_1 yes

bash-3.2#
You have successfully configured the control domain.
Note:Here you havent saved the configuration.These are all in-memory configuration.After the power cycle, you will loose
the configuration if its not saved in sp config profile.
Saving the sp-config file:
You have to save the configuration after every ldom settings that you made. For an example, if you didnt save, ldom
configuration will be flushed after server power cycle.
1.Create a new profile the current configuration.
bash-3.2# ldm add-spconfig unixarena
bash-3.2# ldm ls-spconfig
factory-default
unixarena [current]
bash-3.2#
In the above command output, you can see that currents settings are saved in unixarena and this settings will come up
after the system reboot and power cycle.
For an example , if i do any small changes on ldom ,it will not update in the spconfig file.
bash-3.2# ldm set-vcpu 12 primary
bash-3.2# ldm ls-spconfig
factory-default
unixarena [next poweron]
bash-3.2#
Here we just increased the number vpcus to 12 . The previous number of vpcu was 8. Here sp-config file unixarena will
be activated on next power cycle. Which mean you will loose the cpu setting you just done. In a order to save the
configuration, just re-create the sp-config like below.
bash-3.2# ldm ls-spconfig
factory-default
unixarena [next poweron]
bash-3.2# ldm remove-spconfig unixarena
bash-3.2# ldm ls-spconfig
factory-default [next poweron]
bash-3.2# ldm add-spconfig unixarena_new
bash-3.2# ldm ls-spconfig
factory-default
unixarena_new [current]
bash-3.2#
Checking the OBP,Hypervisor and LDOM version:
-bash-3.2# ldm -V

Logical Domain Manager (v 2.0)
Hypervisor control protocol v 1.4
Using Hypervisor MD v 1.1

System PROM:
Hypervisor v. 1.7.4. @(#)Hypervisor 1.7.4.a 2009/09/21 08:25\015

OpenBoot v. 4.30.4 @(#)OBP 4.30.4 2009/08/19 07:25
-bash-3.2#
Hope this article is first step to learn LDOM for Solaris beginners.In this article we have seen the basic concepts of oracle
VM for SPARC (LDOM), packages installation,configuration the service domain,configuring the control domain or primary
domain and saving the sp-configuration.
I will try to complete the tutorial by adding another 6 to 8 articles. Hope you will like it . Please share it in social media to
reach maximum number of people. Thank you for visiting UnixArena
LDOM Tutorial Part 2 Configuring the guest domain




How to configure the Guest domains on LDOM ?
February 18, 2014 in LDOM


Guest domain is a top level virtual machine which we can create using the control domain. Guest domain will not perform
any administrative operation on ldom but runs its own operating system on it.Once you have completed the Oracle VM for
SPARC tutorial 1 ,then you can start following this article.We will see how we can create a new guest domain and allocate
the resources to it.
Oracle VM for SPARC Tutorial:
Installation and Configuring the Primary domain
Creating Guest Domains (You Are Here)
Configuring I/O Domain
Dynamically adding/removing Resources
Live Migration for Guest Domains
Adding ISO image to Multiple Guests
Configuration Backup & Restore
Removing the LDOM and Services

Creating the new Guest Domain
1. Login to the control domain which we have already configured . (Refer Oracle VM for SPARC tutorial 1 )
2. You need to find out how much resources are available in the system.Use ldm list-devices -a command to get those
details.
PD-bash# ldm list-devices -a
CORE
ID %FREE CPUSET
0 0 (0, 1, 2, 3, 4, 5, 6, 7)
1 100 (8, 9, 10, 11, 12, 13, 14, 15)
2 100 (16, 17, 18, 19, 20, 21, 22, 23)
3 100 (24, 25, 26, 27, 28, 29, 30, 31)
4 100 (64, 65, 66, 67, 68, 69, 70, 71)
5 100 (72, 73, 74, 75, 76, 77, 78, 79)
6 100 (80, 81, 82, 83, 84, 85, 86, 87)
7 100 (88, 89, 90, 91, 92, 93, 94, 95)

VCPU
PID %FREE PM
0 0 no
1 0 no
2 0 no
3 0 no
4 0 no
5 0 no
6 0 no
7 0 no
8 100 --- -------------->Free CPU's cores starts here.
9 100 ---
10 100 ---
11 100 ---
12 100 ---
13 100 ---
14 100 ---
15 100 ---
16 100 ---
17 100 ---
18 100 ---
19 100 ---
20 100 ---
21 100 ---
22 100 ---
23 100 ---
24 100 ---
25 100 ---
26 100 ---
27 100 ---
28 100 ---
29 100 ---
30 100 ---
31 100 ---
64 100 ---
65 100 ---
66 100 ---
67 100 ---
68 100 ---
69 100 ---
70 100 ---
71 100 ---
72 100 ---
73 100 ---
74 100 ---
75 100 ---
76 100 ---
77 100 ---
78 100 ---
79 100 ---
80 100 ---
81 100 ---
82 100 ---
83 100 ---
84 100 ---
85 100 ---
86 100 ---
87 100 ---
88 100 ---
89 100 ---
90 100 ---
91 100 ---
92 100 ---
93 100 ---
94 100 ---
95 100 --- -------------->Free CPU's cores ends here.

MAU
ID CPUSET BOUND
0 (0, 1, 2, 3, 4, 5, 6, 7) primary
1 (8, 9, 10, 11, 12, 13, 14, 15)
2 (16, 17, 18, 19, 20, 21, 22, 23)
3 (24, 25, 26, 27, 28, 29, 30, 31)
4 (64, 65, 66, 67, 68, 69, 70, 71)
5 (72, 73, 74, 75, 76, 77, 78, 79)
6 (80, 81, 82, 83, 84, 85, 86, 87)
7 (88, 89, 90, 91, 92, 93, 94, 95)

MEMORY
PA SIZE BOUND
0x0 512K _sys_
0x80000 1536K _sys_
0x200000 94M _sys_
0x6000000 32M _sys_
0x8000000 96M _sys_
0xe000000 32M primary
0x10000000 28416M ------------------------------> Free Physical memory
0x700000000 4G primary

IO
DEVICE PSEUDONYM BOUND OPTIONS
pci@400 pci_0 yes
pci@500 pci_1 yes

PD-bash#

#
3.Create a new Guest domain using the below command.
PD-bash# ldm add-domain ua1
PD-bash# ldm list-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 8 4128M 0.3% 17m
ua1 inactive ------
PD-bash#
4.Add 8 vcpus to guest domain ua1.
PD-bash# ldm add-vcpu 8 ua1
PD-bash# ldm list-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 8 4128M 0.5% 17m
ua1 inactive ------ 8
PD-bash#
5.Add 4G memory to logical domain ua1.
PD-bash# ldm add-memory 4G ua1
PD-bash# ldm list-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 8 4128M 0.4% 18m
ua1 inactive ------ 8 4G
PD-bash#
6.Create a new virtual NIC on top the virtual switch and assigned to ua1. List the logical domain bindings to verify the
configuration.
PD-bash# ldm add-vnet vnet0 primary-vsw0 ua1
PD-bash# ldm list-bindings ua1
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
ua1 inactive ------ 8 4G

UUID
1e8baa72-a123-6184-f724-feebad1585eb

CONTROL
failure-policy=ignore

DEPENDENCY
master=

NETWORK
NAME SERVICE ID DEVICE MAC MODE PVID VID
MTU LINKPROP
vnet0 primary-vsw0 0 00:14:4f:fb:54:92 1

PD-bash#
7.Now its time to add the new disks to logical domain ua1 . Here I am using /dev/dsk/c4t0456RTE000005400002d0s2
LUN for ua1 domain .
8.First i am allocating the LUN to virtual disk server.
PD-bash# ldm add-vdiskserverdevice /dev/dsk/c4t600144F0B8E706DA00005300F4350005d0s2 vol0@primary-vds0
PD-bash# ldm list-services vds0
LDom "vds0" was not found
PD-bash# ldm list-services
VCC
NAME LDOM PORT-RANGE
primary-vc0 primary 5000-5100

VSW
NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID
PVID VID MTU MODE
primary-vsw0 primary 00:14:4f:fa:12:0e nxge1 0 switch@0 1 1
1500

VDS
NAME LDOM VOLUME OPTIONS MPGROUP DEVICE
primary-vds0 primary vol0
/dev/dsk/c4t0456RTE000005400002d0s2

PD-bash#
9.Now bind the virtual disk to logical domain ua1 which you have created in step 8 .
PD-bash# ldm add-vdisk vdisk0 vol0@primary-vds0 ua1
PD-bash# ldm list-bindings ua1
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
ua1 inactive ------ 8 4G

UUID
1e8baa72-a123-6184-f724-feebad1585eb

CONTROL
failure-policy=ignore

DEPENDENCY
master=

NETWORK
NAME SERVICE ID DEVICE MAC MODE PVID VID
MTU LINKPROP
vnet0 primary-vsw0 0 00:14:4f:fb:54:92 1

DISK
NAME VOLUME TOUT ID DEVICE SERVER MPGROUP
vdisk0 vol0@primary-vds0 0

PD-bash#
10.The same way you can map the ISO image file to guest domain using the below set of commands.
PD-bash# ldm add-vdiskserverdevice /export/home/sol-10-u10-ga2-sparc-dvd.iso
iso0@primary-vds0
PD-bash# ldm add-vdisk iso0 iso0@primary-vds0 ua1
PD-bash# ldm list-bindings ua1
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
ua1 inactive ------ 8 4G

UUID
1e8baa72-a123-6184-f724-feebad1585eb

CONTROL
failure-policy=ignore

DEPENDENCY
master=

NETWORK
NAME SERVICE ID DEVICE MAC MODE PVID VID
MTU LINKPROP
vnet0 primary-vsw0 0 00:14:4f:fb:54:92 1

DISK
NAME VOLUME TOUT ID DEVICE SERVER MPGROUP
vdisk0 vol0@primary-vds0 0
iso0 iso0@primary-vds0 1

PD-bash#
11.Just bind the logical domain to the primary domain.
PD-bash# ldm list-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 8 4128M 0.4% 26m
ua1 inactive ------ 8 4G
PD-bash#
PD-bash# ldm bind-domain ua1
LDom ua1 started
PD-bash# ldm list-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 8 4128M 0.7% 30m
ua1 bind ------ 5000 8 4G
PD-bash#
12.Set the auto-boot is false to logical domain ua1 for Operating system installation.
PD-bash# ldm set-variable auto-boot\?=false ua1
13.Boot/start the logical domain ua1.
PD-bash# ldm start-domain ua1
LDom ua1 started
PD-bash# ldm list-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 8 4128M 0.7% 30m
ua1 active -t---- 5000 8 4G 12% 1s
PD-bash#
14. Take the console of the logical domain ua1 .It uses the console port 5000 (see step:13)
ok boot iso0
PD-bash# telnet 192.168.2.11 5000
Trying 1192.168.2.11...
Connected to 192.168.2.11.
Escape character is '^]'.

Connecting to console "ua1" in group "ua1" ....
Press ~? for control options ..

{0} ok
15. List the devices to boot.
{0} ok devalias
iso0 /virtual-devices@100/channel-devices@200/disk@1
vdisk0 /virtual-devices@100/channel-devices@200/disk@0
vnet0 /virtual-devices@100/channel-devices@200/network@0
net /virtual-devices@100/channel-devices@200/network@0
disk /virtual-devices@100/channel-devices@200/disk@0
virtual-console /virtual-devices/console@1
name aliases
{0} ok
16.Boot the system from iso.
{0} ok boot iso0
Boot device: /virtual-devices@100/channel-devices@200/disk@1 File and args:
SunOS Release 5.10 Version Generic_147440-01 64-bit
Copyright (c) 1983, 2011, Oracle and/or its affiliates. All rights reserved.
Configuring devices.
Using RPC Bootparams for network configuration information.
Attempting to configure interface vnet0...
|
You have successfully created the guest domain and assigned the resources to it. You have also mapped the virtual disks
to the virtualdisk server and mapped to the guest domain or logical domain. In the end of the article ,we have see how to
access the console of the guest domains and mapped the ISO file for OS installation.
Hope this article is informative to you .Please share it in social networks. Thank you for visiting UnixArena.


How to Configure the I/O Domain on Oracle VM for SPARC ?
February 19, 2014 in LDOM


If you allow any of the guest domain or control domain to access the any I/O (PCI) cards directly from the hardware
,usually that domain is called I/O domain or root domain.By default, two virtual I/O services that you need to configure on
primary domain to provide the I/O to the guest domains.One is virtual disk (vds) for storage and another one is virtual
switch(vsw) for network communication.But in I/O domain,you can directly access the real physical LUN or physical
network port.But the typical guest domain will be using the virtual I/O services.For an example, SAN LUNs will be mapped
to control domain and from there , you can map the physical LUNS to guest domains as virtual services.
Oracle VM for SPARC Tutorial:
Installation and Configuring the Primary domain
Creating Guest Domains
Configuring I/O Domain (You Are Here)
Dynamically adding/removing Resources
Live Migration for Guest Domains
Adding ISO image to Multiple Guests
Configuration Backup & Restore
Removing the LDOM and Services

IO Domain Example
In the above image, control domain aka primary domain and SOL10 Guest domain also called I/O domains since its
directly accessing the PCI card(HBA). Here we will see how we can configure the guest domain as I/O domain.
1. Login to control domain and list the complete hardware details of the server .
-bash-3.2# ldm list-devices -a
CORE
ID %FREE CPUSET
0 0 (0, 1, 2, 3)
1 0 (4, 5, 6, 7)
2 0 (8, 9, 10, 11)
3 0 (12, 13, 14, 15)
4 0 (16, 17, 18, 19)
5 0 (20, 21, 22, 23)
6 0 (24, 25, 26, 27)
7 0 (28, 29, 30, 31)

VCPU
PID %FREE PM
0 0 no
1 0 no
2 0 no
3 0 no
4 0 no
5 0 no
6 0 no
7 0 no
8 0 no
9 0 no
10 0 no
11 0 no
12 0 no
13 0 no
14 0 no
15 0 no
16 0 no
17 0 no
18 0 no
19 0 no
20 0 no
21 0 no
22 0 no
23 0 no
24 0 no
25 0 no
26 0 no
27 0 no
28 0 no
29 0 no
30 0 no
31 0 no

MAU
ID CPUSET BOUND
0 (0, 1, 2, 3) primary
1 (4, 5, 6, 7)
2 (8, 9, 10, 11)
3 (12, 13, 14, 15)
4 (16, 17, 18, 19)
5 (20, 21, 22, 23)
6 (24, 25, 26, 27)
7 (28, 29, 30, 31)

MEMORY
PA SIZE BOUND
0x0 512K _sys_
0x80000 1536K _sys_
0x200000 62M _sys_
0x4000000 64M _sys_
0x8000000 2G primary
0x88000000 512M ldom1
0xa8000000 512M ldom3
0xc8000000 896M ldom2

IO
DEVICE PSEUDONYM BOUND OPTIONS
pci@780 bus_a yes
pci@7c0 bus_b yes

-bash-3.2#
In the bottom of the above command output shows that IO (pci@780 and pci7c0) have already bind to some of the
domain.
2.To find where the PCI IO card has been bound , you need to use ldm list-bindings command.
-bash-3.2# ldm list-bindings
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -ndcv- SP 8 2G 0.6% 4h 58m

UUID
0066bb91-df76-c6ca-a92b-f18faf256978

MAC
00:14:4f:9d:ca:6a

HOSTID
0x849dca6a

CONTROL
failure-policy=ignore

DEPENDENCY
master=

CORE
CID CPUSET
0 (0, 1, 2, 3)
1 (4, 5, 6, 7)

VCPU
VID PID CID UTIL STRAND
0 0 0 0.2% 100%
1 1 0 0.2% 100%
2 2 0 0.1% 100%
3 3 0 0.1% 100%
4 4 1 0.1% 100%
5 5 1 0.4% 100%
6 6 1 0.1% 100%
7 7 1 0.1% 100%

MAU
ID CPUSET
0 (0, 1, 2, 3)

MEMORY
RA PA SIZE
0x408000000 0x8000000 2G

VARIABLES
keyboard-layout=US-English

IO
DEVICE PSEUDONYM OPTIONS
pci@780 bus_a
pci@7c0 bus_b
The above outputs confirms both the PCI card has been assigned to primary domain.So our primary domain is also called
I/O domain or root domain.
3.We will remove one the PCI card from primary and we will assign to one of the guest domain. Before removing,please
make sure that card is not used in primary .If its in use, please un-configure it .Let me confirm which card can be removed
.(pci@780 and pci@7c0)
pci@780 is used for internal disks. We are good to move the pci@7c0 to one of the guest domains since there is no
device is currently using it from primary domain.
AVAILABLE DISK SELECTIONS:
0. c0t0d0
/pci@780/pci@0/pci@9/scsi@0/sd@0,0
1. c0t1d0
/pci@780/pci@0/pci@9/scsi@0/sd@1,0
-bash-3.2# dladm show-dev
vsw0 link: up speed: 1000 Mbps duplex: full
e1000g0 link: up speed: 1000 Mbps duplex: full
e1000g1 link: unknown speed: 0 Mbps duplex: half
e1000g2 link: unknown speed: 0 Mbps duplex: half
e1000g3 link: unknown speed: 0 Mbps duplex: half
-bash-3.2# cat /etc/path_to_inst |grep e1000g
"/pci@780/pci@0/pci@1/network@0" 0 "e1000g"
"/pci@780/pci@0/pci@1/network@0,1" 1 "e1000g"
"/pci@7c0/pci@0/pci@2/network@0" 2 "e1000g"
"/pci@7c0/pci@0/pci@2/network@0,1" 3 "e1000g"
-bash-3.2#
4.List the configured guest domain.
-bash-3.2# ldm list-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 8 2G 0.5% 4h 51m
ldom1 bound ------ 5000 8 512M
ldom2 bound ------ 5001 8 896M
ldom3 active -n---- 5002 8 512M 0.0% 3h 20m
-bash-3.2#
5. Remove or unbind the pci@7c0 to from primary domain.You can see couple of physical network adapters has been
removed from primary.You may need to reboot the server to release the pci cards sometimes.
-bash-3.2# ldm remove-io pci@7c0
-bash-3.2# dladm show-dev
vsw0 link: up speed: 1000 Mbps duplex: full
e1000g0 link: up speed: 1000 Mbps duplex: full
e1000g1 link: unknown speed: 0 Mbps duplex: half
-bash-3.2#
-bash-3.2# ldm list-bindings
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 8 2G 0.2% 3h 57m

IO
DEVICE PSEUDONYM OPTIONS
pci@780 bus_a
6.Now add the pci card pci7c0 to ldom1 .sometimes you may need to reboot the system to see the new pci cards.
# ldm add-io pci@7c0 ldom1
Initiating a delayed reconfiguration operation on the primary domain.
All configuration changes for other domains are disabled until the primary
domain reboots, at which time the new configuration for the primary domain
will also take effect.
-bash-3.2#
7.You can list the binding using ldm list-bindings command to confirm the changes.sometimes you need to reboot the
system to see the devices in operating system level.
ldm list-bindings ldom1
NAME SERVICE PORT
SP

------------------------------------------------------------------------------
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
ldom1 bound ------ 5000 8 512M

UUID
613e0c0c-3554-610d-e36f-9441124ce387

MAC
00:14:4f:fb:11:83

HOSTID
0x84fb1183

CONTROL
failure-policy=ignore

DEPENDENCY
master=

CORE
CID CPUSET
2 (8, 9, 10, 11)
3 (12, 13, 14, 15)

VCPU
VID PID CID UTIL STRAND
0 8 2 100%
1 9 2 100%
2 10 2 100%
3 11 2 100%
4 12 3 100%
5 13 3 100%
6 14 3 100%
7 15 3 100%

MEMORY
RA PA SIZE
0x8000000 0x88000000 512M

VARIABLES
auto-boot?=false
boot-device=vdisk1:a disk net
keyboard-layout=US-English

IO
DEVICE PSEUDONYM OPTIONS
pci@7c0 bus_b
8. Login to ldom1 and see the pci card devices .
-bash-3.2# dladm show-dev
vsw0 link: up speed: 1000 Mbps duplex: full
e1000g2 link: up speed: 1000 Mbps duplex: full
e1000g3 link: up speed: 1000 Mbps duplex: full
-bash-3.2#
Now ldom1 is also called as I/O domain since its accessing the network port directly without hypervisor. The same way if
your system have multiple HBA-cards , you can assign one of the cards to guest domain and SAN team can directly
allocate tothe luns to guest domain.(ohhh sorry now its called I/O domain)
You have successfully removed the pci card from primary domain and assigned to the guest domain. So the guest domain
also become I/O domain now. These I/O domain will be used on special application requirements. Otherwise we will
configure only the guest domains.

How to Dynamically add the resources in Oracle VM for SPARC
?
February 20, 2014 in LDOM


The beauty of LDOM or Oracle VM for SPARC is that resources can be dynamically added or removed from the
domains(Primary/Guest). This will help you to perform the load balancing between the guest domains on the physical
server. Dynamic resource allocation is only possible if the guest domain operating system is booted.If you add
the resource during the guest domain OS start-up, allocation will fail.
Note:Solaris operating system support dynamic resources allocation.
Oracle VM for SPARC Tutorial:
Installation and Configuring the Primary domain
Creating Guest Domains
Configuring I/O Domain
Dynamically adding/removing Resources (You Are Here)
Live Migration for Guest Domains
Adding ISO image to Multiple Guests
Configuration Backup & Restore
Removing the LDOM and Services

Here we will see how to add/remove resources dynamically when the guest domain is up and running.Before performing
any resource management, please make sure you have enough resources on the system .(use # ldm list-devices -a to
find free resources)
1.List the running domains from the control domain aka primary domain.
bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 14 4128M 0.2% 7h 48m
ldom2 active -n---- 5000 4 4G 0.1% 9h 28m
ldom2clone active -n---- 5001 8 8G 0.1% 7h 7m
As per the above output,all the domains are in active state.Which means domains are in powered.(Login to the ldoms and
verify OS is up or not .)
2.Add four more virtual CPUs to ldom2 guest domain.
-bash-3.2# ldm add-vcpu 4 ldom2
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 14 4128M 0.3% 7h 48m
ldom2 active -n---- 5000 8 4G 0.3% 9h 28m
ldom2clone active -n---- 5001 8 8G 0.1% 7h 7m
You can see number of vcpu has been increased from 4 to 8 for ldom2.
3.Number CPUs can added by directly setting the new value.For an example, Currently ldom2 has just 8 vcpu .If you want
to add four more, you just give the below command.
-bash-3.2# ldm set-vcpu 12 ldom2
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 14 4128M 0.3% 7h 49m
ldom2 active -n---- 5000 12 4G 0.1% 9h 29m
ldom2clone active -n---- 5001 8 8G 0.1% 7h 8m
4.CPU also can be removed using set command. Currently ldom2 has 12 vcpus. If you want to remove 8 from that, just
give the below command.
-bash-3.2# ldm set-vcpu 4 ldom2
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 14 4128M 0.4% 7h 49m
ldom2 active -n---- 5000 4 4G 3.4% 9h 29m
ldom2clone active -n---- 5001 8 8G 0.1% 7h 8m
-bash-3.2#
5.Removing vcpu also can be done using remove-vpcu command.
-bash-3.2# ldm remove-vcpu 2 ldom2
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 14 4128M 0.2% 7h 52m
ldom2 active -n---- 5000 2 6G 0.2% 9h 32m
ldom2clone active -n---- 5001 8 8G 0.1% 7h 11m
-bash-3.2#
6. Dynamic memory management also same like the cpu management. You need to use memory keyword instead of
vcpu. To set the new memory size to the running domain,use the below commands.If the new value size is higher than
the current one ,memory will be added .If the new value is lesser than the existing one, memory size will be reduced to the
guest domains.
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 14 4128M 0.2% 7h 51m
ldom2 active -n---- 5000 4 4G 0.1% 9h 30m
ldom2clone active -n---- 5001 8 8G 0.1% 7h 10m
-bash-3.2# ldm set-memory 8G ldom2
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 14 4128M 0.2% 7h 51m
ldom2 active -n---- 5000 4 8G 0.2% 9h 31m
ldom2clone active -n---- 5001 8 8G 0.1% 7h 10m
7.We can also use the below method to increase the physical memory of guest domains.
-bash-3.2# ldm add-memory 2G ldom2
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 14 4128M 0.3% 7h 51m
ldom2 active -n---- 5000 4 10G 0.1% 9h 31m
ldom2clone active -n---- 5001 8 8G 0.2% 7h 10m
8. Like remove option in vcpu, remove-memory option is used to reduce the memory size to the guest domain.
-bash-3.2# ldm remove-memory 4G ldom2
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 14 4128M 0.4% 7h 52m
ldom2 active -n---- 5000 4 6G 0.5% 9h 32m
ldom2clone active -n---- 5001 8 8G 0.2% 7h 11m
-bash-3.2#
Other resources like virtual disk also can be added on fly to the virtual machine.Here I am creating a ZFS level LUN for
test purpose.
-bash-3.2# zfs create -V 25G pool1/uavol1
-bash-3.2# zfs list
NAME USED AVAIL REFER MOUNTPOINT
pool1 25.4G 248G 31K /pool1
pool1/clonevol1 96.3M 248G 4.75G -
pool1/uavol1 25.3G 248G 4.70G -
pool1/vol1 25.3G 269G 4.70G -
Adding the disk to virtual disk service. (vds0)
-bash-3.2# ldm add-vdiskserverdevice /dev/zvol/dsk/pool1/uavol1 uavol1@primary-vds0
Assigning the LUN to guest domain from the virtual disk service.
-bash-3.2# ldm add-vdisk uavdisk uavol1@primary-vds0 ldom2
Connect to the server and scan the newly assiged LUN.
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 14 4128M 0.2% 8h
ldom2 active -n---- 5000 2 6G 0.2% 9h 40m
ldom2clone active -n---- 5001 8 8G 0.1% 7h 19m
-bash-3.2# telnet 192.168.2.28 5000
Trying 192.168.2.28...
Connected to 192.168.2.28.
Escape character is '^]'.

Connecting to console "ldom2" in group "ldom2" ....
Press ~? for control options ..

ldom2-208 console login: root
Password:
# devfsadm
# echo |format
To terminate the ldom console, use ctrl + ] key sequence to back to the control domain.
You can also see the lun from the guest domain OK prompt. Let me halt the domain and start it.
-bash-3.2# ldm stop-domain ldom2
LDom ldom2 stopped
-bash-3.2# ldm start-domain ldom2
LDom ldom2 started
Connect to the guest domain console again.
-bash-3.2# telnet 192.168.2.28 5000
Trying 192.168.2.28...
Connected to 192.168.2.28.
Escape character is '^]'.

Connecting to console "ldom2" in group "ldom2" ....
Press ~? for control options ..

{0} ok devalias
uavdisk /virtual-devices@100/channel-devices@200/disk@0
iso_ldom2 /virtual-devices@100/channel-devices@200/disk@2
vdisk1_dom2 /virtual-devices@100/channel-devices@200/disk@1
vnet0 /virtual-devices@100/channel-devices@200/network@0
net /virtual-devices@100/channel-devices@200/network@0
disk /virtual-devices@100/channel-devices@200/disk@1
virtual-console /virtual-devices/console@1
name aliases
You can see the newly assigned disk in the OBP level as well.So the bottom line is that resource management is very
simple and effective in LDOM aka oracle VM for SPARC.
Hope you too liked the Oracle VM for SPARC resource management. Thank you for visiting UnixArena.




How to perform Live domain migration on LDOM ? Oracle VM
SPARC
February 20, 2014 in LDOM


In Oracle VM for SPARC,we can migrate the guest domains to another system without any downtime. This is similar to
vmotion on VMware ESXi. This feature provides the operational flexibility to the customers who dont want the system to
be down for system maintenance.Prior to the system maintenance ,we can easily move the guest domains to another
system without any downtime using this live migration.Once you have completed the system maintenance , you can just
migrate the guest domains back to the original systems.
Oracle VM for SPARC Tutorial:
Installation and Configuring the Primary domain
Creating Guest Domains
Configuring I/O Domain
Dynamically adding/removing Resources
Live Migration for Guest Domains (You Are Here)
Adding ISO image to Multiple Guests
Configuration Backup & Restore
Removing the LDOM and Services

Normally we would allocate the same set of LUNS to source and target machines. The live-migration feature is just only
for the Guest domain migration not for the storage.So when the control domains verify the remote system, it will check
whether the guest machine storage is accessible or not.
Prerequisites:
Network access between Control Domains
Shared Storage
Compatible versions of Oracle VM Server for SPARC and Firmware.
Similar Hardware configuration on both the systems
Identical Virtual services between the nodes.
How the migration is happening without any downtime ?
Migration has been initiated by one of the control domain root user
Control domain performs the nessasary check prior to the migration.It checks server compatibitliy,network
connnectivity and available resources
Virtual machine configuration is copied from source to target
Virtaul machines in-memory contents will be copied to target system
Guest domain will be suspended on source system
Finally guest will be started on the target system
Here we will see how we can perform the live migration between two T5140 systems.
1. Login to the source system and list the guest domains.
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 12 4128M 0.5% 1d 15h 45m
ua1guest active -n---- 5000 12 6G 0.1% 1h 35m
-bash-3.2#
2.List the guest domain ua1guest bindings.Make sure the remote system should has enough enough resource to handle
this guest domains. (Use # ldm list-devices -a)
-bash-3.2# ldm ls-bindings ua1guest
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
ua1guest active -n---- 5000 12 6G 0.1% 1h 36m

UUID
7b74fd6b-88ea-c578-d0df-efe202fb8c12

MAC
00:14:4f:f9:87:e7

HOSTID
0x84f987e7

CONTROL
failure-policy=ignore

DEPENDENCY
master=

CORE
CID CPUSET
1 (8, 9, 10, 11, 12, 13, 14, 15)
3 (24, 25, 26, 27, 28, 29, 30, 31)

VCPU
VID PID CID UTIL STRAND
0 8 1 0.2% 100%
1 9 1 0.1% 100%
2 10 1 0.1% 100%
3 11 1 0.0% 100%
4 12 1 0.0% 100%
5 13 1 0.3% 100%
6 14 1 0.0% 100%
7 15 1 0.0% 100%
8 24 3 0.0% 100%
9 25 3 0.0% 100%
10 26 3 0.0% 100%
11 27 3 0.1% 100%

MEMORY
RA PA SIZE
0x10000000 0x10000000 4G
0x400000000 0x110000000 2G

VARIABLES
auto-boot?=false
keyboard-layout=US-English

NETWORK
NAME SERVICE ID DEVICE MAC MODE PVID VID
MTU LINKPROP
vnet0 primary-vsw0@primary 0 network@0 00:14:4f:fa:b2:d6 1
1500
PEER MAC MODE PVID VID MTU LINKPROP
primary-vsw0@primary 00:14:4f:fa:12:0e 1 1500

DISK
NAME VOLUME TOUT ID DEVICE SERVER MPGROUP
ua1guestdsk ua1guest@primary-vds0 0 disk@0 primary

VCONS
NAME SERVICE PORT
ua1guest primary-vc0@primary 5000

-bash-3.2#
3.List the source systems services. You should also have the same services name configured on the target system.
-bash-3.2# ldm list-services
VCC
NAME LDOM PORT-RANGE
primary-vc0 primary 5000-5100

VSW
NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID
PVID VID MTU MODE
primary-vsw0 primary 00:14:4f:fa:12:0e nxge1 0 switch@0 1 1
1500

VDS
NAME LDOM VOLUME OPTIONS MPGROUP DEVICE
primary-vds0 primary ua1guest
/dev/rdsk/c4t600144F0C1135B490000530383E50007d0s2
GSTISO ro /export/home/sol-10-u10-
ga2-sparc-dvd.iso

-bash-3.2#
4.Now login to target system and check whether the live migration can be done or not.
-bash-3.2# ldm list-services
VCC
NAME LDOM PORT-RANGE
primary-vc0 primary 5000-5100

VSW
NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID
PVID VID MTU MODE
primary-vsw0 primary 00:14:4f:fb:a5:6d nxge1 0 switch@0 1 1
1500

VDS
NAME LDOM VOLUME OPTIONS MPGROUP DEVICE
primary-vds0 primary ua1guest
/dev/rdsk/c4t600144F0C1135B490000530383E50007d0s2

-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 8 8G 0.2% 1d 6h 23m
subuldom inactive ------ 4 4G
-bash-3.2#
As per the above command output, all the services names are matching with source system. We also have the enough
resource to handle the guest domain.
5. Login to the source systems control domain and execute the migration command.It will prompt for target system root
password. Just enter continue.
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 12 4128M 0.3% 1d 15h 53m
ua1guest active -n---- 5000 12 6G 0.0% 1h 42m
-bash-3.2# ldm migrate ua1guest 192.168.2.29
Target Password:
6.Login to the target systems control domain and list the domains.
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 8 8G 0.2% 1d 6h 23m
subuldom inactive ------ 4 4G
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 8 8G 14% 1d 6h 26m
ua1guest bound -----t 5000 1 6G
subuldom inactive ------ 4 4G
-bash-3.2#
In the above output, you can see , ua1guest domain is copying from the source system.
7.In the source system , you may get a warning like the below one after successfully completion of live-migration.
Domain ua1guest is not running an operating system that is
compatible with the latest migration functionality.

Upgrading to the latest software will remove restrictions on
a migrated domain that are in effect until it is rebooted.
Consult the product documentation for a full description of
these restrictions.
-bash-3.2#
8.In target system, the guest domain flag will be changed from t to n .
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 8 8G 44% 1d 6h 28m
ua1guest bound -----t 5000 1 6G
subuldom inactive ------ 4 4G
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 8 8G 0.9% 1d 6h 32m
ua1guest active -n---- 5000 12 6G 0.1% 12h 11m
subuldom inactive ------ 4 4G
-bash-3.2#
9.You didnt believe the live migration ? Let me double check.
-bash-3.2# telnet localhost 5000
Trying ::1...
telnet: connect to address ::1: Network is unreachable
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

Connecting to console "ua1guest" in group "ua1guest" ....
Press ~? for control options ..

UAMIGD console login: root
Password:
Last login: Thu Feb 20 00:50:39 on console
Oracle Corporation SunOS 5.10 Generic Patch January 2005
# uptime
2:02am up 1:43, 1 user, load average: 0.00, 0.02, 0.01
#
As per the above command output, nothing has happened to the guest domain. The system is up from last 2hr. The
migration has been initiated few minutes back.So the guest domain was up during the migration time too.
Hope you will also give a try on LDOM guest live migration . Thank you for visiting UnixArena.




How to Add the Same ISO image file in Multiple LDOMS ?
February 20, 2014 in LDOM


In oracle VM for SPARC (LDOM), you need to attach the ISO file to each logical domain, for the first time operating
system installation. If you have pre-configured jumpstart , You can use it for the guest domain operating system
installation. If you dont have jumpstart,then you need to install it using the ISO image. By default LDOM will not allow to
map the same device file to multiple guest domains .Here we will see how we can map the same ISO image to multiple
LDOMS.
Oracle VM for SPARC Tutorial:
Installation and Configuring the Primary domain
Creating Guest Domains
Configuring I/O Domain
Dynamically adding/removing Resources
Live Migration for Guest Domains
Adding ISO image to Multiple Guests (You Are Here)
Configuration Backup & Restore
Removing the LDOM and Services

1. Login to the control domain and list the ISO image which you need to map it to ldoms.
# ls /export/home/sol-10-u10-ga2-sparc-dvd.iso
sol-10-u10-ga2-sparc-dvd.iso
2.List the available services. Create a new virtual device for the virtual disk server using the below command.
-bash-3.2# ldm list-services
VCC
NAME LDOM PORT-RANGE
primary-vcc0 primary 5000-5100

VSW
NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID
PVID VID MTU MODE
primary-vsw0 primary 00:14:4f:fa:fe:49 e1000g0 0 switch@0 1 1
1500

VDS
NAME LDOM VOLUME OPTIONS MPGROUP DEVICE
primary-vds0 primary vol1 /dev/zvol/dsk/pool1/vol1
clonevol1
/dev/zvol/dsk/pool1/clonevol1
vol0
/dev/dsk/c2t600144F0B8E706DA00005300F47B0007d0s2
vol10
/dev/dsk/c2t600144F0C1135B4900005303839A0002d0s2

# ldm add-vdsdev options=ro /export/home/sol-10-u10-ga2-sparc-dvd.iso newiso@primary-vds0
-bash-3.2# ldm list-services
VCC
NAME LDOM PORT-RANGE
primary-vcc0 primary 5000-5100

VSW
NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID
PVID VID MTU MODE
primary-vsw0 primary 00:14:4f:fa:fe:49 e1000g0 0 switch@0 1 1
1500

VDS
NAME LDOM VOLUME OPTIONS MPGROUP DEVICE
primary-vds0 primary vol1 /dev/zvol/dsk/pool1/vol1
clonevol1 /dev/zvol/dsk/pool1/clonevol1
vol0
/dev/dsk/c2t600144F0B8E706DA00005300F47B0007d0s2
vol10
/dev/dsk/c2t600144F0C1135B4900005303839A0002d0s2
iso0 ro /export/home/sol-10-u10-ga2-sparc-dvd.iso
3.Add the newly created virtual read-only iso image to guest domain.
-bash-3.2# ldm list-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 12 4128M 0.3% 5h 45m
ua1 active -n---- 5000 8 4G 0.1% 58m
ua2 active -n---- 5001 8 2G 0.0% 58m
ua3 active -n---- 5002 4 1G 0.1% 30m
-bash-3.2# ldm add-vdisk viso1 newiso@primary-vds0 ua1
-bash-3.2#
4.You can see that ISO file has been attached to the ua1 ldom.
-bash-3.2# ldm ls-bindings ldom1
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
ldom1 bound ------ 5000 8 512M

DISK
NAME VOLUME TOUT ID DEVICE SERVER MPGROUP
vdisk1 vol1@primary-vds0 2 disk@2 primary
viso1 newiso@primary-vds0 0 disk@0 primary
5.When try to map the same iso virtual image to another domain, you will get below error.
-bash-3.2# ldm add-vdisk viso2 newiso@primary-vds0 ua2
The volume "newiso" is already in use.
6.To avoid the above error, you need to create a new virtual read-only ISO image again.
-bash-3.2# ldm add-vdsdev options=ro /export/home/sol-10-u10-ga2-sparc-dvd.iso newiso2@primary-vds0
-bash-3.2#
7.Add the second virtual iso image to the another LDOM using the below command.
-bash-3.2# ldm add-vdisk viso2 newiso2@primary-vds0 ua2
-bash-3.2#
-bash-3.2# ldm list-bindings ua2
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
ua2 active -n---- 5001 8 2G 0.1% 1h 3m
DISK
NAME VOLUME TOUT ID DEVICE SERVER MPGROUP
vdisk1 ua1ldom2@primary-vds0 0 disk@0 primary
viso2 newiso2@primary-vds0 1 disk@1 primary
The same way you can map the same ISO file to N-number of guest domains.(6 and 7 ).
Hope this article is informative to you. Thank you for visiting UnixArena.


Oracle VM SPARC LDOM configuration Backup and Restore
February 21, 2014 in LDOM


Oracle VM for SPARC configuration can be backed up in XML file and it can be restored it whenever required.This
configuration backup will be very useful for future rebuilding. It also provides the option to take the backup of particular
domain and that can be restored using ldm add-domain command.In this article we will how take the individual domain
backup,how to take the complete ldom configuration backup and restore methods.
Oracle VM for SPARC Tutorial:
Installation and Configuring the Primary domain
Creating Guest Domains
Configuring I/O Domain
Dynamically adding/removing Resources
Live Migration for Guest Domains
Adding ISO image to Multiple Guests
Configuration Backup & Restore (You Are Here)
Removing the LDOM and Services
1. Login to control Domain and list the current configuration file.
-bash-3.2# ldm list-spconfig
factory-default
config_new [current]
-bash-3.2#
2.To take the complete LDOM configuration backup,use below command.
# ldm list-constraints -x >config.xml
3.You can also take the configuration backup for specific LDOM,
# ldm list-constraints -x ldom_name > ldom_name.xml
How to restore the system to factory default ?
1. Login to the control domain console and remove the spconfig profile.
bash-3.2# ldm ls-spconfig
factory-default
config_new [current]
bash-3.2# ldm remove-spconfig config_new
bash-3.2# ldm ls-spconfig
factory-default [next poweron]
bash-3.2#
2.Power cycle the server to take the changes effect.
bash-3.2# init 5
bash-3.2# svc.startd: The system is coming down. Please wait.
svc.startd: 101 system services are now being stopped.
Feb 18 10:00:49 sol10-11 syslogd: going down on signal 15
svc.startd: The system is down.
syncing file systems... done

Serial console stopped.

-> show /SYS
Properties:
type = Host System
ipmi_name = /SYS
keyswitch_state = Normal
product_name = T5140
product_part_number = 4613802-42
product_serial_number = FML1015013
product_manufacturer = SUN MICROSYSTEMS
fault_state = Faulted
power_state = Off
-> start /SYS
Are you sure you want to start /SYS (y/n)? y
Starting /SYS

->
-> start /SP/console
Are you sure you want to start /SP/console (y/n)? y

Serial console started. To stop, type #.

Done
0:0:0>Master CPU Tests Basic....Done
0:0:0>Init MMU.....
0:0:0>L2 Tests....Done
0:0:0>Extended CPU Tests....Done
0:0:0>Scrub Memory....Done
0:0:0>Functional CPU Tests....Done
sol10-11 console login:
3. Login to the system and see whether factory default settings has been restored or not .
sol10-11 console login: root
Password:
Feb 18 10:10:58 sol10-11 login: ROOT LOGIN /dev/console
Last login: Tue Feb 18 01:19:27 on console
Oracle Corporation SunOS 5.10 Generic Patch January 2005
You have new mail.
-bash-3.2#

-bash-3.2# ldm list-spconfig
factory-default [current]
-bash-3.2#
Restoring the sp-config :
1.Once the system has restored to the factory default,then you can restore the system backup using the XML files which
we have created. Login to the control domain and restore it.
# ldm init-system -i config.xml
# ldm add-spconfig unixarencfg1
# ldm list-spconfig
factory-default
unixarenacfg1 (current)
2.To restore the specific ldom/primary domain configuration
# ldm init-system -i ldom_name.xml
# ldm add-spconfig unixarencfg2
# ldm list-spconfig
factory-default
unixarenacfg1
unixarenacfg2 (current)
3.To restore only the virtual service,
# ldm init-system -s -i filename.xml
-bash-3.2# ldm list-spconfig
factory-default
unixarenacfg1
unixarenacfg2[active on Reboot]
-bash-3.2# ldm add-spconfig unixarencfg3
-bash-3.2# ldm list-spconfig
factory-default
unixarenacfg1
unixarenacfg2
unixarenacfg2 (current)
The old configuration copies can be removed using ldm remove-spconfig profile_name . If you make any small changes
on LDOM configuration, you must create a new profile to save it.Otherwise you may loose the new changes after the
system power cycle.
Thank you for visiting UnixArena.




How to Remove Logical Domains and Services ? Oracle VM for
SPARC
February 20, 2014 in LDOM


Destroying the infrastructure is a easy task in all the environments.In oracle VM for SPARC also ,you can easily remove
the guest domains in few seconds. The same way you can also remove the virtual disk resources and virtual network
resources. In some cases you may not able to remove the resources if its in use or already bound to any guest domains.
In this article ,we will see how to remove the guest domains and how to remove the virtual services like vds and vswitch.
Oracle VM for SPARC Tutorial:
Installation and Configuring the Primary domain
Creating Guest Domains
Configuring I/O Domain
Dynamically adding/removing Resources
Live Migration for Guest Domains
Adding ISO image to Multiple Guests
Configuration Backup & Restore
Removing the LDOM and Services (You Are Here)
1. Login to the control domain and list the available domains.
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 14 4128M 0.2% 8h 7m
ldom2 active -t---- 5000 2 6G 50% 4m
ldom2clone active -n---- 5001 8 8G 0.1% 7h 26m
2.Halt the domain by logging in to the guest domain operating system. (init 5) . You can also perform the force shutdown
from the control domain using the below command.
-bash-3.2# ldm stop-domain ldom2clone
LDom ldom2clone stopped
3.Un-bind the domain from the hypervisor.
-bash-3.2# ldm unbind-domain ldom2clone
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 14 4128M 6.8% 8h 8m
ldom2 active -t---- 5000 2 6G 50% 5m
ldom2clone inactive ------ 8 8G
-bash-3.2#
4.Now you can remove the inactive guest domain from the hypervisor.
-bash-3.2# ldm remove-domain ldom2clone
-bash-3.2# ldm ls-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 14 4128M 0.2% 8h 10m
ldom2 active -t---- 5000 2 6G 50% 7m
-bash-3.2#
You can see ldom2clone has been removed from the hypervisor completely.
If you want to remove the specific resources from the guest domain,follow the below steps.
1. List the bounded resources for one of the guest domain. (ldom2)
-bash-3.2# ldm list-bindings ldom2
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
ldom2 active -t---- 5000 2 6G 50% 8m

UUID
f2aa98a4-ddae-6d93-948f-9e2e9e81ff40

MAC
00:14:4f:fb:b6:94

HOSTID
0x84fbb694

CONTROL
failure-policy=ignore

DEPENDENCY
master=

CORE
CID CPUSET
1 (8, 9)

VCPU
VID PID CID UTIL STRAND
0 8 1 100% 100%
1 9 1 0.0% 100%

MEMORY
RA PA SIZE
0x10000000 0x10000000 4G
0x480000000 0x390000000 2G

VARIABLES
auto-boot?=false
boot-device=disk:a disk net
keyboard-layout=US-English

NETWORK
NAME SERVICE ID DEVICE MAC MODE PVID VID
MTU LINKPROP
vnet0 primary-vds0@primary 0 network@0 00:14:4f:f9:53:57 1
1500
PEER MAC MODE PVID VID MTU LINKPROP
primary-vds0@primary 00:14:4f:fb:e2:f3 1 1500

DISK
NAME VOLUME TOUT ID DEVICE SERVER MPGROUP
vdisk1_dom2 vol1@primary-vds0 1 disk@1 primary
iso_ldom2 iso0@primary-vds0 2 disk@2 primary
uavdisk uavol1@primary-vds0 0 disk@0 primary

VCONS
NAME SERVICE PORT
ldom2 primary-vcc0@primary 5000
2.Try to remove the one of the virtual disk resource from the guest domain.
-bash-3.2# ldm remove-vdisk uavdisk ldom2
Please perform the operation while the LDom is bound or inactive
-bash-3.2# ldm remove-vdisk -f uavdisk ldom2
Please perform the operation while the LDom is bound or inactive
You are not able to remove the virtual disk since its in use on ldom2.
3.You can stop the guest domain and then remove it .
-bash-3.2# ldm stop-domain ldom2
LDom ldom2 stopped
-bash-3.2# ldm remove-vdisk uavdisk ldom2
-bash-3.2#
-bash-3.2# ldm list-bindings ldom2
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
ldom2 bound ------ 5000 2 6G

UUID
f2aa98a4-ddae-6d93-948f-9e2e9e81ff40

MAC
00:14:4f:fb:b6:94

HOSTID
0x84fbb694

CONTROL
failure-policy=ignore

DEPENDENCY
master=

CORE
CID CPUSET
1 (8, 9)

VCPU
VID PID CID UTIL STRAND
0 8 1 100%
1 9 1 100%

MEMORY
RA PA SIZE
0x10000000 0x10000000 4G
0x480000000 0x390000000 2G

VARIABLES
auto-boot?=false
boot-device=disk:a disk net
keyboard-layout=US-English

NETWORK
NAME SERVICE ID DEVICE MAC MODE PVID VID
MTU LINKPROP
vnet0 primary-vds0@primary 0 network@0 00:14:4f:f9:53:57 1
1500
PEER MAC MODE PVID VID MTU LINKPROP
primary-vds0@primary 00:14:4f:fb:e2:f3 1 1500

DISK
NAME VOLUME TOUT ID DEVICE SERVER MPGROUP
vdisk1_dom2 vol1@primary-vds0 1 disk@1 primary
iso_ldom2 iso0@primary-vds0 2 disk@2 primary

VCONS
NAME SERVICE PORT
ldom2 primary-vcc0@primary 5000

-bash-3.2#
4. Removing the devices from the vds service
-bash-3.2# ldm list-services
VCC
NAME LDOM PORT-RANGE
primary-vcc0 primary 5000-5100

VSW
NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID
PVID VID MTU MODE
primary-vds0 primary 00:14:4f:fb:e2:f3 nxge0 0 switch@0 1 1
1500
primary-vsw0 primary 00:14:4f:fa:bf:27 nxge1 1 switch@1 1 1
1500

VDS
NAME LDOM VOLUME OPTIONS MPGROUP DEVICE
primary-vds0 primary vol1 /dev/zvol/dsk/pool1/vol1
iso0 /export/home/sol-10-u10-ga2-
sparc-dvd.iso
clonevol1
/dev/zvol/dsk/pool1/clonevol1
vol0
/dev/dsk/c3t600144F0B8E706DA00005300F4270004d0s2
vol2
/dev/dsk/c3t600144F0C1135B490000530383D70006d0
uavol1 /dev/zvol/dsk/pool1/uavol1

-bash-3.2# ldm remove-vdsdev uavol1@primary-vds0
-bash-3.2# ldm list-services
VCC
NAME LDOM PORT-RANGE
primary-vcc0 primary 5000-5100

VSW
NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID
PVID VID MTU MODE
primary-vds0 primary 00:14:4f:fb:e2:f3 nxge0 0 switch@0 1 1
1500
primary-vsw0 primary 00:14:4f:fa:bf:27 nxge1 1 switch@1 1 1
1500

VDS
NAME LDOM VOLUME OPTIONS MPGROUP DEVICE
primary-vds0 primary vol1 /dev/zvol/dsk/pool1/vol1
iso0 /export/home/sol-10-u10-ga2-sparc-
dvd.iso
clonevol1 /dev/zvol/dsk/pool1/clonevol1
vol0
/dev/dsk/c3t600144F0B8E706DA00005300F4270004d0s2
vol2
/dev/dsk/c3t600144F0C1135B490000530383D70006d0

-bash-3.2#
We have seen how to remove the assigned virtual disk from the guest and removing the disk from the virtual disk server.
Now we will see how to remove the complete service from the service domain aka primary domain.
1.Try to remove the virtual switch service from service domain
-bash-3.2# ldm remove-vsw primary-vds0
Service is active and being used by bound client(s)
Failed to remove virtual switch
2.The above command failed because,that virtual service is used for some of the guest domains. First remove the those
assigned virtual resources from guest domain the n try to remove it.
-bash-3.2# ldm remove-vsw primary-vsw0
-bash-3.2# ldm list-services
VCC
NAME LDOM PORT-RANGE
primary-vcc0 primary 5000-5100

VSW
NAME LDOM MAC NET-DEV ID DEVICE LINKPROP DEFAULT-VLAN-ID
PVID VID MTU MODE
primary-vds0 primary 00:14:4f:fb:e2:f3 nxge0 0 switch@0 1 1
1500

VDS
NAME LDOM VOLUME OPTIONS MPGROUP DEVICE
primary-vds0 primary vol1 /dev/zvol/dsk/pool1/vol1
iso0 /export/home/sol-10-u10-
ga2-sparc-dvd.iso
clonevol1 /dev/zvol/dsk/pool1/clonevol1
vol0
/dev/dsk/c3t600144F0B8E706DA00005300F4270004d0s2
vol2 /dev/dsk/c3t600144F0C1135B490000530383D70006d0
Using the same setup commands (showed in this article), you can remove the other guest domains and virtual services.
Once you have cleared the all services and domains, you can uninstall the oracle VM for SPARC software using below
method.
bash-3.2# pkgrm SUNWldm

The following package is currently installed:
SUNWldm LDoms Manager software
(sparc.sun4v) 2.0,REV=2010.09.02.10.08

Do you want to remove this package? [y,n,?,q] y

## Removing installed package instance

This package contains scripts which will be executed with super-user
permission during the process of removing this package.

Do you want to continue with the removal of this package [y,n,?,q] y
## Verifying package dependencies in global zone
## Processing package information.
## Removing pathnames in class
Waiting up to 60 seconds for svc:/ldoms/ldmd:default to stop...
## Removing pathnames in class
/var/svc/manifest/platform/sun4v
/var/svc/manifest/platform
/var/svc/manifest
/var/svc
<<<<<<<<>>>>
/opt/SUNWldm/bin/schemas/ovf-strings.xsd
/opt/SUNWldm/bin/schemas/ovf-section.xsd
/opt/SUNWldm/bin/schemas/ovf-envelope.xsd
/opt/SUNWldm/bin/schemas/ovf-core.xsd
/opt/SUNWldm/bin/schemas/ldmd-property.xsd
/opt/SUNWldm/bin/schemas/ldmd-binding.xsd
/opt/SUNWldm/bin/schemas/event-v3.xsd
/opt/SUNWldm/bin/schemas/combined-v3.xsd
/opt/SUNWldm/bin/schemas/cli-list-constraint-v3.xsd
/opt/SUNWldm/bin/schemas/cim-vssd.xsd
/opt/SUNWldm/bin/schemas/cim-rasd.xsd
/opt/SUNWldm/bin/schemas/cim-common.xsd
/opt/SUNWldm/bin/schemas
/opt/SUNWldm/bin/ldmd_start
/opt/SUNWldm/bin/ldmd
/opt/SUNWldm/bin/ldmconfig
/opt/SUNWldm/bin/ldm
/opt/SUNWldm/bin
/opt/SUNWldm
/opt
## Updating system information.

Removal of was successful.
bash-3.2# pkgrm SUNWldomu

The following package is currently installed:
SUNWldomu Solaris Logical Domains (Usr)
(sparc.sun4v) 11.10.0,REV=2006.08.08.12.13

Do you want to remove this package? [y,n,?,q] y

## Removing installed package instance
## Verifying package dependencies in global zone
## Processing package information.
## Removing pathnames in class
/usr/lib/sparcv9/libpri.so.1
/usr/lib/sparcv9/libds.so.1
/usr/lib/sparcv9
/usr/lib/libpri.so.1
/usr/lib/libds.so.1
/usr/lib/ldoms/vntsd
/usr/lib/ldoms/ldmad
/usr/lib/ldoms/drd
/usr/lib/ldoms
/usr/lib
/usr
## Updating system information.

Removal of was successful.
bash-3.2# pkgrm SUNWldomr

The following package is currently installed:
SUNWldomr Solaris Logical Domains (Root)
(sparc.sun4v) 11.10.0,REV=2006.10.04.00.26

Do you want to remove this package? [y,n,?,q] y

## Removing installed package instance

This package contains scripts which will be executed with super-user
permission during the process of removing this package.

Do you want to continue with the removal of this package [y,n,?,q] y
## Verifying package dependencies in global zone
## Processing package information.
## Executing preremove script.
## Removing pathnames in class
Waiting up to 60 seconds for svc:/ldoms/vntsd:default to stop...
Waiting up to 60 seconds for svc:/ldoms/agents:default to stop...
Waiting up to 60 seconds for svc:/platform/sun4v/drd:default to stop...
## Removing pathnames in class
/var/svc/manifest/platform/sun4v
/var/svc/manifest/platform
/var/svc/manifest
/var/svc
/var
/platform/sun4v/kernel/misc/sparcv9/platsvc
/platform/sun4v/kernel/misc/sparcv9/ldc
/platform/sun4v/kernel/misc/sparcv9/fault_iso
/platform/sun4v/kernel/misc/sparcv9/ds
/platform/sun4v/kernel/misc/sparcv9/dr_mem
/platform/sun4v/kernel/misc/sparcv9/dr_io
/platform/sun4v/kernel/misc/sparcv9/dr_cpu
/platform/sun4v/kernel/misc/sparcv9
/platform/sun4v/kernel/misc
/platform/sun4v/kernel/drv/sparcv9/vsw
/platform/sun4v/kernel/drv/sparcv9/vnet
/platform/sun4v/kernel/drv/sparcv9/vlds
/platform/sun4v/kernel/drv/sparcv9/vldc
/platform/sun4v/kernel/drv/sparcv9/vds
/platform/sun4v/kernel/drv/sparcv9/vdc
/platform/sun4v/kernel/drv/sparcv9/vcc
/platform/sun4v/kernel/drv/sparcv9/ds_snmp
/platform/sun4v/kernel/drv/sparcv9/ds_pri
/platform/sun4v/kernel/drv/sparcv9/drctl
/platform/sun4v/kernel/drv/sparcv9/cnex
/platform/sun4v/kernel/drv/sparcv9
/platform/sun4v/kernel/drv/ds_snmp.conf
/platform/sun4v/kernel/drv/ds_pri.conf
/platform/sun4v/kernel/drv/drctl.conf
/platform/sun4v/kernel/drv
/platform/sun4v/kernel
/platform/sun4v
/platform
/lib/svc/method/svc-vntsd
/lib/svc/method/svc-drd
/lib/svc/method/ldoms-agents
/lib/svc/method
/lib/svc
/lib
## Updating system information.

Removal of was successful.
bash-3.2# pkginfo | grep -i ldom
application SUNWldmp2v LDoms P2V tool
bash-3.2# pkgrm SUNWldmp2v

The following package is currently installed:
SUNWldmp2v LDoms P2V tool
(sparc) 2.0,REV=2010.09.02.10.08

Do you want to remove this package? [y,n,?,q] y

## Removing installed package instance

This package contains scripts which will be executed with super-user
permission during the process of removing this package.

Do you want to continue with the removal of this package [y,n,?,q] y
## Verifying package dependencies in global zone
## Processing package information.
## Removing pathnames in class
## Removing pathnames in class
/usr/share/man/man1m/ldmp2v.1m
/usr/share/man/man1m
/usr/share/man
/usr/share
/usr/sbin/ldmp2v
/usr/sbin
/usr
/opt/SUNWldmp2v/man/man1m/ldmp2v.1m
/opt/SUNWldmp2v/man/man1m
/opt/SUNWldmp2v/man
/opt/SUNWldmp2v/lib/ldmp2v_convert
/opt/SUNWldmp2v/lib
/opt/SUNWldmp2v/etc/ldmp2v.conf.sample
/opt/SUNWldmp2v/etc
/opt/SUNWldmp2v/bin/ldmp2v_prepare
/opt/SUNWldmp2v/bin
/opt/SUNWldmp2v
/opt
## Updating system information.

Removal of was successful.
bash-3.2# pkginfo | grep -i ldom*
bash-3.2# pkginfo | grep -i ldm*
bash-3.2#
You have successfully uninstall the oracle VM for SPARC software. You can safely recycle the control domain to remove
the complete ldom stuffs.
Thank you for visiting UnixArena.


LDOM console write access issue Solaris
March 27, 2014 in LDOM


Now a days i have mostly dealing with oracle VM for SPARC on T-4 servers.(LDOM). As we all know that hardware
console can be writable for only one user at a time and if you take the second session , it will automatically connect in read
only mode unless you use the force option. The same way oracle VM for SPARC console also behaves even though its a
virtual consoles on the hypervisor levels.
Sometimes , the first console session may terminate unexpectedly due to network issues and if you try take the console
for second time ,you may get error that You do not have write access on the console screen.
root@sun5#ldm list-domain
NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME
primary active -n-cv- SP 8 2G 0.3% 2h 34m
ua1 active -n---- 1 12 3G 0.1% 1h 37m
ua2 active -n---- 2 12 2560M 0.1% 1h 37m
root@sun5#
root@sun5#telnet 127.0.0.1 1
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.

Connecting to console "ua1" in group "ua1" ....
Press ~? for control options ..
You do not have write access
You do not have write access
You do not have write access
You do not have write access
You do not have write access
You do not have write access
You do not have write access
You do not have write access
You do not have write access

telnet> quit
Connection to 127.0.0.1 closed.
To fix this issue, login to the control domain and list the telnet session.
root@sun5#ps -ef |grep telnet
root 6259 6118 0 12:07:22 pts/4 0:00 telnet 127.0.0.1 1
root 6035 374 0 12:01:21 ? 0:00 /usr/sbin/in.telnetd
root 1645 374 0 09:47:55 ? 0:00 /usr/sbin/in.telnetd
root 5867 374 0 11:57:25 ? 0:00 /usr/sbin/in.telnetd
root 6004 5870 0 12:00:49 pts/1 0:00 telnet localhost 2
root 6262 6038 0 12:07:46 pts/2 0:00 grep telnet
Kill the process which is associated to the guest domain. In this case , i am having issue with ua1 guest domain which
has the console port-1.Let me terminate the process of that and try to connect the console again.
root@sun5#kill -9 6259
root@sun5#telnet 127.0.0.1 1
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.

Connecting to console "ua1" in group "ua1" ....
Press ~? for control options ..

ua1 console login: root
Password:
Mar 24 13:44:13 ua1 login: ROOT LOGIN /dev/console
Last login: Mon Mar 24 13:16:41 from 192.168.0.117
Oracle Corporation SunOS 5.11 11.1 September 2012
root@ua1:~# logout

ua1 console login:
telnet> quit
Connection to 127.0.0.1 closed.
root@sun5#
Awesome .We have fixed the issue .This is small trick but you need to remember this when you face this kind of issues
while connecting the console.
Share it ! comment it !! Be sociable !!!




How to configure IPMP on Solaris 10 ?
June 26, 2013 in Solaris Networking



This article describes how to configure link based IPMP interfaces in Solaris 10. IPMP eliminates single network card
failure and it ensures system will be always accessible via network.You can also configure the failure detection seconds
in /etc/default/mpathd file and the default value is 10 seconds.In this file there is an option called FAILBACK to
specify IP behavior when primary interface recovered from the fault. in.mpathd is a daemon which handles
IPMP (Internet Protocol Multi-Pathing) operations.There are two type of IPMP configuration available in Solaris 10.

1.Link Based IPMP
The link based IPMP detects network errors by checking the IFF_RUNNING flag.Normally it doesnt require any test IP
address like probe based IPMP.
2.Probe Based IPMP
The probe based IPMP detects network errors by sending ICMP ECHO_REQUEST messages.It requires test IP Addresses
unlike link based IPMP.
1.Link Based IPMP
Request:
Configure IP address 192.168.2.50 on e1000g1 & e1000g2 using Link based IPMP.


Step:1
Find out the installed NICs on the systems and its status.Verify the ifconfig output as well.
Make sure the NIC status are up and not in use.
Arena-Node1#dladm show-dev
e1000g0 link: up speed: 1000 Mbps duplex: full
e1000g1 link: up speed: 1000 Mbps duplex: full
e1000g2 link: up speed: 1000 Mbps duplex: full
Arena-Node1#ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843 mtu 1500 index 2
inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255
ether 0:c:29:ec:b3:af
Arena-Node1#


Step:2
Add the IP address in /etc/hosts and specify the netmask value in /etc/netmasks like below one.
Arena-Node1#cat /etc/hosts |grep 192.168.2.50
192.168.2.50 arenagroupIP
Arena-Node1#cat /etc/netmasks |grep 192.168.2
192.168.2.0 255.255.255.0
Arena-Node1#eeprom "local-mac-address?=true"


Step:3
Plumb the interfaces which you are going to use for new IP address. check the status in ifconfig output.
Arena-Node1#ifconfig e1000g1 plumb
Arena-Node1#ifconfig e1000g2 plumb
Arena-Node1#ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843 mtu 1500 index 2
inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255
ether 0:c:29:ec:b3:af
e1000g1: flags=1000842 mtu 1500 index 3
inet 0.0.0.0 netmask 0
ether 0:c:29:ec:b3:b9
e1000g2: flags=1000842 mtu 1500 index 4
inet 0.0.0.0 netmask 0
ether 0:c:29:ec:b3:c3

Step:4
Configure IP on Primary interface and add the interfaces to IPMP group with your own group name.
Arena-Node1#ifconfig e1000g1 192.168.2.50 netmask 255.255.255.0 broadcast + up
Arena-Node1#ifconfig e1000g1
e1000g1: flags=1000843 mtu 1500 index 3
inet 192.168.2.50 netmask ffffff00 broadcast 192.168.2.255
ether 0:c:29:ec:b3:b9

Arena-Node1#ifconfig e1000g1 group arenagroup-1
Arena-Node1#ifconfig e1000g2 group arenagroup-1
Arena-Node1#ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843 mtu 1500 index 2
inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255
ether 0:c:29:ec:b3:af
e1000g1: flags=1000843 mtu 1500 index 3
inet 192.168.2.50 netmask ffffff00 broadcast 192.168.2.255
groupname arenagroup-1
ether 0:c:29:ec:b3:b9
e1000g2: flags=1000842 mtu 1500 index 4
inet 0.0.0.0 netmask 0
groupname arenagroup-1
ether 0:c:29:ec:b3:c3

Step:5
Now we have to ensure IPMP is working fine.This can be done in two ways.

i.Test:1 Remove the primary LAN cable and check it.Here i have removed the LAN cable from e1000g1 and let see what
happens.
Arena-Node1#ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843 mtu 1500 index 2
inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255
ether 0:c:29:ec:b3:af
e1000g1: flags=19000802 mtu 0 index 3
inet 0.0.0.0 netmask 0
groupname arenagroup-1
ether 0:c:29:ec:b3:b9
e1000g2: flags=1000842 mtu 1500 index 4
inet 0.0.0.0 netmask 0
groupname arenagroup-1
ether 0:c:29:ec:b3:c3
e1000g2:1: flags=1000843 mtu 1500 index 4
inet 192.168.2.50 netmask ffffff00 broadcast 192.168.2.255
Again i have connected back the LAN cable to e1000g1.
Arena-Node1#dladm show-dev
e1000g0 link: up speed: 1000 Mbps duplex: full
e1000g1 link: up speed: 1000 Mbps duplex: full
e1000g2 link: up speed: 1000 Mbps duplex: full
Arena-Node1#ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843 mtu 1500 index 2
inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255
ether 0:c:29:ec:b3:af
e1000g1: flags=1000843 mtu 1500 index 3
inet 192.168.2.50 netmask ffffff00 broadcast 192.168.2.255
groupname arenagroup-1
ether 0:c:29:ec:b3:b9
e1000g2: flags=1000842 mtu 1500 index 4
inet 0.0.0.0 netmask 0
groupname arenagroup-1
ether 0:c:29:ec:b3:c3
Here the configured IP is going back to original interface where it was running before. Here I had specified
FALLBACK=yes . Thats why IP is moving back to original interface.The same way you can also specify
failure detection time to mpathd using parameter FAILURE_DETECTION_TIME in ms.
Arena-Node1#cat /etc/default/mpathd |grep -v "#"
FAILURE_DETECTION_TIME=10000
FAILBACK=yes
TRACK_INTERFACES_ONLY_WITH_GROUPS=yes
Arena-Node1#

ii.Test:2 Normally most of the Unix admins will be sitting in remote site. So you will be not able to perform the above
test.In this case ,you can use if_mpadm command to disable the interface in OS level.
Fist i am going to disable e1000g1 and let see what happens.
Arena-Node1#if_mpadm -d e1000g1
Arena-Node1#ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843 mtu 1500 index 2
inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255
ether 0:c:29:ec:b3:af
e1000g1: flags=89000842 mtu 0 index 3
inet 0.0.0.0 netmask 0
groupname arenagroup-1
ether 0:c:29:ec:b3:b9
e1000g2: flags=1000842 mtu 1500 index 4
inet 0.0.0.0 netmask 0
groupname arenagroup-1
ether 0:c:29:ec:b3:c3
e1000g2:1: flags=1000843 mtu 1500 index 4
inet 192.168.2.50 netmask ffffff00 broadcast 192.168.2.255

Now i am going to enable it back.
Arena-Node1#if_mpadm -r e1000g1
Arena-Node1#ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843 mtu 1500 index 2
inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255
ether 0:c:29:ec:b3:af
e1000g1: flags=1000843 mtu 1500 index 3
inet 192.168.2.50 netmask ffffff00 broadcast 192.168.2.255
groupname arenagroup-1
ether 0:c:29:ec:b3:b9
e1000g2: flags=1000842 mtu 1500 index 4
inet 0.0.0.0 netmask 0
groupname arenagroup-1
ether 0:c:29:ec:b3:c3
The same way you can manually failover the IP to one interface to another interface.

In the both tests,we can clearly see IP is moving from e1000g1 to e1000g2 automatically without any issues.So we have
successfully configured Link based IPMP on Solaris.
These failover logs will be logged in /var/adm/messages like below.
Jun 26 20:57:24 node1 in.mpathd[3800]: [ID 215189 daemon.error] The link has gone down on e1000g1
Jun 26 20:57:24 node1 in.mpathd[3800]: [ID 594170 daemon.error] NIC failure detected on e1000g1 of group
arenagroup-1
Jun 26 20:57:24 node1 in.mpathd[3800]: [ID 832587 daemon.error] Successfully failed over from NIC e1000g1 to
NIC e1000g2
Jun 26 20:57:57 node1 in.mpathd[3800]: [ID 820239 daemon.error] The link has come up on e1000g1
Jun 26 20:57:57 node1 in.mpathd[3800]: [ID 299542 daemon.error] NIC repair detected on e1000g1 of group
arenagroup-1
Jun 26 20:57:57 node1 in.mpathd[3800]: [ID 620804 daemon.error] Successfully failed back to NIC e1000g1
Jun 26 21:03:59 node1 in.mpathd[3800]: [ID 832587 daemon.error] Successfully failed over from NIC e1000g1 to
NIC e1000g2
Jun 26 21:04:07 node1 in.mpathd[3800]: [ID 620804 daemon.error] Successfully failed back to NIC e1000g1
To make the above work persistent across the reboot create the configuration files for both the network interfaces.
Arena-Node1#cat /etc/hostname.e1000g1
arenagroupIP netmask + broadcast + group arenagroup up
Arena-Node1#cat /etc/hostname.e1000g2
group arenagroup up
Thank you for reading this article. Please leave a comment if its useful for you.

Das könnte Ihnen auch gefallen