Virtual Machine Technology

Wei Wang

CS 4630 CS 6501 1
Spring 2016 Defense Against the Dark Arts
 Why Virtual Machine for this
course?
● To protect you from your own attacks
– Your VM is your victim
● We use Linux as our main system
– You probably use a different one
● Tool of the trade for computer security research
– A controlled environment to study viruses
– Widely adopted for virus detection
– Isolation and encapsulation, more secure
– Also used by virus to avoid detection and defy analysis

CS 4630 CS 6501 2
Spring 2016 Defense Against the Dark Arts
 Outline
● History
● Key concepts
● Terminology
● Virtual machine basics
● Virtual machine taxonomy

CS 4630 CS 6501 3
Spring 2016 Defense Against the Dark Arts
 History of Virtual Machines

CS 4630 CS 6501 4
Spring 2016 Defense Against the Dark Arts
 1960s and 1970s – Hardware
utilization
● IBM VM/370 operating system (1972)
– Companies and other organizations required to run
multiple OSes.
– Hypervisor (control program) creates virtual
machine environment
● Each user has own virtual machine with guest OS, own
address space, virtual devices, etc.
● Official Name: Virtual Machine Monitor or
Hypervisor

CS 4630 CS 6501 5
Spring 2016 Defense Against the Dark Arts
 1980s and 1990s – VMs decline in
popularity
● Poor performance
● Client-server applications
● Inexpensive PCs
● Distributed computing

CS 4630 CS 6501 6
Spring 2016 Defense Against the Dark Arts
 Late 1990s – Renewed Interest
●
Improved CPU performance renewed interest in virtual machine
● Improve Infrastructure Utilization
– Multiple VMs share hardware
– improve utilization
– Reduce installation and operational cost
– Old business: web hosting
– New business: Cloud computing
● Failover, Security and Disaster Protection
– Isolated Environments
– Encapsulation
● Flexibility
– We are in an era of mixed environments: SW and HW, need the ability to go cross-platform
– Easy for system testing

CS 4630 CS 6501 7
Spring 2016 Defense Against the Dark Arts
 Key Concepts

aka. The Source of Virtual Machine’s Power

CS 4630 CS 6501 8
Spring 2016 Defense Against the Dark Arts
 Key Concepts
● Levels of abstraction
● Well-defined interfaces
● Virtualization

CS 4630 CS 6501 9
Spring 2016 Defense Against the Dark Arts
 Levels of Abstraction
● Allow implementation details at lower levels of a
design to be ignored or simplified
● Arranged in a hierarchy
– Lower levels – Hardware
● Physical components with real properties
– Higher levels – Software
● Logical components

CS 4630 CS 6501 10
Spring 2016 Defense Against the Dark Arts
 Levels of Abstraction: HW/SW
Interface
Application

Algorithms

High Level Language (HLL)

Assembly Language

Operating System

Instruction Set Architecture (ISA)

Digital Logic

Electronics – (transistors etc.)

CS 4630 CS 6501 11
Spring 2016 Defense Against the Dark Arts
 Levels of Abstraction: HW/SW
Interface
Application
Data Strutures/Operations
Algorithms
HLL Specifications
High Level Language (HLL)
Assembly Specifications
Assembly Language
System Calls / Linking Spec
Operating System
Instructions Set
Instruction Set Architecture (ISA)
Digital Signals
Digital Logic
0/1, Voltage etc.
Electronics – (transistors etc.)

CS 4630 CS 6501 12
Spring 2016 Defense Against the Dark Arts
 Well-defined interfaces: Challenges
● Reduced interoperability
– Processors support limited instruction sets
● IA-32 vs. PowerPC
– Different operating systems
● Windows vs. Linux
– Application binaries
● Dependent on OS and instruction set
● Hardware resource dependency
– OS
– Applications

CS 4630 CS 6501 13
Spring 2016 Defense Against the Dark Arts
 Virtualization
● Formally:
– Virtualization involves the construction of an
isomorphism that maps a virtual guest system to a
real host. (Popek and Goldberg, 1974)

CS 4630 CS 6501 14
Spring 2016 Defense Against the Dark Arts
 Virtualization cont’d
● Function V maps
guest state to the host Si
e(Si)
Sj
state GUEST
● For a sequence of
V(Si) V(Sj)
operations e that
modify the guest
state, a corresponding Si’
e’(Si’)
Sj’
e’ in the host modify HOST
the host state

CS 4630 CS 6501 15
Spring 2016 Defense Against the Dark Arts
 Virtualization cont’d
● Abstractions and Well-defined Interfaces allows
the introduction of a new virtualization layer
● Virtualization helps mitigate the reduced
interoperability and hardware dependency
problems of layers and well-defined interfaces

CS 4630 CS 6501 16
Spring 2016 Defense Against the Dark Arts
 Virtualization cont’d
● Mapping of virtual resources or state to real
resources on underlying machine
– E.g., registers, memory, files
● Emulation of the virtual machine ABI or ISA
– Use of real machine instructions and/or system
calls to carry out actions specified by virtual
machine instructions and/or system calls

CS 4630 CS 6501 17
Spring 2016 Defense Against the Dark Arts
 Terminology

CS 4630 CS 6501 18
Spring 2016 Defense Against the Dark Arts
 Terminology
● Computer architecture
– Functionality and appearance of a computer or
subsystem
● Implementation
– Actual embodiment of an architecture
– Architectures may have several implementations
● E.g., high-performance or low-power

CS 4630 CS 6501 19
Spring 2016 Defense Against the Dark Arts
 Terminology (cont’d)
● Implementation layers
– Correspond to levels of abstraction in a computer
system – each layers is essentially a state-machine
Application
Algorithms
High Level Language (HLL)
Assembly Language
Operating System
Instruction Set Architecture (ISA)
Digital Logic
Electronics – (transistors etc.)

CS 4630 CS 6501 20
Spring 2016 Defense Against the Dark Arts
 ISA, ABI and API
● Instruction Set Architecture Applications
(ISA)
API
– User ISA + Sys ISA Libraries
(c-runtime etc.)
● Application Binary Interface ABI/Sys call
(ABI) OS
– Sys call + User ISA Sys ISA ABI/User ISA
● Application Programming Hardware (CPUs etc.)
Interface
– Defined with high level
language
– Standard libraries

CS 4630 CS 6501 21
Spring 2016 Defense Against the Dark Arts
 Terminology: Virtual Machine (VM)
● An emulation of a particular computer system
– Any layer is a system itself
– Almost any layer can be emulated: hardware, OS,
libraries …
– Remember each layer is a (state-) machine itself
– VM provides an implementation for each (state-)
machine (layer)

CS 4630 CS 6501 22
Spring 2016 Defense Against the Dark Arts
 Terminology
● Machine
– Matter of perspective
● process: OS and underlying user-level hardware
● OS: underlying hardware with ISA as the interface
● Host
– Underlying platform
● Guest
– Software that runs in the VM environment

CS 4630 CS 6501 23
Spring 2016 Defense Against the Dark Arts
 Process VM vs System VM
● Process VM ● System VM (VMM)
– Simulates ABI (sys – Simulates full ISA
call & user ISA) and
API
Guest Applications Applications

Guest
Proc. Virtual Machine
OS
VM (E.g., JVM)

Virtual Machine Sys

OS VM
(E.g., VMWare)
HOST
Hardware or
Hardware HOST
Another OS

CS 4630 CS 6501 24
Spring 2016 Defense Against the Dark Arts
 Process VM Example 1
● Multiprogramming Applicatio Applicatio
Guest n/Process n/Process
– Each user process 1 2
given the illusion of Proc. Process
Scheduler
having complete VM
machine to itself OS

– OS supporting HOST
Hardware or
multiple user Another OS
processes

CS 4630 CS 6501 25
Spring 2016 Defense Against the Dark Arts
 Process VM Example 2
Application with ARM
● Emulators and Diff- Guest
instructions
ISA Dynamic Binary
Translator
Proc. QEMU:
– Emulate one VM Translate from ARM to
x86
instruction set on
hardware with another
instruction set
OS
– Example: QEMU HOST
X86 Processor

CS 4630 CS 6501 26
Spring 2016 Defense Against the Dark Arts
 Process VM Example 3
Application with x86
● Same-ISA Dynamic Guest
instructions
Binary Translator
– Same ISA Dynamo (Performance)
or
– Translate code block to Proc.
Strata (Security)
VM
another code block with or
Pin (Analysis)
the same functions but,
– Performance: Dynamo
OS
– Security: Strata
HOST
– Analysis: Pin X86 Processor

CS 4630 CS 6501 27
Spring 2016 Defense Against the Dark Arts
 Process VM Example 4
● High-level language Guest Jave Application w/.
Java Byte Code
VMs
JIT Compiler:
– Cross platform Byte Code to x86 insn
portability Proc.
VM JRE
– Minimize hardware-
specific and OS- JVM
specific features
OS
– Example: Java VM HOST
Hardware

CS 4630 CS 6501 28
Spring 2016 Defense Against the Dark Arts
 Process VM Example 5
● Libraries Emulation Guest Windows Applications

– Emulation the APIs of

another operating
system Proc. Winehq:
VM Provides Windows APIs
– Supporting cross-OS
portability
– Same ISA
Linux
– Example: Wine HOST
Hardware

CS 4630 CS 6501 29
Spring 2016 Defense Against the Dark Arts
 Process VM Example 6
●
VM-based Obfuscator Guest
– Code Obfuscation: make code Obfuscated Virus
hard to understand to defy Code using special
analysis ISA
– Obfuscation techniques: junk Proc.
instructions, complex logic, VM
custom ISA, etc. VM Obfuscator:
convert special ISA
– Commonly used by virus as anti- to x86 ISA
detection and anti-analysis
techniques
Windows
– Also employed to protect private
HOST
software
– Example: VM Protect, Themida Hardware

CS 4630 CS 6501 30
Spring 2016 Defense Against the Dark Arts
 Process VM vs System VM
● Process VM ● System VM (VMM)
– Simulates ABI (sys – Simulates full ISA
call & user ISA) and
API
Guest Applications Applications

Guest
Proc. Virtual Machine
OS
VM (E.g., JVM)

Virtual Machine Sys

OS VM
(E.g., VMWare)
HOST
Hardware or
Hardware HOST
Another OS

CS 4630 CS 6501 31
Spring 2016 Defense Against the Dark Arts
 System VM Example 1
● Classic system
– VM directly on
hardware, e.g., VMWare
ESX, Xen, KVM
Applications
● Hosted VM
– VM on host operating
system, e.g., VMWare
Workstation, VirutalBox
Applications

Guest Guest
Guest OS Guest OS

Sys Sys
VMWare ESX VirtualBox VM
VM
Host OS
HOST Hardware HOST
Hardware
CS 4630 CS 6501 32
Spring 2016 Defense Against the Dark Arts
 System VM Example 2
● Codesigned VMs x86 Applications
(Hardware Guest
optimization) x86 OS

– No native applications.
VM software is part of x86 to Berkeley Sys
hardware RISC decoder VM
implementation AMD K5 Processor:
Berkeley RISC HOST
● AMD Processors
● Modern Intel
Processors

CS 4630 CS 6501 33
Spring 2016 Defense Against the Dark Arts
 System VM Example 3
● Whole System VMs Applications
(Emulation) Guest
– Complete software Windows
system (OS and Sys
Virtual PC
applications) VM
supported on a host Mac OS X
system running a HOST
PowerPC
different ISA and OS
– VirtualPC (Windows
on Mac)

CS 4630 CS 6501 34
Spring 2016 Defense Against the Dark Arts
 Full Virtualization vs
Para-Virtualization
● Full Virtualization
– Provide same
functions as real
hardware
– Allows unmodified
guest OS
– Example: VMware
● Para-Virtualization
– Provide similar, but no
identical functions as
real hardware
– May need to modified
guest OS
– Example: Xen (both
full and para)

CS 4630 CS 6501 35
Spring 2016 Defense Against the Dark Arts
 Taxonomy
● Process vs. system
● Same ISA vs. different ISA

CS 4630 CS 6501 36
Spring 2016 Defense Against the Dark Arts
 CS 4630 CS 6501 37
Spring 2016 Defense Against the Dark Arts