M5-32 Server Overview Welcome to the M5-32 Server Overview Training.

Schedule: Timing
57 minutes 15 minutes 70 minutes

Topic
Audio Practice Total

Course Objectives

M5-32 Server Overview 1 - 2

M5-32 Server Introduction The M5-32 Server is a Datacenter server that was designed with the highest reliability and availability features. It has a high redundancy of components and performs numerous health checks.

M5-32 Server Overview 1 - 3

M5-32 Server Overview The M5-32 server has very fast CMT (Chip-level Multi-threading) processors with a lot of memory and the latest industry standard I/O. As with previous servers, the M5-32 is modular with shared common components for investment protection. The domaining of the server is highly flexible with hard partitions, soft partitions, and application containers resulting in an industry leading virtualization strategy. The M5-32 Server features include application acceleration which means the M5-32 Server increases application performance and user response times. It uses six core M5 processors, high performance interconnections and industry standard PCIe Gen 3 I/O. The M5-32 Server include State-of-the-Art RAS (Reliability, Availability and Serviceability) features.

M5-32 Server Overview 1 - 4

SPARC M5-32 Server The M5-32 Server is based on the SPARC M5 processor and designed with emphasis on core thread performance. The 6 S3 M5 CPU processor cores run at 3.6 Ghz (gigahertz) with each core having its own dedicated 128K L2 cache. It runs as a single or multi-threaded operation with up to 8 threads per core. This means one CPU with 6 cores can run 6 threads minimum and up to 48 threads maximum. The M5-32 Server using its maximum CPU capacity has a total of 192 cores. (Maximum CPU core calculation: 6 cores * 32 CPU core processors = 192 cores) The 6 CPU processor cores can communicate between themselves at a very fast speed since they share 48MB L3 cache (with four 12 MB L3 16-way cache) via a crossbar. These features give the M5 CPUs high throughput with multi-threading and excellent single thread performance. On the right of the slide, you see a photo of the SPARC M5 processor.

M5-32 Server Overview 1 - 6

SPARC M5-32 Server Make-Up The M5-32 Server has the best features of the T-series servers with the high throughput S3 SPARC processor core, Oracle VM Server for SPARC (LDOMs) virtualization and ILOM system management. M-series servers are built on this extensive SPARC heritage. The M5-32 Server has the best features of the M-series servers with hardware domains, flexibility and massive scalability since it is expandable to 32 processors and 32TB of system memory. It has high availability design, redundancy of all of its major components and hot-plug replace or upgrade of these major components. Maximum system memory calculation: 32 DIMM slots per CPU * 32-Gbyte capacity DIMMs = 1024 Gbytes per CPU (2 CPUs per CMU * 1024 Gbytes = 2048 Gbytes per CMU) * 16 CMUs = 32,768 or 32 TB for M5-32 Server

M5-32 Server Overview 1 - 7

M5-32 Server Comparison to M9000-32 The M5-32 Server is highly scalable and it has high availability providing the Data Center a server with up to 32 CPU sockets and 1024 DDR3 DIMMs. Compared to the M9000-32, the M5-32 Server provides six times better throughput performance. It has three times more threads (1536) than M9000-32 with half the number of sockets (32). It has four times more shared cache as the M9000-32. It has eight times the memory bandwidth of M9000-32.

M5-32 Server Overview 1 - 8

High-End Product Comparison The M5-32 server most closely resembles the M9000-32. In this table you can see a high-level comparison of the M5-32 Server to the M9000-32 server. Processor - The M5-32 is a 3.5 GHz (gigahertz) chip multiprocessor while the M900032 SPARC 6 VII (or 7) is 3.0 GHz (gigahertz) chip multiprocessor. There are 6 cores per M5 processor so the M5-32 Server has a total of 192 cores (6 cores/CPU * 2 CMPs per CMU * 16 CMUs maximum per M5-32 Server = 192 total core processors in the M532 Server). Memory The total maximum system memory for the M9000-32 is 4 TB using 8GB DIMMs * 512 DDR2 DIMM slots. The M5-32 Server has a total maximum system memory of 32 TB using 32GB DIMMs * 1024 DDR3 DIMM slots.

M5-32 Server Overview 1 - 9

Key Differences From Previous M-Series In this table you can see how the M5-32 server differs from the previous M-series servers. SP - Administrators use XSCF (eXtended System Control Facility) running XCP (XSCF Control Package) firmware for M8000/M9000 models to oversee their systems on-site as well as from remote locations while they use the ILOM for the M5-32. Domains - There are 24 hard domains (or Physical Domains) available for the M8000/M9000 models while there are up to 4 Physical Domains (or PDoms) for the M532 server along with up to 128 Logical Domains (or LDoms) per Pdom or up to 512 Logical Domains per M5-32 Server. Memory - DDR3 is the system memory, or to get more specific, SDRAM (Synchronous Dynamic Random Access Memory), used in the M5-32 with the older DDR2 in the M8000/M9000 models. (SDRAM synchronizes the memory's responses to control inputs with the system bus, allowing it to queue up one process while waiting for another.)

M5-32 Server Overview 1 - 11

Terminology It is very important to understand the terminology used in the M5-32 Server before going any further. Some of the terms may be familiar to you from other SPARC server lines. Please take a minute to read the terms and their definition before continuing with the training.

M5-32 Server Overview 1 - 13

M5-32 Server Front Learn about the components the front of the M5-32 Server by engaging in this Student Interaction.

M5-32 Server Overview 1 - 14

M5-32 Server Rear Learn about the components the rear of the M5-32 Server by engaging in this Student Interaction.

M5-32 Server Overview 1 - 15

CMPs in CMUs At the top of the slide, you see the CMU numbering as you look left to right at the rear of the M5-32 Server. You see 16 CMUs numbers CMU 0 through CMU 15. At the bottom of the slide, you see one CMU shadowed which is CMU 0 with an enlarged view. The CMU contains memory and the two SPARC M5 processors in each CMP (Chiplevel multiprocessing or multicore processor) which are labeled CMP 0 and CMP 1. Each SPARC M5 processor has 6 CPU cores and each core has L2 8-way 128K cache.

M5-32 Server Overview 1 - 16

CMU Memory Components Each CMU contains two memory boards with each memory board connecting to a CMP. The two memory boards are shown as light gray rectangles on the CMU diagram. There is one memory board for each CMP (Chip-level multiprocessing or multicore processor). The two CMPs are shown as two orange rectangles on the CMU diagram. Each memory board has eight BoB (Buffer-on-Board) controllers. Four of the eight BoB controllers are connected directly to the CMP while the other four BoB controllers cascade into the first set of four BoBs. The four BoB controllers connected directly to the CMP are shown as light blue boxes on the diagram. The four cascading BoB controllers are shown as dark blue boxes in top of the diagram. Notice, BoB number ones cascading connection to BoB number zero and BoB number zero connected directly to the CMP. There are a total of 4 DIMMs for each Buffer-on-Board controller. These are Oracle Certified DDR3L (1.35V Low-voltage) DIMM modules that run at a data transfer rate of 1066 MHz. DIMM capacities that are supported are 16GB and 32GB. In a CMU, there are a total of 64 DIMM slots.

M5-32 Server Overview 1 - 17

I/O Unit Components On this slide you see the IOU. On the bottom left of the IOU, you see an I/O Board in IOB 0. On the top left of the IOU, you see an I/O Board in IOB 1. The I/O boards are required in order to access the internal HDDs/SSDs. Each I/O board provides the connection to two EMS cards. The 10GbE interfaces are on the EMS cards. In the center of the IOU, you see two cages with each cage containing four HDD/SSD Drives. The left cage is cage 1 and the right cage is cage 2. The slots are labeled HDD 0 through HDD 3 (left to right) in cage 1 and the slots are labeled HDD 4 through HDD 7 in cage 2. On the bottom right of the IOU, you see eight PCIe Gen3 cards in slots labeled PCIe 1 through PCIe 8. On the top right of the IOU, you see 8 PCIe cards in slots labeled PCIe 9 through PCIe 16. Below and above the HDDs in the center of the IOU, you see four EMSs. Labeling slots starts at the bottom with EMS 1 through EMS 4.

M5-32 Server Overview 1 - 18

Domain Configuration Unit (DCU) DCUs (Domain Configuration Units) are the hardware building blocks of Physical Domains (PDoms). Each DCU includes two or four CMUs, one SPP, one IOU & eight fans. A Physical Domain (PDom) can be made from one or up to four DCUs. Each PDom operates like an independent server that has full hardware isolation from other PDoms on the M5-32 Server.

M5-32 Server Overview 1 - 19

Follow the instructions in this Student Interaction to become familiar with M5 CPU processors use of Coherency Links.

M5-32 Server Overview 1 - 20

Follow the instructions in this Student Interaction to become familiar with M5 CPU processors use of Scalability Links.

M5-32 Server Overview 1 - 21

CMUs Connected to Scalability Switch In this diagram you see 4 DCUs which is the maximum that can be configured in a M532 Server. Each DCU has 4 CMUs which are connected to one of the 12 Scalability Switch Boards (SSBs). Each SSB has a Bixby (BX) ASIC (Application-Specific Integrated Circuit). The ASIC connects the CPU, memory, and I/O. The Bixby ASIC also controls the cache coherency and holds a portion of the Central Coherency Directory (reverse directory of all L3 caches on all the processors). In a directory-based coherence, the data being shared is placed in a central directory that maintains the coherence between caches. The directory acts as a filter through which the processor must ask permission to load an entry from the primary memory to its cache. When an entry is changed the directory, it either updates or invalidates the other caches with that entry. The ASIC determines who requested the data, where the data is located and where to send the data. Each of the 12 Scalability Switch boards has a slot number labeled according to its position in the assembly. The first SSB slot is BX 0 and the last SSB slot is BX 11. Even numbered CMUs are connect to odd numbered SSB boards. For example, CMU 0 is connected to BX 1, BX 3, BX 5, BX 7, BX 9 & BX 11 ASICs (shown with black connecting lines). Odd numbered CMUs are connect to even numbered SSBs. For example, CMU 1 is connected to BX 0, BX 2, BX 4, BX 6, BX 8 & BX 10 ASICs (shown with red connecting lines).

M5-32 Server Overview 1 - 22

Two CMU Connections to PCIe Slots In the M5-32 Server, there is a relationship between a particular CMPs pci port, the PCIe switch port in the I/O board and the PCIe slot in the IOU. The diagram on the slide depicts those relationships with its color coding. On the diagram, you see one DCU with two CMUs labeled CMU0 & CMU3. Each CMU has two CMPs labeled CMP0 and CMP1 with labeled pci slots (e.g. CMU0s CMP0 has pci slots 0 and 1). In the center of the diagram, you see two I/O Boards (IOB) labeled IOB 0 and IOB 1. Each I/O board has 4 PCIe switches. One switch in the group four switches are populated with connections from the CMP pci ports to the PCIe slots shown at the bottom of the diagram. On the left side of the diagram, you see CMU 0 CMP 0 (shown in blue) pci port 0 is connected to IOB 0s second switch from the left and then routed to PCIe slots 1 and 2. CMU 0 CMP 0 pci port 1 is connected to IOB 0s leftmost switch and then routed to PCIe slots 3 and 4. You also see an EMS (Ethernet Module and Storage) connection from this switch to EMS 1. EMSs are PCI compliant connectors for PCI signals to be accepted by the EMS and for it to route SAS signals to the IOB Back-Plane. CMU 0 CMP 1 (shown in yellow) pci ports 2 & 3 are routed into the other two switches in IOB 0.

M5-32 Server Overview 1 - 23

Four CMU Connections to PCIe Slots On the diagram, you see one DCU with four CMUs labeled CMU0, CMU1, CMU2 & CMU3. You see that there are only two PCIe slots connected to each CMPs pci port instead of four PCIe slots you saw used in the DCU configuration with only two CMUs on the previous slide. While the previous examples configuration provides the best I/O availability, this DCU configuration with 4 CMUs provides the most redundancy.

M5-32 Server Overview 1 - 25

M5-32 Server Overview 1 - 26

Two Clock Boards There are two clock boards in the M5-32 Server so one clock board is active (Clock 0) and the other clock board (Clock 1) is on standby for failover in this redundant configuration. Each clock board has a single clock source per board. If the active clock board fails, the system to reboots and it uses the redundant clock board and marks the failed clock board as inactive. The failed board can be replaced while the system is running on the alternate clock board. An ILOM command is required to be run to replace the failed clock board with a new one. After its replacement, the new clock board is recognized and can now be used for failover if needed.

M5-32 Server Overview 1 - 27

Service Processor (SP) The Service Processor is redundant in the M5-32. It provides the primary platform configuration and management for the M5-32 Server. It works with the SPP (Service Processor Proxies) in each DCU to configure and monitor the components in each DCU. It runs the ILOM (Integrated Lights Out Management) software to manage the M5-32 Server. There are two service processors (SPs) in the M5-32 server. One acts as the primary and the other as the standby to support automatic failover.

M5-32 Server Overview 1 - 28

Service Processor Proxy (SPP) A Service Processor Proxy (SPP) is a local service processor dedicated to a DCU. The SPPs purpose is to manage the CPU, memory controllers and DIMMs within a DCU. Also, it does environmental monitoring of internal sensors (e.g. temperature sensors and adjusts the 8 connected fans according to need). It also provides the rKVMS (Remote Keyboard/Video/Mouse/Storage) console for remote management support of the SPP (e.g. remote booting of physical & virtual servers after a firmware update). In a M5-32 Server, there are four SPPs with each SPP being dedicated to one of the four DCUs and commanded by the Main Service Processor (SP). Audit logs, faults, etc. are forwarded from the SPPs to the Main-SP. In a multi-DCU domain scenario, the SPP with the lowest number available in the domain will become the Golden SPP that will manage the Physical Domain for all the DCUs in the domain. The configuration from the Golden SPP is backed-up to the MainSP.

M5-32 Server Overview 1 - 29

SP and SPP Connectivity The SP and SPPs communicate via internal 100 Base-T Ethernet connections. The 100 in the media type refers to the transmission speed of 100 Mbit/s for the 100BASET twisted pair Ethernet cables. The SP manages the M5-32 Server chassis along with the 16 port Ethernet switch. The Ethernet switch is a VLAN switch so it can be partitioned, etc. The SPPs are connect to the Ethernet switch. Each SPPs manage 8 CPUs (or 4 CMUs), monitor the 8 fans, send I/O signals to I/O boards, manage the rKVMS activity, etc. If there is an issue with a fan, the SPP informs the SP of any cooling issues and the SP adjusts fans if needed. If the fans are not working for a CMU, there will be no airflow so the CMU will be shutdown.

M5-32 Server Overview 1 - 30

rKVMS The rKVMS (Remote Keyboard/Video/Mouse/Storage) is a virtual device that is part of the SPP. The virtual rKVMS do NOT exist on the SPs. The rKVMS provides a virtual USB keyboard, video, mouse and storage for the user. The M5-32 Server supports both serial-redirection and video-redirection for each configured Physical Domain. However, the domain console is only supported via serialredirection. The domain console is NOT supported via video-redirection. The video window allows a user to connect to the domain but it is only an X session connection. A user can launch JRC (Java Remote Console) on their laptop or server from the BUI on the Main-SP. The user selects the desired domain, selects either "serial-redirection" or "video-redirection' and then clicks the Launch button. JRC is loaded on the users local system and this initiates a connection to that domain's Golden-SPP. All communication via JRC and the ILOM is to the Golden-SPP for that domain. None of the rKVMS network traffic goes to the Main-SP. JRC is launched from the Main-SP but it connects to the Golden-SPP which 'is unique to the M5-32 Server. The Golden-SPP manages rKVMS the same as any single-SP ILOM system such as the T5 Server. On M5-32 Server, you can remove the I/O path to the rKVMS devices just like you can with the T5-8 where the CMUs are also removable.

M5-32 Server Overview 1 - 31

Service Processor Software (ILOM) The Oracle ILOM (Integrated Lights Out Management) software enables active management and monitoring of the M5-32 Server. The ILOM on the M5-32 Server looks and behaves just like ILOM on other platforms except is it also has a simple (user-visible) set of extensions to support Enterprise features, such as, Physical Domains, Service Processor Proxies and redundant Service Processors. These extensions have a minimal impact on user experience. The ILOM runs on the SP (Service Processor).

M5-32 Server Overview 1 - 32

Oracle ILOM Key Functions The Oracle ILOM is the system management firmware that is preinstalled on the M5-32 Server. ILOM functions provide the user the ability to perform firmware updates, Remote Host Management, Inventory and Component Management, System Monitoring and Alert/Fault Management, User Account Management and Power Consumption Management. So, with all these functions, the ILOM enables you to actively manage and monitor components installed in your M5-32 Server. The Oracle ILOM SP runs independently of the server and regardless of the server power state as long as AC power is connected to the server. When you connect the server to AC power, the ILOM service processor immediately starts up and begins monitoring the server. All environmental monitoring and control are handled by Oracle ILOM. Oracle ILOM provides a browser-based interface, a command-line interface, as well as SNMP and IPMI interfaces. So, you see the ILOM is available through a variety of interfaces. More information on the ILOM interfaces follows: BUI interface - The web interface provides an easy to use BUI (Browser User Interface)

M5-32 Server Overview 1 - 33

Advanced Virtualization Management Advanced Virtualization Management provides a central interface for VM lifecycle management to provide quicker VM deployments and therefore increase productivity. This is accomplished via Solaris Zones, Oracle VM for SPARC & Dynamic Domains. The Customer is able to monitor VM or system-level utilization, reconfigure VMs dynamically, create resource pools and migrate VMs across servers.

M5-32 Server Overview 1 - 35

Virtualization Software The M5-32 Server has a high degree of virtualization including Physical Domains (PDoms). The granularity of the Physical Domains is 8 CPU sockets (or 2 CMUs). Logical Domains (LDoms) are supported within Physical Domains and Oracle Solaris Zones for OS virtualization. Oracle Enterprise Manager Ops Center provides an administrator a user-friendly interface for these different virtualization levels.

M5-32 Server Overview 1 - 36

What is a Physical Domain? There can be up to four physical domains (PDoms or PDomains) in the M5-32 Server. Each PDom operates like an independent server that has full hardware isolation from other PDoms in the chassis. Domain configurable units (DCUs) are the building blocks of PDoms. Each PDom is represented as /HOSTx in Oracle ILOM where x ranges from 0 to 3 (PDomain_0, PDomain_1, PDomain_2, PDomain_3).

M5-32 Server Overview 1 - 37

Two Types of Physical Domains There are two Types of Physical Domains. The first type is an expandable unbound Dynamic Domain or just a Pdom or Physical Domain. The second type is a Bounded Dynamic Domain or Bounded PDom. The System Administrator sets the attribute of the Physical Domains to select the type (e. g. attribute expandable=true).

M5-32 Server Overview 1 - 38

Unbound Dynamic Domains An expandable unbound Dynamic Domain or Physical Domain or Pdom allows each PDom to be different. A Physical Domain consisting of a single DCU is permitted to add new DCUs via a DR operation. In this way, the Physical Domain can expand the number of M5 CPU processors beyond its original 8 in its one DCU. A Physical Domain greater than one DCU must use the SSBs Bixby ASIC with its portion of the Coherency Directory to perform a lookup for remote DCUs so latency is higher for Dynamic Domains in comparison to Bounded Dynamic Domains. Each of the 6 Bixby ASICs hold 50% of the system directory (or Coherency Directory) for a M5-32 Server. Logically, the 12 SSBs are managed as two groups of 6. Each group of 6 can loose 1 SSB and it can run in a degraded mode of 5 and the system directory (or Coherency Directory) is still available. One negative is that due to the system directory (or Coherency Directory) requirement, the loss of a Bixby ASIC causes a PDom reset.

M5-32 Server Overview 1 - 39

Bounded Physical Domains A Bounded Dynamic Domain (or Bounded PDom) does not use the Bixby ASICs Coherency Directory for CMU to CMU access within the DCU. Because it does not use the Bixby ASIC, it does less I/O and therefore has a lower latency compared to a Dynamic Domain PDom. Another benefit to Bounded Dynamic Domains is that it is not impacted by loss of SSBs. The disadvantage of a Bounded Dynamic Domain is that is can never grow beyond a DCU so it can not expand beyond 8 M5 CPU processors.

M5-32 Server Overview 1 - 40

Physical Domain Examples On the slide, you see nine PDom configuration examples for a M5-32 Server. Each example uses 4 DCUs with each DCU configured with 4 CMUs. As the legend indicates, the solid line around the DCU(s) represents a Bounded Dynamic Domain and a dashed line around the DCU(s) represents an expandable unbound Dynamic Domain. Starting with the three PDom configuration examples at the top of the slide, you see a Dynamic Domain in the upper left example. PDom 0 has 4 DCUs configured as one Dynamic Domain. The middle configuration shows a Bounded Dynamic Domain labeled PDom 0 with one DCU and a Dynamic Domain labeled PDom 1 with three DCUs. The rightmost configuration shows two Dynamic Domains labeled PDom 0 and PDom 1. PDom 0 has one DCU and PDom 1 has three DCUs.

Take a minute to review the remaining six examples shown on the slide so you know what Physical Domain configurations are possible within the M5-32 Server.

M5-32 Server Overview 1 - 41

What Is a Logical Domain? Logical Domains (or LDOMs) allow running multiple instances of Solaris on a single physical platform. Logical Domains is the server virtualization and partitioning technology based on SPARC V9 architecture. Oracle rebranded this technology as Oracle VM Server for SPARC (after Oracles acquisition of Sun in January 2010). Each domain is a virtualized environment with a reconfigurable subset of hardware resources and an OS that can be started, stopped, and rebooted independently of the host system or any other domains. Also, a running domain can be dynamically reconfigured to add or remove CPUs, RAM, or I/O devices without requiring a reboot.

M5-32 Server Overview 1 - 42

Hypervisor and Logical Domains Hypervisor implements the software component of the virtual machine, providing low overhead hardware abstraction. It enforces hardware and software resource access restrictions for guests, including inter-guest communication, to provide isolation and security. It also performs initial triage and correction of hardware errors.

M5-32 Server Overview 1 - 43

What Is a Zone? A zone is a virtual operating system abstraction that provides a protected environment in which applications run. The applications are protected from each other to provide software fault isolation. To ease the labor of managing multiple applications and their environments, they coexist within one operating system instance, and are usually managed as one entity. The original operating environment, before any zones are created, is called the global zone to distinguish it from nonglobal zones. The global zone is the operating system instance. Oracle Solaris Zones enable you to isolate one application from others on the same OS, allowing you to create an isolated environment in which users can log in and do what they want from inside an Oracle Solaris Zone without affecting anything outside that zone. Oracle Solaris Zones are secure from external attacks and internal malicious programs. Each Oracle Solaris Zone contains a complete resource-controlled environment that enables you to allocate resources such as CPU, memory, networking, and storage.
M5-32 Server Overview 1 - 44

Oracle VM Server for SPARC: M5 Servers Oracle VM Server for SPARC, previously called Sun Logical Domains, provides highly efficient, enterprise-class virtualization capabilities for the M5-32 server. A logical domain is a discrete logical grouping with its own operating systems, resources, and identity within a single computer system. Application software can run in logical domains. Each logical domain can be created, destroyed, reconfigured, and rebooted independently. Oracle VM Server for SPARC leverages the built-in hypervisor to subdivide system resources (CPUs, memory, network, and storage) by creating partitions called logical (or virtual) domains. Each logical domain can run an independent operating system. Oracle VM Server for SPARC provides the flexibility to deploy multiple Oracle Solaris operating systems simultaneously on a single platform. This is the virtualization solution that fully optimizes Oracle Solaris and SPARC for your enterprise server workloads.

M5-32 Server Overview 1 - 45

M5-32 Server RAS by Hardware or Software Component On this slide, you can see the many RAS features in each component of the M5-32 system. Take a minute to read the features listed for Oracle Solaris 11 FMA (Fault Management Architecture), M5 processor, Hypervisor, central directory and switch, Service Processor, Power and Cooling, System I/O and Memory components. For example, Oracle Solaris 11 FMA (or S11 FMA) provides FRU Monitoring for hotupgradable components. The M5 processor has parity protection. The M5 memory has extended ECC (Error-correcting code) error correction protection. Hypervisor enables failure containment in the software partitions (Logical Domains). The central directory and switch has CRC protection. There are two Service Processors with automatic failover. The M5-32 system hardware has hot-plug CMU (CPU Memory Unit) modules, disk drives, PCIe cards, Service Processor modules, power supplies and fans. So, these are just a few examples of the RAS hardware features for processor, memory, I/O, storage, power and cooling for the fault tolerant M5-32 system design.

M5-32 Server Overview 1 - 46

Optionally Download Training Use the ATTACHMENT link in the upper right-hand corner to optionally download this trainings slides and notes in PDF format. Note: Student Interactions cannot be viewed.

M5-32 Server Overview 1 - 47

M5-32 Server Overview 1 - 48