Release 9.

Hardware Sizing Guide for Sun Deployments

2010 Blackboard Inc. Proprietary and Confidential

Publication Date: June 11, 2010 Worldwide Headquarters Blackboard Inc.

650 Massachusetts Avenue NW Sixth Floor Washington, DC 20001-3796 +1 800 424 9299 toll free US & Canada +1 202 463 4860 telephone +1 202 463 4863 facsimile www.blackboard.com +31 20 5206884 (NL) telephone +31 20 5206885 (NL) facsimile www.blackboard.com

International Headquarters Blackboard International B.V.

Dam 27 2nd Floor 1012 JS Amsterdam The Netherlands

Copyright 1997-2010. Blackboard, the Blackboard logo, BbWorld, Blackboard Learn, Blackboard Transact, Blackboard Connect, the Blackboard Outcomes System, Behind the Blackboard, and Connect-ED are trademarks or registered trademarks of Blackboard Inc. or its subsidiaries in the United States and other countries. U.S. Patent Numbers: 6,988,138; 7,493,396; 6,816,878. Sun and Java are registered trademarks of Sun Microsystems, Inc. in the United States and other countries. Microsoft, Windows, and SQL Server are registered trademarks of Microsoft Corporation in the United States and other countries. Oracle is a registered trademark of Oracle Corporation and its affiliates. UNIX is a registered trademark in the United States and other countries, licensed exclusively through X/Open Company Ltd. Other product and company names mentioned herein may be the trademarks of their respective owners. No part of the contents of this manual may be reproduced or transmitted in any form or by any means without the written permission of the publisher, Blackboard Inc.

Blackboard Learn Hardware Sizing Guide for Sun Deployments

2010 Blackboard Inc. Proprietary and Confidential

Page 2

Contents
Introduction ...............................................................................................................................4 About Blackboard Learn ........................................................................................................4 About this Guide ...................................................................................................................4 Recommended Configurations .................................................................................................5 Deploying Blackboard Learn ....................................................................................................6 Sizing the Application Server ...................................................................................................7 Sun Configuration Information ...............................................................................................7 Will Tomcat Clustering Make a Difference? ........................................................................... 7 Sun Configurations for Blackboard Learn Release 9.1 ........................................................... 9 Sizing the Database ................................................................................................................ 11 Sizing Storage ......................................................................................................................... 12 Media and Content Storage................................................................................................. 12 Database Data File Storage ................................................................................................ 12 Storage Performance .......................................................................................................... 13 Appendix A: Sizing Methodology ........................................................................................... 14

Blackboard Learn Hardware Sizing Guide for Sun Deployments

2010 Blackboard Inc. Proprietary and Confidential

Page 3

Introduction

Introduction
A paradigm shift is underway in online learning. Online learning tools that were once used primarily to augment classroom learning have matured to support an online model in which the Course Management System serves as the primary medium for teaching and learning. As the prevalence of high-speed Internet connections and home computer ownership has grown, so too has the demand for distance learning. Today's learners are turning to web-based distance learning programs as a means to balance other obligations with the desire to further their education, or because they are too far away from the institutions that can best serve their needs. The movement toward fully online learning models creates new demands for the IT environment and opens up new opportunities for institutions to use technology as a competitive tool. Demand for higher performance and faster response times are being driven by two key factors: greater numbers of users, and increased activity by users. Not only are institutions growing and supporting greater numbers of online users, but users are also interacting with the system in new ways that generate substantially greater system load. Users tend to stay online for much longer periods of time and have greater interactivity with their online courses. The technology must therefore support large numbers of users who concurrently access the system and require rapid response times so that they can interact without noticeable delays. When users have consistent and responsive page load times, online learning is easier and more effective, new forms of teaching and learning can also evolve when interaction has the feeling of being real-time. Traditional online lectures and online documents can be blended with interactive media, allowing students to share experiences with each other across geographical boundaries. This can increase the effectiveness of the learning environment and expand the available market the institution serves. Institutions want their online learning technology infrastructure to support these new forms of teaching and learning on a scale that allows many more students than could traditionally participate in a classroom approach. The online learning architecture must therefore be highly scalable as well as cost-effective to maintain so that institutions can focus on teaching instead of technology. The right technology infrastructure for online learning can create a competitive advantage for the institution.

About Blackboard Learn

Blackboard Learn includes Blackboards course management system, Course Delivery, as well as other core applications. Additional capabilities that can be licensed include community engagement, content management, and outcomes assessment.

About this Guide

This guide helps Blackboard clients achieve high service levels and reduce risk by properly configuring and sizing the implementation of Blackboard Learn software on Sun application and database servers. The information in this guide is based on analysis and benchmarking models from the Blackboard Performance Team in conjunction with Sun. For information about Blackboards sizing methodology, see Appendix A: Sizing Methodology. This guide is intended to provide guidance. It is not intended as a service level agreement. Deployments will differ from institution to institution based on a variety of factors including the usage of the application. For more information about systems architecture design and detailed sizing questions, contact your Blackboard Account Manager.

Blackboard Learn Hardware Sizing Guide for Sun Deployments

2010 Blackboard Inc. Proprietary and Confidential

Page 4

Recommended Configurations

Recommended Configurations
Blackboards recommended configuration is based on a scalable reference architecture that takes advantage of todays latest technologies to support a rich online learning or distance learning model. In addition, this architecture offers superior performance and scalability as well as predictable service levels. This information is based on joint performance testing and benchmarking between Blackboard and Sun to achieve maximum performance throughput for each configuration.

Blackboard Learn Reference Architecture on Sun Hardware

Blackboard Learn Hardware Sizing Guide for Sun Deployments

2010 Blackboard Inc. Proprietary and Confidential

Page 5

Deploying Blackboard Learn

Deploying Blackboard Learn

Customers who deploy Blackboard Learn on Sun hardware have the flexibility to choose from a variety of server architectures that support Solaris, Linux, and Windows. Sun offers hardware models built on UltraSPARC T2/T1, AMD Opteron, and Intel Xeon processor platforms. The UltraSPARC architecture is well suited for Blackboard Learn. Clients can deploy either the Sun Fire T5120/T5220 server for price and throughput performance, or the Sun Fire T5140/T5240 for overall throughput performance. All 6 units have the capacity to scale to 6 cores or 8 cores with a minimum of 16GB of memory. The Sun Fire T5140/T5240 has the capacity of 2 sockets per physical machine. Blackboard recommends leveraging both physical and virtual configurations. For years, the Blackboard products have been deployed in distributed manner across multiple physical servers. This configuration remains as the primary option for availability and scalability. In recent releases, many Blackboard customers have introduced virtualized configurations within a physical server using technology such as Solaris Zones. The combination of both physical and virtualized configurations has introduced resiliency, scalability, and high performance. In Blackboard Learn Release 9.1, Blackboard introduced a 100% 64-bit offering for all certified and supported operating systems (OSs) as an additional deployment approach to integrate with physical and virtual configurations.

Blackboard Learn Hardware Sizing Guide for Sun Deployments

2010 Blackboard Inc. Proprietary and Confidential

Page 6

Sizing the Application Server

Sizing the Application Server

Sun Configuration Information
Application tier scalability is achieved using a highly flexible, horizontally scaled infrastructure. Multiple instances of Blackboard Learn can be deployed on a single Sun server using either a virtualization package or bare-metal configuration to isolate Blackboard application instances from each other and to manage allocation of CPU and memory resources. Each Sun server in the application tier can run multiple zones each with its own instance of the Blackboard Learn software. To achieve a high volume of concurrent user sessions, the architecture employs load balancing to distribute requests across all of the Blackboard Learn application instances deployed on the zones. The architecture takes advantage of 64-bit computing by using Java heap sizes that range from 2G to 16GB for each Blackboard application instance. Some customers on rare occasions have deployed Java Virtual Machines (JVMs) as large as 32GB in size. The Java heap had been limited to 1.7GB in a 32-bit environment, but this limitation is overcome in the 64-bit environment. Because this feature is now available for all platforms with Blackboard Learn Release 9.1, Blackboard strongly recommends that customers adopt 64-bit for their primary deployment OSs. The architecture includes a recommended 8GB of memory for each VM, leaving up to 2GB for the Apache Web server, and up to 2GB for the OS and monitoring tools. This assumes a JVM heap size of 4GB. Obviously, a larger JVM heap size will require additional memory for the VM. The larger memory footprint for Blackboard Learn enables each application instance to more efficiently service a high volume of user requests. In other words, each application instance can scale vertically while the architecture can also be scaled horizontally by deploying additional VMs. This best of both worlds approach takes advantage of many application threads in the application tier to service thousands of requests per minute. The recommended servers for the application tier are Sun SPARC Enterprise T5120 and T5140 models, which were chosen for their small footprint, moderate cost, and powerful throughput performance. The highly flexible and scalable application tier can help customers achieve: Lower costs: Increased utilization of servers enables good performance on a low-cost consolidated infrastructure. Increased flexibility: VMs can be moved easily to other physical servers to redistribute workloads or recover quickly from a hardware failure. Faster Deployment: With fewer physical systems to setup and configure, less time is required for deployment.

Will Tomcat Clustering Make a Difference?

Customers should begin migrating from Tomcat clusters. Tomcat clustering was introduced for scalability purposes when the Blackboard Learn architecture was 32-bit and customers wanted the ability to increase their memory usage on a single server. With the option to virtualize on both 32-bit and 64-bit, Blackboards benchmarking efforts have moved away from Tomcat cluster deployments. Customers can achieve similar performance in a virtual environment on the same physical server with multiple virtual instances as with a bare metal configuration with many Tomcat cluster instances. The difference in configurations is a smaller demand on the Apache or IIS web server fronting the Tomcat instance(s). The option to deploy a 64-bit JVM with larger heap sizes has suppressed the need for customers to run in a cluster option.

Blackboard Learn Hardware Sizing Guide for Sun Deployments

2010 Blackboard Inc. Proprietary and Confidential

Page 7

Sizing the Application Server

Blackboard recommends that customers consider a deployment approach consisting of larger 64-bit JVMs that are distributed across physical servers with the option to virtualize the hardware to take advantage of the CPU and Memory capacity of these systems.

Blackboard Learn Hardware Sizing Guide for Sun Deployments

2010 Blackboard Inc. Proprietary and Confidential

Page 8

Sun Configurations for Blackboard Learn Release 9.1

Sizing Category Description Standard The Standard sizing configuration is designed for availability and performance. Options exist for both physical distributions of application servers and/or virtualized instances of the application. The institution has minimal deployment of distance learning or connected classroom initiatives (less than 5%). Advanced The Advanced sizing configuration is designed for high-availability and high-performance. The model is designed for heavy adoption of the product across institutions of varying sizes. Customers deploying distance learning programs or have connected classroom initiatives to a portion of the community fall into this category. Complex The Complex sizing configuration is designed for massive adoption, high-availability and highperformance. Customers deploying predominately distance learning programs or have connected classroom initiatives fall into this category. Customers with larger community sizes (greater than 250,000 users) fall into this category.

App li cati on S erver C onf ig ur ati on Number of OS Instances (Physical or Virtual) Minimum CPU Configuration Per OS Instance 2 to 4 1 x 8-Core 1.2 GHz UltraSPARC T2 64 Threads Minimum CPU Collapse Ratio (Zone to Physical Server) Ideal CPU Configuration Per OS Instance 4:1 (Zone Configuration) 1 x 8-Core 1.6 GHz UltraSPARC T2 64 Threads Ideal CPU Virtualization Collapse Ratio (Zone to Physical Server) 64-bit Memory Requirements Per OS Instance I/O Workload at Peak Hour Hits/Hour at Peak Hour Characteristics 4:1 (Zone Configuration) 4 to 8 1 x 8-Core 1.4 GHz UltraSPARC T2 64 Threads 4:1 ( Zone Configuration ) 2 x 8-Core 1.6 GHz UltraSPARC T2 Plus 128 Threads 6:1 ( Zone Configuration ) 8 to 16 2 x 8-Core 1.4 GHz UltraSPARC T2 Plus 128 Threads 6:1 ( Zone Configuration ) 2 x 8-Core 1.6 GHz UltraSPARC T2 Plus 128 Threads 6:1 ( Zone Configuration )

8GB to 16GB recommended 300 to 500 I/Os Up to 4 Million Hits/Hour With 8 CPU threads per instance and a 64-bit deployment across 4 instances, this configuration will support as many as 20,000 concurrent sessions for a peak hour. A concurrent session is a user who has logged into the application within the last hour. Results will vary depending on the use of memory intensive areas of the application such as Assessment and Grade Center. Memory has the greatest impact on the session concurrency and responsiveness of the application, followed by CPU capacity. Consider a larger memory footprint for even smaller workloads to deliver faster page responsiveness.

8GB to 16GB recommended 500 to 1000 I/Os Up to 8 Million Hits/Hour With 16 to 20 CPU threads per instance and a 64-bit deployment across 6 instances, this configuration will support as many as 40,000 concurrent sessions for a peak hour. A concurrent session is a user who has logged into the application within the last hour. Results will vary depending on the use of memory intensive areas of the application such as Assessment and Grade Center. Memory has the greatest impact on the session concurrency and responsiveness of the application, followed by CPU capacity. Consider a larger memory footprint for even smaller workloads to deliver faster page responsiveness.

8GB to 16GB recommended 1000 to 2000 I/Os Up to 12 Million Hits/Hour With 16 to 20 CPU threads per instance and a 64-bit deployment across 6 instances, this configuration will support as many as 80,000 concurrent sessions for a peak hour. A concurrent session is a user who has logged into the application within the last hour. Results will vary depending on the use of memory intensive areas of the application such as Assessment and Grade Center. Memory has the greatest impact on the session concurrency and responsiveness of the application, followed by CPU capacity. Consider a larger memory footprint for even smaller workloads to deliver faster page responsiveness.

Blackboard Learn Hardware Sizing Guide for Sun Deployments

2010 Blackboard Inc. Proprietary and Confidential

Page 9

Sun Configurations for Blackboard Learn Release 9.1

Sizing Category Description Standard The Standard sizing configuration is designed for availability and performance. Options exist for both physical distributions of application servers and/or virtualized instances of the application. The institution has minimal deployment of distance learning or connected classroom initiatives (less than 5%). Advanced The Advanced sizing configuration is designed for high-availability and high-performance. The model is designed for heavy adoption of the product across institutions of varying sizes. Customers deploying distance learning programs or have connected classroom initiatives to a portion of the community fall into this category. Complex The Complex sizing configuration is designed for massive adoption, high-availability and highperformance. Customers deploying predominately distance learning programs or have connected classroom initiatives fall into this category. Customers with larger community sizes (greater than 250,000 users) fall into this category.

Dat ab ase S erv er C on fi gur at io n Minimum CPU Configuration 1 x 8-Core 1.6 GHz UltraSPARC T2 64 Threads Ideal CPU Configuration 1 x 8-Core 1.6 GHz UltraSPARC T2 64 Threads Memory Requirements I/O Workload at Peak Hour Availability Approach 8GB to 16GB 600 to 1,200 I/Os Failover clustering approaches that leverage Active/Passive database configurations should be considered for this deployment. 16GB to 64GB 1,500 to 3,000 I/Os Failover clustering approaches that leverage Active/Passive database configurations should be considered for this deployment. Solaris deployments may consider Oracle Real Application Clusters (Oracle RAC) 2 x 8-Core 1.6 GHz UltraSPARC T2 Plus 128 Threads 8 x 2.53 GHz SPARC64 VII on 4 Boards 2 x 8-Core 1.6 GHz UltraSPARC T2 Plus 128 Threads 4 x 8-Core 1.6 GHz UltraSPARC T2 Plus 256 Threads 48GB to 128GB 4,500 to 10,000 I/Os Failover clustering approaches that leverage Active/Passive database configurations should be considered for this deployment. Solaris deployments should consider Oracle Real Application Clusters (Oracle RAC)

Or a cl e M e mory Re qu ir em e nt s Estimated SGA Size Processes Estimated Process Size Estimated PGA Memory 1GB to 4GB 600 to 1,200 3M to 7M 2GB to 8GB 4GB to 16GB 2,000 to 4,000 3M to 7M 10GB to 20GB 16GB to 32GB 5,000 to 10,000 3M to 7M 25GB to 50GB

Blackboard Learn Hardware Sizing Guide for Sun Deployments

2010 Blackboard Inc. Proprietary and Confidential

Page 10

Sizing the Database

Sizing the Database

The Sun Fire T5220 and T5240 are ideal for running Oracle 10gR2 or 11gR1 in small and medium size institutions. Both systems have 6-core and 8-core configurations available. Memory requirements will vary depending on deployment within a 32-bit or 64-bit configuration. Each of these systems can be used for availability purposes in an Oracle Real Application Clusters (Oracle RAC) configuration. Blackboard offers support for RAC with all versions of Oracle supported by Blackboard. For information about running Blackboard Learn with RAC, see the Blackboard Learn Server Administrator Guide available on Behind the Blackboard. For large campus configurations, a Sun Fire T5440 or Sun SPARC Enterprise M5000 server is used instead of the Sun Fire T5220 or T5240 server to provide increased capacity. A high availability option is supported through an optional cluster configuration with a redundant instance of the database running on a second server using Oracle RAC to provide the redundant database functionality. Sizing for memory is absolutely critical for ensuring a high-performing and reliable configuration. Oracle configurations require a set of inputs to calculate the SGA and PGA spaces of memory. The SGA will contain both the SQL area for caching statements, as well as the buffer cache for storing blocks of data in memory. The PGA space will maintain the process memory requirements, as well as hash and sorting space for query execution. The combination of these two regions makes up the requirements of Oracle memory.

Blackboard Learn Hardware Sizing Guide for Sun Deployments

2010 Blackboard Inc. Proprietary and Confidential

Page 11

Sizing Storage

Sizing Storage
Media and Content Storage
Blackboard Learn uses a non-relational file system for the storage of multi-media and binary files such as text files, images, word processing documents, spreadsheets, and other file types used in teaching and learning. Blackboard recommends that clients on UNIX platforms use a network file share (NFS). Blackboard recommends that clients on Windows use a common internet file system (CIFS). Both of these network-based file system protocols allow for simplified management, ease of data expansion, and multiple access points from applications servers. Load balanced installations can use both file system types. Deploying storage on a local application server can also be implemented, but Blackboard does not recommend doing so. File system content can range from 5100 times the size of database content. File system content from a block perspective is touched less frequently than the database file system. Clients can opt to configure their systems to a lesser performing RAID configuration with slower spindles because I/O performance is less of a concern. Blackboard recommends that clients determine a storage quota per student and faculty member as well as account for passive users that require less storage quota. Assume that faculty will have greater storage requirements. Following is a simple example:
Pr of i l e Faculty Student Other Q uo t a 5GB 500MB 20MB Us e r s 500 10,000 1,000 St or a g e Ne e d s 2.5 TB + RAID Considerations 5TB + RAID Considerations 20GB + RAID Considerations

Sizing the file system depends on archival strategies, data management policies, RAID configuration, and I/O performance standards. Blackboard typically assumes that the file system will require about 100+ I/O per second per application server at peak. To calculate your I/O per second needs, multiply this metric against the number of application servers in your deployment.

Database Data File Storage

Blackboard Learn uses a relational database system (Oracle and SQL Server) for the storage of database content. Clients on UNIX platforms may use one of the following: A network file share (NFS) ISCSI (networked block-level) Direct attached storage (block-level)

Clients on Windows SQL Server may use one of the following: ISCSI (networked block-level) Direct attached storage (block-level)

Blackboard Learn Hardware Sizing Guide for Sun Deployments

2010 Blackboard Inc. Proprietary and Confidential

Page 12

Sizing Storage

The database storage requirements of institutions can vary. Typically, database content can range from 5100 times less than file system content. Sizing the database depends on archival strategies, data management policies, RAID configuration, and, most importantly, I/O performance standards. Blackboard typically assumes that the file system will require between 350 to 600 I/O per second per application server at peak. To calculate your I/O per second needs, multiply this metric against the number of application servers in your deployment. The primary driver for database storage should be performance.

Storage Performance
The database tier has variable I/O performance needs, therefore, Blackboard recommends using fibre channel drives with 10,000 to 15,000 RPM capacity. To deliver adequate I/O throughput, Blackboard also recommends using smaller capacity drives so that there will be more spindles to reduce seek times and improve data transfer rates. The I/O performance requirements are not the same for the shared file system. Slower, condensed storage options using SATA or SAS disks in the range of 7,200 or 10,000 RPM are more than adequate. The configurations defined in this guide have been validated to support adequate I/O throughput for the user loads defined for each configuration. When sizing storage, be sure to do the following: 1. Determine how much storage your institution will need. 2. Determine how this storage can be spread across multiple trays and disks to optimize performance throughput. Remember that the shared file system can require upwards of 5100 times the amount of storage of the database. However, the database can use 520 times the number of I/O operations per second than the shared file system.
De s c r i pt i on Content Storage Tier St a nd a r d UNIX: Network Attached Storage Architecture or ZFS for Solaris only Windows: CIFS Architecture Up to 2.2 TB of usable storage 7.2k to 10k RPM SATA or SAS disks Database Storage Tier UNIX: Network Attached Storage Architecture, ISCSI, or FC-SAN Windows: ISCSI or FC-SAN Up to 500 GB of usable storage 10k to 15k RPM FC Adv a nc e d UNIX: Network Attached Storage Architecture or ZFS for Solaris only Windows: CIFS Architecture Up to 5 TB of usable storage 7.2k to 10k RPM SATA or SAS disks UNIX: Network Attached Storage Architecture, ISCSI, or FC-SAN Windows: ISCSI or FC-SAN Up to 1 TB of usable storage 10k to 15k RPM FC Com pl e x UNIX: Network Attached Storage Architecture or ZFS for Solaris only Windows: CIFS Architecture Up to 10 TB of usable storage 7.2k to 10k RPM SATA or SAS disks UNIX: Network Attached Storage Architecture, ISCSI, or FC-SAN Windows: ISCSI or FC-SAN Up to 2 TB of usable storage 10k to 15k RPM FC

Blackboard Learn Hardware Sizing Guide for Sun Deployments

2010 Blackboard Inc. Proprietary and Confidential

Page 13

Appendix A: Sizing Methodology

Appendix A: Sizing Methodology

Sizing is a three-step process consisting of two modeling exercises and performance testing. The modeling exercises are used to gather statistical evidence about how users interact with the system. The data generated from these exercises is subsequently used in a series of performance tests. Performance tests help quantify what the system will look like under hypothesized workloads and scenarios. The process begins with understanding how Blackboard clients have used the product in the past. This form of sampling is called behavior modeling. The objective of this form of sampling is to gather meaningful data representing the following: Who is using the system? What is being done? Where are they performing their operations? When are they performing their operations? How long are users spending performing their operations?

Predictive modeling is used for new performance testing new features. Little information can be collected about a feature that has not been built. Because of this, Blackboard hypothesizes user interactions with these new features. The data collected from both modeling exercises is then used for performance testing and benchmarking. Performance benchmarking is conducted by Blackboard with a selected partner of choice (such as Sun Microsystems, Dell, Microsoft, or Oracle) at the Blackboard Performance Laboratory using a combination of purchased and donated equipment from a partner. Performance testing and benchmark activities focus primarily on the performance (response times exhibited by users) and scalability of the system (utilization of system resources such as CPU, memory, and I/O). HP/Mercury LoadRunner is the simulation tool of choice for generating workload.

Blackboard Learn Hardware Sizing Guide for Sun Deployments

2010 Blackboard Inc. Proprietary and Confidential

Page 14