Course : Database Administration

Effective Period : September 2015

OPTIMIZING SQL SERVER

2014

Session 5
 Learning Objectives
To explain the strategy to manage and monitor the
Database Server

To apply SQL Server Security Features

To apply Strategies for Performance Tuning

 Contents
 Application Optimization
 The Silent Killer: I/O Problems
 SQL Server Internals and File Allocations

 Table and Index Partitioning

 Data Compression
 Understanding SQL Server and CPUs
 Memory Considerations and Enhancements
 Resource Governor
 Acknowledgement

These slides have been adapted from:

Adam Jorgensen, Bradley Ball, Steven Wort,

Ross LoForte, Brian Knight - Professional
Microsoft SQL Server 2014 Administration 1st
ed.
ISBN: 978-1-118-85913-1

Chapter 11
 Application Optimization
The goal of performance tuning SQL Server 2014 is to minimize the
response time for each SQL statement and increase system throughput

Maximize the scalability of the entire database server by reducing network-

traffic latency, as well as optimizing disk I/O throughput and CPU processing
time.

1. Defining a Workload

In most cases, it is not possible to test SQL Server to scale with the actual
demands on the application while in production
We must set up a test environment that best matches the production system,
and then use a load generator such as Quest Benchmark Factory, Idera
SQLscaler, or the built-in Distributed Replay Adapter to simulate the database
workload
 The Silent Killer: I/O Problems
1. SQL Server I/O Process Model
 Windows Server I/O Manager handles all I/O operations and fulfills all I/O
(read or write) requests by means of scatter-gather or asynchronous
methods
 The SQL Server storage engine manages when disk I/O operations are
performed, how they are performed, and the number of operations that
are performed.
 The job of the database storage engine is to manage or mitigate as much of
the cost of these I/O operations as possible
 In-memory OLTP represents a major advance for SQL Server in the effort to
optimize servers, as well as the code they run, and alleviate I/O contention
2. Database File Placement
 SQL Server stores its database on the operating system files—that is, physical
disks or Logical Unit Numbers (LUNs) surfaced from a disk array

 Database file location is critical to the I/O performance of the Database

Management System (DBMS).

 Using a fast and dedicated I/O subsystem for database files enables it to
perform most efficiently
 To maximize the performance gain, make sure you place the individual data
files and the log files all on separate physical LUNs
 We can place reference-archived data or data that is rarely updated in a
read-only filegroup
3. tempdb Considerations

 tempdb performance has a rather large impact on system performance

because it is the most dynamic database on the system, and needs to be
the quickest
 tempdb is used to store user objects and internal objects and has two
version stores
A version store is a collection of data pages that hold data rows required to
support particular features that use row versioning
two version stores are as follows:
a) Row versions generated by data modification transactions in tempdb that
use snapshot or read-committed row versioning isolation levels
b) Row versions in tempdb generated by data-modification transactions for
features such as online index operations, Multiple Active Result Sets
(MARS), and AFTER triggers
 tempdb has added support for the following large set of features that create
user and internal objects or version stores:

 Placing the tempdb database on a dedicated and extremely fast I/O

subsystem can ensure good performance.

 At a minimum, perform the following

a) Take into consideration the size of your existing tempdb.
b) Monitor tempdb while running the processes known to affect tempdb
the most
c) Rebuild the index of your largest table online while monitoring tempdb.

 The output on your system depends on your database setup and usage. The
output of this query might appear as follows:
We must perform the following actions when configuring tempdb:
 Pre-allocate space for tempdb files based on the results of your testing,
but to prevent SQL Server from stopping.
 Per SQL Server instance, as a rule of thumb, create one tempdb data file
per CPU or processor core, all equal in size up to eight data files.
 Ensure that tempdb is in simple recovery mode, which enables space
recovery.
 Place tempdb on a fast and dedicated I/O subsystem.
 Use instant database file initialization. See Chapter 10 for more
information on setting up instant database file initialization
  Windows System Performance Monitor:

 Another great tool is the sys.dm_db_task_space_usage Dynamic Management

View (DMV), which provides insight into tempdb's space consumption on a per-
task basis
 SQL Server Internals and File Allocations
 The reason for having multiple tempdbs within any user SQL Server
database is because of the way that data is allocated within SQL Server

Pages—Data within SQL Server is stored on 8KB pages

Extents—An extent is a grouping of eight contiguous pages within SQL Server
(Mixed extents and Dedicated/uniform extents

Three most pertinent allocation bitmaps when dealing with with databases
and tempdb
1. Page Free Space (PFS)—This tracks the allocated pages and the page status
2. Global Allocation Map (GAM)—This tracks dedicated extents with 1 bit per
extent and Shared Global Allocation Map (SGAM)—This tracks mixed extents
with 1 bit per extent
3. Index Allocation Map (IAM)—This page tracks all other pages that are
allocated to one particular object
 Two processes based on algorithms used to allocate space within all SQL
Server data files
1. Proportional fill determines how much data is written to each of the files
in a multi-file filegroup based on the proportion of free space within each
file
2. Round Robin is the pattern in which a new filegroup is selected in a multi-
file filegroup once that file has met its proportional fill limit before a
growth operation is required
 It is important to remember that the files maintain an even size in order to
keep an even distribution of data to each file in a multi-file filegroup
 Table and Index Partitioning
 Partitioning is the breaking up of a large object (such as a table) into
smaller, manageable pieces. A row is the unit on which partitioning is
based.

1. Why Consider Partitioning?

When configuring partitioning we should plan out our partitioning strategy for
the data we want to keep on hand, as well as a potential archiving strategy
Following is a high-level process for partitioning:

1. Create a partition function to define a data-placement strategy.

2. Create filegroups to support the partition function.
3. Create a partition scheme to define the physical data distribution strategy
(map the function data to filegroups).
4. Create a table or index on the partition function.
5. Examine query plans to see partitions accessed as verification that only a
subset of data was returned.
Few processes that may be affected by partitioning your environment:

1. Database backup and restore strategy (support for partial database

availability)
2. Index maintenance strategy (rebuild), including index views
3. Data management strategy (large insert or table truncates)
4. End-user database workload
5. Concurrency: Parallel partition query processing and Table partition lock
and
6. Enhanced distribution or isolated database workloads using filegroups

2. Creating a Partition Function

 A partition function is your primary data-partitioning strategy
 When creating a partition function, the first order of business is to
determine your partitioning strategy
 After you select your strategy, you must create a partitioning function that
matches that strategy.
Under a sliding-window scenario, every month (or quarter), a new partition
would be introduced to the environment as a retainer for the current (Hot)
data for the tickets and activities tables

CreatePartitionFunction
CREATE PARTITION FUNCTION
PFL_Years (datetime)
AS RANGE RIGHT FOR VALUES
( '20050101 00:00:00.000', '20070101 00:00:00.000',
'20090101 00:00:00.000', '20110101 00:00:00.000',
'20120101 00:00:00.000')
3. Creating Filegroups
 We should create filegroups to support the strategy set by the partition
function
 user objects should be created and mapped to a filegroup outside of the
primary filegroup, leaving the primary filegroup for system objects

4 Creating a Partition Scheme

A partition scheme is what maps database objects such as a table to a physical
entity such as a filegroup, and then to a file

CREATE PARTITION SCHEME CYScheme

AS
PARTITION PFL_Years
TO ([CY04], [CY06], [CY08], [CY10], [CY11], [CY12])
CREATE PARTITION SCHEME CYScheme2
AS
PARTITION PFL_Years
TO ([Default], [Default], [Default], [Default], [Default], [Default])

 Partitioning enables us to delete or insert gigabytes of data on a 500+GB

partitioned table with a simple metadata switch and will only affect around
125 allocation bitmaps,
 reducing the transaction from 500GB to less than 1MB, provided that the
delete or insert is based on the partitioned column
 Data Compression
You can use data compression on the following database objects:

1. Tables (but not system tables)

2. Clustered indexes
3. Non-clustered indexes
4. Index views
5. Partitioned tables and indexes where each partition can have a different
compression setting

1. Row Compression
Row compression affects data at the row level and changes the internal
structure completely
uncompressed data record compared to the
structure of a compressed data record.
2. Page Compression

Page compression includes row compression and then implements two other
compression operations:
 Prefix compression—For each page and each column, a prefix value is
identified that can be used to reduce the storage requirements. values are
replaced by a reference to the prefix stored in the CI.
 Dictionary compression—This involves searches for repeated values
anywhere in the page, which are replaced by a reference to the CI.

To create a new compressed table with page compression, use the following
commands:
To change the compression setting of a table, use the ALTER TABLE command.

an example of applying
compression to a
partitioned table and index:
 Table partition operations on a compression partition table have the following
behaviors:

• Splitting a partition—Both partitions inherit the original partition setting.

• Merging partitions—The resultant partition inherits the compression setting
of the destination partition.
• Switching partitions—The compression setting of the partition and the table
to switch must match.
• Dropping a partitioned clustered index—The table retains the compression
setting.

3. Estimating Space Savings

Prior to enabling data compression, you can evaluate the estimated
compression cost savings.
Following is the syntax of the sp_estimate_data_compression_savings
stored procedure:
4. Monitoring Data Compression

 For monitoring data compression at the SQL Server 2014 instance level, two
counters are available in the SQL Server:Access Method object that is found in
Windows Performance Monitor:
• Page compression attempts/sec counts the number of page compression
attempts per second.
• Pages compressed/sec counts the number of pages compressed per
second

 The sys.dm_db_index_operational_stats Dynamic Management Function

(DMF) includes the page_compression_attempt_count and
page_compression_success_count columns, which are used to obtain page
compression statistics for individual partitions
5. Data Compression Considerations
When deciding whether to use data compression, keep the following items in
mind:
 Data compression is available with SQL Server 2014 Enterprise and
Developer Editions only.
 Enabling and disabling table or clustered index compression can rebuild all
nonclustered indexes.
 Data compression cannot be used with sparse columns.
 LOBs that are out-of-row are not compressed.
 Nonleaf pages in indexes are compressed using only row compression.
 Nonclustered indexes do not inherit the compression setting of the table.
 When you drop a clustered index, the table retains those compression
settings.
 Unless specified, creating a clustered index inherits the compression setting
of the table.
When deciding what to compress, do the following:

 Look for the largest tables in your database.

 Look at the type of allocation units used for each table. Remember,
IN_ROW_Data compresses, LOB and ROW_OVERFLOW pages do not.
 Look at the usage patterns of your tables using
sys.dm_db_index_operational_stats. Look for high SEEK and SCAN
utilization. High update utilization indicates that you may want to skip
compressing the object if you are CPU-bound.
 Test and monitor the execution plans of queries and their performance
before and after to validate that compression has helped your performance.
  Understanding SQL Server and CPUs
 SQL Server 2014 has a special algorithm for scheduling user processes
using the SQLOS

There are two types of schedulers within SQL Server:

• Hidden schedulers are used by internal SQL Server processes.
• Visible schedulers are available for user queries and user processes.
 Windows utilizes preemptive scheduling, and the SQLOS uses cooperative
(or non-preemptive) scheduling.

1. NUMA and Hot Add CPUs

 For years, the standard hardware configuration has been Non-Uniform
Memory Access (NUMA). Fortunately, SQL Server has been NUMA-aware
since SQL Server 2005
 With Windows 2012 and SQL Server 2014, SQL Server supports hot-add
CPU,
With Windows 2012 and SQL Server 2014, SQL Server supports hot-add
CPU

Following are the system requirements to support hot-add CPU:

• Hardware support for hot-add CPU.

• Supported by the 64-bit edition of Windows Server 2012.
• Supported by SQL Server 2014 Enterprise Edition.
• SQL Server 2014 must not be configured for soft NUMA.
2. Cache Coherency
 For reasons of data integrity, only one processor can update any piece of data
at a time. Other processors that have copies in their caches can have their
local copy “invalidated” and thus must be reloaded. This mechanism is
referred to as cache coherency
 We can use the affinity mask configuration setting to assign a subset of the
available processors to the SQL Server process

 When the Windows kernel transfers control of the processor from an

executing process to another that is ready to run, this is referred to as a
context switch.
 Context switching is expensive because of the associated housekeeping
required to move from one running thread to another
 Housekeeping refers to the maintenance of keeping the context or the set
of processor register values and other data that describes the process state
 In addition to this user mode time, context switching can take place in the
Windows operating system (OS) for privileged mode time.
The total processor time is equal to the privileged mode time plus the
user mode time.
 Privileged mode is a processing mode designed for operating system
components and hardware-manipulating drivers. It enables direct access to
hardware and all memory.

 In most environments, context switching should be less than 1,000 per

second per processor. The SQL Server lightweight pooling option
provides relief for this by enabling tasks to use NT “fibers,” rather than
threads, as workers.
 A fiber is an executable unit that is lighter than a thread, and operates in
the context of user mode.
3. Affinity Mask
 The affinity mask configuration option restricts a SQL Server instance to
running on a subset of the processors

 In a server consolidation or
multiple-instance
environment, for more
predictable performance,
SQL Server may be
configured to bind CPUs to
specific instances, reducing
the chance for cross-instance
contention
4. Parallelism
Max Degree of Parallelism (MAXDOP)
 By default, the MAXDOP value is set to 0, which enables SQL Server to
consider all processors when creating an execution plan.
 In most systems, a MAXDOP setting equivalent to the number of cores in
one NUMA node is recommended

Cost Threshold for Parallelism

 The Cost Threshold for Parallelism is exactly what it sounds like

 When SQL Server looks at the overall cost of a serial query plan,
depending on the level of optimization, the SQL Server optimizer can also
generate a parallel plan to compare which is the “cheaper” and thus
easiest to execute
the results of this query. We see that two stored procedures each have a
cost of over 5, but under 15.
the Cost Threshold for Parallelism
 Memory Considerations and
Enhancements
There are a few memory considerations and enhancements in SQL Server
2014

1. Buffer Pool Extensions

 With Buffer Pool Extensions (BPEs), we now have the capability to extend
the buffer pool.
 The main system memory will be considered the L1 (Level 1) buffer pool,
and the BPE will be considered the L2 (Level 2) buffer pool.
 Using the following script, we can create a BPE and place it on a physical
disk
 The second query can be used to query what is being stored in your buffer
pool by database, and whether it is stored in the regular buffer pool or in a
BPE.
 The multilevel caching hierarchy created by the BPE occurs when clean,
non-dirty pages, are written to the L2 BPE cache
 Dirty pages will remain written to the L1 buffer pool

An example of how data flows throughout SQL Server with a BPE

DMVs and Extended Events for Tracking and Monitoring BPEs
2. Tuning SQL Server Memory
A number of performance counters are available from the System Performance
Monitor.
With Windows Server 2012, SQL Server 2014 has support for hot-add memory,
a manager framework, and other enhancements.

SQLOS components that

perform thread scheduling
and synchronization,
perform SQL Server memory
management, provide
exception handling, and host
the Common Language
Runtime (CLR)
The architecture also enables SQL Server 2014 to better support both Cache
Coherent Non-Uniform Memory Access (CC-NUMA).
The architecture uses the concept of a memory node, which is one hierarchy
between memory and CPUs. There is a memory node for each set of CPUs to
localize memory and its content to these CPUs.

A CPU node is also a hierarchical structure designed to provide logical

grouping for CPUs

A memory-tracking enhancement called the Memory Broker, which enables

the tracking of OS-wide memory events
3. Data Locality
 Data locality is the concept of having all relevant data available to the
processor on its local NUMA node while it's processing a request
 Near memory and far memory. Near memory is the preferred method
because it is accessed by a processor on the same NUMA node
 Far memory is expensive because the request must leave the NUMA node
and traverse the system interconnect crossbar to get the NUMA node that
holds the required information in its memory
4. Max Server Memory
When “Maximum server memory” is kept at the default dynamic setting, SQL
Server acquires and frees memory in response to internal and external pressure
SQL Server uses all available memory if left unchecked, so the “Maximum server
memory” setting is strongly recommended

Configuration Guidelines for Maximum Server Memory

Example of setting Maximum Server memory on SQL Server 2014.
 Resource Governor
The main elements of Resource Governor include resource pools, workload
groups, and classification support

1. Resource Pools
The resource pools are the physical resources of the SQL Server. During SQL
Server installation, two pools are created: internal and default
We can create user-defined pools using the CREATE RESOURCE POOL DDL
statement, or by using SQL Server Management Studi
2. Workload Groups
A workload group is a container for similar sessions according to the defined
classification rules, and applies the policy to each session of the group
We can create the user-defined workload group by using the CREATE
WORKLOAD GROUP command, modify it by using the ALTER WORKLOAD
GROUP command, and drop it by using the DROP WORKLOAD GROUP
command.
You can apply several configuration settings to a workload group:
 Maximum memory allocation per request
 Maximum CPU time per request
 Maximum IOPS per request/per second
 Minimum IOPS per request/per second
 Resource timeout per request
 Relative importance setting per request
 Workgroup limit per number of requests
 Maximum degree of parallelism
 Specific resource pool
3. Classification
 Only one user-defined classification function can be designated as a
classifier; and after registering it, it takes effect after an ALTER
RESOURCE GOVERNOR RECONFIGURE command is executed.

 In the absence of a user-defined

classification function, it can use
the default workload group

the components of Resource

Governor and their
relationships.
4. Using Resource Governor from SQL Server 2014 Management Studio
 From inside SQL Server 2014 Management Studio in Object Explorer,
Resource Governor is in the Management node
 From the Resource Governor Properties dialog box, we can see that two
resource pools, Internal and Default, with two corresponding workload
groups, are visible
To create the three user-defined resource pools, workload groups, and
the user-defined classification function, and after enabling the Resource
Governor, the properties should match what is shown below
5. Monitoring Resource Governor

 SQL Server 2014 includes two performance objects to collect workload

group and pool statistics for each SQL Server instance:
- SQLServer—Resource Pool Stats for resource-specific statistics
- SQLServer—Workload Group Stats for workload-specific statistics

Resource Governor DMVs

 References
• Adam Jorgensen, Bradley Ball, Steven Wort, Ross
LoForte, Brian Knight - Professional Microsoft SQL
Server 2014 Administration 1st ed.
ISBN: 978-1-118-85913-1