SQL Server Summit 2005

SQL Server 2005 What is new for the DBA?

Cheryl Stepney 3/31/05
Microsoft SQL Server Summit 2005 Minneapolis

What is new for the DBA?

High availability for enterprise applications Key security and performance features Focus on self manageability and optimization

Integration with Visual Studio and .NET Native XML technology Interoperability via Web Services

Real time decision making: reporting, data mining Rich client and SharePoint solutions Comprehensive ETL solution: up to 600% faster

Availability and security advancements

Database mirroring, online operations for increased availability Data security and privacy: encryption, enhanced auditing

Performance and scalability

Scales as the business grows: partitioning, snapshots

From mobile devices to 64-bit data centers

This sessio n

Continued focus on manageability

Integrated management toolset for relational and OLAP

Expanded self tuning

Topics for the DBA at the Summit

Security Service Broker
Developer Track

9:40 am Sogeti 2:15 pm Born 1:15 pm Magenic

Reporting Services
Developer Track

High Availability/Scalability 1:15 pm Unisys This Session 2:15 pm Microsoft

Agenda: SQL Server 2005 Internals and More

SQL Packaging Editions clarified Integrated Data Management Minimizing the Maintenance Window Deployment Tools Management and Performance Tools New Datatypes SQL and CLR Internals Notification Services Full-Text Engine

Introducing SQL Server 2005 Packaging

Note: All higher editions include same functionality as the edition below it.

New for SQL Server 2005

Express
Fastest way for developers to learn, build & deploy simple data driven applications

Workgroup
Easiest to use & most affordable database solution for smaller departments & growing businesses

Standard
Complete data management & analysis platform for medium businesses and large departments

Enterprise
Fully integrated data management and analysis platform for business critical enterprise applications

1 CPU 1 GB RAM 4GB DB Size Simple Management Tool Report Wizard & Report Controls Replication & SSB Client

2 CPU

4 CPU

3 GB RAM
Management Studio Import/Export Limited Replication Publishing Back-up Logshipping

Unlimited RAM (64-bit)

Database Mirroring OLAP Server Reporting Server New Integration Services Data Mining Full Replication & SSB Publishing

Unlimited Scale + Partitioning Adv. DB mirroring, Complete online & parallel operations, DB snapshot Advanced Analysis Tools including full OLAP & Data Mining Report Builder + Customized & High Scale Reporting Adv SSIS

Integration Services

Whats New for the DBA SQL Server 2005 Management Tools
Computer Manager Management Studio Profiler and Replay Database Tuning Advisor SQL Agent SQLCMD Express Manager

SQL Computer Manager

Combines the functionality of the following SQL Server 2000 tools:
Server Network Utility Client Network Utility Services Manager

Ability to set service properties for:

SQL Server SQL Server Agent Analysis Server Report Server Microsoft Search Microsoft Distributed Transaction Coordinator (MS DTC) Full-Text Search

New Management Tools

SQL Server Management Studio
New Rich GUI Integrated Authoring, Management Support all SQL Server components Managed code, new design patterns Improved Scalability, Scripting SMTP Support, Cluster Support, No Outlook require

Management Objects (SMO)

SQLiMail Replacement

SQL Server Management Studio

Management Studio
Query Analyzer

Enterprise Manager

OLAP Analysis Manager

Report Services

Notification Server SQL Mobile

Integration Services

SQL Server Management Studio

Enhanced Query Authoring
T-SQL Analysis Services MDX and XML/A

Dynamic Help SQLCMD Mode Improved display for XML results Integrated execution plan graphing tool for Enhanced Project Management Script Projects Enhanced Templates

Database Management Infrastructure

Monitoring & Performance Tuning
Dynamic Management Views Catalog Views Default Trace Event Infrastructure Enhancements

Problem Resolution

Dedicated Admin Connection (DAC) Enhanced Mini Dumps Plan Guides & Plan Forcing

Very Large Databases

Partitioning OnLine Index Operations Database Snapshots Page Checksums/Restore

Database Problem Resolution

Dedicated Admin Connection
Connect even to a hung server Uses reserved scheduler, port & resources

Enhanced Mini-Dumps
All interesting internal structures now in mini-dump No more full dumps!

Plan Stability
Plan Guides
controlling optimization with query hints
even when you cant change the application

Plan Forcing
Provide a full XML Showplan as a query hint

Database Tuning Advisor

Time Bound Tuning What-If Analysis

Manageability - Diagnostics
Dynamic Management Views
Server Transparency sys.memory_clerks how much memory is each component consuming ? sys.schedulers Is a scheduler hung ?

SQL Trace
Query data from live traces

SQL Profiler
Deadlock and ShowPlan Visualization

Dynamic Management Views & Functions

Server Level
dm_exec_*
Execution of user code and associated connections

Component Level
dm_repl_*
Replication

dm_os_*
Memory, locking & scheduling

dm_broker_*
SQL Service Broker

dm_fts_*
Full Text Search

dm_tran_*
Transactions & isolation

dm_qn_*
Query Notifications

dm_io_*
Input/Output on network and disks

dm_clr_*
Common Language Runtime

dm_db_*
Databases and database objects

Data Base Encryption

Certificates used by
SQL Server to create SSL connections SQL Service Broker

The certificates created by SQL Server comply with the X.509v3 certificate standard.

Asymmetric Keys
consists of a private key and the corresponding public key Each key can decrypt data encrypted by the other Asymmetric encryption and decryption are relatively resource-intensive, but they provide a higher level of security than symmetric encryption An asymmetric key can be used to encrypt a symmetric key for storage in a database

Symmetric Keys
A single key that is used for both encryption and decryption Encryption and decryption with a symmetric key is fast suitable for routine use with sensitive data in the database

SQL Server 2005 Administration Enhancements

Dynamic AWE Memory Management ATTACH_REBUILD_LOG Instant File Initialization Programming: New System Management Objects (SMO) API
extends and supersedes the Distributed Management Objects (DMO) object model provides classes for configuring and managing instances of the Database Engine

Programming: New SQL Server WMI Provider

The WMI Provider is an instance provider

Transact-SQL New Constructs

Improved Error Handling
use familiar TRY-CATCH logic to trap and handle error conditions

New Metadata Views

System catalog views provide information about the objects

Data Definition Language (DDL) Triggers

DDL triggers can be used to audit or regulate changes to a database schema

Event Notifications
Send information about Transact-SQL data definition language (DDL) statements and SQL Trace events to a service broker service Can be used to log and review changes or activity occurring on the database

Queue Processing Extensions

The INSERT, UPDATE, and DELETE now support an OUTPUT clause that return data based on the rows modified by the statement READPAST can now be specified on UPDATE and DELETE statements

Transact-SQL Enhancements
Bulk Operations on OPENROWSET
OPENROWSET now supports a BULK operation Controlled using new table hints & BULK_BATCHSIZE and FORMATFILE

TOP Enhancements
The TOP operator has been enhanced to take any numeric expression

Distributed Query
The EXECUTE statement now supports an AT LinkedServer clause that specifies executing a stored procedure on a linked server Distributed queries also support the new large object data types nvarchar(max), varchar(max), and varbinary(max)

TABLESAMPLE
New TABLESAMPLE clause limits the number of rows processed by a query Unlike TOP, which returns only the first rows from a result set

New CASCADE Integrity Constraints

REFERENCES clause now supports the SET NULL and SET DEFAULT cascading referential integrity actions

Overflow Data Can Exceed Page Size

Transact-SQL New Constructs

Unified Large Object Programming Model
Now supports a MAX length specification for the shorter data types When MAX is specified, the data types can store the same size strings as text, ntext, and image values (up to 2 GB),
but are processed in the same way as when they store shorter strings

Recursive Queries
Stores a Common Table Expressions (CTE) is a results set that is stored temporarily during the execution of a SELECT, INSERT, UPDATE, or DELETE statement

New APPLY Operator

Invokes a table-valued expression (function or subquery) for each row returned by an outer table reference

New PIVOT and UNPIVOT Operators

Rotate result sets, where columns become rows and rows become columns

Query Notifications
Allow an application to cache a result set and request the Database Engine to notify it if any of the underlying data is modified by another application

Index Enhancements
Disabling Indexes
Indexes can be disabled to aid in administration and diagnostics When rebuilding a non-clustered index
disabling the index before rebuilding the index significantly reduces the amount of disk space required

Reorganizing and Rebuilding Indexes

SQL Server 2005 introduces the ALTER INDEX statement that can reorganize or rebuild indexes, or set index options

Online index maintenance

Create, Rebuild, Reorganize, Drop, Add or drop constraint

Fully parallel
Indexes can be rebuilt online, pages containing LOB data can be compacted, and partitioned indexes can be reorganized or rebuilt on a single partition basis

Online/offline are both supported

New XML Support

New XML Data Type XQuery Support
based on working drafts of the W3C XQuery language definitions

XML Data Manipulation Language (XML DML)

The current W3C definition of XQuery does not include the ability to modify XML documents Microsoft has extended the XQuery implementation in SQL Server 2005 to include the ability to insert, update, and delete XML documents or fragments

Transact-SQL Enhancements: FOR XML Enhancements

can nest FOR XML statements to create a hierarchy of XML documents

Indexes on XML Type Columns Bulk Copy Format Files in XML

an XML schema that can be used to specify bulk copy format files as XML documents

New Utilities
New sqlcmd Utility
The sqlcmd utility replaces the osql and isql utilities sqlcmd uses the OLE DB API to communicate with an instance of the SQL Server Database Engine
while the earlier utilities used the ODBC or DB-Library APIs.

sqlcmd also introduces a richer set of commands that allow it to operate better in application scripts,
such as Visual Basic for Applications (VBA) scripts

New sqlwb Utility

The sqlwb utility can be used to launch the new SQL Server Management Studio from the command prompt

New Profiler90 Utility

The Profiler90 utility can be used to launch the SQL Profiler tool from the command prompt

New dta Utility

used to launch the Database Tuning Advisor from the command prompt

Notification Services
Rule Types The operation of the generator is controlled by the Transact-SQL rules defined for the application
Event chronicle rules store or update the history of events in chronicle tables defined by the application developer Subscription event rules generate notifications for event-triggered subscriptions Subscription scheduled rules generate notifications for scheduled subscriptions.

Full Text Search

Snapshot Isolation
Increased data availability for read applications
Allows non-blocking consistent reads in an OLTP environment Copy on write mechanism Writers dont block readers Readers dont block writers Read-mostly database with relatively few writes Reporting Migration from Oracle to SQL Server

Scenarios

Database Snapshots
Snapshot of a database at a point in time
Created instantly Read only

Does not require a complete copy of the data

Shares unchanged pages of the database Requires extra storage only for changed pages Uses a copy-on-write mechanism

Recover from User, Application or DBA error

Rewind database to Database Snapshot

Other replication changes

Stand alone replication monitor DDL on published objects supported seamlessly while replicating Merge performance and scalability improved 2-4x New security architecture New data types supported
XML, CLR, varchar/varbinary(max) etc.

New .Net API (RMO) for setup, admin and synchronization

Partitioning
Range Partitioning breaks a single object into multiple manageable pieces Partitioning is transparent to the application Partitioning applies to
Tables Indexes Indexed views

The row is the unit of partitioning

All partitions reside in a single database

Partition Manageability
Common operations
Split, merge, add, and drop partitions Bulk data loading, Index maintenance

Sliding Window Time

Obsolete Near-Line Read-Only Read-Mostly Hot

Archived Switch data instantly between

Table and partition of another table Two partitions of different tables

SQL Server Data Access API Layer

SQL Native Client replaces Microsoft Data Access Components (MDAC)
a stand alone data access API that is used for both OLE DB and ODBC It combines the SQL OLE DB Provider and the SQL ODBC Driver into one native dynamic link library (DLL) while also providing new functionality that is separate and distinct from MDAC create new applications to take advantage of new SQL Server features such as Multiple Active Result Sets (MARS), User-Defined Types (UDT), and XML data type support

SqlClient Namespace and SQL Native Client- Multiple Active Result Sets (MARS)
introduces the ability for multiple statements to return result sets at the same time on the same connection

SqlClient Namespace and SQL Native Client: New Data Types

A new xml data type that supports storing XML documents and fragments in databases ability of the char, nchar, varchar, nvarchar, binary, and varbinary data types to support strings longer than 8,000 bytes

Web Access: Native HTTP SOAP Access

Instances of the Database Engine can be configured with HTTP endpoints, or addresses to which SOAP requests can be sent from applications NO Need for IIS

HTTPS Synchronization
Merge Replication
Firewall

Subscriber

IIS

Publisher/Distributor

-InternetUrl https://myserver.mydomain.com

Replisapi.dll

No VPN or port required for SQL Server Eases deployment of mobile applications Off by default Secure by default Only authenticated users can send request to web server Requires HTTPS Wizard turns off anonymous access

Programmability Enhancements - .Net

Common Language Runtime Integration
The Microsoft .NET Framework Common Language Runtime V2.0 (CLR) is now hosted in the SQL Server Database Engine .NET languages such as C# and VB.NET User-defined types and aggregates can also be written in .NET languages

User-defined Aggregates
User-defined aggregate functions written in a .NET language extend the basic aggregate functions, such as SUM, MIN, and MAX An aggregate function is one that processes several data values and returns a single value

User-defined Data Types

CLR user-defined data types (UDTs) allow you to implement database data types in any of the languages supported by the .NET CLR CLR UDTs are integrated with the Database Engine type system and can be used in all the contexts where the built-in types are used

Stored Procedures, Triggers, and User-defined Functions

Database procedural objects can be written in .NET languages to incorporate more complex logic than is supported by the Transact-SQL language

Common Language Runtime Internals

SQL Engine
CLR goes through SQL OS for Memory Threads/fibers Synchronization SQL-CLR host co-ordinates Assembly Loading Security management App domains Escalation policy for exceptions

CLR
Hosting Layer

SQL OS Windows

CLR Memory Management

All memory allocation from CLR through SQL Server Most of CLR memory (GC Heap) comes from multi-page memory allocator (outside Buffer-pool or MemToLeave)
Max server memory setting does not cover CLR memory

On memory pressure
SQL informs CLR in order to initiate a GC

DMVs for monitoring CLR memory usage

Sys.dm_os_memory_clerks Sys.dm_os_memory_objects

Perf counters: # GCs, amount of memory allocated Memory clerks of interest: SQLCLR, SQLCLRASSEMBLY

CPU: threads / synchronization

All managed threads are mapped to tasks managed by the SQL Server scheduler
Co-operative scheduling Exception: GC threads

SQL Scheduler detects non-yielding tasks (managed or TSQL) and punishes them
Force the task to stop, put back in the queue and miss some turns

Callback to SQL Server on P/Invoke

(when calling native code) Switched to pre-emptive mode

CLR lock requests go through SQL Server lock manager

Unified deadlock detection between locks held by CLR and native SQL

Application Domain Management

App-domains
CLR mechanism for code isolation, code unloading

SQL Server uses two kinds of app-domains

DDL-time: temporary use during assembly DDL for verification Execution time: actual code execution

Granularity
One per assembly owner in each database All assemblies owned by same user are in same app domain

Cross-app domain mgd mgd calls not allowed DMVs of interest: Sys.dm_clr_appdomains

Summary: DBA Productivity

Reliable and secure Ease-of-use and TCO leader for database management Achieve high availability without added complexity, cost Scales from mobile device to the datacenter

SQL Server Summit 2005