The Froogle DBA

MANAGI NG AND MONI TORI NG SQL SERVER AT SCALE WI TH

SQL SERVER FEATURES AND OPEN SOURCE TOOLS
J OHN GREESON
Introduction
This presentation is
About a framework for managing and monitoring SQL Server at scale
Based on an actual framework in use in an environment with
Over 70 SQL Server Instances
About 2000 SQL Server Databases
2.5 DBAs
Complementary to a series on my blog at moderndba.com
A high level overview of the framework architecture
Discussion on how the different components of the architecture work together
This presentation is not
A listing of the scripts needed to make the framework work
Scripts and links are available on moderndba.com
A presentation summary post will be made Monday (9/8)
Overview
About the Framework
Guiding Principles
Why Bother?
Just one Possible Solution
Components
Central Management Servers
Extended Properties
Management Data Warehouse
Data Collection
Policy Based Management
Enterprise Policy Framework
SQL Server Agent
Zenoss
Why the name Froogle?
The framework is Frugal
The framework was developed with the help of Internet Search
Sorry I dont use Bing
Once the framework is in place it is easy to take information learned
and target it to the appropriate instances in your environment
Framework Guiding Principles
Use SQL Server features before adding something else
Centralized Administration
Centralized Monitoring
Decentralized Execution
Self-Documentation
Why Bother?
There are a lot of great tools out there for SQL Server why spend the
time?
SQL Server is expensive
It is a great value, but a sizeable investment. Especially at scale
Getting the most out of SQL Server increases the value to your company
The more you rely on SQL Server to manage SQL Server, the more you
learn about well SQL Server
Easier to onboard a new team member since there are fewer
applications to learn
Just One Possible Solution
There are probably hundreds of different solutions that would provide
the same or similar benefits
No need to implement the whole framework
Take the pieces that fit in your environment and use them
Many of the components are useful even if you are already invested in
3
rd
party tools
Framework Components
Central Management Servers
Extended Properties
Management Data Warehouse
Data Collection
Policy Based Management
Enterprise Policy Framework
Zenoss
Central Management Servers (CMS)
An instance of SQL Server that is designated as a Central Management
Server maintains server groups that contain connection information for
one of more instances of SQL Server (SQL Server BOL)
Introduced in SQL Server 2008 so CMS has to be 2008 or greater
Can add instances at least as far back as SQL Server 2000
CMSs Role in the Framework
SQL Server Management Studio is the primary management tool used
in the framework
Central Management Server is the method all DBAs use to identify
instances
Central Management Server allows DBAs to run scripts against all or
some of the instances in their environment
Used for deploying scripts
Used for reporting
Central Management Server allows DBAs to verify policy using Policy
Based Management against all or some of the instances
Central Management Servers (CMS)
Central Management Servers (CMS)
Central Management Servers (CMS)
Central Management Servers (CMS)
Extended Properties (EP)
A user defined name and value (sql_variant) which can be associated
with a database or object inside of a database
Can be used to store information about a database, data file, table,
columns, constraint,
Can be used to make your environment self-documenting
They move with the database
Maintenance of documentation is one of the most difficult things to
keep up with in a large rapidly changing environment
EPs Role in the Framework
Use Extended Properties to document
things that are important to know about
your database
What application(s) rely on this database?
Who is the contact for this database?
Where is the documentation located for this
database/application?
When was the last time this information
was validated?
Some/All of this information can be used
to dynamically create groups in Central
Management Server
Management Data Warehouse (MDW)
A relational database that contains the data that is collected from a
server that is a data collection target (SQL Server BOL)
SQL Server 2008 or later required to serve as MDW
Relies on Data Collection running on target servers to populate the
MDW
Data Collection only available for SQL Server 2008 or newer
Possible to gather on 2005?
Provides canned reports for information collected with provided Data
Collectors
Focus of reporting is on performance
Data Collection
Either runs constantly and cached, or runs on a schedule
Cached data uploaded to MDW on a schedule
May require the use of agent proxies depending on your environment
Different Collection Types
Generic T-SQL Query Collector Type
Generic SQL Trace Collector Type
Performance Counters Collector Type
Query Activity Collector Type
Definition is in XML format
Format is specific to the Collection Type being used
Data Collection
Data Collection
Management Data Warehouse (MDW)
Management Data Warehouse (MDW)
Management Data Warehouse (MDW)
Management Data Warehouse (MDW)
Management Data Warehouse (MDW)
Management Data Warehouse (MDW)
Management Data Warehouse (MDW)
Management Data Warehouse (MDW)
Management Data Warehouse (MDW)
Management Data Warehouse (MDW)
Management Data Warehouse (MDW)
Management Data Warehouse (MDW)
MDWs Role in the Framework
Used to troubleshoot performance issues in the environment
Central Repository used to lookup information needed when an
instance goes offline
Use custom collection sets to gather this information
MDW Scaling Lessons
Disk Performance will be crucial
Stagger your collections across your environment
Only keep what you absolutely need
Dont use failed job notification
Check for missing collection sets in the MDW on a periodic basis
CMS, EP, and MDW Together
Critical component of the Froogle DBA Framework
The Magic
Brings the three components previously discussed together and adds
additional benefit
By ensuring CMS groups are kept up-to-date we reduce the
maintenance of supporting CMS
CMS, EP, and MDW Together
Getting EP into MDW
Get Extended Properties into MDW using
data collection
Collect them at least daily (need custom
schedule for this)
No real value in history unless you need to
know how a value changes over time
Make sure that you gather a row with null
for a database if no Extended Properties are
set
Loading CMS Groups Based on EP
Script available at moderndba.comwhich can be used to automatically
create/recreate groups based on custom_snapshots.ExtendedProperties
table in MDW
Script uses a group inside of CMS you designate as representing your
current instances you manage (Instance List)
It then uses the Extended Properties most recently gathered in MDW for
each instance on the list to dynamically generate the other groups as
appropriate
Just schedule script to run after your last Extended Property data collection
runs
MONITOR YOUR GROUP IDS
server_group_id is stored as an INT
You may need to reseed identity at some point
Identifying Databases Missing EP
Keep a table of expected Extended Properties
Check for missing Extended Properties by querying
custom_snapshots.ExtendedProperties table as needed
Policy Based Management
System for managing one of more instance of SQL Server to verify that
components and setting meet or enforce standards
Allows you to define the desired state of a system and check it against
the actual state
Evaluation Mode
On demand
On change: prevent
On change: log only
On schedule
Policy Based Management
Terms
Target Entities in SQL Server to be checked by the policy (instance,
database, table, index, etc.)
Facet set of logical properties that model the behavior or characteristics for
certain types of managed targets (hardcoded by the maker of the facet)
Condition Boolean expression that indicated a set of allowed states by a
facet
Policy Condition and expected behavior
Policy Category Way to group similar policies
Enterprise Policy Management Framework
PowerShell script which runs policy group(s) against CMS group(s)
Results are stored in a database
Reports allow you to analyze your environment
Available on CodePlex
Enterprise Policy Management Framework
Enterprise Policy Management Framework
Zenoss
Open source and commercial monitoring options
Agentless monitoring
Written in Python
Only runs on Linux/Unix platforms
MySQL database only
Wraps modelers and monitors into packs called ZenPacks
Modelers identify services running on a system and add appropriate
monitors
Multiple Zenpacks available to monitor SQL Server
Microsoft SQL Server Zenpack WMI monitoring
Microsoft Windows Zenpack - WinRM
Zenosss Role in the Framework
Monitor Server, SQL Server Service, and SQL Server Agent
Monitor Disk space and growth
Some SQL Server specific performance counters of limited interest
Zenoss
Zenoss
Zenoss
Zenoss
Overview
About the Framework
Guiding Principles
Why Bother?
Just one Possible Solution
Components
Central Management Servers
Extended Properties
Management Data Warehouse
Data Collection
Policy Based Management
Enterprise Policy Framework
SQL Server Agent
Zenoss
Questions?
CMS Stored Procedures and Views
Views
msdb.dbo.sysmanagement_shared_server_groups
msdb.dbo.sysmanagement_shared_registered_servers
Stored Procedures
sp_sysmanagement_add_shared_registered_server
sp_sysmanagement_add_shared_server_group
sp_sysmanagement_delete_shared_registered_server
sp_sysmanagement_delete_shared_server_group
sp_sysmanagement_move_shared_registered_server to new group
sp_sysmanagement_move_shared_server_group to new parent group
sp_sysmanagement_rename_shared_registered_server only changes name
sp_sysmanagement_rename_shared_server_group only changes name
sp_sysmanagement_update_shared_registered_server changes connection, etc.
sp_sysmanagement_update_shared_server_group changes description
EP Stored Procedures and Function
Stored Procedures (in each database)
sp_addextendedpropety
Adds a new extended property to a database object
sp_updateextendedproperty
Updates the value of an existing extended property
sp_dropextendedproperty
Drops an existing extended property
Table Valued Function
fn_listextendedproperty (..)
Returns a table variable containing all the properties found that meet the criterion passed in the
parameters
Catalogue View
sys.extended_properties
SQL Server Agent
Windows service that executes scheduled administrative tasks (SQL
Server BOL)
Main components
Jobs
Alerts
Operators
Agents Role in the Framework
Jobs
Run data collection process
Rebuilds CMS groups based on EP
Runs Enterprise Policy Framework check(s)
Alerts