RDBMS Industry and technology trends

Guy Harrison Chief Architect, Database Solutions

Copyright 2006 Quest Software

Agenda
Review of market share and competitive landscape Open source and disruptive technology Technical directions
Grids & clusters Self-managing databases Application development technologies

Industry trends:
Security and compliance Outsourcing Globalization of data and the database

Visions for the future of the DBMS

History of RDBMS competition

1990: Client server revolution
Sybase vs. Ingres vs. Oracle Multi threaded servers (SMP support) Stored procedures/Client server capabilities Informix vs. IBM vs. Oracle OODBMS vs. Object relational vs. relational Internet and Java compatibility Best of breed configurations with EMC, Solaris & Oracle Infinite scale-up anticipated Oracle price gouging creates some drift to IBM and SQL Server Battle not over capability but over cost SQL Server disrupts Oracle but constrained by Windows OS market Oracle disrupting IBM/MF and the high end via grid/RAC. IBM pursues enigmatic Information As A Service (IAAS) strategy

1996: Object Oriented Database distractions

2000: Internet gold rush

2005: ROI/ TCO/ Compliance

Market share and competitive landscape

2005 RDBMS market share

17%

45% Oracle Corp. IBM Microsoft Corp. 17% Others

26%

26%

21%
Sybase Inc.

Source: IDC 2006

0% 1%

NCR Teradata Progress Software Corp. SAS Institute M ySQL Ingres Corp. Fujitsu 12% 22%

13%

Revenues by platform
2005 RDBMS revenues by platform

Other

Revenue ($M)

Microsoft

IBM

Unix Windows Linux Mainframe Other

Oracle

0.00

500.00

1,000.00

1,500.00

2,000.00

2,500.00

3,000.00

Revenues by company size

80 70 60 SQL Server Oracle DB2

50
40 30

SQL Server Oracle

SQL Server Oracle

SQL Server

SQL Server

20
10

Oracle DB2

Oracle DB2

DB2

DB2

SQL Server Oracle

Less than $100 million

$100 million $500 million to $1 billion to to less than less than $10 billion $500 million $1 billion

More than $10 billion

Source: Forrester

DB2

Market growth by platform 2004 predictions

$4,500 $4,000 $3,500 $3,000
Millions

Windows Unix Linux Mainframe Other

$2,500 $2,000 $1,500 $1,000 $500 $0 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

Source: IDC

Market growth by platform 2006 predictions

$14,000

$12,000

$10,000 Mainframe $8,000 Unix Linux/other open source Windows 32 and 64 $6,000 Linux+Unix Other $4,000

$M
$2,000

$0 2005 2006 2007 2008 2009 2010

Market share conclusions and speculations

Oracle continues to dominate non-mainframe RDBMS landscape. However, most shops support >1 RDBMS type (Oracle/SQLServer) and DBA managers at least need to understand more than one technology Growth of Windows as a server platform ensures a healthy growth trend for SQL Server Similarly, Oracle stands to be the main beneficiary from the growth of Linux No compelling reason to believe that either vendor is going to dominate Server platform choices are key in RDBMS vendor decisions and viceversa. The above ignores the possibility of sudden disruption.

Disruptive technology
Disruptive innovation occurs when a technology or a technical approach emerges that offers a radically cheaper way of meeting a need.
Lower cost alternatives to low-end utilization Lower cost creates new consumers

Established players are motivated to move towards the more profitable high-end of the market.
Existing customers tend to demand features at the high end High end has higher profit margins High end is less effected by disruptive technology

But as both the established and disruptive technologies advance, the established technology overshoots while the disruptive technology gains the mainstream.
See The Innovators Dilemma, Clayton Christensen, HUP

Disruptive Technology
Functionality demanded at high end of market

Functionality

Sustaining Technology

Disruptive
Technology

Functionality demanded at low end of market

Time The Innovators Dilemma, Clayton Christensen, Harvard University Press

OSDBMS - MySQL
Advantages:
Huge install base Many mission-critical deployments (Sabre, Yahoo, NASA, etc) Critical part of the LAMP stack Well placed to leverage Linux server growth But dont forget WAMP Disruptive both as low-cost innovation and competing against nonconsumption Providing 90s style RDBMS for free (internal) or <10% of Oracle cost (commercial) Attempts to monetize the install base not yet successful MySQL users are completely satisfied by the free offering Many OSS companies are disturbingly reminiscent of Y2000 Dot.coms (millions of non-paying customers; VC funded; unproven business model) Commercial vendors have all read The Innovators Dilemma All have a free version for entry level use Oracle aggressively counter-disrupting via strategic acquisition Many competing demands on MySQL R&D Unlike Red Hat, MySQL dont get the software for free

Challenges:

Industry trends - Outsourcing

15% of companies report they plan to outsource DBA roles (Gartner 2004) b/c of:
Reduced cost Difficulty maintaining expertise in house

But BIG obstacles to widespread adoption:

High risk (cost of database failure > savings from outsourcing) Security implications Quality of service

Adoption is relatively narrow and shallow:

Minority of companies (but tend to be large) outsourcing only routine DBA activities

Industry trends - Security and compliance

A perfect storm accelerated interest in Database security:
9/11 The legislative response to Enron et al Sarbanes-Oxley, HIPPA, VISA High profile database break-ins, slammer worm, etc Outsourcing (challenge of external DBAs).

Industry responses:
Database Encryption Vulnerability Assessment Fine grained auditing Intrusion detection and prevention Separation of duties: Privileges to administer a database do not automatically imply privilege to view or alter data Oracle leading in inbuilt security features

Industry Trend Autonomic / self managing computing

All vendors especially Oracle are motivated to compete on ease of administration
Oracle ADDM (Automatic Database Diagnostic Manager) SQL Server DTA (Database Tuning Advisor) UDB Leo (Learning Optimizer)

Evolutionary changes for the DBA As legacy becomes automated, leading edge still requires intensive manual administration
Oracle 10g RAC, for instance

Overall effect of automation will be to slightly reduce DBA market growth and to shift demand to higher end skills

Technical trends grids / utility computing

Computing resources (IO, storage, memory, CPU) allocated on demand across the enterprise
Analogy to the electricity grid

Economic benefits will be irresistible once the technical challenges overcome. Grids have been viable only for CPU-bound applications until recently To create a database-enabled grid we need:
A way to shift CPU/memory (eg blades) efficiently between databases A way to shift IO & storage efficiently between databases

Grids, RAC and VMs

Oracle RAC is a step towards CPU on demand for databases
In some future release (possibly Oracle 11) blades will migrate between RAC clusters on demand

ASM provides a disk-grid solution

Although there are non-Oracle technologies that can achieve this in a heterogenous manner

RAC and ASM are not quite there yet

Nevertheless, RAC changes the economics of providing HA VLDB in a way that competitors cannot currently address

Virtualization offers an alternative utility computing vision

Resources can be shifted between VMs on demand However, not able to migrate VMs across hosts instantaneously (so scalability limited to size of single host) Databases currently perform poorly inside VMs Hypervisor technologies and Virtual-aware chipsets will possibly correct this

Technical trends grids

Blade Farm

Blade Rack

Blade Rack

Blade Rack

Blade Rack

Blade Rack

Blade Rack

Blade Rack

RAC Instance

RAC Instance

RAC Instance

RAC Instance

RAC Instance

RAC Instance RAC Instance

RAC Instance RAC Instance

RAC Instance RAC Instance

Disk Farm (ASM) Disk Farm (ASM?)

Disk Disk Disk Disk Disk Disk Disk

Disk Disk

Disk Disk Disk Disk

Disk Disk Disk Disk

Disk Disk

Technical trends Object-Relational Mapping

Increasing trend towards ORM
Dominant paradigm in Java already (Hibernate) Increasing in OSS (Ruby on Rails) Emerging in .NET Increasing prevalence in OSS and .NET environments (LINQ, Rails Active Record) Obfuscation of relational data in some cases (esp. JDO) Harder to perform Business Intelligence and Analytical processing Container generated SQL Harder to tune & debug though often simpler access paths Tendency towards over-simplified and/or unnormalized data models to suit programming models (ORMs tend to prefer single table accesses) RDBMS vendors introducing tune without change and secondary optimizers in response

Technology trends Application development

Multiple factors converging to reduce the significance of stored procedures in modern applications:
No standardization in stored procedure languages Use of SPs increase RDBMS vendor lock-in ORM ignores SPs Packaged applications want to be heterogenous (except for Oracle Fusion ) Middle tier a better choice for business logic

However:
We still see strong growth in the PL/SQL development tools market Oracle is one of the big two ERP vendors and they dont want to be heterogenous Middle tier/OO languages (and programmers!) not optimized for data access

Data volume growth

Drivers
Because we can Fine grained, real world data (RFID) Longer term retention policies (Sarb-Ox, etc) Complex and unstructured data (Flickr, YouTube)

Size of largest production database Source: Forrester (DBMS Survey 2006)

More than 5 TB 21%

Less than 100 GB 12% 100 GB to 499 GB 17%

Implications
Scale out architectures increasingly more attractive (unpredictable future demands) Demand for archiving solutions Suppresses disruptive effect of low end vendors
2 TB to 5 TB 19% 1 TB to less than 2 TB 7%

500 GB to less than 1 TB 24%

Growing demand for data search and linking

Islands of information need to be broken down Motivations for globalising and unifying data:
RFID/Supply chain Web services/mash-ups/co-operative e-commerce Expectations raised by internet content search

But:
No clear technical solution Significant societal issues in respect of security and privacy

Visions of the future of DBMS

Larry Ellison: A single, global, logical (Oracle) instance/cluster tied together with grids and data pump technology Both data and computing resources will be made available across the network on demand Data sharing through consolidation Web services standards bodies: All interactions will occur through well-defined Web Service interfaces utilizing specific specifications such as WS-transaction, WSsecurity, etc. RDBMS is a local persistence store only Open Source/Web 2.0 community: Same as above, but mash-ups not Standardsbased WS

Visions of the future of DBMS

Adam Bosworth (Google)
Something radically different is going to emerge. Orders of magnitude more data is going to be stored on the net in the near future and the expectation is going to be that we can find and possibly modify it from anywhere Formal, tightly coupled web services will give way to simple, sloppy (maybe RSS based) protocols Matching will result not from a worldwide standardization of data or a semantic web, but from stupid but powerful algorithms (similar to Google spell check) Centralized relational Databases of today are going to seem so twentieth century and one of those technologies that never got the internet

Visions of the future of DBMS

Michael Stonebraker (creator of Posgres/Ingres) et al
One Size Fits All (OSFA) RDBMS architecture cannot meet the needs of current and emerging demands: OLTP, stream processing (telco, web), OLAP/DW, Unstructured, mobile, embedded, multi-dimensional, etc Specialized databases can generated 10x performance improvements For instance, column based organization instead of row based The competing demand of data integration will probably preclude a re-fragmentation of data They suggest either: Hybrid system with various underlying storage engines (a la MySQL) Data federation A new from scratch DBMS system with relational features but also able to perform column based operations

Questions?