Previous Next

Previous Next
Download Download
Register Register
Subscribe Subscribe
Previous Next
Previous Next
MARCH 2014

2014 State
Of Database
Tech
DOWNLOAD PDF
Conventional databases from Microsoft, Oracle, and IBM
still dominate the enterprise. What will it take for NoSQL,
DBaaS, and distributed systems to break through? >>
N
owhere in technology is the
chasm between state of the art
and actual use so wide as with
databases. Tech journalists and
industry pundits feed that gap,
expending barrels of virtual ink on innova-
tions including Hadoop, in-memory systems,
and Amazons Redshift. At publication time, a
search for Hadoop on Google News yielded
just shy of 15,000 results versus 13,400 for
Oracle database.
Yet despite its online popularity, Hadoop
is in production or pilot by only 13% of the
956 respondents to our 2014 State of Da-
tabase Technology Survey, all of them in-
volved with their organizations database
strategies. Compare that with Microsoft SQL
Server (75%) or Oracle (47%). Just 5% use
MongoDB, 3% have bought SAP Hana, and
1% use Vertica to name three databases
getting big play in the press. Even FileMaker
beats startup darlings Cassandra, Riak, and
MariaDB.
And were not talking about small shops
here: 48% of respondents hail from organiza-
tions with more than 1,000 employees (23%
have more than 10,000), and 37% have at
least $100 million in annual revenue.
Todays database landscape isnt just static.
Its positively retro. Remember 2004? Face-
book had just launched, the iPad wasnt even
a twinkle in Steve Jobss eye, and Gartners da-
tabase market share report put IBM (34.1%),
Oracle (33.7%), and Microsoft (20%) in the
top spots. In our survey, Microsoft, Oracle,
and IBM still hold the top spots; we do add
MySQL, but thats about it for innovation. The
top six databases in use are all relational; you
have to go down to the 10th most widely
used database in our survey to find NoSQL
(MongoDB, at 5%).
And those relational databases from Mi-
crosoft, Oracle, and IBM? Theyre essentially
just updated versions of the companies 2004
offerings.
Oracle gets the lions share of respondents
database spending, with 46% of those using
Oracle devoting more than half of their da-
tabase budgets to it, followed by SQL Server
(34% spend more than half ), Access (25%),
and DB2 (24%). Yet despite stubbornly high
pricing and concerns about using older tech-
nology as data variety and volume expand,
IBM, Microsoft, and Oracle neednt worry
about losing customers fast. Among respon-
dents, 56% of SQL Server shops, 44% of both
Oracle and MySQL users, and 29% of DB2
March 2014 2 informationweek.com
Previous Next
Previous Next
Download Download
Register Register
Subscribe Subscribe
Previous Next
Previous Next
By Joe Masters Emison
DOWNLOAD PDF
2014 State Of Database Tech
Conventional databases from Microsoft, Oracle, and IBM still dominate the enterprise.
What will it take for NoSQL, DBaaS, and distributed systems to break through?
Previous Next
Previous Next
Download Download
Register Register
Subscribe Subscribe
Previous Next
Previous Next
Next Wave Of Business Tech
Engage with Oracle president
Mark Hurd, Box founder Aaron
Levie, UPMC CIO Dan Draw-
baugh, GE Power CIO Jim Fowler,
former Netflix cloud architect
Adrian Cockcroft, and other lead-
ers of the Digital Business move-
ment at the InformationWeek
Conference and Elite 100 Awards
Ceremony, to be held in conjunc-
tion with Interop in Las Vegas,
March 31 to April 1. Click here for
full agenda.
March 2014 3 informationweek.com
shops plan to increase their use of those
databases.
Wheres the disruption?
IT Stronghold
Cloud computing and shadow IT have
shaken up many enterprise tech fief-
doms. Not so in databases. The difference
is that cloud and shadow IT are driven
by employees and developers getting
work done without ITs involvement. Da-
tabases, however, are different for three
big reasons. First, the constituency that
most cares about databases, database ad-
ministrators, is usually part of IT. Second,
enterprise application developers dont
much care which database sits under the
abstraction layer, at least not enough to
go to battle. IT is charged with making
code live, so again, the database decision
remains with IT.
Finally, database hardware choice is
insanely important. When youre using
one server, or a small set of colocated
systems, to handle all of an applications
writes, the hardwares speed and uptime
are crucial. Very few respondents produc-
tion databases even run on virtual ma-
chines forget about hybrid or public
cloud. And thus, the status quo sticks.
So do you think you can hold pat for an-
other 10 years?
Maybe. But its no mystery why venture
capitalists, cloud providers, and startups
are bullish for NoSQL, and companies like
ConAgra are ponying up for expensive in-
memory technology for a reason. While
conventional relational databases can,
theoretically, serve any data store use,
theyre not always the best choice for to-
days global, varied, and mobilized work-
loads. And thats not just because the
licensing and hardware demands of con-
ventional relational databases are limiting
and expensive. More often, alternatives
make sense because modern applications
have requirements that simply didnt exist
10 years ago.
For example, Forbes estimates that for
every minute Amazon.com is down, the
company loses $66,240. We live in an al-
ways-on world, where nightly scheduled
maintenance windows are verboten. A
high volume of writes, broad geographic
distribution, and frequent upgrades
are facts of life. New databases such as
Mongo and Riak were built from the
ground up to run on a distributed archi-
[2014 STATE OF DATABASE TECH]
Previous Next
Previous Next
Download Download
Register Register
Subscribe Subscribe
Previous Next
Previous Next
Previous Next
Previous Next
Download Download
Register Register
Subscribe Subscribe
Previous Next
Previous Next
Education And Networking
Learn how cloud computing,
software-defined networking,
virtualization, wireless, and other
key technologies work together
to drive business at Interop Las
Vegas. It happens March 31
to April 4.
March 2014 4 informationweek.com
tecture and address all of these factors.
Setting up a distributed database to pro-
vide high availability is simple in concept. In
practice, however, there are a lot of choices
to be made. You need significantly different
setups based on factors like the amount and
type of data youre writing, how consistent
you need the data to be, and the volume of
writes. (We explore these and other consid-
erations here.)
Theres one core architectural difference
between conventional RDBMSes and those
created to handle distributed architectures.
The latter group doesnt require a single
master node that handles all writes. They
accept writes across multiple nodes. While
there are multiple-master options for RDBM-
Ses Oracle RAC, MySQL Cluster, SQL Serv-
ers Peer-to-Peer Transactional Replication
all have limitations because their core
architectures were designed with a single
node in mind. Oracle RAC requires machines
connected on a private network and shared
disk. MySQL Cluster doesnt perform well for
large transactions or joins. And in Peer-to-
Peer Transactional Replication, you can get
conflicts if a row is updated at the same time
on different servers.
Newer databases arent just structured
differently. They deviate from the licens-
ing models IT has used with conventional
RDBMSes. And thats a good thing given that
only MySQL and IBM DB2 surpass 50% of re-
spondents saying theyre somewhat or very
satisfied with their vendors licensing deals.
Surprisingly, many respondents also still pur-
chase CPU/core licenses for Microsoft SQL
Server, Oracle, and IBM databases. Worst case,
these shops may be forced to compromise on
architectural decisions simply to afford a par-
ticular database.
From a hardware perspective, while bare
metal is still the most popular way to run IBM
DB2 and Oracle, a surprising 14% of respon-
dents host Microsoft SQL Server on private
clouds, and 30% run MySQL in hybrid or pub-
lic clouds. These numbers will go up as dis-
[2014 STATE OF DATABASE TECH]
Previous Next
Previous Next
Download Download
Register Register
Subscribe Subscribe
Previous Next
Previous Next
Get This And
All Our Reports
Our full report on enterprise
mobile app dev is free with reg-
istration. This report includes 37
pages of action-oriented analysis,
packed with 30 charts.
What youll find:
> 7 biggest native vs. browser app
dev challenges
> 13 most important IDE features
> 7 business drivers for mobility
Previous Next
Previous Next
Download Download
Register Register
Subscribe Subscribe
Previous Next
Previous Next
March 2014 $99
Report ID: R7770314
Next
reports
2014Stateof
DatabaseTechnology
ITs stuck in neutral. Exciting techs like NoSQL, DBaaS, and
distributed architectures are in use by fewof our 956
respondents. Just 5%have Hadoop in production, and a
mere 1%use Amazon DynamoDB for critical functions.
What will it take to jump-start innovation?
By Joe Masters Emison
repor ts. i nformati onweek. com
Which of these databases are used for your organizations most critical functions?
Top 10 Databases
Microsoft SQL Server
Oracle Database
MySQL (Oracle or Community)
IBM DB2
Microsoft Access
PostgreSQL
SAP Sybase ASE
Teradata
MongoDB
SAP Hana
Data: InformationWeek 2014 State of Database Technology Survey of 956 business technology professionals, January 2014
57%
38%
13%
11%
11%
3%
3%
3%
1%
1%
March 2014 5 informationweek.com
tributed (read: cloudified) architectures grow
in popularity. Moreover, IT can get huge di-
saster recovery/business continuity and geo-
graphic distribution benefits from running
databases across a private or hybrid cloud.
In past years weve highlighted respondents
dissatisfaction with licensing, but that angst
hasnt been enough to push IT to try a new da-
tabase model. Oracle, as usual, gets the most
bad vibes, with 27% saying theyre dissatisfied
with licensing. Its no surprise that none of the
27 PostgreSQL users had complaints.
Whats Up With Hadoop?
In contrast to the Big Three application da-
tabases, the state of the analytical database
market has changed quite a bit in the past
decade. Theres one primary driver: the shift
away from complex and pricey enterprise
data warehouses and toward analytical data-
bases, where costs have dropped significantly.
Consider Hewlett-Packards popular Vertica
analytical database. HP offers a free commu-
nity version that will run up to 1 TB of data on
as many as three nodes. Amazons Redshift
lets IT run an analytical database-as-a-service
for as little as $0.25 per hour. Such advances
empower customers of older-era analytical
databases and enterprise data warehouses to
switch and save a lot of money. And the shift
has created opportunity for a new breed of
analytical database vendor.
These newcomers the likes of Actian Ma-
trix (formerly ParAccel), Pivotal GreenPlum,
Teradata Aster, and Vertica use a column-
oriented design. That is, instead of data stored
as rows, its stored as columns, where each
column has the same data type. This enables
fast summarizing over many rows and often
returns a result more quickly than a conven-
tional enterprise data warehouse with OLAP
cubes. Loading data into a columnar database
is likely a much faster process as well. So while
there are still cases for star schemas, one of
the newer columnar databases is likely a bet-
ter choice than a data warehouse if you need
to run general analytical queries.
The other thing thats changed dramati-
cally in the analytical space is ETL, in two
areas. The first is a growing trend toward
ELT instead of ETL that is, extract, load,
transform instead of extract, transform,
load. Essentially, instead of using code to
manipulate data before loading it into a da-
[2014 STATE OF DATABASE TECH]
Previous Next
Previous Next
Download Download
Register Register
Subscribe Subscribe
Previous Next
Previous Next
How satisfied are you with the licensing agreements for these databases?
20% 5% 30% 10% 35%
35% 4% 20% 6% 35%
36% 5% 25% 4% 30%
19% 6% 26% 13% 36%
17% 7% 18% 20% 38%
Very dissatisfied Somewhat dissatisfied Neutral Somewhat satisfied Very satisfied
MySQL (Oracle or Community)
IBM DB2
Microsoft Access
Microsoft SQL Server
Oracle Database
Data: InformationWeek 2014 State of Database Technology Survey of 956 business technology professionals (539 respondents using Microsoft
SQL Server, 359 using Oracle Database, 124 using MySQL, 108 using IBM DB2, and 102 using Microsoft Access for their organizations most
critical functions), January 2014
Satisfaction Varies
tabase, you load semistructured (or perhaps
unstructured) data into a database and then
use database commands to transform the
data into a query-able form. Shilpa Lawande,
VP of software engineering and customer
support for Vertica, says this is what Zynga
does for its analytical needs. Theres a great
benefit to ELT from an ease-of-operation
standpoint as well; it should be easier to
maintain and store a series of database com-
mands than it is proprietary code that ma-
nipulates arbitrary files.
The second big change related to ETL is
Hadoop. While only 5% of our survey respon-
dents use Hadoop in production, its one of
the most widely discussed software packages
today. Hadoop makes it (relatively) easy to
write massively scalable software that pro-
cesses big input files. If you need to extract
parts of an exabyte of data and load that in-
formation into your analytical database, its
likely that your source data is just too big for
ELT. Or perhaps you have a constant fire hose
of data streaming in, and you need to pull
out particular chunks for analysis. These are
ideal cases for Hadoop, where you can write
processing code and effortlessly scale it to as
many nodes as you wish.
Hadoop is wonderfully fault-tolerant, and it
[2014 STATE OF DATABASE TECH]
March 2014 6 informationweek.com
Previous Next
Previous Next
Download Download
Register Register
Subscribe Subscribe
Previous Next
Previous Next
R
espondents are on top of database
security just 18% dont encrypt, and
80% say they have not been breached.
But theres always room to improve. Here are
a few points to consider.
Point No. 1: Its usually far easier to com-
promise an application than to crack the
database directly. If the outer shell of se-
curity for your application works, and you
encrypt traffic between the application and
the database, youve got most of the secu-
rity battle won.
Point No. 2: The biggest database security
risks come from insiders developers or sys-
tems administrators with the wrong access
levels (think Edward Snowden) or the wrong
execution permissions. Here are some factors
to think about as you consider whether a da-
tabase has appropriate security features:
>> The three As: authentication (verify the
connecting users identity), authorization
(verify that the user is allowed to do the ac-
tion), and accounting (log what happens).
>> Verify granular, role-based authentica-
tion within the database platform itself. Dont
let developers implement security. Note that
many newer databases had if you have cre-
dentials, you have all access when first re-
leased. Ensure that any databases youre con-
sidering have moved past that stage.
>> Demand encrypted communications
channels, preferably with TLS.
>> Security and governance must extend
across the whole application. Dont consider
database security a separate operation. Your
databases should fit within your whole-stack
security plan.
>> Watch permissions around code execu-
tion as well as data access. Consider built-in
calls (like mass deletes), stored procedures,
and user-defined functions.
>> Penetration testing is critical. In addition
to running intrusion-detection/intrusion-
prevention systems, hire penetration experts
to attempt to compromise your application
stack. Do this on a regular basis, even if its
not required by compliance mandate. This is
money well spent. Joe Masters Emison
Database Security: Progress, Problems
DATA PROTECTION
March 2014 7 informationweek.com
isnt just for ETL. Among respondents us-
ing Hadoop, the most common applica-
tion is for running analytics. You may not
need an analytical database at all if you
have Hadoop to handle your input data
and your analytical output is fairly simple
to code from the data thats coming in.
Database-As-A-Service
In theory, DBaaS offers compelling ben-
efits: automated scaling, backup, and re-
covery; no need for database adminis-
tration skills; and better business agility
because IT doesnt need to set up or test
database servers.
But in practice, its more complicated,
and thus unlike SaaS and IaaS, DBaaS re-
mains a niche market. Most databases
still run on servers completely controlled
by the same group that runs application
servers. To understand why, lets take a
look at the two types of DBaaS offerings
available.
The first option essentially comprises
managed versions of standard database
servers. The most-well-known of these is
[2014 STATE OF DATABASE TECH]
Previous Next
Previous Next
Download Download
Register Register
Subscribe Subscribe
Previous Next
Previous Next
Does Your Organization Run Hadoop?
22%
5%
44%
21%
8%
Data: InformationWeek 2014 State of Database Technology Survey of 956 business technology professionals, January 2014
Yes, in production
Yes, in pilot
No, but were considering it
Dont know
No, and were not considering it
March 2014 8 informationweek.com
[2014 STATE OF DATABASE TECH]
Previous Next
Previous Next
Download Download
Register Register
Subscribe Subscribe
Previous Next
Previous Next
Amazons Relational Database Service. Cus-
tomers trade some control to get some ben-
efits, including automated security patches
and backups, and easier distribution of nodes
across datacenters. Other examples of this type
of service include Herokus PostgreSQL host-
ing and Google Cloud SQL. The problem: This
type of DBaaS takes only half-measures toward
the theoretical benefits of DBaaS. It doesnt
eliminate the need for database administration
skills in-house, because functions like creating
databases and designing tables are really im-
possible to outsource. And to use one of these
services, you must have your application serv-
ers running on the same cloud, which is often
a nonstarter.
A second category of DBaaS gets you a lot
closer to the full package of theoretical ben-
efits. But its firmly a NoSQL, key-value-pair
world. The most-well-known examples here
are Amazons DynamoDB and Firebase. These
systems abstract the database server such
that your data is stored across many nodes,
and all backups, restores, and scaling are
done without IT being involved.
The first challenge here is whether you
can wedge your applications data into one
of these databases. Its likely youll need to
re-architect any application built for a rela-
tional database. On the other hand, if youre
building a new application, or your current
database just cant get the job done (too
slow, cant handle the traffic), these may
be great choices, at least for certain types
of data.
And this brings us to the biggest obstacle
to DBaaS today: IT tends to set up database
servers to match application servers. If the ap-
plication server is in a local datacenter, so is
the database server. Concerns around latency
and speed, and the desire for homogeneous
architectures, partly drive this tendency. But its
also because database servers are the ultimate
set it and forget it device. The average server
at my company, BuildFax, has been running for
less than a month; our average database server
has been running for more than a year. We set
them up, they work, we dont mess with them.
If it aint broke, why fix it?
The Next 10 Years
The biggest questions about the state
of databases today are: When, if ever, will
conventional relational databases lose
their stranglehold on enterprises? Will dis-
Does your organization encrypt all databases that contain sensitive information?
28%
26%
20%
8%
18%
Database Encryption
Data: InformationWeek 2014 State of Database Technology Survey of 956 business technology professionals, January 2014
Yes; we use native database
encryption capabilities
Yes; we use a third-party database
encryption product
Yes, via a combination of native
and third-party capabilities
We do not encrypt
We encrypt only some databases
containing sensitive information
tributed databases become the norm
for large organizations in two years?
Five? Or will relational databases on
bare metal still be the standard 10 years
from now?
Its difficult to oust an incumbent data-
base. The stakes are high, and youre likely
to be second-guessed. If you ask a DBA
who works with only two databases what
the best choice is for a web-scale applica-
tion its going to be one of those two
databases. Dont allow comfort zone to
trump progress. If youre looking to pilot
a distributed architecture, or just go cold
turkey from Oracle, you need to budget
time to do your own research. Enterprise
use of new databases is likely broader
than our survey reflects; generally, inno-
vation is happening in specific greenfield
projects, so talk to peers.
However, because the Big Three ven-
dors will continue to innovate, theyll
keep sizeable market share. Almost all of
our respondents have at least one DBA
on staff, and this translates to entrench-
ment. The benefits of a distributed sys-
tem also seem questionable to many
CIOs, given the performance of single-
master architectures. Newer high-end
servers dont go down often, and the
majority of core enterprise applications
can be run off a single master, especially
considering how high IT can vertically
scale todays RDBMSes. Its likely that only
a small percentage of applications will
require a new database and IT will
admit that only after repeated failures of
the incumbent. That means the top five
databases in the 2024 State of Databases
Survey will likely have at least a few in
common with 2014.
Joe Masters Emison oversees the award-winning Build-
Fax cloud architecture and frequently speaks on cloud
topics. Write to us at iwletters@ubm.com.
Previous Next
Previous Next
Download Download
Register Register
Subscribe Subscribe
Previous Next
Previous Next
March 2014 9
[2014 STATE OF DATABASE TECH]
informationweek.com