NoSQL Databases
Vamshi Krishna Reddy V
International Institute of Information Technology
Bangalore, India
vamshikrishnareddy.v@iiitb.net
ABSTRACT
From the day of invention of internet to period around 2003,
websites were mostly read only where a publisher would post
some content and users would be reading or using that con-
tent. This kind of applications needed a different kind of
data storage which is predefined and predictable. Thus ma-
jor part of research went in storing data and presenting it to
users. But now due to increase in social interactive trend on
web, every user has become a publisher, where the databases
changed from read heavy architectures to read-write and
write-heavy architectures. In this paper I am presenting the
basic need for alternate database systems, purpose of using
NoSQL databases, available options with their distinctive
features and challenges involved in implementing them in
real world occupied by Relational databases.
Keywords
NoSQL Databases, Non Relational Databases
1. INTRODUCTION
In the present scenario of information explosion, enormous
amount of data is being generated by different applications
and organizations over the internet. Traditional ways of
storing data about payrolls, employee management etc., are
have remained same from past decades without much change
in the way they are being done. But in the areas where the
user generated content is more and applications which en-
abled the users to post their content online, the data being
generated is humongous and more over its a new kind of
problem which doesn’t existed before few years. For this
kind of needs where data generated is about the users, data
is important but not safety critical, the storage systems
need not be so stringent with rules and schemas upon them.
These in turn reduce the performance in the process of en-
forcing many facilities which are not necessary. So there is a
need for a different genre of data storage systems for specific
usage requirements.
2. NOSQL
We are using traditional SQL databases also known as Re-
lational Database systems for managing our data all these
days. But due to above mentioned reasons and informa-
tion explosion there is a need for systems which are more
flexible compared to SQL databases. Huge amounts of data
also results in large amount of requests for it, which requires
swift response i.e. the performance of the system should be
high enough to handle the load. Data is becoming more and
more connected (more and more joins where relationships
are exploding). Datasets are becoming larger and larger.
Data is becoming less and less structured. As the existing
systems are unable to provide optimized solutions for these
challenges, we are in quench for features more than SQL
databases are providing.
As a solution for the above mentioned requirements, NoSQL
has come into picture in recent days. A lot of SQL Server
professionals themselves do not really know what NoSQL is.
Some think it is a product, but most think it is a competitor
and potential threat to SQL Server and relational databases
in general. NOSQL literally means Not Only SQL but not
an “Anti Sql” or “No SQL” which is misinterpreted by lot of
people.
NoSQL Definition
Next Generation Databases mostly addressing some of the
points like being non-relational, distributed, open-source and
horizontal scalable. The term covers a wide range of tech-
nologies, data architectures and priorities. The original in-
tention has been modern web-scale databases. The move-
ment began in early 2009 and is growing rapidly. Some
more characteristics such as schema-free, easy replication
support, simple API, eventually consistent are also featured
by NoSQL databases. NoSQL is not a competitor for tradi-
tional RDBMS/SQL rather It is actually a solution to many
use cases where using RDBMS was perhaps a poor fit. Thus
the decision for an architect is not to decide which of the
two competing options (RDBMS or NoSQL) is better but
to choose one of them as a preferred standard storage strat-
egy. This simply is a design decision of choosing a more
appropriate storage system for the application under con-
sideration.
In these lines, many of the online giants who are generat-
ing data, themselves have come up with new ways of storing
data for efficient lookup and high performance. They are
all not that different and many of them are inspired by one
another and borrow features from each other. Most of them
are open source. Memcached was the first one to come up
in this category, which was developed by livejournal about
a decade ago. The most influential developments in this
area which suddenly catalysed the development of NOSQL
databases are Google’s BigTable and Amazon’s paper on Dy-
namo. Later facebook, the largest social networking website
as of today has come up with its own way of storing and
querying data. This is called Cassandra which was open
sourced. They have developed this in order to increase the
performance of the inbox messages. Just as NoSQL presents
new challenges it also offers significant rewards to those who
can successfully incorporate it into their solution portfolio.
The key benefits are going to emerge around improved data
comprehension, flexible scaling solutions and productivity.
3. DRIVING FACTORS OF NOSQL
There are three key drivers behind the increased interest in
NoSQL.
1. The whole new form of traffic profile driven by what
might be referred to as Web 2.0 or the Social Web
which is maturing now over the internet. As the world
becomes more connected it is possible for sites to ex-
perience massive variations of traffic. This may be ex-
pected and unexpected, such as on events like Christ-
mas or world cup, the traffic may increase enormously
but some of the unexpected events like 9/11 attacks,
where many of the websites faced a gruesome challenge
in serving the traffic and most of them actually failed
to do so.
2. The second factor is the fact that data changes over
time. As the business model evolves concepts and data
models often struggle to evolve and keep pace with
changes.
3. The final factor is that the NoSQL technology is now
starting to become a commodity. Once an Amazon or
Google had no choice but to create a solution for them-
selves that answered their problems of scale. Apache
Foundation or other open source groups are providing
community-driven support and development and has
lead to the possibility of using extremely sophisticated
code by small businesses as they are being provided
in a smart simpler way by abstracting the complexity
through RESTful API’s.
Some use cases to show the need of NoSQL Databases
• An application generating a large amount of temporary
data which is not actually a part of main database such
as in shopping cart, retained search, personalization
and incomplete questionnaires by users.
• Web applications like twitter where people follow each
other as well as get followed by some body. Main-
taining these relations in relational tables will be a big
problem as we need to have redundant data and lot of
joins and foreign keys. This can be easily done with
graphs. So we go for graph databases.
• In write heavy architectures like twitter where users
write content to the data store where there happens
to be 600 tweets per second i.e those many writes per
second. But this is not the only job of that database,
it has to also cater to the read requests made to it by
the third party applications.
• In Facebook, the number of messages exchanged are
about 50 Million per day which is more than the num-
ber of mails exchanged over internet.
4. TYPES OF NOSQL SOLUTIONS
Different types of NoSQL databases currently available are
1. Column-Oriented Databases They store all the col-
umn data in order sequentially on disk, you can read
all the data for that column very quickly. Some of
the examples for this kind of databases are SenSage,
SybaseIQ, FluidDB. Typical application scenario may
be statistics.
2. Document Oriented Databases eliminates “gaps”
left by lots of empty fields in db records. Lack of
enforced structure makes it very flexible. Examples
are CouchDB, MongoDb etc., Typical applications are
most of web applications. Tolerant of incomplete data
is its strength.
3. Key-Value Stores will maintain relation of the keys
and their corresponding values. These values may be
in turn documents etc., Major example in this area
are Tokyo Cabinet/Tyrant, Redis, Voldemort, Oracle
BDB
4. Graph Databases are those which use the graph the-
ory concepts to store the data. Data is stored in terms
of nodes and the relations between them are realized in
the form of edges. Examples for graph databases are
Neo4J, InfoGrid, Infinite Graph. Typical Applications
include Social networking, Recommendations. Some of
the strengths are Graph algorithms e.g. shortest path,
connectedness, n degree relationships, etc.
5. Distributed Databases are the databases in which
data may be stored in multiple computers located in
the same physical location, or may be dispersed over a
network of interconnected computers. Examples: Cas-
sandra, HBase, Riak. Typical Applications include im-
plementation of Distributed file systems. Fast lookups,
good distributed storage of data add to its strengths.
5. TARGETED USERS
As said earlier, different types of these NoSQL databases
have been developed in order to cater specific needs of the
users. The decision of using this kind of databases also de-
pends upon the user base and the frequency of usage. Also
users need to drill down some points of concern of their
application and the extent of relaxation they can allow on
several factors like reliability or fault-tolerance etc. If it is
going to be a safety critical system, RDBMS may be the
best choice. Choosing a NOSQL is to solve a problem that
relational databases are a bad fit for. SQL will be around
for a while. It’s good at doing what it was designed to do.
However, there are many times when people use SQL simply
because there is nothing better out there. As data complex-
ity rises, a new method for accessing and persisting that
data will have to be investigated.
Most of the Social networking websites, Music streaming
and video streaming websites can use this NoSQL databases
as eventual consistency is required rather than over all con-
sistency. The read write access for these applications will
be in order of millions of requests per day and size of data
will be in order of Petabytes. More over user may not af-
ford to miss a comment but he will be fine if a comment
shows up on second refresh though it was made by the time
you refreshed it first. This is not the case in stocks, safety
critical systems like air traffic control databases and orders
through secure payment etc. These are Not meant for 100%
transactional reliability But for scalability and performance.
Not all organizations or applications will have have 100 mil-
lion users accessing at same time. So Data intensive or-
ganizations/applications can use the NoSQL databases and
Normal predictive usage with limited records can be well
supported by RDBMS with more efficiency.
6. FEATURES OF NOSQL
Horizontal Scalability: Most of the NoSQL databases
such as Document oriented databases, key value stores, dis-
tributed databases are designed for horizontal scalability.
This means as your database grows, you can simply add
more hardware, or more resources from the cloud. This
can be achieved as these databases operate on something
similar to Distributed Hash Tables (DHTs). DHTs store a
key/value pair in hash buckets. These buckets hold a num-
ber of key/value pairs indexed by ”hash value.” This hash
value is a number generated from the data in such a way
that all key/value pairs are distributed evenly among the
hash buckets. For example, if the DHT has 5 hash buckets
and 50 key/value pairs are stored, each hash bucket should
have about 10 key/value pairs. One of the advantages here
is that this is extremely easy to parallelize. If application de-
mands for more database servers, adding more hash buckets
would solve the problem. As the database grows, just add
more servers, and none of them need to be super-computer.
This is what a “Horizontally Scalable” system means.
One of the problems with SQL servers is they don’t work
well in a cloud. As databases grow and as traffic increases,
larger and faster computers are required. Load balancing
can be achieved by mirroring the servers, but they still need
to be large and fast. This just doesn’t fit with the cloud
model. NoSQL servers, on the other hand, can simply add
more nodes.
Schema-Less: Another differentiating feature of NoSQL
databases is that they are schemaless. This is somewhat
different from the designing of traditional databases so it
would be hard to digest for the users who were using rela-
tional databases for a long period of time. Instead of each
record existing in a row of carefully designed columns, each
record exists in a document. This is like a file on the file sys-
tem. Each document can store any amount of data without
following a schema. Though these documents are schema-
less, they’re not freeform. Many databases opt to use the
open formats such as JSON, which helps you store key/value
pairs in a formatted way. A document can have any number
of key/value pairs and these entries need not be fixed.
7. CONFLICTS OF NOSQL WITH SQL
ACID Properties
• The major appeal of relational databases is they make
the ACID promise:
– Atomicity - a transaction should be either commit
or roll back.
– Consistency - valid data should only be written
to the database
– Isolation - Ensure even all transactions are hap-
pening serially, the data is correct and one trans-
action should not be able to see uncommitted
writes.
– Durability - When once committed or written, it
should be persistent.
• The problem with ACID is that it provides much more
than what an application requires. It trips the applica-
tion when tried to scale a system across multiple nodes.
Down time is unacceptable. So the system needs to be
reliable. Reliability requires multiple nodes to handle
machine failures. To make scalable systems that can
handle lots and lots of reads and writes, many more
nodes are required.
• Scaling ACID across many machines will cause prob-
lems with network failures and delays. The algorithms
don’t work in a distributed environment at any accept-
able speed.
Brewer’s CAP Theorem
Most of the NoSQL databases are compromising on ACID
properties in order to provide high performance and scala-
bility. In this process, it can be assured that out of the fol-
lowing three properties only two can be satisfied by a given
database.
• Consistency: The data should be same all the time. In
simple words “what you write is what you see”. A sys-
tem is consistent if an update is applied to all relevant
nodes at the same logical time.
• Availability: All operations on the data store should
return successfully. For a distributed system to be con-
tinuously available, every request received by a non-
failing node in the system must result in a response.
• Partition-Tolerance: The capability of a system to be
consistent and deliver messages and respond to clients
requests even when one or more nodes in the network
fails to transport messages.
From ACID and CAP theorem, following conclusions can
be made about the database systems which are assigned an
acronym called BASE.
• Basically Available - System should be up and running
all the time.
 • Soft State - Consistency can be compromised at a given
point of time.
• Eventually Consistent - But the system should become
consistent at some later time.
Some of the big applications such as systems by Google,
Yahoo, ebay, Facebook, Amazon, etc., are being built using
CAP and BASE properties.
Eg: In Facebook, a person may afford to get a comment in
his second refresh even though its being made by the time
he refreshed first. So Overall system should be consistent
but can be relaxed at a given point of time.
8. CHALLENGES INVOLVED IN NOSQL
In the process of incorporating the NoSQL strategy for a
large scale monolithic systems having relational data stor-
age, systems may actually be depending on certain prop-
erties of relational data, for example data types or trans-
actional consistency, then the problem is much harder. In
some ways decoupling data provision needs to be the first
task rather than migrating data storage
While migration there must be a clear analysis of what data
is relational and what is stored in relational stores currently,
but only due to the lack of alternatives. It is also important
to review historic decisions to see if they were made with
historical constraints in mind.
Businesses mine information in corporate databases to im-
prove their efficiency and competitiveness, and business in-
telligence (BI) is a key issue in Information Technology for
all medium to large companies. NoSQL databases offer few
facilities for ad-hoc query and analysis. Even a simple query
requires significant programming expertise, and commonly
used BI tools do not provide connectivity to NoSQL.
Lack of experienced experts in the area of NoSQL databases,
because all the NoSQL developers are in the learning mode,
where as it is easy to find experienced RDBMS program-
mers.
9. CONCLUSION
Ease of use is the major key concern for any kind of sys-
tem, similarly in this case, simpler setup and ease-of-use of
NOSQL databases is attracting lot of users and developers
towards it. Traditional RDBMS/SQL needs more stringent
rule building as in building of schemas and defining key con-
straints etc., but those schemas are exactly what give rela-
tional databases higher performance opportunity in parallel
DBMS implementations.
NoSQL offers the chance to think differently about data i.e.
how to store and query the data efficiently. It allowed the
users to understand what is required and what can be ig-
nored while building a database. NoSQL is just another tool
in the application developer’s toolbox. It is like a hammer
cannot be used to fix every job around the house, you cannot
expect that a relational database is always the right choice
for every data access application. NoSQL implementations
solve a data storage problem that relational databases were
not really designed to address. So the bottom line on NoSQL
is that NoSQL is not a technology that is going to replace
relational database systems except in very specialized in-
stances. Relational databases will continue to work effi-
ciently for the tasks they have been designed to and NoSQL
databases will carry out the applications which require fea-
tures that are Not Only SQL.
10. REFERENCES
[1] The end of SQL and relational databases.
http://blogs.computerworld.com/15510/the_end_of
_sql_and_relational_databases_part_1_of_3, 2010.
[2] Why Enterprises Are Uninterested in NoSQL -
Communications of the ACM.
http://cacm.acm.org/blogs/blog-cacm/99512-why-
enterprises-are-uninterested-in-nosql/fulltext,
2010.
[3] CAP Theorem, Eventual Consistency, NoSQL.
http://venublog.com/2010/04/07/cap-theorem-
eventual-consistency-nosql/,
2010.
[4] My Thoughts on NoSQL.
http://www.eflorenzano.com/blog/post/my-
thoughts-nosql/,
2009.
[5] The dark side of NoSQL.
http://codemonkeyism.com/dark-side-nosql/, 2009.
[6] CAP Confusion: Problems with âĂŸpartition
toleranceâĂŹ - Cloudera - Apache Hadoop for the
Enterprise.
http://www.cloudera.com/blog/2010/04/cap-
confusion-problems-with-partition-tolerance/,
2010.
[7] NoSQL: A Modest Proposal.
http://voodootikigod.com/nosql-a-modest-
proposal,
2009.