Module 8

Storing Non-Relational Data in

Azure
Module Overview

• What is Azure Storage?

• Azure Storage Tables
• Azure Storage SDK
• Monitoring Table Storage
• Using the Azure Storage Emulator
Lesson 1: What is Azure Storage?

• Azure Storage
• Types of Storage
• Azure Storage vs. Azure SQL Databases
• Geo-Replication in Azure Storage
• Accessing Storage Data
Azure Storage

• Azure Storage services allows you to store

records, files and simple requests in a flexible,
managed and scalable solution
• Separating the storage of your data from your
application allows more flexibility when planning
and scaling different aspects of your cloud
application scenario[s].
• Storage services are massively scalable which
allows you to store large sets of data without
being forced to plan partitioning and sharding of
your data.
Types of Storage

Tables

Blobs

Queues
Types of Storage

Store related records

Unstructured table
Tables storage
Very efficient key lookups

Storage of files, drives

and multimedia
SAS Tokens are available Blobs
to protect your stored
content

Store and mange simple

messages/requests
Queues Unified so your messages
can be distribute among
many worker processes
Azure Storage vs. Azure SQL Databases
Azure Storage
• Table Records
• Store records of different
structures in the same table
• Very efficient composite index
queries
• Files
• Store files in a highly-available
and redundant storage
mechanism
• Queues
• Queue mechanism available to
share requests between many
consumers
Azure SQL Databases
• Table Records
• Enforces schema on table
records
• Allows you to create table
relations and perform joins
• Files
• Store files in a varbinary(max)
• Queues
• You can use locks and
transactions to emulate a
queue
Geo-Replication in Azure Storage

• Locally Redundant Storage (LRS)

• Default option
• Data is replicated to three different nodes within the
same data center
• Zone Redundant Storage (ZRS)
• Data is replicated to three different data centers within
the same region.
• It is possible for data to be replicated across region
boundaries if there are not enough data centers for
storage within the region.
Geo-Replication in Azure Storage

• Geo Redundant Storage (GRS)

• Data is replicated to three different nodes within the
primary data center.
• Data is replicated asynchronously to three nodes within
a fixed redundant data center.
• Data is “eventually consistent”.

• Read Access - Geo Redundant Storage (RA-GRS)

• Similar to GRS, data is replicated to a redundant data
center.
• Access to the secondary data source is read-only.
Accessing Storage Data

• Access to resources are authenticated using a

shared key for your storage account
• Blobs can be configured to allow anonymous access
• Shared Access Signatures can be generated to give a
client controlled, temporary access to a storage
resource
• Storage accounts are given two keys that can be
regenerated at any time.
• This allows you to rotate keys and regenerate keys on a
scheduled basis without your application losing access
to resources
Lesson 2: Azure Storage Tables

• Table Storage
• NoSQL Data in Table Storage
• Web Endpoints for Table Storage
Table Storage

• Table Storage allows you to store flexible

datasets that are not constrained by a schema or
model
• Client applications can dictate the model of data stored
• Complex objects of different schema[s] can be stored
in the same table
• Built for massive scale and very large datasets
• Data is partitioned using a partition key to
support load balancing across different nodes.
• Data is indexed by a combination of the partition
key and a row key for very fast lookup.
NoSQL Data in Table Storage

• Azure Table Storage is a NoSQL database

• Implemented as a Key-Value store

• Tables contain entities that are partitioned across

multiple nodes using the partition key
• Entities have a index composed of the
combination of the partition and row keys
• Properties of the particular entity are
implemented as a collection of key value pairs
• Key is the name of the property.
• Value is the value for the property.
NoSQL Data Structure in Table Storage

PartitionKey: Student PartitionKey: Student PartitionKey: Teacher

RowKey: 145A RowKey: 287C RowKey: 945FT

FirstName FirstName LastName

• Chad • Joann • Summers

LastName LastName Grade

• Drayton • Chambers •5

Age Grade
•8 •3

Grade
•3
NoSQL Data Structure in Table Storage
(optional)

Storage
Table Entity
Account
Name = …
Email = …
customers
Name = ...
Email = …
store RewardID = …

OrderID = …
orders
Date = …
Web Endpoints for Table Storage

• Table Storage can be accessed using the OData

protocol.
• Can return results using AtomPub or JSON.
• JSON data can typically provide up to a 70% reduction in
bandwidth.
• OData queries use the PartitionKey and RowKey as the
index for the entity
• Projection and filters are supported

• SDKs for Storage Access have been provided in

many popular programming languages
• .NET
• Node.js, Java, PHP, Ruby, Python
Lesson 3: Azure Storage SDK

• Getting Started with the Azure Storage SDK

• Creating a Table Storage Client using the SDK
• Creating a Table using the SDK
• Querying Data using the SDK
• Inserting Data using the SDK
• Removing Data using the SDK
Getting Started with the Azure Storage SDK

• The Azure Storage SDK provides convenience

methods and classes for accessing the different
Storage services.
• You can access the assemblies installed on your
machine when you install the Azure SDK for .NET
(Visual Studio 2013)
• Assemblies can also be accessed via NuGet and the
WindowsAzure.Storage package
• You will need your storage account connection
string in order to connect to the storage account
using the SDK.
Creating a Table Storage Client using the SDK

// Retrieve the storage account from the connection

string.
CloudStorageAccount storageAccount =
CloudStorageAccount.Parse(

CloudConfigurationManager.GetSetting("StorageConnectionS
tring"));

// Create the table client.

CloudTableClient tableClient =
storageAccount.CreateCloudTableClient();
Explanation

1. Create a CloudStorageAccount variable using

your connection string.
• You can also use the
CloudStorageAccount.DevelopmentStorage static
property to access your development storage
emulator.
2. Invoke the CreateCloudTableClient method to
return a CloudTableClient variable.
Creating a Table using the SDK

// Create the table if it doesn't exist.

CloudTable table =
tableClient.GetTableReference(“test");
table.CreateIfNotExists();
Explanation

1. Invoke the GetTableReference method using

the name of the table as the first parameter to
return a CloudTable variable.
2. Invoke the CreateIfNotExists method of the
CloudTable variable.
• If the table does not exist, a new table is created
with the name specified.
Querying Data using the SDK

// Create a retrieve operation that takes a customer

// entity.
TableOperation op =
TableOperation.Retrieve<OrderEntity>(“CEA", “14B");

// Execute the retrieve operation.

TableResult retrievedResult = table.Execute(op);

// Retrieve the result.

if (retrievedResult.Result != null)
var entity = (OrderEntity)retrievedResult.Result;
Explanation

1. Invoke the static TableOperation.Retrieve method

using the entity type as the generic parameter, the
partition key as the method’s first parameter and the
row key as the second parameter to return a query
operation.
• The generic type should either inherit from TableEntity or
implement the RowKey, PartitionKey and TimeStamp
properties.
2. Invoke the Execute method on the TableClient variable
using the operation as the first parameter to return a
TableResult variable.
3. The entity is stored in the Result property of the
TableResult variable
Filtering Data in a Query

// Create the table query.

TableQuery<CustomerEntity> rangeQuery = new
TableQuery<CustomerEntity>()
.Where(TableQuery.CombineFilters(
TableQuery.GenerateFilterCondition(
"PartitionKey", QueryComparisons.Equal, “Dulaney“
),
TableOperators.And,
TableQuery.GenerateFilterCondition(
"RowKey", QueryComparisons.LessThan, "E“
)
));

// Loop through the results, displaying information about the entity.

foreach (CustomerEntity entity in table.ExecuteQuery(rangeQuery))
{
// Do something with the entity
}
Inserting Data using the SDK

// Create a new customer entity.

var customer = new CustomerEntity("Harp", "Walter");
customer.Email = "Walter@contoso.com";
customer.PhoneNumber = "425-555-0101";

// Create the TableOperation that inserts the customer.

var insertOperation = TableOperation.Insert(customer);

// Execute the insert operation.

table.Execute(insertOperation);
Explanation

1. Create a new instance of your entity type and

store it in a variable.
• Your entity type should either inherit from
TableEntity or implement the RowKey, PartitionKey
and TimeStamp properties.
2. Invoke the static TableOperation.Insert method
using the entity variable as the first parameter.
3. Invoke the Execute method on the TableClient
variable using the operation as the first
parameter.
Removing Data using the SDK

// Create the Delete TableOperation.

var deleteOp = TableOperation.Delete(entityToDelete);

// Execute the operation.

table.Execute(deleteOp);
Explanation

1. Invoke the static TableOperation.Delete

method using the entity variable as the first
parameter.
2. Invoke the Execute method on the TableClient
variable using the operation as the first
parameter.
Lesson 4: Monitoring Table Storage

• Table Storage Metrics

• Table Storage Logging
• Using Table Storage for Logging
Table Storage Metrics

• Storage Analytics Metrics aggregates all

transactions for a storage account over a time
period
• Resulting metric data can be stored in a blob or
table storage
• Metrics can by Hourly or by the Minute
• Data includes:
• Ingress/Egress
• Availability
• Request Percentages
• Errors
Table Storage Logging

• Logging can be used to store metadata about

each individual transaction to Table Storage
• Logging can include information such as:
• Successful requests
• Failed requests
• Requests using SAS tokens
• Anonymous requests
Using Table Storage for Logging

While table storage can be used to store logs and

metric data for your storage account or other
applications, you should consider storing logs in
Blob Storage.

Reading logs typically requires pulling an entire

collection of logs for a time period and this would
cause you to do a table scan which is inefficient in
Table Storage. Log files can easily be stored in
Blob Storage and scanned on local hardware.
Lesson 5: Using the Azure Storage Emulator

• Storage Emulator
• Differences Between the Emulator and the Cloud
• Table Storage Implementation in the Storage
Emulator
Storage Emulator

• The Azure SDK comes with a built-in Storage

Emulator
• The emulator provides local access to Blobs,
Queues and Tables for use when developing a
Cloud application
Differences Between the Emulator and the Cloud

• The emulator only supports a single static

account and key.
• Name: devstoreaccount1
• Key:
Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6
IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==
• The emulator will not scale to support a large
number of concurrent requests.
• Because domain name resolution is not available
on a local computer, the URI scheme for the
storage emulator is different from the Azure
Storage services.
Table Storage Implementation in the Storage
Emulator

• The emulator creates a database using the

version of SQL installed on your development
machine.
• Typically it uses LocalDB
• The emulator then uses LINQ to SQL on top of
the database to emulate the functionality and
queries provided by the Table Storage service.
• The emulator exposes HTTP endpoints so that
you can use the Azure SDK libraries or the REST
endpoints to access your Table Storage account.
Lab: Storing Event Registration Data in Azure
Table Storage

• Exercise 1: (Optional) Implementing Azure Table

Storage
• Exercise 2: Populating the Sign-In Form with
Registrant Names
• Exercise 3: Updating the Events Website to use
Table Storage
•Logon Information
Exercise 4: Verify that the Events Web Site is using
Virtual Machine:Tables
Azure Storage 10978A-SEA-DEV
for Registrations
User name: Admin
Password: Pa$$w0rd

To proceed with this Lab, you must complete Module 3

Estimated Time: 60 minutes
Lab Scenario

Even though event registrations could be stored in

SQL, you have a unique need. Each event requires
a different registration form that can be changed
at any time. Essentially, registrations could be of
any schema. To facilitate this you have elected to
use Azure Table Storage for your event
registrations.
Lab Review

• Since the Azure Storage SDK shares a

CloudStorageAccount class that is used for all
types of storage, what are some of the ways that
you can refactor your application to take
advantage of this?
Module Review and Takeaways