Sie sind auf Seite 1von 301

4/27/18

Module 1
Application Architecture Patterns
in Azure

Module Overview

• Design Pattern Resources


• Performance Patterns
• Resiliency Patterns
• Scalability Patterns
• Data Patterns

1
4/27/18

Lesson 1: Design Pattern Resources

• Why Patterns?
• Microsoft Patterns & Practices
• Azure Architecture Center

Why Patterns?

• Most problems are already solved by other


professionals in the industry

• Years of experience and research have defined best


practices and the “best practice” when developing
new solutions

• Most patterns are platform-agnostic

2
4/27/18

Microsoft Patterns & Practices

• Engineering focused group at Microsoft


• Collects real-world scenarios from customers
• Engineers solutions using best-practices
• Analyze trends and services used by the community
• Shares findings using
• GitHub
• Whitepapers
• Conference Sessions
• http://docs.microsoft.com

Cloud Design Patterns

• The Cloud Design Patterns is a collection developer-


oriented prescriptive architecture guidance

• Includes topics and patterns that help you design


your cloud solutions

• Patterns are platform and framework-agnostic

• Examples are provided in the context of Azure and C#

3
4/27/18

Patterns & Practices on GitHub

Microsoft Patterns & Practices shares much of their


documentation, projects and findings today on GitHub:

https://github.com/mspnp

For example, the Microservices Reference Implementation


shares best practices when designing a microservices solution
running on Azure using Kubernetes:

https://github.com/mspnp/microservices-reference-
implementation

Azure Architecture Center

Landing page for reference architectures, patterns and


guidance for solutions on the Azure Platform

https://docs.microsoft.com/azure/architecture/

4
4/27/18

Azure Architecture Center Guide

The Architecture Center features an all-up guide to


creating solutions that are scalable, resilient and highly
available

https://docs.microsoft.com/azure/architecture/guide/

Lesson 2: Performance Patterns

• Stateless Applications
• The Valet Key Pattern

5
4/27/18

Stateless Applications

• When designing web applications, split your business


processes into Partitioning Workloads
• Partitioning Workloads:
• Can be handled in modular websites, cloud services, or
virtual machines
• Provides the ability to scale the different components of
your application in isolation

Partitioning Workloads

• Example Photo Sharing Application

Virtual Machine
Web Front-End
SignalR Hub
Image Processor
Thumbnail Storage

6
4/27/18

Partitioning Workloads

• Example Photo Sharing Application


• The Web Front-End shows thumbnails of images users have
uploaded today
• The application code takes an image that a user uploads,
processes it into a thumbnail and then stores the thumbnail
• The SignalR hub notifies the client web browser that the
image is finished processing

Partitioning Workloads

• Example Photo Sharing Application

Virtual Storage Background


Web App
Machine Service Worker

Thumbnail
Signal-R Storage Web Front- Image
Hub and End Processing
Retrieval

• Each component of the application can be scaled in


isolation to meet its specific demand

7
4/27/18

The Valet Key Pattern

• If your application access a resource on behalf of


your clients, your servers take on additional load
• You do not want to make your resources publicly
available so you are typically forced to have your
application validate a client
• The Valet Key pattern dictates that your application
simply validates the client and then returns a token
to the client. The client can then retrieves the
resource directly using its own hardware and
bandwidth

The Valet Key Pattern

• The client requests a resource from your application


• Your application validates the client and then returns an
access token
• The client then directly accesses the resource by using the
provided token

8
4/27/18

CQRS Pattern

The Problem

• Potential mismatch between read and write


• Data Access layer contains largest amount of
requests (for database, identity and security)

CQRS Pattern

• Solves problem by separating read and write models


• Each model can be managed in isolation
• Models can be deployed to separate compute instances
• Example: A write-heavy workload will need more write instances

9
4/27/18

Throttling Pattern

The Problem:
• Load on a cloud application varies:
• By Time
• By Quantity of Users
• By Specific User Activities
• By Performance of Actual Underlying Hardware
• Handling load correctly for one user/client may “starve”
other clients of resources
• Solutions must be able to handle sudden bursts of usage

Throttling Pattern

• Solves problem by creating a “ceiling” on usage

10
4/27/18

Discussion

Which Azure services implement


SAS token features that you can use
in the Valet Key pattern?

Lesson 3: Resiliency Patterns

• Transient Errors
• The Retry Pattern
• Queues

11
4/27/18

Transient Errors

• Transient faults can occur because of a temporary


condition
• Service is unavailable
• Network connectivity issue
• Service is under heavy load
• Retrying your request can resolve temporary issues
that normally would crash an application
• You can retry using different strategies
• Retry after a fixed time period
• Exponentially wait longer to retry
• Retry in timed increments

Transient Fault Handling

• The Transient Fault Handling application block is a


part of the Enterprise Library
• The Enterprise library contains a lot of code that is
necessary to implement the pattern and the retry
strategies
• Retry strategies are prebuilt for common scenarios
including accessing Azure services
• You can build custom strategies and extend the
library for the unique needs of your application

12
4/27/18

Circuit Breaker Pattern

Prevents applications from repeatedly retrying an


operation that will most like fail
• Acts as a proxy between application and service
• Prevents waste of CPU cycles on long-lasting faults

Circuit Breaker Pattern

13
4/27/18

The Retry Pattern

• The Retry pattern is designed to handle temporary


failures
• Failures are assumed to be transient until they
exceed the retry policy
• The Transient Fault Handling Block is an example of a
library that is designed to implement the Retry
pattern (and more)
• Entity Framework provides a built-in retry policy
implementation
• Implemented in version 6.0

The Retry Pattern

• The application sends a request to a hosted service


• The hosted service responds with a HTTP 500 (Internal
Server Error) code
• The application retries until it exceeds the retry count for its
policy or is met with an acceptable status code such as HTTP
200 (OK)

14
4/27/18

Queues

• A modular web application can behave like a


monolithic application if each component relies on a
direct two-way communication with a persistent
connection
• Persistent queue messages allow your application to
handle requests if one of your application
components fail or is temporary unavailable
• An external queue allows your application to audit
requests or measure your load without adding any
overhead to the code of your primary application

Queues

• Queues can be used to communicate in a


disconnected manner between the different
components of your application
• If an instance of your application module fails, another
instance can process the queue messages
• Messages that fail to be processed can either be reinserted
into the queue or not marked as complete by the client
module
• Messages can be measured or audited in isolation from
your application
• Queues become very important when dealing with
background processing (Worker roles and WebJobs)

15
4/27/18

Queue-Based Load Leveling Pattern

Use a queue to act as a “buffer” between requestor


generators and request services

Queue decouples the tasks from the service


• Service can work at it’s own pace

Discussion

If you were to design a retry policy


to connect to an instance of Azure
SQL Database, what parameters
(cool-off time, max retry count)
would you use?

16
4/27/18

Lesson 4: Scalability Patterns

• Asynchronous Messaging
• Cached Data Consistency
• Load Balancing

Asynchronous Messaging

• With synchronous messaging, the service processing


some logic must run immediately when requested
• This becomes a problem with large, varying quantities of
request
• This can prevent an application from scaling both up or
down
• Scaling down can drop requests “in flight”

17
4/27/18

Competing Consumers Pattern

• Application instances (producers) generate messages to be placed in the queue


• Service instances (consumers) poll the queue to see if any messages are waiting to
be processed
• The service instance that receives the message will process the message and then
flag the message as processed in the queue
• If the service instance fails to process the message, the queue will eventually make the message
available to other instances after a period of time

Cached Data Consistency

• Cache data can quickly become stale

• Example:
• Application caches a list of the 10 latest records
• Whenever a new record is added, the list is officially stale
• Do you re-query the database constantly?
• Do you skip using a cache to have real-time data?

18
4/27/18

Cache-Aside Pattern

• Treats cache as a read/write store


• Ensures synced data between cache and data store

Cache-Aside Pattern

1. Determine whether the item is currently stored in cache


2. If the item is not currently stored in ache, retrieve the item from the
source data store
3. Store a copy of the item in the cache

19
4/27/18

Static Content

• Static content is often hosted in a CDN to deliver


content directly to clients in the most efficient
manner possible

• Web servers can serve static content, but they are


often not the right choice
• Developers often serve multiple CSS, JS and HTML files as
part of a single web page “visit”

• Serving static content from a CDN can save on


compute and memory utilization of web servers

Static Content Hosting Pattern

• Minimizes web hosting compute costs


• Especially so for web sites that consist only of static content
• Improves end-user content performance

20
4/27/18

Load Balancing

• Provide the same service from multiple instances and


use a load balancer to distribute requests across all
of the instances
• Considerations for selecting a load balancing
strategy:
• Hardware or software load balancers
• Load balancing algorithms (round robin)
• Load balancer stickiness

• Load balancing becomes critical even if you have a


single service instance as it offers the capability to
scale seamlessly

Load Balancing and Geographic Resiliency

• Load balancing can be combined with geographic


redundancy to help achieve high availability
• You can use a load balancer to direct client requests to their
closest data center
• Traffic Manager is often used for this task
• A load balancer can be used to implement a failover
scenario
• When a data center or compute instance is down, clients can be
directed to the next desirable instance that is available
• Designing a load balancing strategy along with
distributing your application across data centers is
key to high availability across the globe

21
4/27/18

Discussion

How could you implement the


cache-aside pattern using in-
memory caching mechanisms?

Lesson 5: Data Patterns

• Redis Cache
• Database Partitioning

22
4/27/18

Redis Cache

• Based on the open-source Redis platform


• Multiple tiers are available that offer different numbers of
nodes
• Supports transactions
• Supports message aggregation using a publish subscribe
model
• Considered a key-value store where the keys can be simple
or complex values
• Massive Redis ecosystem already exists with many different
clients

Database Partitioning

• Most cloud applications and services store and


retrieve data as part of their operations
• The design of the data stores that an application uses can
have a significant bearing on the performance, throughput,
and scalability of a system
• One technique that is commonly applied in large-
scale systems is to divide the data into separate
partitions
• Partitioning refers to the physical separation of data
for scale
• Modern data stores understand that data may be spread
across many different instances

23
4/27/18

Sharding Pattern

A map is implemented that contains lookup data mapped by


shard key. With a multi-tenant application, data using the same
shard key will be stored in an identical shard. In this example,
the tenant ID is the shard key.
1. An application instance will make a request to the map for
the shard which contains tenant #55. The map will return "Shard A".
2. The application instance will then make a request directly to the
database at "Shard A" for records related to tenant #55.
3. A new application instance will make a request to the map for
tenant #227 and will receive a response of "Shard C".
4. The new application instance makes a request directly to the
database at "Shard C" for records related to tenant #227.
5. As new tenants are added and more space is necessary, new shards
can be added to the map. Tenant IDs can than be associated with
the new shards.

Sharding Pattern - Lookup

24
4/27/18

Database Query Performance

• Database Administrators and Developers focus on


how data is stored not read
• Database systems are designed for fast data writes
• Reading data can require complex queries that uses
relationships between multiple collections
• RDBMS systems are notorious for having 3+ JOINs in a single
query
• NoSQL systems make require cross-partition querying to
find relevant data for a record

Materialized View Pattern

• Pre-built view of most queried data


• Automatically updated when source data is changed
• Functions as a specialized cache

25
4/27/18

Discussion

For a new application and database


design, how would you select a
partition key?

Module Review and Takeaways

• Review Questions

26
4/27/18

Module 2
Deploying Resources with
Azure Resource Manager

Module Overview

• ARM Templates
• Role-Based Access Control (RBAC)
• Resource Policies
• Security
• Building Blocks

27
4/27/18

Lesson 1: ARM Templates

• Azure Resource Manager


• JSON

Azure Resource Manager

28
4/27/18

Azure Resource Manager Templates

Provide a scalable, repeatable method for deploying


Azure resources

All resources in the ARM model are built using JSON


templates

https://github.com/Azure/azure-quickstart-templates

Deploying Resources

• PowerShell
• Cross Platform Command-Line Interface
• Client Libraries
• Visual Studio
• Portal template deployment

• All use the REST API: The REST API is available


here: https://docs.microsoft.com/rest/api/resources

29
4/27/18

Resource Group Deployment

JSON

What is JSON?

JavaScript Object Notification (JSON) is a method for


passing data and objects in a formatted style

Similar to XML but “lightweight”

30
4/27/18

What Is a JSON Template?

{
"$schema":
"http://schema.management.azure.com/schemas/2015-01-
01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
},
"resources": [
],
"outputs": {
}
}

Template Complexity

31
4/27/18

Template Driven Resources

Parameters

Resources Output

Variables

Template Driven Resources

32
4/27/18

Discussion

How could your organization best


author, deploy and work with JSON
templates?

Lesson 2: Role-Based Access Control (RBAC)

• Role-Based Access Control


• Role Assignment
• Resource Scope

33
4/27/18

Role-Based Access Control

Azure role-based access control allows


granular access by users, groups and
applications to resources

Available through Portal.azure.com, each


resource has an Access Control (IAM) blade

Roles

34
4/27/18

Role Assignment

• Users: From the same Azure AD and same subscription

• Groups: If a role is assigned to a group, a user receives the


rights of the role when added to the group. The user also
automatically loses access to the resource after getting
removed from the group

• Service principals: Services can be granted access to Azure


resources by assigning roles via the Azure module for
Windows PowerShell to the Azure AD service principal
representing that service

Resource Scope

35
4/27/18

Discussion

At what scope is it best to allocate


RBAC, resources, resource groups,
subscription? To whom should you
allocate? Users or User Groups?

Lesson 3: Resource Policies

• Azure Resource Policies


• Policy vs RBAC
• Built-In Policies
• Policy Definition
• Policy Assignment
• Policies for Naming Conventions

36
4/27/18

Azure Resource Policies

• Provides resource conventions in an organization and


consists of:
• policy definition - describe when and what action to take
• policy assignment - apply the policy definition to a scope

Policy vs RBAC

• RBAC controls user access (need RBAC to create


resources)
• Policies control resources (need RBAC to use policies)

The Contributor role cannot create or apply policies

Permissions
To define requires:
Microsoft.Authorization/policydefinitions/write

To apply requires:
Microsoft.Authorization/policyassignments/write

37
4/27/18

Built-In Policies

Azure provides built-in policy definition limiting the


number users need to define; some examples are:
• Allowed locations
• Allowed resource types
• Allowed storage account SKUs
• Allowed virtual machine SKUs
• Not allowed resource types

Definitions are stored in JSON

Policy Definition

How to define:
• Use All Mode
• Use Parameters
• Policy Rule contains simple if and then blocks

{
"if": {
<condition> | <logical operator>
},
"then": {
"effect": "deny | audit | append"
}
}

38
4/27/18

Policy Assignment

• Using PowerShell
• GUI through Azure Portal

Policy Assignment

• Using PowerShell:

$rg = Get-AzureRmResourceGroup -Name


“ContosoVMS"

$definition = Get-AzureRmPolicyDefinition -Id


/providers/Microsoft.Authorization/policyDefini
tions/a57364a-7474-ed43-c564-bf8b9038c4c

New-AzureRMPolicyAssignment -Name VM Sizes


Assignment -Scope $rg.ResourceId -
PolicyDefinition $definition

39
4/27/18

Policy Assignment

• Using the Portal

Policies for Naming Conventions

Prescribe how organization resources are named:


• Wildcard
• Pattern
• Tags
• Multiple patterns

40
4/27/18

Policies for Naming Conventions

• Pattern:

{
"if": {
"not": {
"field": "name",
"match": "contoso??????"
}
},
"then": {
"effect": "deny"
}
}

Discussion

In what situations could you use


Azure Policy and RBAC, Azure Policy
alone or RBAC alone?

41
4/27/18

Lesson 4: Security

• Azure Key Vault


• Key Vault Use in ARM Templates

Azure Key Vault

When deploying resources, often secrets are required.


These should not be passed but stored in the Azure
Key Vault.

42
4/27/18

Key Vault Use in Azure

• Application access without passing credentials

Key Vault Use in ARM Templates

Top Tip: set Key Vault enabledForTemplateDeployment property to true at creation.


This will permit access from Resource Manager templates during deployment.

43
4/27/18

Discussion

How could you use Key Vault to


deploy secrets along with your
ARM templates?

Lesson 5: Building Blocks

• Azure Building Blocks


• Deploying Resources using Building Blocks

44
4/27/18

Azure Building Blocks

Designed to simplify deployment of Azure resources

Provides a command line tool and set of Azure


Resource Manager templates

https://github.com/mspnp/template-building-blocks/

Supported Resources

Building Blocks support the following resource types:


• Virtual Networks
• Virtual Machines
• Virtual Machine Extensions
• Load Balancers
• Route Tables
• Network Security Groups
• Virtual Network Gateways
• Virtual Network Connection

45
4/27/18

Deploying Resources Using Building Blocks

• Creating a Parameters File

Deploying Resources Using Building Blocks

• Running a Parameters File

46
4/27/18

Deploying Resources Using Building Blocks

• Template Output

Discussion

When would you use the Azure


Building Blocks tooling instead of
manually authored ARM
Templates?

47
4/27/18

Lab: Getting Started with Azure Resource Manager

• Exercise 1: Creating Resource Groups


• Exercise 2: Deploying an Empty Template
• Exercise 3: Deploying a Simple Template
• Exercise 4: Cleanup Subscription

Logon Information
Virtual machine: 20535A-SEA-ARCH
User name: Admin
Password: Pa55w.rd

Estimated Time: 90 minutes

Lab Scenario

As part of your onboarding as a Fabrikam consultant,


your team has asked you to become familiar with the
Azure Resource Manager features and to deploy your
first ARM templates.

48
4/27/18

Lab Review

• In this lab, you created your first ARM templates and


resource groups. How could you use resource groups
to organize your Azure resources in your subscription?

Module Review and Takeaways

• Review Questions

49
4/27/18

50
4/27/18

Module 3
Building Azure IaaS-Based Server
Applications

Module Overview

• High Availability
• Templated Infrastructure
• Domain-Joined Virtual Machines

51
4/27/18

Lesson 1: High Availability

• Azure Availability
• Availability Sets
• Availability Zones

Azure Availability

• Azure provides money-backed SLAs for IaaS services:


• Two Instances or more in an Availability Set = 99.95%
• Single Instance VM using Premium Storage = 99.9%

• Decisions should based on cost and availability


requirements

52
4/27/18

Stand-Alone VMs

• Single instance VM would gain 99.9% SLA if it


complies with:
• Premium Storage for all Operating System Disks and Data
Disks

• Any single instance VM without Premium storage


receives no SLA

Availability Sets

• Availability Sets provide assurance that any multiple


instance VM will be available 99.95%
of the time

Availability Sets cater


for planned and
unplanned maintenance
using Update Domains
and Fault Domains

53
4/27/18

Availability Sets

When planning multiple tier applications use multiple


Availability sets, one per tier

Availability Zones

Service helps to protect resources from datacenter


level failures

54
4/27/18

Discussion

When would you use multiple


Azure Availability Sets in the same
application?

Lesson 2: Templated Infrastructure

• Templated Infrastructure
• Virtual Machine Scale Sets
• Virtual Machines vs. Virtual Machine Scale Sets
• Virtual Machine Scale Set Considerations

55
4/27/18

Templated Infrastructure

While ARM templates are


an excellent resource, for
large scale deployments,
other solutions are
available:

VM Scale Sets allow true auto


scaling to deploy big compute and
big data solutions

Virtual Machine Scale Sets





56
4/27/18

Virtual Machines vs. Virtual Machine Scale Sets

Scale Sets: VMs:


• Easy to grow and shrink on • Attach disks to VMs
demand • Attach non-empty disks
• Easy to reimage • Snapshot a VM
• Easy to overprovision • Capture a VM Image
• Upgrade policies • Migrate from native to managed
disks
• Assign IPv6 public IP addresses to
individual VM NICs

Virtual Machine Scale Sets

57
4/27/18

Virtual Machine Scale Set Considerations

Considerations for “Large” VMSS

• Managed Disks
• Marketplace images scale to 1,000 VMs
• Custom images scale to 300 VMs
• Ensure available IP addresses in subnet
• Ensure your compute limits are high enough
• Fault Domains relate to a single placement group

58
4/27/18

Discussion

What are the benefits and


drawbacks of VMSS for different
types of application and services?

Lesson 3: Domain-Joined Virtual Machines

• Domain and IaaS Applications


• Hybrid Connectivity
• Azure AD Domain Services

59
4/27/18

Domain and IaaS Applications

Azure provides a number of options for Domains:


• Azure AD (and B2B, B2C)
• Hybrid ADDS And Azure AD
• Azure AD Domain Services

Hybrid Connectivity

• Azure AD Connect
• Active Directory Federation Services
• AD Connect Passthrough
• Deploy AD DS to an Azure VM

60
4/27/18

Azure AD Hybrid

• Passthrough vs. ADFS

Azure AD Hybrid

• Passthrough vs. ADFS

61
4/27/18

Azure AD Domain Services

Azure AD Domain Services integrates previously


created Hybrid scenarios or works as a cloud only
solution. The benefits are:
• Simplicity – few clicks to setup
• Integrated – deep Azure AD integration
• Compatible – Windows Server AD
• Cost-effective – no infrastructure burden

Discussion

What business scenarios would


dictate the use of Azure AD Domain
Services over a DC box hosted on
Azure IaaS or existing metal?

62
4/27/18

Lab: Deploying Infrastructure Workloads to Azure

• Exercise 1: Deploying a Virtual Machine using


PowerShell DSC
• Exercise 2: Deploying a Virtual Machine Scale Set using
PowerShell DSC
• Exercise 3: Cleanup Subscription

Logon Information
Virtual machine: 20535A-SEA-ARCH
User name: Admin
Password: Pa55w.rd

Estimated Time: 60 minutes

Lab Scenario

One of your clients wants to build a web application


hosted on Internet Information Services (IIS) that will
scale in response to increases and decreases in usage.
The solution should minimize the amount of manual
setup and maintenance work necessary for each
virtual machine running IIS.

63
4/27/18

Lab Review

• If you are deploying a farm of web servers Virtual


Machines using ARM, when would you use VMSS as
opposed to traditional VMs?

Module Review and Takeaways

• Review Questions

64
4/27/18

65
4/27/18

Module 4
Creating Managed Server Applications
in Azure

Module Overview

• Infrastructure-Backed Platform-as-a-Service (PaaS)


• High-Performance Compute (HPC)
• Migration

66
4/27/18

Lesson 1: Infrastructure-Backed Platform-as-a-Service


(PaaS)
• Infrastructure Backed PaaS
• App Service Environments
• Azure Service Fabric
• Azure Container Service

Infrastructure Backed PaaS

Azure provides several services to cater for highly


scaled isolated applications:
• App Service Environments
• Azure Service Fabric
• Azure Container Service

67
4/27/18

App Service Environments

• Dedicated environment for high scale, secure Apps:


• Very high scale
• Isolation with secure network access
• High memory utilization
• Single or Multi region
• Deployed to a Virtual Network
• An ASE is dedicated exclusively to a single
subscription (Max 100 instances)

68
4/27/18

Azure Service Fabric

• Distributed systems platform to package, deploy and


manage microservices and containers
• Avoids complex infrastructure problems
• High density microservice applications running on a
shared cluster of machines
• Container deployment and orchestration
• Stateless and stateful services

Azure Service Fabric

69
4/27/18

Azure Container Service

Simple management of Cluster of VMs using either


Docker, Mesosphere DC/OS or Kubernetes

• Removes infrastructure complication and planning


• No cluster charges, just used resources
• Secure, reliable, highly scalable

Discussion

What are some of the benefits and


drawbacks of ASE, Service Fabric
and ACS?

70
4/27/18

Lesson 2: High-Performance Compute (HPC)

• High Performance Computing (HPC)


• Azure Batch
• Stateless Component Workloads

High Performance Computing (HPC)

HPC is commonly defined as the use of super


computers and parallel processing techniques for
solving complex computational problems

Azure provides solutions to manage this function:


• Custom Workloads on IaaS
• Azure Batch

71
4/27/18

Custom Workloads on IaaS

HPC Pack using Azure Virtual Machines:


• HPC Pack is Microsoft’s HPC cluster and job management
solution for Windows
• Uses Head Nodes and compute nodes – Suggested as A8
and A9 VM sizes
• HPC Pack can also be used in hybrid scenarios to “burst to
Azure” with A8 or A9 instances to obtain more processing
power

Custom Workloads on IaaS

HPC Pack can also


“burst to
Azure” with A8 or
A9 instances to
obtain more
processing power
through a hybrid
solution

72
4/27/18

Azure Batch

A free service designed for large data set manipulation


and transform workloads:
• Job Scheduling
• Compute resource management
• Large-scale parallel workloads
• Batch API to enable scaling to thousand of compute nodes

Azure Batch

73
4/27/18

Stateless Component Workloads

In addition there are several services that are the


foundations for HPC Solutions in Azure:
• Virtual Machines (VMs)
• VM Scale Sets
• Azure Container Services
• HDInsight
• Machine Learning

Discussion

How would you implement HPC


using large VM instances?

74
4/27/18

Discussion

How would you implement HPC


using VMSS with varying quantities
of VMs based on queue length?

Lesson 3: Migration

• Migration
• On-Premises Lift and Shift
• Migration from Classic IaaS
• Migration from Cloud Services

75
4/27/18

Migration

Several options exist when migrating workloads from


on-premises to Azure and from Azure IaaS
to PaaS:
• Migrate on-premises to Azure
• Migrate from IaaS to PaaS
• Migrate from Cloud Services to PaaS
• Native app or Migration

On-Premises Lift and Shift

First stage of a migration may be move the workload


direct to IaaS in Azure, having completed this the
potential is to follow the modernization maturity
model

76
4/27/18

On-Premises Migration

Two options:
• To IaaS – little code changes – need to manage the OS
• To PaaS – rewrite the code but no OS management required

In either case the data can be hosted in either IaaS or


PaaS SQL databases

Migration from Classic IaaS

For IaaS migration from Classic to Azure Resource


Manager the following can be migrated:

Virtual Machines Network Security Groups


Availability Sets Route Tables
Cloud Services Reserved IPs
Storage Accounts
Virtual Networks
VPN Gateways
Express Route Gateways

Check carefully for unsupported configurations, which could affect


successful migration

77
4/27/18

Migration from Cloud Services

Migrating Web and Worker roles in a cloud service to


Azure Fabric stateless services is the simplest method
of migration to Service Fabric

Discussion

What should you consider when


migrating on-premises container
applications to Azure?

78
4/27/18

Lab: Deploying Managed Server Workloads to Azure

• Exercise 1: Creating Azure Container Service Cluster


• Exercise 2: Docker Image
• Exercise 3: Cleanup Subscription

Logon Information
Virtual machine: 20535A-SEA-ARCH
User name: Admin
Password: Pa55w.rd

Estimated Time: 60 minutes

Lab Scenario

The latest client has internally developed their new


web application using Docker. The developers on the
team have created a Docker container image as their
build output and wished to host this image in Azure.

79
4/27/18

Lab Review

• What type of solutions would benefit from the use of


Azure Container Service and Docker containers?

Module Review and Takeaways

• Review Questions

80
4/27/18

81
4/27/18

Module 5
Authoring Serverless Applications in
Azure

Module Overview

• Azure Web Apps


• Azure Functions
• Integration
• High Performance Hosting
• Mobile Apps Case Study

82
4/27/18

Lesson 1: Azure Web Apps

• Web Apps
• API Apps

Web Apps

• Web Apps:
• Near instant deployment
• SSL and Custom Domain Names available in some tiers
• WebJobs provide background processing for independent
scaling
• Can Scale to larger machines without redeploying
applications
• Virtual Machines:
• Need Availability Sets or Load Balancers to prevent
simultaneous restarts for maintenance or hardware failures
• Additional machines needed for background processing

83
4/27/18

Web App Deployment

• Create Packages:
• Continuous Delivery with VSO or GitHub
• Can use Team Foundation Version Control (TFVC) or Git for
source control
• Deployment Slots:
• Can create slots such as: Staging, Production, Testing
• Web Deploy:
• Older IIS Extension method to Export and Import
• FTP Deployment

Web App for Containers

Deploy applications and solutions that are


containerized directly to App Service Web Apps
• Simplifies deployment
• Matches the already popular container workflow
using:
• CI/CD with Docker Hub, Azure Container Registry or GitHub
• Compatible with existing App Service Features:
• Auto-scale, Deployment Slots, etc.

84
4/27/18

Container Orchestration

Containers can be sourced from your existing


registries

• Docker Hub:
• Deploy images already shared on Docker Hub
• Deploy the most popular official images
• Private images are available on Docker Hub

• Azure Container Registry:


• Managed service for hosting Docker images
• Can deploy to Docker Swarm, Kubernetes or Web App for
Containers

API Apps

• Quickly implement
Custom APIs:
• Publish to External,
Partner and Internal
developers
• Extend Operations for
data and services:
• Each API can have 1 or
more operations
• API Apps can be
integrated into Logic
App workflows

85
4/27/18

Mobile Apps

Native SDKs Offline Sync

Office 365 Salesforce Dynamics

SQL Tables Mongo DB On-premises


REST API

X-Plat SDKs
Facebook Twitter Microsoft Google Azure
Azure Active
Active
Directory
Directory
Node.js
Express

.NET
Web API

iOS Android Windows Kindle In-app


OSX Chrome

Discussion

How would your team make the


decision to use public or private
Docker containers?

86
4/27/18

Lesson 2: Azure Functions

• Serverless Processing
• Event-Based Triggers

Serverless Processing

• Many compute tasks vary in their consumption:


• Sometimes the compute units are unused or unnecessary
• The compute must be around to respond to bursts in usage
• Traditional compute models use auto-scale and metrics to try
and better prioritize compute spend based on utilization:
• The compute spend never “exactly” matches the utilization

87
4/27/18

Serverless Services

Serverless refers to services that offer consumption-


based billing where you only
pay for services you consume

Azure Functions

• Azure Functions:
• Build on WebJobs Technology
• Available in Consumption and App Service Plan billing
modes
• Can be deployed using Scripts or Pre-Compiled
• Managed and Edited directly in the portal:
• Supports CI from GitHub or VSO if preferred

88
4/27/18

Azure Functions Features

• Functions support a wide variety of programming


languages:
• C#
• F#
• Node.js
• Java
• PHP
• Python
• Functions also support scripting languages:
• Bash
• PowerShell

Event-Based Triggers

• Azure Functions features no-code triggers that can


invoke a function based on changes in the following
services:
• Azure:
• Storage Blobs
• Cosmos DB
• Storage Tables
• Mobile Apps
• Office 365 Files
• Third-Party:
• Twilio
• SendGrid

89
4/27/18

Messaging Triggers

• Azure Functions can also integrate into existing


workflows that use Azure messaging services:
• Service Bus
• Event Grid
• Storage Queues
• Event Hubs

Discussion

What could you use a Bash or


PowerShell Azure Function for?

90
4/27/18

Lesson 3: Integration

• API Management
• Logic Apps

API Management

• API Proxy Service:


• Protect your API Endpoints
• Add billing, throttling, monitoring and policies to existing
APIs without changing their code
• Manipulate requests and responses in-flight to meet
business requirements

91
4/27/18

Logic Apps

• Automation workflow solution:


• No-code designer for rapid creation of integration solutions
• Pre-built templates to simplify getting started
• Out-of-box support for popular SaaS and on-premises
integrations
• BizTalk APIs available to advanced integration solutions
• JSON-based workflow definition:
• Can be deployed using ARM templates

Logic App Connectors

92
4/27/18

Logic App Components

• Workflow:
• The business process described as a series of steps
• Triggers:
• The step that invokes a new workflow instance
• Actions:
• A individual step in a workflow, typically a Connector or
custom API App
• Connectors:
• A special case of an API App that is pre-built and ready to
integrate with a specific service or data source:
• For example: Twitter and SQL Server Connectors

Example Logic App

93
4/27/18

API Apps in Logic Apps

Any API App can be used as a first-class Action in a


Logic App Workflow

Discussion

Are there any business reasons to


use API Management to
manipulate the response of your
REST API?

94
4/27/18

Lesson 4: High Performance Hosting

• Best Practices
• Basic Web Application
• Scaling
• Traffic Manager

Best Practices

• In a multi-tier model create each tier in a separate


App Service Tier
• Create Deployment slots in separate App Service
Tiers
• Make use of the VNET Integration with service end-
points to SQL Databases and Storage Accounts
• Distribute your application across Azure regions and
use Traffic Manager to manage it

95
4/27/18

Basic Web Application

Resource group

Azure Active App Service Plan Azure SQ L D atabase


D irectory

Authentication log ical server

App Service app database database

Scaling

• Horizontal scaling allows you to create a multi-


instance app
• Performance and reliability
• Have always at least 2 instances
• App Service can scale up to 100 Instances:
• Basic tier: up to 3 instances (only manual scaling)
• Standard tier: up to 10 instances
• Premium and PremiumV2 tier: up to 20 instances
• Isolated tier: up to 100 instances

96
4/27/18

Scaling

Azure Active Web front end D ata storage


D irectory
App Service App Service Plan
Authentication Plan
S QL D ocumentDB
Database

Redis cache
Web App API App Web Job

Azure
Search

logs Queue static


content
Content Delivery
N etwork (CD N )
Blob
S torage S torage S torage
account account account

Resource
group

Edge
servers

Traffic Manager

• Allows to control user traffic distribution


• Endpoints can include:
• App Services
• Cloud Services (Legacy)
• Other endpoints (even on-premises with internet
connection)
• Can be used in several modes:
• Failover
• Geography
• Distribution

97
4/27/18

Traffic Manager

Multi-Region Model

• Primary and secondary regions:


• Choose regions from the same regional pair (Example: East
US 2 and Central US)
• Azure Traffic Manager
• Geo-replication of data (SQL Database or/and
Cosmos DB)

98
4/27/18

Multi-Region Model

Active region

App Service App Service Plan


Plan

Azure Active
D irectory Redis cache
Web App API App Web Job

Az ure
Authentication S earch
Q ueue static
content CDN

Standby region
Traffic
Manager App Service
Plan
App Service Plan

Redis cache
Web App API App Web Job

Az ure
S earch
Q ueue

Discussion

How could you use Azure CDN and


Traffic Manager to create a “fast”
experience for global users of your
application?

99
4/27/18

Lesson 5: Mobile Apps Case Study

• Case Study Overview


• Case Study Solution

Case Study Overview

• Review the case study requirements


• Design a solution to the customer business problem
• Present your solution
• Respond to questions and objections
• Review a potential solution

100
4/27/18

Customer Business Problem

• Crazy Taxi Cab Company:


• Transportation Provider in Arizona:
• Premier provider of private, low-cost transportation
• Unique Pivot:
• Drivers are employees who work as a team
• Drivers are NOT independent contractors
• Reputation:
• Drivers provide a greater customer experience than traditional
independent contractors who are “racing to the finish line”

Customer Business Problem

• Crazy Taxi Cab Company:


• Grown Too Fast:
• Dispatchers’ ability to multitask is a bottleneck
• Dispatchers must reach drivers to pick up customers
• Dispatchers must field calls from perspective customers

• “While we function like a well-oiled machine, we’re still


running on 20th century equipment and we are already
seeing sign that this is eroding our advantage over the
competition…”
-Christopher Giovanni, COO

101
4/27/18

Customer Inventory

Web & Mobile Web & Mobile Web, Mobile &


Developer Developer .NET Developer

Customer Goals

Dispatch:

Location:

Tracking:

102
4/27/18

Customer Needs

• A back-end as a service that provides both data


storage, API support and push notifications
• A solution that is quick to implement with a small
team that has limited back-end development
experience and capacity
• A back end solution that can be implemented using
.NET
• A solution that is easy to monitor for health and
telemetry data

Customer Objections

• Doesn’t Azure Mobile Apps only work on Windows


devices?
• Our development team doesn’t know node.js. We
had heard mention of Mobile Services, but thought it
only supported JavaScript backends
• Our development team seems to think implementing
push notifications using Apple and Android apps
directly is easy, but we (as the executives) aren’t so
sure. How difficult can it be?
• Can’t we just build all of our backend using Azure
Web Apps?

103
4/27/18

Call to Action

Case Study Timing: 60 Minutes

Who are the business decision makers


and stakeholders?

What customer business needs do you


need to address with your solution?

Diagram your proposed solution

Case Study Solution

• Target Audience

• Solution Architecture

• Benefits

104
4/27/18

Target Audience

• Christopher Giovanni, COO


• Operations Management Team

Solution Architecture

Mobile Apps

Web Apps

105
4/27/18

Benefits

• Authentication:
• Drivers can use Microsoft, Facebook, Twitter or Google
credentials
• Notifications:
• Broadcast device notifications can be sent to each driver
• Offline Data:
• Data is cached in the app to work when devices are out of
signal range
• Custom Back-end Services:
• Custom REST APIs can be hosted by Mobile Apps using Web
API

106
4/27/18

Benefits

• Front-end Website:
• Web Apps can host the secure website for dispatchers
• Scaling:
• Autoscale can be configured to automatically adjust
instance count to match usage
• Back-end Jobs:
• WebJob processes data on a schedule and outputs to SQL
Database for reports

Lab: Deploying Serverless Workloads to Azure

• Exercise 1: Creating Web App


• Exercise 2: Deploying Web App Code
• Exercise 3: Deploying Function App and Code
• Exercise 4: Cleanup Subscription

Logon Information
Virtual machine: 20535A-SEA-ARCH
User name: Admin
Password: Pa55w.rd

Estimated Time: 90 minutes

107
4/27/18

Lab Scenario

A startup company has hired your organization to help


them take their web application, stored in a Git
repository, and deploy the application and its API to
Azure.

Lab Review

• How can you reuse Function App Code that you


deployed using ARM?

108
4/27/18

Module Review and Takeaways

• Review Question

Module 6
Backing Azure Solutions with
Azure Storage

109
4/27/18

Module Overview

• Pricing
• Blob Storage
• Files
• StorSimple

Lesson 1: Pricing

• Azure Storage
• Storage Account Security
• Storage Account Replication
• Storage Performance & Pricing

110
4/27/18

Azure Storage

Azure provides a variety of storage features

Storage, like other services is provided in differing


performance and cost levels. In addition, storage is
broken down into four discrete services provided
within Storage Accounts:
• Blobs
• Tables
• Queues
• Files

Azure Storage Accounts

Storage accounts are further split into General


Purpose and Blob Storage

Blob Storage (hot


General Purpose General Purpose
Type of Account and cool access
Standard Premium
tiers)

Blob, File, Queue


Services Supported services
Blob service Blob service

Block blobs, Page


Types of Blobs Block blobs and Append
blobs and Append Page blobs
supported blobs
blobs

111
4/27/18

Storage Account Security

Storage accounts can be secured by Azure AD or by


Shared Access Signatures:
• Azure AD RBAC controls management functions when
applied to a Storage Account
• Azure AD RBAC can be used to read data objects when
applied to storage account keys
• Shared Access Signatures and Stored Access Polices further
secures data objects to dates times and permissions
• Azure Storage can be accessed by any HTTP/HTTPS requests
and has multiple storage libraries for popular languages

Storage Account Replication

Storage account replication can be changed after


creation except for Zone Redundant Storage (ZRS)

Replication LRS ZRS GRS RA-GRS


Data stored in multiple datacenters No Yes Yes Yes
Data read from secondary & primary location No No No Yes
No of copies of data stored in separate nodes 3 3 6 6

Data transfer costs my be incurred if you change from Locally


redundant storage (LRS) to Geo redundant storage (GRS) - this
would be a one time cost

112
4/27/18

Storage Performance & Pricing

Premium Storage is:


• For page blobs and VM Disks
• Only available as a Locally Redundant storage account
• Only available for certain VM series

Discussion

When would storage performance


become an issue over storage
replication?

113
4/27/18

Lesson 2: Blob Storage

• Blob Storage
• Un-Managed Disks
• Managed Disks
• Deployment Considerations

Blob Storage

All VM Disks are stored within


the Azure Blob Service:
• Unmanaged disks require the
user to provision Storage
Accounts and manage throughput
• Managed disks allow Azure
to handle all storage and
provisioning jobs and IOPs
is not a consideration

114
4/27/18

Un-Managed Disks

• Require a storage account


• Management overhead
• Storage account IOPS limits
• Choose between Standard and Premium account at
creation

Un-Managed Disk Pricing

115
4/27/18

Managed Disks

• Standard and Premium disks at a disk level


• Azure handles storage account and limits
• Transaction billing (standard only)
• Snapshots
• Images

Managed Disk Pricing

116
4/27/18

Deployment Considerations

Managed disks removes complexity from multiple disk


VM deployments:
• Can deploy with templates
• Can manage with:
• PowerShell
• Azure CLI
• Portal
• Easy snapshot creation and management
• Rapid performance changes

Discussion

What are the benefits and


drawbacks of IaaS un-managed disk
deployments?

117
4/27/18

Discussion

What are the benefits and


drawbacks of IaaS managed disk
deployments?

Lesson 3: Files

• Azure Files
• Azure File Sync

118
4/27/18

Azure Files

An SMB 3.0 file service providing reliable network file


shares without infrastructure:
• File Shares
• File Sync
• IaaS File Shares

Sharing Files in Cloud Infrastructure

• Azure Files
• Azure IaaS VM File Share
• Azure File Sync for Hybrid and DR

119
4/27/18

Azure File Shares

Components

URL or server / application file share access

Azure File Sync

File Sync Service:


• NTFS volumes only
• Dedupe supported (not with Cloud Tiering)
• Cloud Tiering for cold files
• DR feature for failed servers

120
4/27/18

Azure IaaS File Sharing

Azure AD Domain Services integrates previously


created Hybrid scenarios or works as a cloud only
solution. The benefits are:
• Simplicity – few clicks to setup
• Integrated – deep Azure AD integration
• Compatible – Windows Server AD
• Cost-effective – no infrastructure burden

Discussion

When creating a file sharing


solution, how do you decide
between Azure Files, a IaaS File
Server or a third-party file sharing
solution?

121
4/27/18

Lesson 4: StorSimple

• StorSimple
• Data Tiering

StorSimple

Hybrid file storage solution:


• Cost saving solution
• Accelerate Disaster Recovery
• Automate Data Management

122
4/27/18

Architecture

Features

• Transparent integration – iSCSI protocol to invisibly


link data storage facilities
• Reduced storage costs – Allocates sufficient local or
cloud storage to meet demands, extends cloud
storage when necessary
• Simplified storage management – standard tools
• Improved disaster recovery and compliance –
Restores data as it is needed
• Data mobility – Can be accessed from other sites for
recovery and migration purposes

123
4/27/18

Data Tiering

Discussion

Your organization has a content-


management solution that stores
documents. How can you
implement StorSimple in your
organization to reduce costs and
improve document access
performance?

124
4/27/18

Lab: Deploying Azure Storage to Support Other


Workloads in Azure
• Exercise 1: Creating Required Resources for a Virtual
Machine
• Exercise 2: Creating a VM With a Storage Account
• Exercise 3: Creating a VM With a Managed Disk
• Exercise 4: Cleanup Subscription

Logon Information
Virtual machine: 20535A-SEA-ARCH
User name: Admin
Password: Pa55w.rd

Estimated Time: 60 minutes

Lab Scenario

An established business has hired your company to


help it transition from using Storage Accounts and
blobs to managed disks in Azure.

125
4/27/18

Lab Review

• If you are reusing VHDs across different VMs created


by various ARM templates, should you use managed
or un-managed disks?

Module Review and Takeaways

• Review Questions

126
4/27/18

127
4/27/18

Module 7
Comparing Database Options
in Azure

Module Overview

• Relational Databases
• NoSQL Services
• Azure Cosmos DB
• Data Storage & Integration
• Data Analysis
• Web Apps & SQL Case Study

128
4/27/18

Lesson 1: Relational Databases

• Azure SQL Database


• Third-Party Databases in Azure

Azure SQL Database

• SQL-as-a-Service Offering:
• Fully managed
• Automatically replicated
• Compatible with existing TDS-capable software:
• Visual Studio
• SQL Server Management Studio
• Entity Framework
• Managed using existing tools, the CLI, PowerShell or the
Portal
• Performance measured in a predictable manner:
• Database Throughput Units (DTUs)

129
4/27/18

Tiers

• • •
• • •
• • •
• • •
• •

Stretch Database

• Dynamically migrate transactional data from SQL


Server to Azure SQL Database:
• Keep cool data in lower-cost Azure storage
• Keep hot data “closer” to your users in local server storage
• Compatible with Always Encrypted and Row-Level Security

130
4/27/18

Elastic Scale

• Elastic Scale simplifies the scaling out (or in) of data


in Azure SQL Database
• Composed of two parts:
• An Elastic Scale library for client applications to configure
shards and access shards
• The Elastic Scale features in Azure SQL Database that
implements the any changes requested by your application

Third-Party Databases in Azure

• Azure Database for MySQL:


• MySQL Community Version
• phpMyAdmin Already Installed
• Azure Database for PostgreSQL:
• Supports PostgreSQL Extensions

131
4/27/18

Discussion

What strategy would you use to


stretch your existing SQL Server
data to Azure SQL Database?

Lesson 2: NoSQL Services

• Azure Storage
• Azure Search

132
4/27/18

Azure Storage

Service in Azure to store various media and files

Storage Architecture

REST REST REST REST SMB 2.1

Front End Layer

Partition Layer

Stream Layer

133
4/27/18

Storage Tables

• NoSQL database service using the key/value


(or Dictionary) design paradigm:
• Can be accessed using dedicated clients or OData protocol
• Built for massive scale

Migration to Cosmos DB

• All existing Storage Accounts are migrating to Cosmos


DB:
• Existing applications can migrate to Cosmos DB with no
code changes
• The Cosmos DB Tables API introduced new features
not found in standard Storage:
• Turnkey global distribution
• Dedicated throughput worldwide
• Single-digit millisecond latencies at the 99th percentile
• Guaranteed high availability
• Automatic secondary indexing

134
4/27/18

Azure Search

• Search-as-a-Service:
• Delegates server and infrastructure management to
Microsoft
• Immediately ready-to-use service that you populate with
search data, and then access from your application:
• Accessible via REST APIs or Client SDKs
• Standard search is fully scalable, with options to increase
storage or service replicas for handling larger query loads

Azure Search Indexers

• Search can index existing data stores including:


• CosmosDB DocumentDB API
• Azure SQL Database

135
4/27/18

Time Series Insights

• Manages IoT time-series data stored in the cloud:


• End-to-end service that manages:
• Analytics
• Storage
• Visualization
• Designed for terabytes of time-series data
• Integrates with IoT Hubs to ingest data

Lesson 3: Azure Cosmos DB

• Cosmos DB
• Consistency Levels

136
4/27/18

Cosmos DB

SQL

Azure Cosmos DB
Key-Value Column-family Documents Graph

Global distribution Elastic scale out Guaranteed low latency Five consistency models Comprehensive SLAs

Global Distribution

Mission-critical applications for a global userbase need …

Global distribution Fast, Responsive millisecond latency

Elasticity of compute and storage Durable, Consistent and Highly available

137
4/27/18

APIs

Column-family

Key-Value

Graph

Documents

SQL

Consistency Levels

• The consistency levels range from:


• Strong consistency where reads are guaranteed to be visible across replicas
before a write is fully committed across all replicas
• Eventual consistency where writes are readable immediately and replicas are
eventually consistent with the primary

138
4/27/18

Throughput

• Each collection is assigned a performance level and


that performance level dictates throughput for that
collection and its corresponding documents:
• If a particular collection is seeing spikes in throughput, you
can manage it’s performance level in isolation by increasing
or decreasing the performance level. This change to the
performance level of a particular collection will not cause
side effects for the other collections. This allows you to
adjust to meet the performance needs of any workload in
isolation.

Discussion

What are some business scenarios


where you would want to use more
than one Cosmos DB API?

139
4/27/18

Lesson 4: Data Storage & Integration

• Data Storage & Integration Options


• Data Integration

Data Storage & Integration Options

Ingest Store Prep & Train Model &


• Data • Big data store • Hadoop/Spark Serve
orchestration and and Machine • Data warehouse
monitoring Learning

140
4/27/18

SQL Data Warehouse

• Model data from various sources:


• Data can be ingested from sources including:
• IoT
• Devices
• CRM
• Graph Database
• APIs
• Other Databases

SQL Data Warehouse

• Query and serve data to your workloads and


applications:
• You can query data from Data Warehouse using PolyBase
• PolyBase uses T-SQL language

• Data stored on well-known platform:


• Data is stored, partitioned and managed using compute
infrastructure nodes backed by Azure Storage

141
4/27/18

SQL Data Warehouse

SQL Data Warehouse

Azure Storage

142
4/27/18

Data Lake

• Integrate a wide variety of data stores into existing


workloads and applications:
• Ideal for large volume scenarios that require high speed
analysis and access
• Store various business data stores in a centralized
location:
• Search and analyze the data using a single platform
a U-SQL

Data Lake

Analytics

U-SQL

YARN

WebHDFS

Unstructured Semi-Structured Structured

Storage

143
4/27/18

Data Lake

• New language designed specifically for Azure Data


Lake:
• Combines best parts of SQL and C#
• Can process any data type even without schemas
• Can be used to write expressive custom code using C#
syntax

Data Integration

Azure Data Factory:


• Compose data processing, storage and movement services
to create & manage analytics pipelines
• Originally focused on Azure & hybrid movement to/from on
premises SQL Server:
• Over time, expanded to more storage & processing systems
throughout
• End-to-end pipeline monitoring and management

144
4/27/18

Data Factory

Connect & Transform


Publish Monitor
Collect & Enrich

Data Factory

Connect &
Collect

Transform
& Enrich

Publish

Monitor

145
4/27/18

Data Factory

Azure Data Factory is an option to migrate data from


Azure Data Lake to Azure SQL Data Warehouse

Lesson 5: Data Analysis

• Data Analysis Options

146
4/27/18

Data Analysis Options

Analysis Services:
• Enterprise BI-as-a-Service
• Increases efficiency of queries:
• Complex raw data is optimized “behind the scenes” for search and
processing
• DirectQuery-caliber speeds are achievable on many data sources
• Easier for users to surface data:
• Data is surfaced in user-friendly business models
• Users can use well-known tools, like Excel or Power BI, to query the
models

Analysis Services

SQL Database

Security
SQL Data Warehouse In-Memory
Cache Power BI
Data Lake Data
Modeling
HDInsight/Spark

Other
Azure Analysis Services Third-Party

Power BI Desktop

Lifecycle
Business Logic
SQL Server/Oracle Management Excel
& Metrics
Other

147
4/27/18

HDInsight

• Three Core Focuses:


• Elastic:
• Store any type of data you like (structured or unstructured)
• Store when or how you like
• Simple:
• You should be able to create a Hadoop cluster in three minutes
• Secure:
• Instances are isolated by default
• Built-on top of Azure Storage and Azure IaaS which both have well-
known security best practices

HDInsight

• Build your solutions on Hadoop:


• Storage + proliferation of compute models for data
processing at scale
• Began life as an open source implementation of Google’s
Map/Reduce and GFS papers
• In use at many major web companies at massive scale
(1000s of node, PB’s of storage)

148
4/27/18

HDInsight

Big data processing on top of Azure Storage

Shape

Experiment Query

HDInsight

Authoring Jobs App Integration


Extend breadth & depth
Enable new scenarios
End User Tooling (IDE’s, Analyst tools, Command lines)
Integrate with current tool chains

Lightweight
Low cost to extend Breadth of Clients (Java, JS, .NET, etc.)
Scenario oriented
Connectivity
Programmability
Consistent REST API’s Security
Loosely coupled

Innovation flows upward


New compute models Authoring frameworks and languages
Perf enhancements Core Hadoop

149
4/27/18

HDInsight

• Sentiment Analysis
• Clickstream Processing
• Machine/Sensor
• Server Logs
• Geo-Location

Finance Telco Retail Manufacturing Transportation Web

Prevent • Securities • Security --- • Machine • Driver • Application


fraud breaches failures & fleet failures
• Compliance • Network issues • Operational
Monitor violations outages issues
real-time
data to…

• Order • Bandwidth • Offers • Routes


Optimize • Site content
routing allocation • Pricing • Supply • Pricing
• Pricing • Customer chain
service

HDInsight

You can use Azure Data Lake with an Azure HDInsight


cluster

Azure HDInsight

Hadoop WebHDFS Client

WebHDFS-compatible REST API

Azure Data Lake Store

150
4/27/18

Data Catalog

• Surfaces enterprise data for workloads and


ad-hoc queries:
• Employees can find data sets that are normally difficult to
find
• Data Assets can be shared among enterprise applications
• Control and delegate access to data assets
• Integrate data assets into existing processes using
REST APIs

Data Catalog

• Search
Understand • Data Assets
Contribute
• Browse • Metadata • Familiar Tools • Tag
• Filter • Experts • Existing • Document
• Context Processes • Publish

Discover Consume

151
4/27/18

Lesson 6: Web Apps & SQL Case Study

• Case Study Overview


• Case Study Solution

Case Study Overview

• Review the case study requirements


• Design a solution to the customer business problem
• Present your solution
• Respond to questions and objections
• Review a potential solution

152
4/27/18

Customer Business Problem

• Boutique Manufacturer of high-quality bicycles and


parts:
• One of the world’s largest makers of premium race and
commuter bicycles
• Mission:
• To passionately pursue advanced, innovative technologies
that help cyclists of all abilities find more enjoyment in the
sport
• Manufactures and Sales:
• Bicycles
• Bicycle Parts
• Bicycle Accessories

Customer Business Problem

• Data-center:
• As of 2009, most IT infrastructure is located in an on-site
datacenter (Provo, Utah)
• Also has servers hosted in a third-party collocated
datacenter
• Costs USD $30,000 - $40,000 per month
• Servers located throughout the United States

• Capacity:
• Most applications and databases run on underutilized
hardware
• Data scattered throughout multiple geographic locations

153
4/27/18

Customer Business Problem

• Other Brand:
• Tailspin
• Goals:
• Upcoming hardware refresh = significant capital
expenditure
• Looking to eliminated the costly and risky refresh cycles

Customer Inventory

• Existing Applications

Product Catalog Inventory Departmental

154
4/27/18

Customer Goals

Migration:

Performance:

Ease of Use:

Customer Needs

• Improve the performance for customers and resellers


accessing its websites around the world
• Support for easily provisioning resources to meet bursts of
demand
• Consolidate and improve the utilization of website and
database hosting resources
• For their most demanding databases, they want to migrate
without having to re-architect their database structure or
make large changes the application
• Avoid downtime, particularly that caused by web and
database server patching
• Leverage familiarity with Microsoft tools

155
4/27/18

Customer Objections

• Scale & Performance:


• I do not want to have to make code changes (or re-deploy) in order to
change the scale of a website
• I hear Azure Web Sites is only useful for websites with small amounts of
traffic; will it really support the heavy traffic we receive?
• We would prefer to avoid performing a database migration (e.g., to
another server) in order to scale the throughput of our database
• We have heard SQL Database does not provide consistent performance,
is this true?

Customer Objections

• Business Continuity:
• How can we certain our data will survive in the event of a catastrophe in
a certain part of the world?
• We need to be able to recover from mistakes made by administrators
that accidentally delete production data (we know they happen, we
would love an “undo”)
• Do we need to have multiple web server instances for each property to
have a high SLA?

156
4/27/18

Customer Objections

• Tool Familiarity:
• Will we need to learn new tools to develop for Azure Websites and SQL
Database?
• What about diagnosing problems? Are there new tools we need
purchase and learn?

• Connectivity:
• Some of our enterprise web services need to access data and other
services located on-premises, is this supported?
• How can we ensure we are delivering the lowest latency possible to our
website visitors?
• We need to ensure that if we have multiple web servers backing a given
website, that no one web server gets all the traffic

Customer Objections

• Management:
• We would prefer not to have to manage patching of web servers and
databases
• With all of our websites and databases around the world, how do we
keep tabs on which is up and which is down and which is struggling?
• We need a simple solution to schedule and automate backup of the
website and database

157
4/27/18

Customer Objections

• Security:
• Is it possible to allow our visitors to use a mix of legacy and modern
browsers and still provide for secure transactions?
• What does Azure offer to help us with auditing access to our web
servers and databases?
• Our staff is accustomed to accustomed to a single sign-on experience —
will this still be possible?

Call to Action

Case Study Timing: 60 Minutes

Who are the business decision makers


and stakeholders?

What customer business needs do you


need to address with your solution?

Diagram your proposed solution

158
4/27/18

Case Study Solution

• Target Audience

• Potential Solution

• Benefits

• Customer Quote

Target Audience

• Hayley Leigh, Manager of Solution Development


• Business Decision Makers
• Technical Decision Makers
• Manager of Solution Development
• Infrastructure Managers
• Application Sponsors
• Business Unit IT
• Developers

159
4/27/18

Potential Solution

Web Application Hosting

Potential Solution

Database Queries & Sync

160
4/27/18

Benefits

By integrating Microsoft cloud solutions into


its datacenter strategy, Adventure Works Cycles has
been able to reduce IT costs by more than $300,000 a
year while gaining greater datacenter scalability and
datacenter agility. The IT team can respond to server
requests in hours instead of months and “turn off”
servers when they are no longer needed.

Benefits

With servers running in Microsoft datacenters around


the world, Adventure Works can provide better
application performance and availability to offices and
customers that are located far from the company’s
California datacenter

161
4/27/18

Benefits

• Scalability & Agility:


• Can respond to requests and implement features much
quicker than in the past
• Easy to setup infrastructure in other geographies
• Performance & Availability:
• Avoid maintenance-related downtime
• Update regional servers in isolation from other global
locations/offices

Customer Quote

“Previously, when our web traffic spiked, we


experienced slowdowns. With web properties running
in Microsoft Azure, we can scale our infrastructure
quickly and proactively and improve customer
experiences.”

Hayley Leigh
Manager of Solution Development
Adventure Works Cycles

162
4/27/18

Lab: Deploying Database Instances in Azure

• Exercise 1: Deploying a CosmosDB Database Instance


• Exercise 2: Validating the REST API
• Exercise 3: Cleanup Subscription

Logon Information
Virtual machine: 20535A-SEA-ARCH
User name: Admin
Password: Pa55w.rd

Estimated Time: 90 minutes

Lab Scenario

A local app development firm has hired your company


to show them how to automate the deployment of
their API and databases. The team has used Cosmos
DB for a few months to power their mobile
application’s REST API. They have a test team, located
on another continent, that will need the ability to
deploy instances of their backend API application and
database multiple times a month to perform
integration and regressions testing.

163
4/27/18

Lab Review

• How could you use multiple Cosmos DB APIs with the


same database?

Module Review and Takeaways

• Review Question

164
4/27/18

Module 8
Networking Azure Application
Components

Module Overview

• Virtual Networks
• Load Balancing
• External Connectivity
• Secure Connectivity
• Networking Case Study

165
4/27/18

Lesson 1: Virtual Networks

• Azure Virtual Network (VNET) Architecture


• Multi-Region Virtual Network Architecture
• VNETs & Subnets

Azure Virtual Network (VNET) Architecture

166
4/27/18

Multi-Region Virtual Network Architecture

Multi-Region Virtual Network Architecture

• Traffic Manager provides DNS based traffic


distribution & failover across Azure Regions
• IAAS & PAAS VNet inter-communication
• Isolate VM workloads in SubNets/Vnet
• ExpressRoute and/or S2S VPN for CorpNet
connectivity or Azure-to-Azure Region traffic
• NSGs secure the in/outgoing traffic on VNet or NIC
level

167
4/27/18

VNETs & Subnets

• Networking Topology:
• Define 1 or more VNets within an Azure Region, and configure an address
space for each
• Define 1 or more SubNets within a VNet, and configure address space
within the VNet range
• VNets and SubNets are using CIDR notation (x.x.x.x/24, x.x.x.x/16,…)
• Configure Network Security Group settings on VNet level
• Attach a NIC to a SubNet

• SubNet IP Addressing:
• IP-address gets allocated to a NIC during provisioning of the NIC
• First available IP-address in a SubNet range is x.x.x.4
• Azure SubNets support dynamic (=default) and static IP addressing

Public & Private IP-addressing

• Public IP-addressing:
• Used for all public internet-facing
communication
• Required parameter when creating
a VM from the portal

• Private IP-addressing:
• Used for all inter-VNet
communication
• Used for all communication
between an Azure VNet and
an on-premises VNet

168
4/27/18

Azure DNS Resolving

• DNS Server settings are configured on VNET level


• Use Azure DNS (Default)
• Or use your custom DNS configuration:
• Azure DNS Appliance (from Azure MarketPlace)
• Azure VM (e.g. Windows ADDS with DNS)
• On-premises DNS solution (requires connectivity)

• Public DNS names (available for


VMs and App Services) must be
unique across Azure regions:
<host.region.cloudapp.azure.com>

Discussion

What business scenarios would


dictate the use of a multi-region
VNET architecture?

169
4/27/18

Lesson 2: Load Balancing

• Load Balancing Solutions


• Azure Load Balancer
• Azure Application Gateway
• Azure Load Balancing Marketplace Appliances
• Azure Traffic Manager

Load Balancing Solutions

• Azure Load Balancer (layer 4)

• Azure Application Gateway (layer 7)

• Azure MarketPlace Load Balancing Appliance (layer 7)

• Azure Traffic Manager (DNS-based)

170
4/27/18

Azure Load Balancer

• Load balancer with a Public IP-address, sending


traffic along to the back-end pool servers
• TCP, UDP traffic
• Azure Platform management
• Support for Availability Sets

10X

Load Balancer Basic

Load Balancer Basic can be used for most load


balancing scenarios:
Basic

Up to 100 backend
instances
Non-zonal frontend

Availability Set (single)

Basic NAT and Probe


health status
-

NSG optional

Free

171
4/27/18

Load Balancer Standard

You can use Load Balancer Standard for TCP & UDP
scenarios with:
• Larger scale Standard
• Greater flexibility Up to 1000 backend
• HA Ports instances
Zone-redundant frontend
• New metrics
Zonal frontend
• Availability zones Availability Sets not
required and Availability
Zones
Integrated Frontend and
Backend health metrics
Supports HA Ports

NSG required

Internal Load Balancer

• Load balancer with a Private IP-address, sending traffic


along to the back-end pool servers
• TCP, UDP traffic
• Azure Platform management

An Azure Load Balancer cannot both be external and internal

172
4/27/18

Azure Application Gateway

URL-based Routing

173
4/27/18

SSL Termination

Internet (www.domain1.com)
(www.domain2.com)

Application GW

http://www.domain1.com https://www.domain2.com

http://www.domain2.com =>
https://www.domain2.com

Web Application Firewall (WAF)

XSS Attack Valid Requests


SQL Injection

174
4/27/18

Web Application Firewall

Pre-configured WAF Rules

Azure Load Balancing Marketplace Appliances

• Preconfigured vendor VM appliances, supported by


Azure
• BYOL or Pay-per-use
• Can be an alternative for
Azure Platform provided
options

175
4/27/18

Azure Traffic Manager

Global Resiliency and 4 Load Balancing options:


Performance, based • Priority
on DNS • Weighted Round Robin
• Geographical
• Performance

Lesson 3: External Connectivity

• On-Premises to Azure Connectivity


• VNET Peering
• Multi-Region VPN Connectivity

176
4/27/18

On-Premises to Azure Connectivity

Connectivity Options

Connectivity Benefits
ExpressRoute • ExpressRoute as primary cross-premises connectivity
• Multiple circuits for redundancy & better routing
• ExpressRoute-VPN co-existence for highly available,
redundant paths

Site-to-Site VPN • S2S VPN over Internet for remote branch locations
• BGP & active-active configuration for HA and transit

Point-to-Site VPN • P2S VPN for mobile users & developers to connect from
anywhere with macOS & Windows
• AD/radius authentication for enterprise grade security

177
4/27/18

High-Performance VPN Gateway SKUs

Scenarios:
• High throughput, hybrid workload over VPN tunnels
• Failover from ExpressRoute circuits to S2S VPN tunnels
• P2S for dev/test connectivity from anywhere

SKU Workload Throughput S2S/V2V P2S SLA


VpnGw1 Production 650 Mbps Max. 30 128 99.95%
VpnGw2 Production 1 Gbps Max. 30 128 99.95%
VpnGw3 Production 1.25 Gbps Max. 30 128 99.95%
Basic Dev/Test 100 Mbps Max. 10 128 99.9%

VNET Peering

• VNET Peering allows you to interconnect 2 Azure


VNETs, as if they are 1 large VNET

• VNET Peering is possible within the same Azure


region, or across Azure regions (using MS Backbone,
no public internet)

• VNET Peering is supported to interconnect an Azure


Classic VNET with an ARM VNET (e.g., for migrating
workloads)

178
4/27/18

VNet Peering

Multi-Region VPN Connectivity

• Before Vnet Peering, the only possible way to


interconnect 2 Azure Regions, was Site-to-Site VPN
Gateway tunneling

• This is still a valid option, if your traffic between both


Azure regions must be encrypted (outside of the
already encrypted Microsoft Backbone, no public
internet)

179
4/27/18

Multi-Region VPN Connectivity

Forced Tunneling

• Challenges:
• IaaS services accessible through internet
• Customers may require their VMs to be only accessed from
on-premises VNET

• Solution—Forced Tunneling:
• IaaS services only accessible from a VNET
• Site-to-Site VPN
• Or ExpressRoute

180
4/27/18

Forced Tunneling

Securing Access to PaaS Services

• Challenges:
• PaaS services accessible through internet
• Customers may require their services endpoints
to be only accessed from their VNETs

• Solution—VNEt Service Endpoints:


• PaaS services only accessible from a VNET
• Available now for Storage and SQL DB
• Will roll out to other PaaS services in the future

181
4/27/18

Securing Access to PaaS Services

Discussion

Why would you want to secure a


PaaS service using VNET Service
Endpoints?

182
4/27/18

Lesson 4: Secure Connectivity

• Network Security Groups

Network Security Groups

• A network security group (NSG) is a top level object


that is associated to your subscription:
• It can be used to control traffic to one or more virtual
machine (VM) instances in your virtual network
• An NSG contains access control rules that allow or deny
traffic to VM instances
• The rules of an NSG can be changed at any time, and
changes are applied to all associated instances

183
4/27/18

Default Inbound Rules

SOURC DESTINATION
NAME PRIORITY SOURCE IP DESTINATION IP PROTOCOL ACCESS
E PORT PORT

ALLOW VNET VIRTUAL_ VIRTUAL_


INBOUND 65000 * * * ALLOW
NETWORK NETWORK

ALLOW AZURE
LOAD AZURE_
BALANCER 65001 * * * * ALLOW
LOADBALANCER
INBOUND

DENY ALL
INBOUND 65500 * * * * * DENY

Default Outbound Rules

DESTINATION
NAME PRIORITY SOURCE IP SOURCE PORT DESTINATION IP PROTOCOL ACCESS
PORT

ALLOW VNET VIRTUAL_ VIRTUAL_


OUTBOUND 65000 * * * ALLOW
NETWORK NETWORK

ALLOW
INTERNET 65001 * * INTERNET * * ALLOW
OUTBOUND

DENY ALL
OUTBOUND 65500 * * * * * DENY

184
4/27/18

Lesson 5: Networking Case Study

• Case Study Overview


• Case Study Solution

Case Study Overview

• Review the case study requirements


• Design a solution to the customer business problem
• Present your solution
• Respond to questions and objections
• Review a potential solution

185
4/27/18

Customer Business Problem

• Fabrikam Residences:
• National Real Estate Services Group:
• Rapid growth slowed by expensive and unresponsive datacenter
infrastructure
• “We are a national real estate firm, we want to make
investments that support our core business, and buying and
managing servers is not our core business.”
-Craig Jones, CIO
• DOS Strategy:
• “Don’t own stuff”
• Focus on investments directly relevant to core business
• Avoid being an asset-intensive organization

Customer Inventory

• Existing Applications

Misc. servers
California Virginia
distributed
Datacenter Datacenter
geographically

250 servers 110 servers


Est. hundreds

186
4/27/18

Customer Goals

Migration:

CRM:

Scale to usage:

Customer Needs

• Reduce on-premise server presence through public


cloud consolidation
• Migrated servers should be located in closest Azure region
• Maintain security and privacy of their infrastructure connect-
throughs
• CRM application must be able to scale to meet peak demand
while not being overprovisioned during liger usage periods
• The CRM application must be highly available and only
accessible from the corporate intranet

187
4/27/18

Customer Objections

• We have a national business and we need connectivity that


can accommodate connectivity from coast-to-coast
• Our workloads are very seasonal. I do not want to pay for
more resources than I need
• The data that crosses our network is very confidential. Is Azure
secure?
• I need to deploy an intranet-based solutions and I have heard
that Azure requires an on-premises load-balancer for internal
facing workloads

Customer Objections

• I have heard that the public IP address of an Azure


deployment can change and break my application
• My workloads require static IP addresses. I have heard Azure
does not support this scenario
• I have some workloads that require multiple network
interfaces on my virtual machines
• Some deployments require the segmenting of network traffic.
Does Azure support this?

188
4/27/18

Call to Action

Case Study Timing: 60 Minutes

Who are the business decision makers


and stakeholders?

What customer business needs do you


need to address with your solution?

Diagram your proposed solution

Case Study Solution

• Target Audience

• Potential Solution

• Benefits

• Customer Quote

189
4/27/18

Target Audience

• Craig Jones, CIO


• IT Directory
• Network Administrator
• Security Lead
• Business Decision Makers
• Technology Decision Makers

Potential Solution

Deploying ExpressRoute Over an Existing MPLS


Network

190
4/27/18

Potential Solution

Active Directory Deployment

Potential Solution

CRM Webservers deployment

191
4/27/18

Benefits

• Security/Privacy:
• Request are isolated using industry-standard VLANs.
Requests do not traverse the public internet
• Network Performance:
• Predictable network performance since you are not
competing with other traffic
• Public Peering:
• Private connectivity between Azure services
• Cross-Region Connectivity:
• Connect multiple virtual networks using the same
ExpressRoute circuit

Benefits

• Network Service Provider Model:


• Existing provider is responsible for establishing connectivity
to Azure

192
4/27/18

Customer Quote

“With our Microsoft cloud infrastructure, we have


reduced our IT costs by 75 percent and gotten out of
the business of owning so much IT stuff”

Craig Jones, CIO


Fabrikam Residences

Lab: Deploying Network Components for Use in


Azure Solutions
• Exercise 1: Creating an ARM Template for a
Linux VM
• Exercise 2: Duplicating the VM Resources
• Exercise 3: Creating a Load Balancer Resource
• Exercise 4: Cleanup Subscription

Logon Information
Virtual machine: 20535A-SEA-ARCH
User name: Admin
Password: Pa55w.rd

Estimated Time: 60 minutes

193
4/27/18

Lab Scenario

Your newest client has purchased a third-party web


application that they have been custom branded and is
ready for deployment. The web application is hosted
on Linux servers using nginx as the web server. The
CTO of your client’s company has specifically
requested that the solution is triple-redundant at a
minimum with three load-balanced machines hosting
the web application.

Lab Review

• When you create a load balancer, how should you


probe your servers to ensure they are available?

194
4/27/18

Module Review and Takeaways

• Review Questions

195
4/27/18

Module 9
Managing Security & Identity
for Azure Solutions

Module Overview

• Security
• Identity

196
4/27/18

Lesson 1: Security

• Platform Security
• Securing the Azure Platform

Platform Security

Security Responsibility is SHARED

Microsoft Azure is built with Customers own responsibility


end-to-end security in mind, of their subscription
besides trust. Microsoft gives governance, data, identities,
you a secure foundation, as and how to protect those. In
well as the tooling to control IAAS, customer owns more
your environment control than in PAAS or SAAS

197
4/27/18

Public Cloud Security

Customer’s PAAS Workloads


(Azure Apps, ML, SQL,
Functions, Data,…)

Customer’s IAAS Workloads


(Networking, Storage, VMs,
Backup, ASR,…)

Physical Hypervisor layer

Physical Datacenter Access


Security (=the buildings)

Cloud Platform Security

- Each Azure Tenant is isolated from all


other tenants, using “Azure Fabric
Controller”

- Azure datacenters are mainly based


on Microsoft proprietary hardware,
running an Azure-host-specific version
of Hyper-V

Customer A Customer B

198
4/27/18

Platform Encryption Scenarios

Microsoft Azure

Admin operational sessions to the Azure platform are encrypted

All communication between Azure Regions is encrypted

Hybrid connectivity (Site-to-Site VPN or ExpressRoute) is encrypted


Microsoft Azure

Data Encryption Scenarios

On top of the Azure Platform provided encryption,


customers have additional levels of encryption
techniques available, depending on the Azure
Resource or Data type:
• Offline disk shipment: only encrypted disks are accepted
• SQL Azure provides database content encryption
(SQL DB TDE)
• Azure Storage Accounts can be encrypted
• Azure VMs (guests) allow Bitlocker encryption
• Azure Backup, Azure Recovery Vault data is stored
encrypted
• Archive Data (data at rest) allows encryption

199
4/27/18

Compliance & Certifications

Security Privacy Compliance Transparency


• Services are built • Policies and • Industry- • Azure
from the ground up, processes help verified practices,
to help safeguard keep customer compliance policies and
customer data data private conformity and guidelines are
and in their certifications public, clear
control and accessible

Securing the Azure Platform

• Azure Subscription Governance


• Limit Admin Access using RBAC (Role Based Access Control)
• Limit VM Admin Access using JIT (Just in Time) Access
• Enable (force) Multi-factor Authentication for
Azure Admin Accounts
• Customize RBAC roles where needed for your
organizational compliance

200
4/27/18

Securing the Azure Platform

• Azure Storage Accounts:


• Enable Storage Account Encryption:
• using your encryption keys
• Access Keys:
• key1/key2 -> regenerate periodically
• Shared Access Signatures (SAS) to narrow
Application service access to the storage object
and data
• Storage Access Policies:
• timestamp
• permissions

Securing the Azure Platform

• Azure SQL-as-a-Service:
• Apply RBAC to limit SQL Resources Admin-level access
• Be cautious with the “Allow Azure Services” access
• Features:
• SQL Database Encryption At Rest (TDE)
• SQL Database Encryption In Transit
• SQL Auditing & Threat Detection
• SQL Dynamic Data Masking
• SQL Row Level Security
• SQL Vulnerability Assessment

201
4/27/18

Securing the Azure Platform

• Azure Networking:
• Isolate VM traffic by deploying multiple VNETs and
separate Subnets within
• Use Network Security Groups to limit traffic allow/deny
• Integrate Forced Tunneling, User Defined Routing to control
traffic outside from the default Azure Routes
• Explore Azure Marketplace Virtual Appliances:
• Load Balancers
• Firewalls

Securing the Azure Platform

• Azure Key Vault:


• Security Keys are stored in a vault and invoked by URI when
needed
• Keys are safeguarded by Azure, using industry-standard
algorithms, key lengths, and hardware security modules
(HSMs)
• Keys are processed in HSMs that reside in the same Azure
datacenters as the applications.

202
4/27/18

Azure Key Vault

Developer accesses
Microsoft the Keys using URI
Azure request

Azure Admin Keys are securely Security Officer


manages Azure stored in Azure Key inspects where Keys
Key Vault and Vault are being used
generates keys

Lesson 2: Identity

• Azure Active Directory


• Azure AD Authentication Strategies
• Azure AD B2B & B2C
• Azure AD Identity Protection
• Azure AD Domain Services

203
4/27/18

Azure Active Directory

Long gone is the time


where all/most
applications were
“only” AD integrated

Microsoft Azure
Active Directory

=>

Windows Server
Active Directory User identities from
multiple repositories Hybrid
identity
LDAP v3
LOB App On-Premises
Servers ADDS
Windows Windows Server
LOB Web File Shares,
PowerShell Active Directory
Servers Printers,…
Web services
(SOAP, Java, Generic SQL
REST) via ODBC

Cloud Authentication

The “cloud way of authenticating:

1. Azure ADConnect using Password Hash Sync

2. Azure ADConnect using Federation (ADFS)

3. Azure ADConnect using Azure AD Passthrough

Authentication Agent

204
4/27/18

Single Sign-On

3rd Party
Web Applications
Cloud Applications
Intranet Windows 10
Azure AD Applications Desktop

Implement
SSO
Everywhere

Directory Sync
Mobile
IoT Devices
Applications
STS (Trust)

Security Token

Azure AD Application Proxy

Microsoft Azure Application


Active Directory Proxy
https://appX-mydomain.msappproxy.net/
connecto
r

Azure or 3rd Party IaaS

DMZ

connector connector connector

app app app app

205
4/27/18

Azure AD Authentication Strategies

• In any of the “cloud” scenarios, AD Connect


User/Group object sync is required
• Replaces legacy tools:
• DirSync, ADSync, FIM with AD Connector
• Benefits:
• Allows for write-back (passwords, devices, groups) to on-
premises AD
• Built-in deployment wizard for on-premises ADFS
infrastructure
• Azure AD Connect Synchronization Services dashboard
• Managed user sign-in options

Azure AD Connect

1st option: Identity + Password (Hash) synchronization

Microsoft Azure
Active Directory

User Identity +
Password Hash synchronization

Azure Active Directory


authenticates user

206
4/27/18

Azure AD Connect

2nd option: Identity synchronization + ADFS

Microsoft Azure
Active Directory

User Identity
synchronization

Authentication passed to ADFS


Windows Server Active Directory
via ADFS

Azure AD Connect

New option: Identity synchronization + Pass-through authentication with Seamless SSO

Microsoft Azure
Active Directory

User Identity
synchronization

Pass-through Seamless
authentication SSO
Authentication passed to
Windows Server Active Directory
via Pass-through authentication

Pass-through authentication
agent

207
4/27/18

Azure AD Connect

Seamless SSO is now enabled for the 1st option, too: Identity + Password (Hash) synchronization

Microsoft Azure
Active Directory

User Identity +
Password Hash synchronization

Azure Active Directory Seamless


authenticates user SSO

Azure AD Connect

More options than ever!

Identity Synchronization + Identity Identity Synchronization +


Password Hash Synchronization+ Synchronization Pass-through Authentication +
Seamless SSO + ADFS Seamless SSO
Microsoft Azure
Active Directory

User

Seamless Identity
ADFS Seamless synchronization
SSO SSO

Identity + Identity Pass-through


Password Hash synchronization Authentication
synchronization

208
4/27/18

Hybrid Identity

• Consolidated deployment
assistant for your identity
bridge components
• All currently available sync
engines will be replaced by the Azure Active
DirSync Directory Connect
sync engine included in the
Connect tool Azure Active Directory
Sync Sync engine
• Assisted deployment of ADFS
will be available through Azure
FIM+Azure Active
Directory Connector

Active Directory Connect ADFS


• ADFS is an optional component
for authentication in hybrid ADFS
implementation. Password sync
can replace ADFS for more
scenarios

Azure AD B2B & B2C

• B2B (Business to Business):


• Collaborate between organizations
• Avoid federation and extra servers
• B2C (Business to Customer):
• Use their existing identities
• Avoid creating additional identities

• MFA (Multi-Factor Authentication):


• Further authenticate users
• Avoid compromises due to simple password constraints

209
4/27/18

Azure AD B2B

• Inviting users from other


Azure AD Tenants into
your own organization
tenant
• User provisioning is
done by the invited
party
• You as an organization
are in control to invite
the other side’s user

Azure AD B2C

• Inviting users from other


social media Identity
Tenants (e.g. Facebook,
Twitter, Google,
LinkedIn, Microsoft
Account) into your own
organization tenant
• User provisioning is
done by the invited
party
• You as an organization
are in control to invite
the other side’s user

210
4/27/18

Multi-Factor Authentication

• What is it?:
• An authentication method, which requires an additional
validation item, besides your username and password
combination:
• Text message
• Azure Authentication App
• How does MFA work?
• Requires 2 or more (configurable) account validation
options:
• Something you know (typically user/password combination)
• Something you have (Mobile authenticator app)

Azure AD Identity Protection

• Automatic detection of vulnerabilities in


your organization’s identity objects (e.g.,
compromised user accounts)
• Define configuration alerts and automatic responses
(runbooks), to detected suspicious
and malicious actions that occur in your
organization’s identity solution
• Recognize, audit and inspect suspicious activity, and
take appropriate action to resolve them

211
4/27/18

Azure AD Privileged Identity Management

• Detect privileged users in Azure Active Directory


• Enable “Just-in-time” administrative level access to
Microsoft Cloud Services
• Detailed reporting related to who got what
administrative access level
• Automatically give users permission to have
permanent admin-level right access, or allow for self-
service group membership

Azure AD Domain Services

• Some (non-cloud native) applications don’t “speak”


cloud:
• The application relies on Active Directory protocols (LDAP,
Kerberos,…)
• Azure AD doesn’t provide Group Policies
• Azure AD doesn’t provide Organizational Units
• You cannot “join” servers into an Azure AD Tenant

212
4/27/18

Azure AD Domain Services

• Key Characteristics:
• Provides a compatibility layer for Active Directory
integrated applications, on top of Azure AD
• Takes resources from Azure AD to “emulate” an Active
Directory domain (users, groups, memberships, passwords,
limited GPOs)
• One AAD DS per Azure AD
• High Availability built-in

Azure AD Domain Services

Azure

Azure AD
Domain Services

Azure AD
Azure ADDS
domain joined

213
4/27/18

Lab: Deploying Services to Secure Secrets in Azure

• Exercise 1: Deploying Key Vault using ARM Template


• Exercise 2: Deploying Virtual Machine using Key Vault
Secret
• Exercise 3: Cleanup Subscription

Logon Information
Virtual machine: 20535A-SEA-ARCH
User name: Admin
Password: Pa55w.rd

Estimated Time: 60 minutes

Lab Scenario

A local credit union client has been excitedly using


Azure Resource Manager to deploy virtual machines to
the cloud with ARM templates. Unfortunately, they
quickly discovered that sending virtual machine
passwords as part of the JSON object in an HTTP
request body violates some of their industry’s best
practices. While they already use SSL for in-flight
request security, they would like to use the Azure Key
Vault to store their virtual machine passwords moving
forward. The client has hired you to put together a
prototype showing them how to accomplish this task.

214
4/27/18

Lab Review

• What type of secrets could you store for container-


based VMs in an Azure Key Vault?

Module Review and Takeaways

• Review Question

215
4/27/18

Module 10
Integrating SaaS Services Available on
the Azure Platform

Module Overview

• Cognitive Services
• Bot Services
• Machine Learning
• Media Processing

216
4/27/18

Lesson 1: Cognitive Services

• Cognitive Services
• Intent Detection
• Cognitive APIs

Cognitive Services

People

Agent Applications Services Infrastructure


Cortana Office 365 Bot Framework Azure Machine
Learning
Dynamics 365 Cognitive Services
Azure N Series
SwiftKey Cortana Intelligence
FPGA
Pix Cognitive Toolkit
Customer Service
and Support

217
4/27/18

Cognitive Services

Cognitive Services

Vision
Computer Vision | Content Moderator | Emotion | Face | Video |
Video Indexer | Custom Vision Service

Speech
Bing Speech | Custom Speech Service | Speaker Recognition

Microsoft Language
Bing Spell Check | Language Understanding | Linguistic Analysis |

Cognitive Text Analytics | Translator Text & Speech | Web Language Model

Knowledge
Services Academic Knowledge | Entity Linking | Knowledge Exploration |
Recommendations | QnA Maker | Custom Decision Service

Search
Bing Autosuggest | Bing Image Search | Bing News Search | Bing Video
Search | Bing Web Search | Bing Custom Search

Labs
Project Prague | Project Cuzco | Project Johannesburg | Project Nanjing |
Project Abu Dhabi | Project Wollongong

218
4/27/18

Bing Search APIs

Enhanced Search and Filtering Capabilities

Web- Secure High


Scale (HTTPs) Performance

REST schema.org JSON

Ongoing Improvements and Support

Intent Detection

• To determine intent in speech or text, we need to


perform linguistic analysis:
• Our tooling will need to perform analysis of natural
language:
• In the industry, this is referred to as natural language processing
• We should be able to flag, tag and identify parts-of-speech:
• Identify actions or concepts from

219
4/27/18

Linguistic Analysis

• Natural language processing tools that identify the


structure of text:
• Sentence separation and tokenization
• Part-of-speech tagging
• Constituency parsing

“Great to meet you!


I need to extend my
booking next week
by one day. Can you
also book me a car?”

Language Understanding (LUIS)

Train by providing
examples

Deploy to an HTTP
Create your endpoint and
own LU model activate on any
device

Maintain model with


ease

220
4/27/18

Language Understanding (LUIS)

Focus on Intents, Utterances and Entities

Intent Utterance Entities


BookFlight "Book a flight to Seattle?" Seattle

StoreHoursAndLocation "When does your store open?" open

ScheduleMeeting "Schedule a meeting at 1pm with Bob in Distribution" 1pm, Bob

Language Understanding (LUIS)

Remind me to
Book me a Order me 2 Where is the
call my dad
flight to Cairo pizzas nearest club?
tomorrow

221
4/27/18

Pre-built Domains

Pre-built Entities

Age Dimension Email

Encyclopedia Geography Money

Number Ordinal Percentage

Phonenumber
(US) Temperature Url

222
4/27/18

Cognitive APIs

Computer Vision Translator Text


Content Moderator Custom Decision Service
Emotion API Entity Linking
Face API Knowledge Exploration
Video Indexer Service (KES)
Bing Speech Service Recommendations
Speaker Recognition Academic Knowledge
Translator Speech Language Understanding
Linguistic Analysis (LUIS)
Text Analytics QnA Maker

Face API

• Cloud-based service to detect and recognize faces:


• Detects up to 64 human faces
• Provides a “face location” within the image
• Image can be provided as part of the request (bytes) or
externally (URL)

223
4/27/18

Face API

Face
location

Submitted
faces

Metadata
about face

Face API

• Face Verification:
• Authentication using a face
• Finding Similar Faces:
• Find faces that look most similar
• Face Grouping:
• Groups faces based on similarities
• Face Identification:
• Detect a face based on a “known people” database

224
4/27/18

Speech Translation

• Machine Speech Translation:


• Send voice
• Receive translated text
• Receive text-to-speech translation
• Optimized for conversations:
• Full conversations handling
• Handles more than simple utterances
• Multiple Languages supported

Speech Translation

TrueText

can can you


here me

Automatic
Speech
Recognition

Can you hear


me?

Machine
Translation

225
4/27/18

Speech Translation

TrueText

can can you


here me

Automatic
Speech
Recognition

Can you hear


me?
Text to
Speech

Machine
Translation

Discussion

If you were building a mobile app


that manages real estate listings,
what are some of the LUIS intents
you could create?

226
4/27/18

Lesson 2: Bot Services

• Bot Services
• QnA Maker

Bot Services

227
4/27/18

Bot Framework

• Tools for building bots:


• Build web sites with card-based UI
• Build services to enrich existing applications
• Implement mechanisms to receive events and act
on them
• Uses industry-standard protocols
• Built-in conversation modeling tools
• Integrated LUIS
• Built-on common patterns

Bot Framework

Platform Platform Services

HTTP
AI
{ Your Code }
REST Endpoint
Intelligent Tools

SDK

Bot Builder SDK

228
4/27/18

Bot Services

• Develop using App service-compatible programming


languages
• Quick-start templates
• Integrated chat window:
• Embeddable web chat
• Support for channels:
• Kik, Twilio, Skype, Slack, Microsoft Teams, etc.
• Cognitive Services integration
• Direct Line support

Bot Services

• Powered by Azure Functions:


• Serverless
• Continuous Deployment
• Auto-scale
• Built-in code editor
• Integrates with various databases and services

229
4/27/18

One Bot, Many Canvases

Custom Skills Business Bots LOB Bots

Memory / Profile Bot Directory IT Managed

Speech Bots in Bing Search Integrated Auth

One Bot, Many Canvases

Notification Microsoft Teams Skype Android iOS

230
4/27/18

QnA Maker

• Web-based service that simplifies the process of


creating a bot to answer simple questions for your
users:
• Bot's knowledge can be imported from an existing FAQ
document or web page
• Service creates a Bot for you without the need to write
custom code
• Bot can be trained and modified using the QnA Maker
portal
• Bot is available as a REST API

QnA Maker

231
4/27/18

Lesson 3: Machine Learning

• Machine Learning

Machine Learning

How can we
make it
happen?

What will
happen?

Why did it
happen?
Value

What
happened?

Difficulty

232
4/27/18

Machine Learning

• 25 different machine learning algorithms


• Modules for importing and cleaning data
• Create experiments to train and test models
• Publish models as web service (REST)

Machine Learning

• Example of Algorithms:
• Classification: Assign a category to each item
• Regression: Predict a real value for each item
• Ranking: Order items according to some criterion
• Clustering: Partition items into homogeneous groups
• Dimensionality reduction: Transform an initial
representation of items into a lower-dimensional
representation while preserving some properties

233
4/27/18

Machine Learning

Data set

Data cleaning

Algorithm Split
Training data
Test data

Train model

(Another model)
Score model

Evaluate model

Discussion

What are examples of business use


cases for machine learning
algorithms?

234
4/27/18

Lesson 4: Media Processing

• Media Services
• Computer Vision API

Media Services

• Traditional video management and delivery


challenges:
• Infrastructure costs:
• Specifically, bandwidth and storage
• Maintenance and management complexity:
• How do you maintain millions of clips?
• Monetizing content:
• Can’t simply host a MP4 file out in the open
• DRM:
• How do you protect content from theft?
• Security:
• How do you protect privileged content?

235
4/27/18

Media Services

“Build-On” Media Partners and Customers

Partner CDNs
Secure Encoding On-Demand Live Ingest Live Origin Advertising
Media Ingest Origin

Azure CDN
Media Job Static/Dynami Content Live Encoding Analytics Identity
Scheduling c Packaging Protection Management

Partner Technologies (Media Processors, Origin Servers, Live Encoders etc.)

(Fabric, Storage, Compute, Database) Origin


Caching

Media Services

• Video Processing
Ingest
• Video-on-demand
Encode
• Video analytics
Package
Encrypt
Deliver

236
4/27/18

Live Streaming

Video Format Conversion

237
4/27/18

Dynamic Video Packaging

Video Indexer

• Natural Language Processing technology


• Automatic Vocabulary adaption / Custom Metadata
• Lattice based index for higher accuracy, store word
alternatives
• Possible tasks:
• Catalog vast content libraries
• Generate transcripts from multimedia
• Keyword Search
• Deep linking

238
4/27/18

Computer Vision API

• Algorithms for Processing Images:


• Tag images based on content
• Categorize images
• Identify the type and quality of images
• Detect human faces and return their coordinates
• Recognize domain-specific content
• Generate descriptions of the content
• Use optical character recognition to identify printed text
found in images

Computer Vision API

• Algorithms for Processing Images:


• Recognize handwritten text
• Distinguish color schemes
• Flag adult content
• Crop photos to be used as thumbnails

239
4/27/18

Image Tagging

'tags': [
{
"name":"grass",
"confidence":0.999999761581421 },
{
"name":"outdoor",
"confidence":0.999970674514771 },
{
"name":"sky",
"confidence":999289751052856 },
{
"name":"building",
"confidence":0.996463239192963 },
{
"name":"house",
"confidence":0.992798030376434 },
{
"name":"lawn",
"confidence":0.822680294513702 },
{
"name":"green",
"confidence":0.641222536563873 },
{
"name":"residential",
"confidence":0.314032256603241
},
]

Description Generation

"description": {
"captions": [
{
"type":"phrase",
"text":"a black and white photo of
a large city",
"confidence":0.607638706850331
},
{
"type":"phrase",
"text":"a photo of a large city",
"confidence":0.577256764264197
},
{
"type":"phrase",
"text":"a black and white photo of
a city",
"confidence":0.538493271791207
}
],
"tags": {
"outdoor",
"city",
"building",
"photo",
"large"
}
}

240
4/27/18

Color Schemes

Image Foreground Background Colors

Black Black White

White, Black,
Black White
Green

Black Black Black

Optical Character Recognition (OCR)

241
4/27/18

Discussion

If you are hosting video using


Media Services, what parameters
would you use to decide between
statically converted or dynamically
packaged videos?

Lab: Deploying Service Instances as Components of


Overall Azure Solutions
• Exercise 1: Deploying Function App and Cognitive
Service using ARM Template
• Exercise 2: Cleanup Subscription

Logon Information
Virtual machine: 20535A-SEA-ARCH
User name: Admin
Password: Pa55w.rd

Estimated Time: 60 minutes

242
4/27/18

Lab Scenario

A local ESL (English-as-a-second-language) outreach


group wants to add a quick translation tool to their
website. To accomplish this, they would like to use the
Translation API in Cognitive Services and wrap it with a
simple proxy so that they can use it on their website
and potentially a future mobile application.

Lab Review

• What other scenarios would require you to deploy a


Cognitive Services instance in an ARM Template?

243
4/27/18

Module Review and Takeaways

• Review Question

Module 11
Integrating Azure Solution
Components Using Messaging
Services

244
4/27/18

Module Overview

• Event Messaging
• Integration
• Internet of Things (IoT)

Lesson 1: Event Messaging

• Storage Queues
• Service Bus
• Event Grid

245
4/27/18

Storage Queues

Queue Message Handling

• Create/Delete Queue
• Measure Queue Length
• Insert Message into Queue
• Retrieve the Next Message
• Extend Message Lease
• Peek at the Next Message
• Update a Message
• Delete a Message

246
4/27/18

Storage Access Control

Service Bus

• Service Bus is a managed messaging infrastructure:


• Massive in scale and completely managed
• Allows you to scale out your applications and consumers
knowing that the messaging platform will scale out with
your application
• Allows decoupled components to communicate
asynchronously and synchronously

247
4/27/18

Service Bus Queues

• Service Bus queues offer a brokered messaging


communication model:
• Distributed applications can share messages in a First In
First Out (FIFO) pattern
• Individual messages are only received by one message
consumer
Message Service Bus Message
Sender Namespace Receiver

Web App

Service Or
Mobile App Queue
Application

Service

Service Bus Relay

• Relays provide a mechanism to connect distributed


client applications or cloud services to a projected
on-premises endpoint:
• It allows for unidirectional or bi-directional communication
• It relays messages directly to an endpoint without any
brokering of the message
• Applications establish an outbound connection to the
relay and the relay manages the transport of the
messages

248
4/27/18

Service Bus Relay

Service Application
Send/Receive

Service Bus
Relay

Client Client
Application Application

Event Grid

249
4/27/18

Event Grid and ARM Integration

• Automate Operations:
• Event Grid can publish ARM events including:
• Resource creation
• Resource modification/deletion
• Deployment of multiple resources to a resource group
• Creation or deletion of a resource group
• Azure services can respond to an Event Grid resource-based
event by performing automation actions:
• A Logic App can modify a newly created database
• Azure Automation can manage a new VM
• Metadata about a resource deployment can be stored in
Azure Storage using an Azure Function

Discussion

What types of application scenarios


would require you to send
messages from many producers to
a single consumer?

250
4/27/18

Lesson 2: Integration

• Serverless Integration
• Notification Hubs

Serverless Integration

251
4/27/18

Logic Apps

• Cloud APIs and platform:


• Supports over 125 built-in connectors
• Scales to meet your needs
• Enables rapid development
• Extends with custom APIs and
Functions
• API connections:
• Authenticate once and reuse

Logic App Connectors

252
4/27/18

Azure Functions

Apps

eCommerce Digital Global Presence Custom Apps LOB API / Services / ISV

App Service

Web Apps Mobile Apps API Apps Functions

Fully Managed Platform Development Enterprise Grade

• Limitless/Auto • Languages and Framework • Enterprise grade SLA


• OS and Framework • Superior DevOps • Secure and Compliance
• Load balance • Self served • On-Premise Connectivity
• Something else • Something else • Something else

Azure Functions

• Methods of Execution:
• Triggers
• WebHooks
• Language of Choice:
• C#, F#, Node.js, Python, PHP, batch, bash, Java
• Pricing Options:
• Dynamic (pay-per-use)
• App Service Plan

• Integrations:
• DocumentDB, Event Hubs, Mobile Apps (tables),
Notification Hubs, Service Bus, Storage
• GitHub (webhooks), On-premises (using Service Bus)

253
4/27/18

Connecting Serverless Components

Serverless Business Scenarios

254
4/27/18

Notification Hubs

• Managed infrastructure for sending push


notifications to mobile devices:
• Multiplatform
• Scalable
• Simple SDK:
• Available on many major mobile platforms
• Broadcast to many users or target specific users

Benefits

• Managed Infrastructure:
• You don’t have to worry about scaling your application
yourself
• You can focus on messages and templates, not the
mechanics of your service
• SDKs available for major platforms
• Template support
• Support for filtering recipients by tag

255
4/27/18

Platform
Retrieve Handle

Send Message

Discussion

How can you enhance an off-the-


shelf software solution using both
Logic Apps and Notification Hubs?

256
4/27/18

Lesson 3: Internet of Things (IoT)

• Event Hubs

Event Hubs

Event Hubs is a partitioned consumer messaging


services:
• Publish and subscribe to streams of records:
• Similar to a message queue or enterprise messaging system
• Store streams of records in a fault-tolerant manner
• Process streams of records “as they occur”

• Ideal for building applications that transform or


react to streams of data

257
4/27/18

Event Hubs

• Input Streaming:
• Receives high-velocity message streams in a
multi-consumer group
• Isolated Read:
• Stores “pointers” for each reader so they can resume at a
specific point-in-time in reading time-based messages from
the queue
• Open Protocols:
• Supports AMQP 1.0
• REST API for management

Event Hubs Conceptual Diagram

HTTP
AMQP

258
4/27/18

IoT Hubs

IoT Hubs builds on the features in Event Hubs by


adding additional functionality that is commonly
needed in IoT applicatons:
• Support across a wider variety of platforms and SDKs:
• Ex. JavaScript and Java Support, RTOS and ARM Platform support
• Device-facing and Service-facing SDKs for registration and
management
• Identity and access management across all devices
connected to Hub

IoT Hubs

2
RTOS, Linux, Windows, Android, iOS

Protocol
Adaptation
1. Direct connection (HTTP, AMQP)
2. Cloud Protocol Adaptation (e.g.
Devices

3
MQTT)
Field Protocol 3. Field Protocol Adaptation (e.g.
Gateway Adaptation
DDS)
4. Gateway-assisted (e.g. Bluetooth
LE)
4

Field IoT Hub


Gateway (Cloud Gateway)

Device
Connectivity & Management

259
4/27/18

Azure IoT Device SDK

• Enable simple, secure device <-> cloud connectivity &


management

• Client “agent” software for devices and gateways

• Libraries that OEMs/SIs/ISVs can use in new and


existing systems

• Open source software framework

IoT Remote Monitoring

Iot Hub

Device

Notification Hub

Logic App

MySQL DB

260
4/27/18

Example IoT Solution

Lab: Deploying Messaging Components to Facilitate


Communication Between Azure Resources

• Exercise 1: Deploying Service Bus Namespace


• Exercise 2: Deploying Logic App
• Exercise 3: Cleanup Subscription

Logon Information
Virtual machine: 20535A-SEA-ARCH
User name: Admin
Password: Pa55w.rd

Estimated Time: 60 minutes

261
4/27/18

Lab Scenario

An established healthcare client has used Service Bus


extensively throughout the years to manage
communication between their custom software
components. After learning about Logic Apps, the
client would like to create Logic Apps that can
consume Service Bus queue messages so that they can
minimize the amount of code that their team needs to
write and maintain annually.

Lab Review

• What are some scenarios where non-developers in


your company can gain value from using Logic Apps?

262
4/27/18

Module Review and Takeaways

• Review Question

Module 12
Monitoring & Automating
Azure Solutions

263
4/27/18

Module Overview

• Monitoring
• Backup
• Automation
• Business Continuity Case Study

Lesson 1: Monitoring

• Azure Network Watcher


• Azure Security Center
• Azure Monitor & Diagnostics
• Azure Advisors
• Azure Service Health
• Operations Management Suite – Log Analytics
• Application Insights
• Power BI

264
4/27/18

Azure Network Watcher

• Networking feature, providing:


• Topology
• Variable Packet Capture
• IP Flow Verify
• Next Hop
• Diagnostics Logging
• Security Group View
• NSG Flow Logging
• VPN Gateway Troubleshooting
• Network Subscription Limits
• Role Based Access Control
• Connectivity

Azure Network Monitor

• Centralized hub for different Azure Resources


Monitoring aspects:
• Alerts
• Metrics
• Log Analytics
• Service Health
• Application Insights
• Network Watcher

265
4/27/18

Azure Security Center

• Centralized Dashboard, focusing on Security posture


of Azure and hybrid systems and applications
• Active in 3 different areas:
• General Security View
• Prevention
• Detection
• Networking Features:
• Networking Recommendations
• Internet Facing Endpoints security view
• Networking Topology security view

Azure Security Center

266
4/27/18

Azure Monitor & Diagnostics

“Highly granular and real-time monitoring data for any


Azure Resource”

View and manage


Set up alerts and Diagnose
all your Integrate with
take automated operational issues
monitoring data your existing tools
actions quickly
easily

Get the granular, up-to-date monitoring data you need—all in one place

Azure Monitor & Diagnostics

“Dev: My Application is slow…, Ops guy, can you figure


out why”

267
4/27/18

Azure Advisors

“Optimize your Azure Resources, according to


Microsoft Best Practices”

Improve the Enhance Optimize


Maximize the ROI
availability of protection from Performance to
of your IT and
business-critical potential security run healthy
business budget
applications threats applications

Get the granular, up-to-date monitoring data you need—all in one place

Azure Advisors

Recommendations are sourced from the Azure


Architecture Center

268
4/27/18

Azure Service Health

“Provides timely and personalized information when


problems in Azure services impact your services.”

Azure Portal
Personalized Service Health Dashboard

Azure Service Health

Azure Monitor
Service Health Alerts
- Service Issues
- Maintenance
- Advisories
http://status.azure.com
General Health Overview of all Azure Services

Azure Service Health

• Service Issues:
• Shows any ongoing problems in the Azure Platform, having
impact on YOU
• Planned Maintenance:
• Provides information on scheduled maintenance of YOUR
impacted Azure Resources
• Health History:
• Shares feedback of past issues with impact on YOUR Azure
Resources

269
4/27/18

Operations Management Suite – Log Analytics

“Gain visibility and control across your hybrid cloud


with simplified security and operations management”

Gain immediate Enable consistent Respond


Ensure availability
insights across control and immediately to
of apps and data
workloads compliance security threats

Monitoring, Management + Business Continuity & Disaster Recovery

Operations Management Suite – Log Analytics

• Separate the signal from the noise


• See the full picture in meaningful detail
• Integrating application monitoring
• Azure Resources & Hybrid
• OMS Agents
• Supports “Any” Log File format

270
4/27/18

Operations Management Suite – Log Analytics

• Powerful Query Language


• Click-to-filter scenarios
• Saved Search
• Export to CSV
• Use “Azure Log Analytics” for
more advanced querying
(portal.loganalytics.io)

Application Insights

“Get actionable insights through application


performance management and instant analytics”

Integrated in the Diagnostics, DevOps


Web App
Web App code, proactively integration from
Performance
running in Azure detecting lifecycle within VS2017,
Management
or on-premises issues GitHub,…

Monitoring, Management + Business Continuity & Disaster Recovery

271
4/27/18

Application Insights

• Application Map:
• Diagram of App Components and interactions between all
services
• Live Metrics:
• Real-Time Requests information
• Servers:
• Detailed Performance per Instance view
• Availability:
• Run scheduled tests for Uptime

Application Insights

272
4/27/18

Power BI

“Workspace approach, integrating with Power BI


Apps, allowing for detailed reporting and data
analytics”

Embed
Connect to Get access to Simplify Mgmt,
interactive data
different data powerful expose IT data to
visuals and
sources, create dashboards, non-IT teams,
reporting
reports and data alerts and drill achieve
features into your
charts down for info compliance
apps

Monitoring, Management + Business Continuity & Disaster Recovery

Power BI

Integrating with Industry-Standard Data Sources

273
4/27/18

Power BI Integrations

Lesson 2: Backup

• Azure Backup
• Backup Options
• Specialized Backup
• Site Recovery

274
4/27/18

Azure Backup

• There are three popular scenarios where Azure is


selected as the ideal backup target:
1. On-Premises backups of Files & Folders into Azure Backup
Vault
2. On-Premises backups of full Windows & Linux VMs into
Azure Backup Vault
3. Azure VM backup to Azure Backup Vault

Backup Options

Azure Backup / Restore of On-Premises Files & Folders

Azure Site Recovery

Microsoft Azure
Data Channel Backup Vault
Source: Windows OS
Public Internet or ExpressRoute
with Public Peering

Microsoft Azure Recovery


Services Agent

275
4/27/18

Backing up OS, Sysvol and Applications

Azure Backup / Restore of On-premises running full workloads


(OS, Sysvol, and Applications)

Azure
Backup
Server
or DPM

Azure Site Recovery

Microsoft Azure
Data Channel Backup Vault
Source: VMware, AWS
& Physical Servers Public Internet or ExpressRoute
with Public Peering

Azure Backup Server SysCtr DPM Azure Backup


Recovery Services
Agent

Backing Up Full VMs

Azure VM Backup / Restore to Azure Backup Vault

Azure Azure Site Recovery


VM
Microsoft Azure
Backup Vault
Data Channel
Source: Azure
Azure Backbone

Microsoft Azure Backup


Extension for VMs

276
4/27/18

Specialized Backup

You can do more than simply backup VMs or Data


using Azure Backup:
• Hybrid Backup Encryption
• Azure Backup Monitoring with Log Analytics
• Azure Backup Reports with Power BI
• Linux Application Consistent Azure Backup

Hybrid Backup Encryption

277
4/27/18

Azure Backup Monitoring with Log Analytics

No infrastructure Enterprise Wide Custom Queries (KQL) ITSM Integration

Azure Backup Reports with Power BI

No infrastructure Enterprise Wide Custom Reports Access Control

278
4/27/18

Linux Application Consistent Azure Backup

NEW

Azure Data Box

Ship

No procuring of disks Parallel transfers Safe and secure

Offline Data Shipping

279
4/27/18

Site Recovery

• Designed for zero-data loss during migration

• Near-zero downtime for their users

• Comprehensive coverage for all applications

• Ability to test application in the new cloud before


migration

Site Recovery Advantages

• Zero application data loss during migration


• Near-zero application downtime during migration

• Broad coverage for hypervisors, applications,


operating systems, and Azure features

• No-impact application testing in Azure

280
4/27/18

Disaster Recovery or Workload Migration from


Hyper-V/SCVMM

Hyper-V Azure Site Recovery


Server
Microsoft Azure
Data Channel
Source: Hyper-V/SCVMM
Public Internet or ExpressRoute
with Public Peering

Microsoft Azure Recovery


Services Agent

Disaster Recovery or Workload Migration from


VMware/AWS/physical

Config
Server
Process
Server

Azure Site Recovery

Microsoft Azure
Data Channel
Source: VMware, AWS
& Physical Servers Public Internet or ExpressRoute
with Public Peering

Process Server Configuration Server Mobility Service

281
4/27/18

Disaster Recovery or Workload Migration from Azure


to Azure

Azure
VM
Azure Site Recovery
Microsoft Azure
Same or different Region
Data Channel
Source: Azure
Azure Backbone

Microsoft Azure Recovery


Services Agent

Lesson 3: Automation

• Azure Automation
• Automation Flow
• Configuration Management

282
4/27/18

Azure Automation

• Configuration and control plane for Azure,


on-premise and other cloud providers:
• Robust configuration management toolkit built-in
• Access governance and control
• Serverless execution of management scripts
• Integration with existing platforms, systems and OS features

Features

• Process Automation:
• Author runbooks - PowerShell, scripts PowerShell
workflow, Graphical, Python2
• Hybrid Runbook Workers with Proxy support
• Configuration Management:
• DSC Configurations, Pull service
• Node Management & Reporting
• Change tracking & Inventory
• Update Management:
• Insights across a hybrid Environment
• Orchestrated updates and troubleshooting

283
4/27/18

Cross-Cloud

Third party
clouds

Microsoft Azure
Automation
On-premises Datacenters
Branch offices datacenters
Secured connection using the
Azure Automation Hybrid
Worker
Automation tasks are running on-premises, but
triggered from Azure Automation
Azure Stack

284
4/27/18

Azure Automation Desired State Configuration

• Host DSC Scripts and clients pull their configurations


automatically
• Support for cloud or on-premises VMs and hosts
• Simple onboard process for Azure Virtual Machines

• Characteristics & Use Cases:


• Import, Authoring, Compiling
• Integrated source control,
• Controlled Distribution to nodes
• Reporting

Desired State Configuration

Automation DSC can be used to manage various


machines:
• Azure virtual machines running Windows or Linux
• Amazon Web Services (AWS) virtual machines
running Windows or Linux
• Physical/virtual Windows computers on-premises, or
in a cloud other than Azure or AWS
• Physical/virtual Linux computers on-premises, or in a
cloud other than Azure or AWS

285
4/27/18

Desired State Configuration

• Built-in integration with on-premises systems


and PowerShell DSC nodes
• Run Azure Automation runbooks
on-premises
• Automation accessible via new REST API (including
GitHub, VSO and ARM)
• Graphical workflow-authoring tool
• Runbook Management from the new Microsoft
Azure portal

Hybrid Runbook Worker

• An on-prem server running MS Mgmt Agent


• Executes runbooks downloaded from AA
• Reports results back to AA and OMS
• Can be deployed in groups for high availability
• Requires no ports (outside-in)

On-premises

Runbook
Environment
Azure
Automation

Microsoft
Mgmt
Agent
Operations
Hybrid Runbook Management
Worker Suite

286
4/27/18

Automation Flow

Azure Portal

Azure Automation
Webhooks Runbooks

PowerShell
A

Alerts
Run actions against
Azure Resources

Automation Flow

Run actions against


on-premises running
Resources, using Hybrid
runbook worker group

287
4/27/18

Configuration Management

Provision/ Manage
Infrastructure

Bootstrap Agents

Customize VM

Chef

Chef is a configuration management tools for


deploying & managing infrastructure and applications

Key Capabilities:
• Infrastructure as code
• Declarative interface to resource
• Policy based configuration management

288
4/27/18

Puppet

Puppet is a configuration management system that


allows you to define the state of your IT infrastructure,
then automatically enforces the correct state

• Key Capabilities:
• Supports easy to read declarative language
• Enforces desired state on the system
• Puppet Forge supports many ready to use modules

Cloud Shell

• Authenticated shell access to Azure from virtually anywhere


(browser)
• Choice of shell experience that best suits the way
you work:
• PowerShell or Bash CLI
• Common tools and programming languages included that’s
updated and maintained by Microsoft
• Persist your files across sessions in attached Azure File storage

289
4/27/18

Lesson 4: Business Continuity Case Study

• Case Study Overview


• Case Study Solution

Case Study Overview

• Review the case study requirements


• Design a solution to the customer business problem
• Present your solution
• Respond to questions and objections
• Review a potential solution

290
4/27/18

Customer Business Problem

• Media and Publishing Company:


• Approximately 5,000 employees
• Located in Seattle, WA

• Specialty:
• Technical and Scientific Books
• Catered to Academic Authors for Years
• Extensive Historical Catalog of Technical and Scientific
Books

Customer Inventory

• Existing Applications

Active Directory Exchange 2013

IIS SQL Server 2014

291
4/27/18

Customer Inventory

• Backup Workflow

Physical Servers Tape Backup

System Center Virtual Machine Manager

Customer Goals

Disaster Recovery:

Budget Conscious:

Limited Resources:

292
4/27/18

Customer Needs

• The ability to perform data-center level recovery for critical


workloads that can be executed in the event of a data center
failure
• Automated and orderly recovery process so different tiers of
the application start in the correct order and remain in a
consistent state
• The ability to perform failback following restoring on-premises
data center functionality that can be executed in the
automated and orderly manner
• The ability to perform multi-tier application and individual
server-level recovery of critical workloads
• Support for server-level and application-level high availability
whenever possible

Customer Needs

• Quick testing and validation of recovery processes with


minimal interruption to the production environment
• Minimized capital and operational expenses
• Optimized authentication for AD-integrated services and
applications
• Centralized management of backups and reduced or
eliminated dependency on offsite tape storage
• The level of security and privacy commensurate with highly
sensitive and competitive nature of the business

293
4/27/18

Customer Objections

• Solution must significantly improve their current recovery


point/time objectives (which today is a manual process)
• Overall cost of the solution
• Protecting a diverse environment such as physical servers or
other hypervisors
• The management tools for the solution must be available in
the event one of the data centers is unavailable
• Protect data that is not hosted within a virtual hard disk
(VHD/X)

Customer Objections

• The protected data must be secure


• Unsure about which workloads are supported on Azure

294
4/27/18

Call to Action

Case Study Timing: 60 Minutes

Who are the business decision makers


and stakeholders?

What customer business needs do you


need to address with your solution?

Diagram your proposed solution

Case Study Solution

• Target Audience

• Potential Solution

• Benefits

• Customer Quote

295
4/27/18

Target Audience

• Anthony Ciske, IT Director


• Network Administrator
• Publishing Application owners:
• (Exchange, SQL, n-tier applications)

Potential Solution

Using Azure Site Recovery and multiple virtual


networks for failover

296
4/27/18

Potential Solution

Using Azure Backup to store servers’ data offsite in


Azure

Benefits

Fabrikam Publishing is using Microsoft Azure Site


Recovery to implement their Disaster Recovery and
Business Continuity strategy. By leveraging built-in
features of Azure Site Recovery, they are able to
accomplish their recovery and resiliency objectives
efficiently and with a minimal cost, giving them a
competitive advantage.

297
4/27/18

Benefits

• Recovery Objectives:
• Site Recovery allows for planned, unplanned and test
orchestrated recovery operations
• Cost Effective:
• Eliminates upfront capital expenses normally related to
creating a secondary datacenter
• Security and Privacy:
• Data is encrypted during transit and at rest
• Non-Disruptive Testing of Disaster Recovery Solution:
• Can validated DR orchestration without disrupting
production environment

Customer Quote

“Hopefully we’ll never have to use it, but we have


peace of mind in knowing that if something terrible
happens, we won’t have to engage in a mad scramble
to recover our workloads.”

Anthony Ciske, IT Director


Fabrikam Publishing

298
4/27/18

Lab: Deploying Configuration Management Solutions


to Azure
• Exercise 1: Deploying a Chef Management Server using
ARM
• Exercise 2: Configuring Management Server
• Exercise 3: Deploying a VM Scale Set using
Chef-Configured VMs
• Exercise 4: Cleanup Subscription

Logon Information
Virtual machine: 20535A-SEA-ARCH
User name: Admin
Password: Pa55w.rd

Estimated Time: 90 minutes

Lab Scenario

A cutting-edge insurance organization would like to


create multiple virtual machines to process insurance
claims. Today, those virtual machines are managed in
the organization’s datacenter using Chef. The client has
reached out to you to create a prototype solution
where the machines are created automatically and
have the Chef agent and configuration installed as part
of the automatic deployment.

299
4/27/18

Lab Review

• Do you find it easier to use PowerShell DSC or Chef to


configure a VM in a VM Scale Set?

Module Review and Takeaways

• Review Question

300
4/27/18

Course Evaluation

• Your evaluation of this course will help Microsoft


understand the quality of your learning experience.
• Please work with your training provider to access the
course evaluation form.
• Microsoft will keep your answers to this survey
private and confidential and will use your responses
to improve your future learning experience. Your
open and honest feedback is valuable and
appreciated.

301

Das könnte Ihnen auch gefallen