Cloud Computing design

patterns blueprints

Rohit Bhardwaj

Principal Cloud Engineer

Kronos Incorporated
 Agenda

• Why build for Cloud Design?

• Dockers
• Cloud design patterns
• Sharing, Scaling and Elasticity Patterns
• Reliability, Resiliency and Recovery Patterns
• Cloud Security
• 12 factor application guidelines
A
A
Supportability
Testability
Vendor lockin
A
A
https://eng.uber.com/wp-content/uploads/2015/01/SOA_overview.png
initial über
https://creately.com/jupiter/diagram/image/i9rx49yi1
1000 services challenges
• Trip tracking

• Mapping and Routing

• Realtime Market place + Matching (Supply vs Demand)

• Reputation tracking (Ratings)

• Global Payment Processing

• Data Warehousing + Modeling

• Supply + Demand prediction

• Fraud Detection
availability
operational cost per trip
response time
https://creately.com/jupiter/diagram/image/ihsiv6cf1
http://microservices.io/patterns/apigateway.html
http://www.powderfool.com/wp-content/uploads/2015/08/2a0b6d5.jpg
twitter

https://media.licdn.com/mpr/mpr/p/6/005/048/236/071c448.jpg
scalability challenge
Traditional Solutions
3 TIER ATCHITECTURE
MULTI-TIERED APP
 Dockers

• Docker allows you to package an application with all of its

dependencies into a standardized unit for software development.
 Dockers demo
• To deploy a single Cassandra node to this docker host, run this command:

• docker run --name=n1 -d cassandra

• You need the IP of this first container to pass as a parameter to subsequent containers

• docker inspect -f '{{ .NetworkSettings.IPAddress }}' n1

• To add a second node to the Cassandra cluster, run this command:

• docker run --name=n2 -d cassandra -seeds IP

• docker exec -it n1 nodetool status

 PRINCIPLES OF CLOUD
APPLICATION ARCHITECTURE
Cloud computing scenarios

http://www.cloudcomputingpatterns.org/
Cloud offerings

http://www.cloudcomputingpatterns.org/
Cloud App Architectures

http://www.cloudcomputingpatterns.org/
Cloud Application
Management
Principle 1: Resilient to
Failure

http://cdn3.positivepsychologyprogram.com/wp-content/uploads/2013/12/resilience-in-positive-psychology-.jpg
 Vulnerabilities

• Hardware failure of a compute network, or storage

physical appliance

• A software failure in an application component

• Cloud service, or a network failure due to an

unresponsive component or a transient network
connectivity problem
 APPLICATION MUST
WORK!

• Degraded performance is fine

• Degraded functionality
 Loose coupling

• Components to be substituted without impact

Manage failure and latency

• Component not statically bind to instance

• Another component can take over

 What if critical data is not
written to database

• How to handle the Mobile devices?

• Offline storing of data

• Automatic sync
Principle 2: Resilient to
Latency
http://officeoffinance.com/wp-content/uploads/2014/02/tenancy.png
 Issue: Infrastructure failures
causing delays
• Network congestion or network partition

• Request contention in non-scalable infrastructure

services

• I/O bandwidth saturation in storage systems

• Software failures in shared services

• Denial-of-service attacks that trigger service failure

or resource exhaustion
Developers: Queue requests
Retry mechanism
Handle failure gracefully

https://media.licdn.com/mpr/mpr/p/6/005/048/236/071c448.jpg
 Issue: Latency between
New York and San Francisco

• 13.8 ms

• 27.6 ms (Round trip)

• As per AT&T: 70 ms

Distance, as the crow flies, between New York and San Francisco =
2,565 miles; speed of light = 186,000 miles/second.
Service A depends on B
which depends on C

http://www.stratoscale.com/wp-content/uploads/imageblog14.2.png
Solution 1: Co-locate

http://associationnow.wpengine.netdna-cdn.com/wp-content/uploads/2012/09/0928_team-800x480.jpg
 Solution 2: Reducing the
number of network requests

• REST API design

• Ask for Zip and phone in same REST request

• Consolidate small details into one form

Bandwidth Aware
 Reduce chattiness

http://techblog.netflix.com/2013/01/optimizing-netflix-api.html
 Principle 3: Secure

• KPMG data life cycle

Data is not secure!

http://mobileapplicationdeveloperscompany.weebly.com/uploads/3/9/9/2/39922959/8888446_orig.jpg
Solution 1: Secure data in
transit

http://underpop.online.fr/j/java/intro/ror/ssh/figs/ssh_0102.gif
Solution 2: Control API
Access
Threat modeling
Principle 4: Location
Independent
 Solution 1: No hardcoded
config: End point addresses

http://www.telerik.com/sfimages/default-source/blogs/not-hard-coded-jpg-jpg
Solution 2: Register service
http://mesosphere.github.io/presentations/hackers-at-berkeley-2015/assets/mesos-animation/mesos-marathon-animation_23b.png

http://mesos.apache.org/documentation/latest/architecture/
 Solution 3: Application
components should be stateless

https://cldup.com/YgsLg7gM2L.png
Solution 4: Applications should
use established protocols

• HTTPS

• Dockers
Solution 5: API management
through proxy
http://microservices.io/patterns/apigateway.html
Principle 5: Elastically
Scalable

https://cloud.google.com/solutions/images/scaling-capacity-to-meet-demand.png
 Principle 6: SOA/Compose-
ability

Discoverability enables an application to dynamically locate

components
Principle 7: Designed for
Manageability
Principle 8: Infrastructure
Independent
Cost and Resource
Consumption Aware
Minimizing Transport Cost

• Minimize payload sizes.

Provide APIs that return just the data the consumer needs (“partial
response”).

Consider data compression, but balance against CPU costs for encoding
and decoding.
Minimize data transfers

Cache immutable data.

Replace or avoid “chatty” protocols.

Netflix case study
Component microservices
Chaos Gorilla
Cassandra maintenance
Isolated Regions
Failure Modes and Effects
Stateless Micro-Service
Architecture
Cloud Application Design
Patterns
Circuit Breaker
 • A request to a remote service timed out.
• The request queue for the remote service is full, indicating that the
remote service is unable to handle additional requests.
 Fallback Strategy

• Fail transparently. Generates a custom response

• Fail silently. Returns a null value for the request.

• Fail fast. Generates a 5xx response code

Circuit Breaker demo

https://github.com/mstine/2016_CloudNativeAppArchWorkshop
https://github.com/Netflix/Hystrix/tree/master/hystrix-dashboard
CIRCUIT BREAKER DESIGN
PATTERN
• Benefits

• Improved resilience to failure

• To provide visibility into component failures (though visibility

into tripped breakers)

• Where to use this pattern

• To eliminate timeout delays due to network or VM instance

failures

• To prevent errors cascading and complicating upstream error

handling (fail transparently, fail silently)
Request Queuing Design
Pattern
http://lostechies.com/derekgreer/files/2012/03/TopicExchange2.png
 REQUEST QUEUING
DESIGN PATTERN
• Benefits

• Improved fault tolerance to ensure high availability

• Improved performance for highly requested APIs

• Where to use this pattern

• To manage application failover (fault tolerance)

• To reduce end point load on a heavily requested API(s)

• To apply alternate strategies to auto-scaling via throttling

Request Collapsing Design
Pattern
Object Change Notification
Pattern
 Netflix change movie to
different formats
• Benefits

• Improved resilience to failure

• Increased scalability

• More efficient resource utilization

• Where to use this pattern

• Where compute intensive processing can be parallelized

• To eliminate single points of failure

• To reduce tight coupling between components to allow for

implementations to be replaced
Service Discovery Pattern
Netflix Eureka architecture

https://github.com/Netflix/eureka/wiki/Eureka-at-a-glance
 Service benefits

• Provide a way for components to direct requests to

available instances.

• Support dynamic registration and de-registration of

service instances.

• Maintain awareness of the state of any given

instance.
Serverless Architectures

https://www.martinfowler.com/articles/serverless.html
AWS Lambda

https://aws.amazon.com/lambda/
Open Tracing

http://opentracing.io/documentation/
Open Tracing
AWS Device Farm

https://aws.amazon.com/device-farm/
Microservices Design
Pattern
http://martinfowler.com/articles/microservices/images/decentralised-
data.png
 MICROSERVICES DESIGN
PATTERN
• Benefits

• Reduces overhead of deploying updates

• Reduces side effects across co-resident services

• Enables elasticity

• Greater visibility into source of issues

• Improved developer productivity

• Where to use this pattern

• In large systems that need to scale elastically and cost effectively

• To eliminate single points of failure

• To improve performance

• To support frequent updates to a critical production system

 Stateless Services Design
pattern

• Reliability

• Scalability

• Visibility

• Simplicity: Easy debug

Configuration Service
pattern
 Current app

• Config changes are made at startup?

• Redeploy / Startup
External Configuration Store
Runtime Reconfiguration: No
downtime

• New Nodes receive new config changes

• Application can subscribe to configs

Authorization Pattern
Operational Strategies
http://mattturck.com/wp-content/uploads/2016/03/Big-Data-Landscape-2016-v18-FINAL.png
Big data characteristics

https://imasaikirangeek.files.wordpress.com/2014/05/defining-big-data1.png
Database Sharding design
replication
 CAP Theorem

BASE (Basically Available, Soft-State, Eventual Consistency) data store

lambda architecture spark
and Cassandra
lambda architecture spark
and Cassandra
 Spring Cloud

• http://start.spring.io/
The Twelve Factors
 Factor 1: Codebase

One codebase tracked in revision control, many deploys

Factor 2: Dependencies

• Explicitly declare and isolate dependencies

• Repeatable deployment
 Factor 3: Config
• Store config in the environment, no properties files

• Resource handles to the database, Memcached,

and other backing services

• Credentials to external services such as Amazon

S3 or Twitter

• Per-deploy values such as the canonical hostname

for the deploy
Spring – Picking up Env Vars

...
import org.springframework.context.EnvironmentAware;
import org.springframework.core.env.Environment;
@RestController
public class HelloController implements EnvironmentAware {
private String name;
@Override
public void setEnvironment(Environment environment) {
this.name = environment.getProperty("who");
} ...
Factor 4: Backing Services
Factor 5: Build, release, run

Strictly separate build and run stages

Strictly separate stages
Factor 6: Processes: Design
for failure

• Execute the app as one or more stateless

processes

• Share nothing

• No Sticky sessions
 Factor 7: Port binding

• Export services via port binding

• Completely self-contained and does not rely on

runtime injection of a webserver into the execution
environment to create a web-facing service.
 Export Services via Port
Bindings

Ÿ Apps are deployed into containers

Ÿ Multiple containers per host
Ÿ Platform to handle port assignments and mappings
http://www.api-first.com/blog/engineering_overview.html#.Vl5voYT8SxI
http://i0.wp.com/blog.restlet.com/wp-content/uploads/2015/04/try-apispark-swagger-ui-small.png
Factor 8: Concurrency

the unix process model for running service daemons.

 Factor 9: Disposability

• Maximize robustness with fast startup and graceful

shutdown

• Processes are disposable, meaning they can be

started or stopped at a moment’s notice.
Factor 10: Dev/prod parity

Keep development, staging, and production as similar as

possible
Faster commits (Quick CI)
 Factor 11: Logs
• Treat logs as event streams

• Routing or storage of its output stream

• Finding specific events in the past.

• Large-scale graphing of trends (such as requests per

minute).

• Active alerting according to user-defined heuristics

(such as an alert when the quantity of errors per
minute exceeds a certain threshold).
Spring – Using ENV to config logging

In application.yml
logging:
level:
org.springframework: ${SPRING_LOG_LEVEL:INFO}
hello: ${LOG_LEVEL:INFO}
Factor 12: Admin processes

• Run admin/management tasks as one-off

processes
The Twelve Factors
 Must haves for Cloud
Dev Ops automation is must

API Support: consuming and supporting APIs

Microservices

HTML 5 Applications: Run Apps in any device

Blue-Green Development: New functionality deployment

validate and gradually grow it

App scaling and Elasticity

Pass with Cloud Foundry

Big Data: Near real time analytics

ASSESSING APPLICATION
CLOUD MATURITY
 References
Cloud Computing Design Patterns,

by Robert Cope; Amin Naserpour; Thomas Erl

The 12 factors: http://12factor.net/

http://www.opendatacenteralliance.org/docs/
architecting_cloud_aware_applications.pdf
www.modsummit.com

www.developersummit.com