Beruflich Dokumente
Kultur Dokumente
Relational Non-Relational
Rational Agile
Predictable Flexible
Traditional Modern
2
Agenda
Tips for
Big Data
Big Data Case Designing
Reference
Challenges Architectures Studies Big Data
Solutions
3
Big Data Challenges
UNSTRUCTURED
STRUCTURED
HIGH
MEDIUM
LOW
4
Big Data Analytics
5
Big Data Analytics Use Cases
LowLatency
Reliability
RealTime
Intelligence
Consumers Intelligent Agents
Volume DataQuality
Performance Data Business SelfService
Discovery Reporting
6
Big Data Analytics Reference Architectures
ArchitectureDrivers: ReferenceArchitectures:
Volume Extended Relational
Sources Non-Relational
Throughput Hybrid
Latency
Extensibility
Data Quality
Reliability
Security
Self-Service
Cost
7
Relational Reference Architecture
Semi- Native
Structured Messaging Data Marts OLAP Cubes Desktop
8
Extended Relational
Reference Architecture
Data Sources Integration Data Storages Analytics Presentation
Semi- Native
Structured Messaging Data Marts OLAP Cubes Desktop
Mobile
Unstructured API Search Engines Devices
Advanced
Analytics Web Services
Selfservice (adhocreporting)
Unstructureddataprocessing
Highdatamodelextensibility
Highdataqualityandconsistency
Extensivesecurity
Reliabilityandfaulttolerance
Lowlatency(nearrealtime)
Lowcost
Skillsavailability
11
Extended Relational vs. Non-Relational Architecture
Extended
ArchitectureDrivers NonRelational
Relational
Largedatavolume
Selfservice (adhocreporting)
Unstructureddataprocessing
Highdatamodelextensibility
Highdataqualityandconsistency
Extensivesecurity
Reliabilityandfaulttolerance
Lowlatency(nearrealtime)
Lowcost
Skillsavailability
12
Extended Relational vs. Non-Relational Architecture
Extended
ArchitectureDrivers NonRelational
Relational
Largedatavolume
Selfservice (adhocreporting)
Unstructureddataprocessing
Highdatamodelextensibility
Highdataqualityandconsistency
Extensivesecurity
Reliabilityandfaulttolerance
Lowlatency(nearrealtime)
Lowcost
Skillsavailability
13
Relational vs. Non-Relational Architecture
Relational Non-Relational
Rational Agile
Predictable Flexible
Traditional Modern
14
Big Data Analytics Use Cases
RealTime
Intelligence
Consumers Intelligent Agents
Performance
Volume Data Business
Discovery Reporting
15
Data Discovery: Non-Relational Architecture
Mobile
Unstructured API Search Engines Devices
Advanced
Analytics Web Services
16
Big Data Analytics Use Cases
RealTime
Intelligence
Consumers Intelligent Agents
DataQuality
Data Business SelfService
Discovery Reporting
17
Business Reporting: Hybrid Architecture
Mobile
Unstructured API Search Engines Devices
Advanced
Analytics Web Services
RealTime
Intelligence
Consumers Intelligent Agents
Data Business
Discovery Reporting
19
Lambda Architecture
Source:
20
Case Study #1: Usage & Billing Analysis
Business Goals:
Provide visual environment for building
Business Area:
custom mobile application Cloud based platform for building, deploying,
Charge customers based on the platform hosting and managing of mobile applications
they are using, number of consumers
applications etc.
21
Architectural Decisions
Architecture Drivers:
Trade-off:
Extended
// Non-Relational
Relational
Extensibility + ExtendedRelationalArchitecture
Data Quality + ExtensibilityviaPreallocated
Self-Service + Fields pattern
22
Technologies:
Solution Architecture Amazon Redshift
Amazon SQS
Amazon S3
Elastic Beanstalk
Jaspersoft BI Professional
Python
23
Case Study #2: Clickstream for retail website
Business Goals:
Build in-house Analytics Platform for ROI measurement Business Area:
and performance analysis of every product and feature
delivered by the e-commerce platform;
Retail. A platform for e-commerce and
Provide the ability to understand how end-users are collecting feedbacks from customers
interacting with service content, products, and features on
sites;
Do clickstream analysis;
Perform A/B Testing
24
Architectural Decisions
Architecture Drivers:
Trade-off:
Extended Non-
//
Relational Relational
Volume/Scalability +/ + NonRelationalArchitecture
Throughput + + ReportingviaMaterializedView
pattern
Self-Service + +/
Extensibility +
25
Technologies:
Solution Architecture Amazon S3
Flume
Hadoop/HDFS, MapReduce
HBase
Oozie
Hive
Node1
Node2
NodeN
26
Tips for Designing Big Data Solutions
27
Clients include:
SoftServe US Office
One Congress Plaza,
111 Congress Avenue, Suite 2700 Austin, TX
78701
Tel: 512.516.8880
Contacts
Serhiy Haziyev: shaziyev@softserveinc.com
Olha Hrytsay: ohrytsay@softserveinc.com
29