Teradata and Hortonworks

The Unified Data Architecture (UDA)

16th October, 2014

Shift from a Single Platform to an Ecosystem

"Logical" Data Warehouse

The hype around replacing the data

warehouse gives way to the more
sensible strategy of augmenting it
The influence of the logical data
warehouse has created a situation in
which multiple repository strategies are
now expected.
2

Big Data requirements are solved by

a range of platforms including
analytical databases, discovery
platforms, and NoSQL solutions
beyond Hadoop.
Source: Big Data Comes of Age. EMA and 9sight
Consulting. Nov 2012.

UNIFIED DATA ARCHITECTURE

System Conceptual View

ERP

MOVE

MANAGE

ACCESS
Marketing

Marketing
Executives

Applications

Operational
Systems

Business
Intelligence

Customers
Partners

SCM

INTEGRATED DATA WAREHOUSE

CRM

Images

DATA
PLATFORM

Audio
and Video

Machine
Logs

Data
Mining

INTEGRATED DISCOVERY PLATFORM

Frontline
Workers

Business
Analysts
Math
and Stats
Data
Scientists

Text
Languages
Web and
Social

Engineers

USERS
SOURCES

ANALYTIC TOOLS
& APPS

UNIFIED DATA ARCHITECTURE

Business Conceptual View

ERP

MOVE

MANAGE

ACCESS
Marketing

Marketing
Executives

Applications

Operational
Systems

Business
Intelligence

Customers
Partners

SCM

INTEGRATED DATA WAREHOUSE

CRM

DATA
PLATFORM
Images

Audio
and Video

Machine
Logs

Text

Web and
Social

Fast Data Loading

& Availability

Business Intelligence
Predictive Analytics
Operational Intelligence
Data
Mining

Filtering &
Processing
Data Mgmt.
(data lake)
Deep History:
Online Archival

INTEGRATED DISCOVERY PLATFORM

Business
Analysts
Math
and Stats

Data Discovery
Fast-Fail Hypothesis Testing

Frontline
Workers

Data
Scientists
Languages

Path, graph, time-series analysis

Engineers

Pattern Detection
USERS
SOURCES

ANALYTIC TOOLS
& APPS

Discovering Deep Retail Insights with UDA

Transforming Web Walks into DNA Sequences
Situation
Largest German online retailer, conglomerate with numerous
brands and 50 websites. 1 Millions visitors, viewing 2M
products.
Problem
Needed a better way of analyzing consumer behavior on the
websites, communicating with category managers
Solution
Treat each web visit sequence like DNA sequence. Built a fast
query tools so analysts can express queries easily for their
categories, get deeper insights

Impact

Leverage Aster platform to generate rapid path insights

Drives 15% increase in market baskets through personalization
Drives 10-20% increase in conversions by shortening paths
Can now see what does and doesnt lead to sales
Widening use across all the Corporate Group websites

Modern Data Architecture: Teradata

TVI Proactive system monitoring tied to Teradata customer support

Alerts

Viewpoint

Services

System
Health

Node
Health

Space
Usage

DB

Customer/
Inventory
Data
Clickstream
Data

KNOX
JDBC/ODBC Compliant
Tool

AMBARI

File

MAPREDUCE
YARN

Bidirectional

JMS

BULK COPY
HDFS

REST

EXTRACT

Flat Files

LOAD

REFINE

SQOOP

HIVE

FLUME

PIG

NFS

ETL

Web HDFS

CUSTOM

HTTP

Sentiment
Analysis
Data

Metrics
Analysis

Query/Visualization/
Reporting/Analytical
Tools and Apps

SOURCE DATA

Sensor Log
Data

Capacity
Heatmap

DISTCP
STRUCTURING
HCATALOG
EXPORT
SQOOP / HIVE

Streaming

AFS

Analytical
Platforms

INTERACTIVE
QueryGrid

Aster Discovery
Platform

LOAD
TDCH
Teradata IDW

EXTRACT

Teradata Portfolio for Hadoop

Bringing Hadoop to the Enterprise

Most Trusted and Flexible Hadoop Platforms for Your

Next-Generation Unified Data Architecture
1. Teradata Aster Big Analytics Appliance
2. Teradata Appliance for Hadoop
3. Teradata Commodity Offering with Dell
4. Hortonworks Data Platform software-only support resell

Complete consulting and training capability

>Big Analytics Servicesacross the UDA
>Data Integration OptimizationETL, ELT across the UDA
>Hadoop deployment and mentoring
>Teradata delivering Hortonworks training
>Hadoop Managed Servicesoperations and administration

Customer Support for Hadoop

>World-class Teradata customer support, backed by Hortonworks
7

Teradata Loom 2.3

Integrated metadata management, data lineage

and data wrangling for Enterprise Hadoop
Loom is a platform for profiling, preparing and tracking data lineage for data
in Hadoop
Hadoop Data Governance and Metadata Management
Rich information model for capturing and managing the relationships
Data dictionary for the big data landscape
Support for non-Hadoop sources

Free version of Loom pre-installed with

Hortonworks Sandbox

Automation (Activescan)

Discovering and introspecting new data in the cluster

Triggering external processing (e.g. Oozie script for ETL)
Automatically collecting metadata about the job - lineage, statistics
Polling YARN job history for lineage

User Interactivity (Workbench)

Advanced user interfaces for data exploration, profiling and preparation
Data wrangling for interactively cleaning/reshaping raw data into useable data
8

Teradata Appliance for Hadoop

Teradata QueryGrid

Teradata Studio with

Smart Loader

Value Added Software from Partners

Kerberos

Teradata Viewpoint

HCatalog

Teradata Connector for Hadoop (TDCH)

Intelligent Start and Stop
NameNode Failover
Teradata Distribution for Hadoop
(Based on Hortonworks HDP)
Optimized hardware for Hadoop
BYNET V5 40GB/s InfiniBand interconnect

Teradata Vital Infrastructure

Teradata Loom ( for data management )

Teradata QueryGrid Vision

IDW

Business users

TERADATA
DATABASE

10

HADOOP

TERADATA
ASTER
DATABASE

Push-down
to Hadoop
System

SQL,
SQL-MR,
SQL-GR

Discovery

Data Scientists

TERADATA
ASTER
DATABASE

TERADATA
DATABASE

RDBMS
DATABASES

MONGODB
DATABASE

COMPUTE
CLUSTER

Multiple
Teradata
Systems

Push-down
to Other
Database

Push-down
to NoSQL
Databases

Run SAS, Perl,

Ruby, Python, R

Teradata QueryGrid: Teradata - Hadoop

Give business users on-the-fly access to data in Hadoop

Trusted: Use existing tools/skills and enable

self-service BI with granular security

QueryGrid: Teradata-Hadoop
QueryGrid: Aster-Hadoop

Standard: 100% ANSI SQL access to

Hadoop data
Data Filtering

Efficient: Intelligent data access

leveraging the Hadoop HCatalog

Data

Fast: Queries run on Teradata or Aster,

data accessed from Hadoop

Hadoop
MR

HCatalog

Hive

Pig

Hadoop Layer: HDFS

11

Teradata Viewpoint
Single Operational View (SOV)
for Teradata, Aster, & Hadoop
Hadoop Portlets:
Node Monitor (Aster & Hadoop)
Hadoop Services

Integration into existing:

Monitoring: System Health, Metrics
Analysis, Metrics Graph, Capacity
Heatmap, Space Usage.
Admin: Alert Viewer, Alert Setup,
Teradata Systems, Role Manager

12

Teradata Connector for Hadoop (TDCH)

Key Features
High-speed connector between Teradata and
Hadoop based on Apache Sqoop framework
Both import and export data between Teradata and
Hadoop
Leverages the JDBC-FastLoad/FastExport mechanism
from Teradata
Import/export Hive rcfile/sequencefile/textfile format
and Hive partitioned files

INTEGRATED
DATA WAREHOUSE

CAPTURE | STORE | REFINE

Available through Hortonworks

> Hortonworks
Teradata Connector for Apache Hadoop (Release v1.2.0)
Download link: http://hortonworks.com/download/

13

Teradata Studio: Smart Loader for Hadoop

Self-Service Load
Hadoop View
Browse through tables
within the Hadoop cluster
- Views table properties

Bi-directional table copies

- Drag and drop interface
- Maps data types between Hadoop
and Teradata tables

Transfer Status and History

- Track load status

Benefits
Simplifies Hadoop browsing
Ad hoc data movement between
Teradata and Hadoop
No scripting required
Point and click

14

Questions and Next Steps

More about Teradata & Hortonworks
http://www.hortonworks.com/partner/teradata/

Teradata Loom for HDP

http://www.teradata.com/tryloom

Find Us
@Strata
15

Booth # 324
Teradata Hadoop Station