June2017 IoT Edge Data Management

Informix
IoT Data Management & Analytics Solution

for the Edge & Cloud
June 2017
Shawn Moe – Informix Engineering Lab
Informix 12.1: The Intelligent Database

Simply Powerful
Agenda
§ The Internet of Things Landscape

§ Data Management at the IoT Edge
§ What about the Cloud?
§ Why Informix 12.1 for both?
§ Informix in the field – IoT applications
§ Wrap it Up….
2 © 2017 HCL Corporation

Changing Business Model – IoT for Health Care & Assisted
Living
Filter critical and life- Notification to Assisted
Ø Automatic sensors saving data Living Central Monitoring Change patients medication,
Station closer monitoring, prevent stroke
to monitor patient Blood pressure threshold
exceeded Display Alerts and
well-being 3 Recommended Actions
Data
Ø New devices: 2 Consolidation
Gateway
Pendants, shower & Locally Act Upon
4
bath buttons Insights
Ø Activity sensors –
rising in the
morning, taking Sensor Data Informix Historian
1
meds, using the Input Operational
• Embedded at Analytics
fridge device/gateway
• Local decision making
at Facility
Ø Bed & Chair sensors • Leverage all data: Collection and analysis of
NoSQL/SQL & 5 data for all devices across
for inactivity Patient/Care giver
Timeseries data
assisted living facilities
monitoring Hundreds of
Assisted Living Corporation
patients
changes food sodium usage
Thousands of
Ø Outside alarms to devices
based on trend of high blood
pressure
alert neighbors
IoT applications have a common set of requirements
Opportunities for innovation
§ Quickly and easily provision new sensors
§ Create a real-time communication channel
with the sensor
§ Capture data from the sensor and store it in
a time series database
§ Provide secure access to the collected data
– analytics at the Edge and Cloud, in real-
time & on historical data
§ Trigger events based on specific data
conditions
§ Interact with the sensor from
business/enterprise applications and/or
from mobile devices
§ Monetize the service based on usage
Simplified IoT Data Flow
Sensors Real-time Analytics Operational Analytics Big Data Analytics
Gateways for local

analytics
(Gateways)
HDFS / Hadoop
Real-time Analytics Big Data Analytics
Informix / Node-
Red
InfoSphere
Publish / Streams Watson / DashDB /
Subscribe BigInsights
Sensor Data History
Cloud Infrastructure
Informix / Cloudant / Watson
IOT Platform Service / Informix SoftLayer / Bluemix
MessageSight /
on Cloud Service
MQTT
(no gateway) Predictive Analytics
PMQ / SPSS /
Cognos
In-memory Analytics
Informix Warehouse
Accelerator / DashDB

IoT requires Analytics and Data Management at each level!
Deep Analytics Zone Insights from BigData
Flexible Hybrid
Data Management
Analytics Zone Streaming Analytics
Message Broker
Smart Gateways
Devices/Sensors

Agenda

§ Wrap it Up….

How Do Smart Gateways Help IoT Solutions?
• Reduce Cost
• Reduces cloud storage by filtering/aggregating/analyzing data locally
• Reduces cloud CPU requirements by precomputing values
• Reduce Latency
• Intelligent gateways can detect and respond to local events as they happen
rather than waiting for transfer to the cloud
• Performs 80% simple operations locally
• Security
• Gateways allow customers to capture and get value from their sensors without
sending data to the cloud
• Protocol Consolidation
• Cloud does not need to deal with the hundreds of IoT protocols. Gateway can
“normalize” data before sending to cloud
More and more IoT processing will move from the cloud to gateway devices

What are the Requirements for a Gateway Database?
§ The database management system must:
- Have a small install footprint, less than 100 MB
- Run with low memory requirements – less than 256 MB
- Use lossless compression to minimize storage space
- Have built-in support for common types of IoT data like time series and spatial/GIS data
- Simple application development environment supporting NoSQL, SQL, and REST
- Require absolutely no administration
- Be able to network multiple gateways together to create a single distributed logical

database
The database must be powerful enough to ingest, process and analyze data in real-time

IoT Requires Highly Optimized Spatial & Temporal Data Processing
§ Many IoT applications have a spatial component to them
• Vehicles, cell phones, even pets…
§ In these cases both location and time is important
• Show me the vehicles that have passed by location X in the last hour
• Where has my car been over the last few hours?

IoT requires the Ability to recognize Patterns and Predict Events
• Industrial IoT is reaching point of stabilization and standardization

• Now looking towards predictive analytics for anomalies and deviations
an abnormal power usage pattern
Similar patterns found

Informix: The Ideal Database for Gateways
Simple to use
• Millions of embedded-use installations
• Hands-Free operation – No administration
Invisible
• Supports popular interfaces such as REST & MongoDB as well as
traditional SQL interfaces (ODBC/JDBC)
• Handles SQL and JSON data in the same database
• Seamless data replication and HA solutions to move or copy data
where needed
Performance
• Unique support for TimeSeries and Spatial data
Agile
• Stream data continuously into the database
• Run analytics operations as data arrives
• Dynamically add and update analytics when needed
• Storage is typically 1/3 the size compared to other vendors
Informix is the only database management system perfectly suited to run in Gateways

Why embed Informix 12.1 at the Edge?
§ Informix is proven technology as an embedded
database. Providing a small footprint with a fully
featured enterprise database server.
§ Built in autonomics with self healing, self

configuration and automation with the DB
scheduler
- Automated space provisioning
- Automated Memory Management
- Dynamically tuned Engine parameters § Embedding Informix in consolidation devices
on the IoT edge allows for:
– Complex store and forward capabilities
with transformation and aggregation of
data
– Business decisions made on the edge,
closer to the producer of the data
– Security policies to determine which
sensor data gets captured, processed
and sent to the cloud
Sensor Data is TimeSeries Data
§ What is a Time Series?
• A logically connected set of records ordered by time
§ What are the key strengths of Informix TimeSeries?
• Native data type resulting in significantly less space requirements
• Typically about 1/3 the space required by other vendors
• Queries run orders of magnitude faster
• Unique optimized storage means more data fits in memory
• Purpose built streaming data loader for sensor data
• Automatically run analytic and/or aggregate functions on new data
• Integrates structured (SQL) or unstructured (JSON) data to store metadata for each
time value
• REST/ODBC/JDBC/JSON interfaces available to work with this data
• API contains hundreds of manipulation & analytics functions + APIs to create your
own analytics
Traditional Sensor data storage vs. Informix TimeSeries storage
Traditional (relational) table storage
Meter_ID Time KWH Voltage ColN
1 1-1-11 12:00 Value 1 Value 2 ……… Value N

2 1-1-11 12:00 Value 1 Value 2 ……… Value N One row for
3 1-1-11 12:00 Value 1 Value 2 ……… Value N every meter for
… … … … ……… … every 15
1 1-1-11 12:15 Value 1 Value 2 ……… Value N minute reading!
… … … … ……… …
Informix TimeSeries storage

Meter_ID Series
1 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
Just one row
3 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …] for each meter

TimeSeries Meter Data Management Benchmark
- 30 million smart meters sending data every 15 minutes
- 2.88 billion records inserted each day
- Workload: data ingestion, data cleanup, and a daily billing cycle
Metric Competitor Informix

Daily processing time 11 hours 5 hour 50 min
Maximum number of cores used 62 32
Maximum amount of memory used 192GB 192GB
Size of database per month of data 15TB 5TB
# Records processed each day 2.88 Billion 2.88 Billion
Billing determinants creation (1/21 of the total 51,322 ~2 million reads per
meter population) second

Agenda

§ Wrap it Up….

What are the IoT Requirements for a Cloud data storage solution?
• Requirements are similar to gateways, but for different reasons:
• Potentially 1000’s of servers means zero administration is a must
• Data volume adds up very quickly so low storage overhead is required
• Data flows into the cloud continuously and must be processed in real-time
• Must be able to handle time series, spatial, SQL and NoSQL data natively
• Additional requirements
• Must be able to scale-out & scale-up
• Must be available as a service

Why Informix 12.1 for both Edge & Cloud?
Enterprise class database for the Edge and the Cloud
§ Enterprise database embedded in gateways and
consolidators based on ARM or Intel/Quark
processors – maximizing availability in this space
§ Horizontal scaling with sharded data across ER –
delivering elasticity for the Cloud
§ SoftLayer and Bluemix support – delivering
Informix on Cloud Service for SaaS offering
§ Multi-tenancy support – allowing hosting of
multiple logically independent server instances
within on single physical instance
§ Delivering cost benefits on hardware resources Informix Server Instance
and software licenses
DB DB DB
§ Simplified administration for backup of multiple Tenant Tenant Tenant
database servers in Cloud A B C
§ Accelerate all types of data with in-memory

Informix Warehouse Accelerator

Informix Hybrid Cloud enables seamless control and
flexibility
Put your data where you need it, when you need it
• Utilize any machine from anywhere –

either on or off premise
• Single platform for both on & off-
premise
• Scalability for peak periods
• Interconnectivity with IoT data
• Maximize existing resources
• Reduce costs and IT footprint
• The same hybrid apps (using both
structured and unstructured data) can
run against any machine in your hybrid
cloud (both on & off premise machines)
• Tremendous Flexibility!

l Informix is available on now available on IBM Bluemix
IBM Informix on Cloud
• Cloud hosted service includes Informix license and cloud “hardware”
• T-shirt sizing: S, M, L, XL instances match Informix license and hardware
capacities to provide optimal value at each size
• Informix instance hosted in IBM SoftLayer data centers with world wide
deployment options
• IBM provisions, configures, and tests the instance and then passes the
credentials on to the customer
• Full Informix functionality to support all kinds of work loads:
• OLTP
• Hybrid NoSQL, SQL, TimeSeries and Spatial
• IoT
• IWA
• Rapid application development with support for SQL, MongoDB, REST or
MQTT themed applications
• It’s Informix!
Rationale: deliver high-quality cloud service with low cost of operations

Agenda

§ Wrap it Up….

Why Informix 12.1? – Flexible Set of Data Availability Options
1. High Availability Data Replication (HDR)

2. Enterprise Replication (ER)
Mix and match these technologies
3. Flexible Grid
-Remote Standalone Secondaries (RSS)
-Shared Disk Secondaries (SDS)
4. Sharding
Benefits of an Informix data availability solution:

- Scale globally
- Manage easily
- Balance workloads Informix has the most complete set of
- Rolling version upgrades Data Availability options in the industry!
- Heterogeneous platform support
- Ideal for the Cloud
Shards: Scale-out your Database across Servers or Gateways
• Distribute data among servers by

range or hash partitioning
• Each shard can have an associated
secondary server for high Shards in a Cloud
availability
• Run queries across all shards or a
subset of the shards
• Only shards that could qualify are
searched
• Shards are searched in parallel
• Ignores shards that are offline

Sharding + Informix HA means your data is always available!
Shared Disk
Secondary Remote
Standalone
Secondary
Shard Key
Country = “FR”
Shard Key
Country =“DE”
Shard Key
Country = “UK”
HDR pair

Informix Flexible Grid is ideal for the cloud environment
Sales Database
• Vertical & horizontal scaling Shards
– Adding physical resources in the cloud & with
the Informix Grid easily add a node to the
Informix cluster
• Quick response to business needs

– Allocate or de-allocate resources in the public
or private cloud as needed (matter of minutes)
• Data Flexibility
– Balance data across the nodes in a cluster
– Query sharded data, data spread out across
the nodes in the cluster
– Layer of abstraction between user and where
the data is actually located Informix Connection Manager
Seamless
queries of data
sharded across
26 multiple servers © 2017 HCL Corporation
Why Informix 12.1? – Flexibility for App Developers
Access to Relational Tables & JSON Collections
- Flexibility in schema – or schema-less
Relational Table JSON
applications Collections
• Ease of application development, SQL API Standard ODBC, Direct SQL Access.
reducing cost JDBC, .NET, Dynamic Views
• Faster to market OData, etc. Row types
Language SQL.
- Native support for JSON and binary JSON MongoDB Mongo APIs for Mongo APIs for
(BSON) data types API Java, Javascript, Java, Javascript,
(NoSQL) C++, C#, etc. C++, C#, etc.
- Data Access is not restricted by Data
Models
• TimeSeries and JSON can co-exist with
traditional SQL in the same DB
– Rapid application & services development support
• Enhanced API support
• REST (REpresentational State Transfer) API support
enables developers to use any programming paradigm
that supports HTTP
Why Informix 12.1? - All Clients can access all Data Models
MongoDB • NoSQL ↔ SQL

Client Informix Wire Translation
Listener • Wire Listeners for
Mobile
REST Client MongoDB, REST &
MQTT protocols
MQTT Client
Desktop
• SQLI, DRDA Protocol
Informix Support
SQLI Client DBMS
• Relational, Collection,
Time Series, & Spatial
Web Relational Table Data Support
DRDA Client JSON Collection
Time Series
Spatial

Why Informix 12.1? – Query Acceleration with Informix Warehouse
Accelerator (IWA)
Intelligent Database for Business Intelligence
§ IBM BLU (In-memory) technology runs on the highly

compressed Data Marts
- No indexes required
- Query predicates are evaluated in compressed
format – no need to uncompress to evaluate
§ Storage and compression techniques allow memory
requirements to be 1/3 to 1/5 of underlying OLTP Warehouse Accelerator
storage
§ Accelerates all types of data directly or using views
Data Marts
§ Support for OLAP SQL functions which enhance
performance and integration with BI tools

Why Informix 12.1? – Query & Warehouse Acceleration
You can use IWA’s In-Memory Analytics to Speed Up queries on…
Synonyms NoSQL Data SQL Data

to local or remote tables ex: JSON collections from local Informix tables
External Tables Views Sensor data

ASCII/binary files in file to Informix tables or TimeSeries data,
system or network pipes NoSQL data time-stamped data
Why Informix 12.1? – Query & Warehouse Acceleration
IWA Use Case – Pharmacy Chain in Mexico
Data Volume Tables in Data Mart Number of Rows
800 Gigabyte
Compressed IWA data mart
59 Tables
12 Fact Tables in biggest datamart
1.2B
In largest Fact Table
Rows
Over 2 TB of data in Informix/IDS 10 Fact Tables are for end user queries 3.6 Billion rows in IDS
6 Months of data
IWA has latest 6 months of data
12.10 Ver.
Informix/IDS + IWA 12.10.FC2
75x Faster
23x: 90min (IDS) -> 4min (IWA)
IDS has 18 months of data 75x: 10min (IDS) -> 8sec (IWA)
IWA’ s 6 last months data kept Production Roll-Out Plan: Nov 2013 IDS’ longest queries took 40-90min
through: Month-Cyclic Roll-in/out
No more need to summarize data IWA’s longest queries take 2-6min
Removed some indexes in Informix/IDS
Agenda

§ Wrap it Up….

IoT in Retail – Sample Informix Deployment
Enabling a better consumer experience
Data Input – Sensors Device data
in Retail Store consolidation
Sensors monitoring people
Gateway
Deals/Promotions/
traffic, smart shelves, PoS, Real-time Response Smart Shopper
Coupons
Vending machines, carts.. on Alerts and Streaming data
Analytics InfoSphere Streams
MessageSight
NoSQL/Relational & BigInsights

Timeseries data
Device protocols
including CoAP,
6LoWPAN, ZigBee
etc.. Collection of data for all
devices across the enterprise
Embedded SoftLayer /
Informix Cluster in the Cloud
BlueMix
Hints/Suggestions
Correlation/Comparison
Detection/Predictions
SPSS/Cognos
Store Informix Warehouse Accelerator

Manager/Supervisor Real-Time Analytics

IoT in Agriculture – Sample Informix Deployment
Enabling a smarter grain management system – emphasizing food safety & quality
Data
Center /
Smart Gateway Cloud
Temperature Sensors (wifi) Wifi
Inside and outside the bin module
via EOP
Humidity sensor (wifi) 3G/4G cell
Every 1 minute
Inside and outside the bin communication
module
Fan & sensor (wifi) Web

Server Console Dashboard
Mobile App for

bin operators
OnSite Weather Data
• Temperature
Every 1 minute
• Pressure
• Humidity
• Rainfall
• Wind
Direction
• Wind Speed
IoT in Transportation - Smarter Vehicle Maintenance
Enabling a smarter transportation system – emphasizing safety & cost savings
§ Analyze
- Suspension, wheels, alignment, noise & vibration
- Wear on mechanical components: bearings, gears, belts
- Human responses
§ Insights from data helps with:
- Safety improvements
- Preventive maintenance scheduling
- Testing & simulation scenarios
- Capacity & route planning
- Much more ….
§ Thousands of different types of sensors involved
§ Possible IoT solution architecture could include:
- Informix at the “mobile Edge” to support immediate
actions and near real-time analytics
- Informix in the Data Center/Cloud for operational
analytics
35
Informix offers Enterprise Class Embedded Data Management
Call Management
A Call Management Solution that provides call features, such as call forwarding,
automated voice, etc. in a geographically distributed, highly available
environment
Benefits:
§ Flexible solution to today’s call features
§ Flexible architecture allows solution to fit into a wide variety of environments,
without changes
§ Cost performance allows competitive solution in a battleground technology
Why do Customers choose Informix based solutions?

Low Cost – Low Administration- High Performance – High Availability – 99.999% Uptime- Accelerate time to market

Agenda

§ Wrap it Up….

Summary
§ Informix - best fit for IoT architecture
§ IoT Smart Gateway
§ IoT Cloud analytics
§ Supported on a wide array of platforms, including SoC computers
§ Best in class embeddability
§ Native support for sensor data - TimeSeries & Spatial data
§ Native support for unstructured (JSON) data
§ Ease of application development - REST access
§ Support to receive IoT data via MQTT protocol
§ High availability and dynamic scaling
§ In-memory query acceleration

Some Useful Information
• Bloor White Paper – IBM Informix and the Internet of Things - http://ibm.co/2bITDyU
• IBM Informix - http://www-01.ibm.com/software/data/informix/
• IBM Informix Support - http://www-
947.ibm.com/support/entry/portal/overview/software/information_management/informix_servers
• IBM developerWorks pages for Informix -
http://www.ibm.com/developerworks/data/products/informix/
• Informix International User Group (IIUG) - http://www.iiug.org/index.php
• Planet IDS - http://planetids.com/
• IBM Informix on LinkedIn -
http://www.linkedin.com/groups?home=&gid=4029470&trk=anet_ug_hm
• IBM Informix on Facebook - https://www.facebook.com/IBM.Informix
• IBM Informix on Twitter - https://twitter.com/IBM_Informix
• IBM Informix Blogs (a few of them):
• https://www.ibm.com/developerworks/community/blogs/smoe/?lang=en
• https://www.ibm.com/developerworks/community/blogs/idsteam/?lang=en
• http://www.ibmnosql.com/author/jmiller/
• https://www.ibm.com/developerworks/community/blogs/fredho66/?lang=en_us
• https://www.ibm.com/developerworks/community/blogs/idsdoc/?lang=en_us
IoT Developers - Get Started!
Docker Hub
§ IBM Smart Gateway kit - https://ibm.biz/BdXr2W

§ Code samples - https://ibm.biz/BdX4QV
§ Github - https://github.com/IBM-IoT/
§ Free Informix Developer Edition - https://ibm.biz/BdXp2g
§ Informix on Docker Hub https://registry.hub.docker.com/u/ibmcom/informix-innovator-c/
§ Informix Developer Edition for Raspberry Pi (32bit)
https://registry.hub.docker.com/r/ibmcom/informix-rpi/

Shawn Moe – smoe@hcl.com

Informix 12.10: Simply Powerful
28

June2017 IoT Edge Data Management

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

June2017 IoT Edge Data Management

Hochgeladen von

Copyright:

Verfügbare Formate

Informix

IoT Data Management & Analytics Solution

Informix 12.1: The Intelligent Database

§ The Internet of Things Landscape

2 © 2017 HCL Corporation

Sensors Real-time Analytics Operational Analytics Big Data Analytics

Gateways for local

Sensor Data History

5 © 2017 HCL Corporation

Deep Analytics Zone Insights from BigData

Analytics Zone Streaming Analytics

6 © 2017 HCL Corporation

§ The Internet of Things Landscape

7 © 2017 HCL Corporation

8 © 2017 HCL Corporation

- Have a small install footprint, less than 100 MB

- Run with low memory requirements – less than 256 MB

- Use lossless compression to minimize storage space

- Simple application development environment supporting NoSQL, SQL, and REST

- Require absolutely no administration

- Be able to network multiple gateways together to create a single distributed logical

9 © 2017 HCL Corporation

10 © 2017 HCL Corporation

• Industrial IoT is reaching point of stabilization and standardization

Similar patterns found

11 © 2017 HCL Corporation

12 © 2017 HCL Corporation

§ Built in autonomics with self healing, self

1 1-1-11 12:00 Value 1 Value 2 ……… Value N

Informix TimeSeries storage

15 © 2017 HCL Corporation

Metric Competitor Informix

16 © 2017 HCL Corporation

§ The Internet of Things Landscape

17 © 2017 HCL Corporation

18 © 2017 HCL Corporation

§ Accelerate all types of data with in-memory

19 © 2017 HCL Corporation

• Utilize any machine from anywhere –

20 © 2017 HCL Corporation

Rationale: deliver high-quality cloud service with low cost of operations

§ The Internet of Things Landscape

22 © 2017 HCL Corporation

1. High Availability Data Replication (HDR)

Benefits of an Informix data availability solution:

• Distribute data among servers by

24 © 2017 HCL Corporation

25 © 2017 HCL Corporation

• Quick response to business needs

MongoDB • NoSQL ↔ SQL

28 © 2017 HCL Corporation

§ IBM BLU (In-memory) technology runs on the highly

29 © 2017 HCL Corporation

Synonyms NoSQL Data SQL Data

External Tables Views Sensor data

Data Volume Tables in Data Mart Number of Rows

§ The Internet of Things Landscape

32 © 2017 HCL Corporation

NoSQL/Relational & BigInsights

Store Informix Warehouse Accelerator

33 © 2017 HCL Corporation

Humidity sensor (wifi) 3G/4G cell

Fan & sensor (wifi) Web

Mobile App for