EDR Concepts & Facilities

(c) E-Net 2012

January 2012
Information all day, every day.
Enterprise Data Replicator
Concepts and Facilities
2
Total Application Resilience
E-Nets Enterprise Data Replicator sets the new
standard in database application availability. EDR
provides replication and propagation solutions
designed specifically for real-world applications.
Your applications: High-value. High-volume. Critical
systems which have the ultimate availability
requirements. Not 99%. Not 99.99999%......Systems
that require 100% uptime.
EDR provides industrial-strength real-time peer-to-
peer replication you can deploy with confidence.
If you have an application which absolutely must
be available at all times, you should consider
EDR.
EDR Concepts & Facilities
(c) E-Net 2012
3
EDR: Solutions
VENDOR EXPERIENCE
EDR builds on E-Nets twenty-five years of experience with
mainframe database replication and offsite recovery solutions, for
both continuous availability and disaster recovery applications.
LEAN & CLEAN ARCHITECTURE
EDR uses a completely new and redesigned software architecture
exploiting todays best practices in development and quality
assurance. It provides the leanest and cleanest high-end data
replication capability.
EXTENDED DISTANCE
EDR addresses todays and tomorrows demands for completely
available and responsive mainframe database applications across
potentially many nodes, distributed across the globe, separated by
thousands of miles.
4
EDR: Solutions
GOES BEYOND MIRRORING
EDR goes well beyond the capabilities of any hardware mirroring
approach, which are by design uni-directional. EDR does not
require any proprietary hardware
NETWORK COMPATABILITY
EDR fully exploits todays and tomorrows network configurations
and topologies, with full TCP/IP and Websphere MQ support
(future)
BULLET-PROOF
EDR automatically recovers from unforeseen system outages,
abends and operational errors with no impact on users
EASE OF OPERATION
EDR is easily maintained and operated, enabling true 100%
continuous operation 24 by 7 by 365. Your applications are
always up. Your information is available all day every day
EDR Concepts & Facilities
(c) E-Net 2012
5
Business Requirements
Scenario A Replication
One site normally handles update workload, but
switch-over to backup site is instantaneous
All sites have Referential Integrity, Triggers, Stored
Procedures, etc.
Backup site is ready to go at all times.
Switch-back to original primary site is fast and simple
Load balancing
Inquiry transactions and other read-only work can use
backup copy
Handles both scheduled and unscheduled outages
with identical operational procedures
6
Replication Scenario A
Switch-over & Switch-back
Normal Operations
Site 1 Site 2 Changes from Site 1 are
continually applied to replica
database at Site 2
Database at Site 2 available
for read-only queries/reports
Switch-over
Fail-over
Database at Site 1 unavailable
replica database at Site 2
becomes the production
database instantaneously!
PROD
PROD
Catch-up Phase
Updates which occurred at
Site 2 are applied to original
production database at Site 1
PROD
Switch-back
Fail-back
Return to normal operations
complete
PROD
PROD?
EDR Concepts & Facilities
(c) E-Net 2012
7
Business Requirements
Scenario B Peer-to-Peer Replication
Multiple sites simultaneously handling update
workload
Virtual Data Center concept
Enables unrestricted load balancing across multiple
sites
A given user may be normally assigned to his/her
preferred site but can be immediately redirected to an
alternate site when preferred site is not available
Need to resolve logical collisions which are possible
due to involvement by multiple commit managers
All sites have Referential Integrity, Triggers, Stored
Procedures
8
Peer-to-Peer Scenario B
Multiple Update Locations
East Coast
West Coast Central
EDR Concepts & Facilities
(c) E-Net 2012
9
EDR: Peer-to-Peer Scenario B:
ber-application layer
Data Center A
Application
DB2 for z/OS
EDR
Data Center C
Application
DB2 for z/OS
EDR
Data Center B
Application
DB2 for z/OS
EDR
ber-application layer
10
Enterprise Data Replicator - Goals
Reliability
The cornerstone to E-Net's ongoing success is reliability -
guaranteed data movement from DB2 to DB2 without compromising
data integrity or the operating system environment. In the world of
continuous availability the ability to recover from abends or
unforeseen system outages must have no impact on the users of
the system.
Performance
Reliability is of utmost importance; however, of equal concern is
performance. As the performance leanness of the E-Net Log Apply
engine has always been acknowledged by our customers, it is a key
goal for EDR. E-Net designed and implemented EDR based on
years of experience handling real-world applications having both
high transaction rates and large databases.
EDR Concepts & Facilities
(c) E-Net 2012
11
Enterprise Data Replicator - Goals
Flexibility
EDRs engine is structured and flexible enough to take advantages
of any new or improved features for any DB2 version. An example
would be the ability to adapt to a new DB2 data type.
Maintainability
EDR is easily maintained and by design anticipates true continuous
operations (24 by 7). EDRs engine is able to switch between
software levels without impacting current users of the system. In
addition EDR is software back level tolerant. This means that it
permits different versions of the engine to execute concurrently with
no impact to users. The engine itself contains easily distributable
and maintainable modules implemented and architected by design.
Compatibility
Compatibility with the existing versions, and future versions of DB2
and other DBMS products are a must. Downward compatibility
between and among versions of EDR is also supported.
12
EDR OCEAN and OAR Architecture
EDR: OCEAN
Open Communications
Environment Across Networks
EDR: OAR
Open Architecture Replication
User interface
3270
Web client
EDR Concepts & Facilities
(c) E-Net 2012
13
EDR: OCEAN - Features
TCP/IP or MQ communications layer
Native TCP/IP support
Supports dynamic VIPA
Websphere MQ Series support
Use non-persistent queues for performance
Support for Websphere MQ Series not yet available, no customer
demand
Client/server user interface architecture
XML-like packages
Standard web app (browser) interface
Also providing 3270 (green screen) support via TSO
Extensible and customizable user interface
Batch APIs for UI functions
14
EDR: OCEAN Topology
EDR Concepts & Facilities
(c) E-Net 2012
15
EDR: OCEAN Configurations
16
EDR: OARs
o OAR/DB2
o Basic capture and apply propagation engine
o OAR/Web
o Supports web clients
o Administration, operations, performance management
o OAR/Sync
o Populates target tables from source
o OAR/Security
o Comprehensive control of all resources, via external interface
EDR Concepts & Facilities
(c) E-Net 2012
17
EDR: OAR/DB2 - Features
Real-time DB2 replication
Exploits DB2 Change Data Capture
Uses efficient push log capture technology
Peer-to-peer support
Effective for both Scenario A and Scenario B deployments
Highly automated operation
Fully dynamic and automatic activation for new objects
Ability to tolerate DDL changes at source site with no outage at
target
Data sharing support at both source and target
Enables true parallelism exploitation
Supports LOBs
External Security Interface for RACF, CA-Top Secret, CA-
ACF2
18
EDR: OAR/DB2 - Features
Full DB2 Version 8 and Version 9 exploitation
Supports variety of migration strategies
Different levels of DB2 supported within a sphere
New DB2 V9 data types and Reorder Row Format (RRF)
Highly efficient parallel apply mechanism
Lean and Clean
Partition-level apply for partitioned table spaces
Specially-designed exploitation of static and dynamic SQL to
apply changes at a target location
Support for referential integrity (RI), triggers and
stored procedures
No requirement for a unique index on target object
Ability to handle long running URs
Customizable conflict resolution
EDR Concepts & Facilities
(c) E-Net 2012
19
OAR/DB2 Capture, Apply & User Interface
OAR/DB2
Apply
OAR/DB2
Capture
DB2
for z/OS
OAR/DB2
Apply
OAR/DB2
Capture
DB2
for z/OS
OAR/DB2
Apply
OAR/DB2
Capture
DB2
for z/OS
Ocean A
Ocean C
Ocean B
User interface
20
EDR: OAR for DB2
OAR/DB2
Capture
OAR/DB2
Apply
EDR Concepts & Facilities
(c) E-Net 2012
21
EDR: OAR/DB2 Restart/Reconnect
22
EDR: OAR/DB2 DB2 Restart
EDR Concepts & Facilities
(c) E-Net 2012
23
EDR: OAR/DB2 Data Sharing - 1
24
EDR: OAR/DB2 Data Sharing - 2
EDR Concepts & Facilities
(c) E-Net 2012
25
EDR: OAR/DB2 RRP Sample 1
26
EDR: OAR/DB2 RRP Sample 2
EDR Concepts & Facilities
(c) E-Net 2012
27
EDR: OAR/DB2 RRP Sample 3
28
EDR: OAR/DB2 RRP Sample 4
EDR Concepts & Facilities
(c) E-Net 2012
EDR: OAR/Web Sample 1
29
30
EDR: OAR/Web Sample 2
EDR Concepts & Facilities
(c) E-Net 2012
EDR: OAR/Web Sample 3
31
EDR: OAR/Web Sample 4
32
EDR Concepts & Facilities
(c) E-Net 2012
33
EDR: OAR/DB2 RO Sets
> Synchronization
and apply are done
at the RO Set level
> Insures integrity
among related
objects
34
EDR: Logical Sync Feature
Source
Database
Target
Database
OAR/DB2
OCEAN A
OAR/DB2
OCEAN B
Logical Sync Logical Sync
Facilitates cross-site copying (synchronization)
Typically used to populate target from source
Fully automated
Fully integrated with OAR for DB2 Capture/Apply
Requires no outage of any kind
Uses highly efficient TCP/IP communications
Optional use of DSNUTILS at target DB2 for loading data
Supports Synchronization of Related Objects (RO Sets)
EDR Concepts & Facilities
(c) E-Net 2012
35
EDR: Physical Sync Feature
Source
Database
Image Copy
Target
Database
OAR/DB2
OCEAN A
OAR/DB2
OCEAN B
EDR transmits image copy or
customer can manage FTP
Standard DB2
Image Copy Utility
DSN1COPY
Import Process
Physical Sync Physical Sync
Image Copy
36
EDR: Logical Audit Feature
Database
Copy 1
Database
Copy 2
OAR/DB2
OCEAN A
OAR/DB2
OCEAN B
Logical Audit Logical Audit
Permits non-disruptive comparison of copies
Fully automated
Requires no outage of any kind at either site
Examines tables row-by-row at consistent points in time
Checksums used to validate that data is in sync
EDR Concepts & Facilities
(c) E-Net 2012
37
Conflict Detection & Resolution
Detection Level One - SQL Error
Types of resolution:
Stop on Any SQL error
Able to automatically resolve Row not found (+100) and Duplicate
(-803) SQL errors
Ignore but report error
Advantages
Minimal overhead
Can assure data consistency in peer to peer unidirectional update
environment
Disadvantages
Cannot detect all types of conflicts
Updates may be lost due to undetected conflicts
Conflict Detection & Resolution
Detection Level Two - Content Based
Read/select done before update of a given row
Types of resolution:
Stop on any conflict or SQL error
May pass conflict or SQL error to an object related user defined
exit for resolution
Advantages
Detects all conflicts
User control of conflict or SQL error
Disadvantages
Increased CPU and bandwidth overhead
Unable to resolve conflicts at the column level
Risk of user exit ABEND if not coded properly
38
EDR Concepts & Facilities
(c) E-Net 2012
39
EDR: DB2 Log Capture
Multiple log capture techniques increasing flexibility
Generally preferred technique: DB2 IFI
Instrumentation Facility Interface
DB2 presents change information on objects identified by EDR
Exploits log merge capability in parallel sysplex data sharing
configurations
Alternative Technique: log capture exit
Used successfully by E-Nets RRDF product
Minimal overhead
Alternative Technique: direct reads against DB2 active log
data sets
Access to log archives via BSDS
40
EDR: DB2 Log Capture via Exit
Uses standard
DB2 log capture
interface DSNJL004
Runs in SRB
mode
Highly efficient
push interface
No I/O performed
EDR Concepts & Facilities
(c) E-Net 2012
41
EDR: DB2 Log Deconstruction
42
EDR: Transmit toTarget Nodes
EDR Concepts & Facilities
(c) E-Net 2012
43
EDR: OAR/DB2 Merge Process
44
EDR: OAR/DB2 Statistics & Monitoring
Feature
EDR Concepts & Facilities
(c) E-Net 2012
EDR: OAR/Web Statistics &
Monitoring Feature for OAR/DB2
45
EDR: OAR/Security
The OAR/Security allows for other EDR OARs to
check for resource authorization via external security
packages like RACF, CA-Top Secret, or CAACF2
For TSO user id is collected from the TSO address space
For Batch user id is collected from the batch address space
For WEB a sign on screen is presented to the client
OAR for DB2 resources
OAR/DB2 access
OAR/DB2 menu access
OAR/DB2 database access/update
46
EDR Concepts & Facilities
(c) E-Net 2012
47
EDR: Debugging Feature
EDR: Performance and Tuning
EDR will dynamically tune itself based on the
environment
TCP/IP bandwidth and target data acceptance rates are
determined, and source deconstruction is throttled
accordingly.
TCP/IP buffering for EDR may be changed by
SETSOCKOPT TCP/IP API to maximize communication
performance.
EDR allows for manual intervention.
Operator commands are available to throttle log deconstruct
rates.
User Interface allows for the dynamic setting of internal timer
intervals and some parameter settings.
48
EDR Concepts & Facilities
(c) E-Net 2012
EDR: Messaging and Alerts
All messages from EDR are formed in a standard z/OS format
and are written to a SYSOUT DD statement called SYSTERM.
Example: EDR1804I 3910 00480 DSTARGET Original
source code generated for GBC10D86
EDR1804I is the message number
Last character ( I ) represents the message severity:
I Informational
W Warning
S Severe
Messages may be found in QuickRef
Severe level messages are also written to the system console for
use by automation products
Customer may state in input file to exclude or include messages
to be written to console and/or SYSTERM.
49
50
EDR: Data Sharing Configuration
Scenario B - True Peer-to-Peer Replication
Both Sites Support Updates Simultaneously
Site 1 Data Sharing Group
Parallel Sysplex
Site 2 Data Sharing Group
Parallel Sysplex
DB2 Subsystem
Member #1
DB2 Subsystem
Member #2
DB2 Subsystem
Member #3
EDR OCEAN
Capture Only
EDR OCEAN
Capture
Merge/Apply
EDR OCEAN
Capture Only
EDR OCEAN
Capture
Merge/Apply
DB2 Subsystem
Member #1
DB2 Subsystem
Member #2
DB2 Subsystem
Member #3
EDR OCEAN
Capture Only
EDR OCEAN
Capture Only
Production Update Site Production Update Site
EDR Concepts & Facilities
(c) E-Net 2012
Specialty Engine Exploitation
Exploitation of IBM's System z Integrated Information
Processor (zIIP)
Approximately 70% of EDR CPU overhead eligible
for dispatch on zIIP
Using approved API via partnership with IBM
51
EDR: Real time. Peer-to-peer.
100% Availability.
High performance, rock solid middleware for DB2
replication and change propagation
Flexible and resilient architecture capable of further
adaptation and evolution
Backed by experienced vendor and responsive
technical support
Advanced technology from E-Net Corporation, the
leader in replication and recovery solutions.
For further information, see contact information on
our website at http://www.enet.com
Questions?
52