Database Teradata Support Plan

Platform Support Plan
System Name: Database Teradata Support Plan
Document ID:
Document Revision: 1.0

INFRASTRUCTURE SERVICES SUPPORT Plan
Database Teradata Support Plan, 1.0
DEPARTMENT/DIVISION/SERVICE AREA – TOPIC/SUBJECT/USE:
Infrastructure Engineering & Development – Teradata
Approvals
TECHNICAL UNIT APPROVAL:
By affixing their signature, the individual below acknowledges that they have reviewed and approve the technical
contents of the Infrastructure Support Plan and attest that they are accurate.
Tucker, Michael
{See Appended Electronic Signature Page}
Assoc. Dir, Service Delivery/Management Database Services
BUSINESS UNIT APPROVAL:
By affixing their signature, the individual below acknowledges that they have reviewed and approve the technical
contents of the Infrastructure Support Plan and attest that they are accurate.
Denecke, Charles J.
Dir, Service Delivery/Management, Enterprise Capability MS&O {See Appended Electronic Signature Page}
Database Services
Revision History
ISSUE # DATE PERSON CHANGE
1.0 3-Feb-2014 Puneet Goomer Initial version
Document created to establish support plan for Enterprise Database Services
Page 3 of 58
The information contained herein is the sole property of Merck and is PROPRIETARY. The recipient is responsible for
safekeeping, prevention of unauthorized appropriation, use or disclosure.
TABLE OF CONTENTS
1 INTRODUCTION ............................................................................................................................................................ 7
1.1 PURPOSE ........................................................................................................................................................................ 7
1.2 SCOPE ............................................................................................................................................................................ 7
1.3 AMENDMENTS TO THE SUPPORT PLAN ........................................................................................................................... 7
2 SUPPORT STRATEGY AND ENVIRONMENT ......................................................................................................... 7
2.1 SUPPORT OBJECTIVE ...................................................................................................................................................... 7
2.2 SUPPORT STRATEGY ....................................................................................................................................................... 7
2.3 SUPPORT ENVIRONMENT ................................................................................................................................................ 8
2.3.1 Contact Information ............................................................................................................................................. 8
2.3.2 Scope of Database Services ................................................................................................................................. 9
3 ABBREVIATIONS AND ACRONYMS ........................................................................................................................ 9
4 ROLES AND RESPONSIBILITIES ............................................................................................................................ 10
5 TOOLS, TECHNIQUES AND METHODS*............................................................................................................... 12
5.1 SUPPORT TOOLS ........................................................................................................................................................... 12
5.1.1 Alarm Point for escalation and notification ....................................................................................................... 12
5.1.2 Teradata Viewpoint ............................................................................................................................................ 12
5.1.3 My Support Center (Remedy) ............................................................................................................................. 12
5.2 COMMUNICATION METHOD.......................................................................................................................................... 12
6 IT SERVICE MANAGEMENT PROCEDURES ....................................................................................................... 13
6.1 SERVICE REQUEST MANAGEMENT ............................................................................................................................... 13
6.2 INCIDENT MANAGEMENT ............................................................................................................................................. 13
6.3 PROBLEM MANAGEMENT ............................................................................................................................................. 14
6.4 CONFIGURATION MANAGEMENT .................................................................................................................................. 15
6.5 CHANGE MANAGEMENT............................................................................................................................................... 15
6.6 IT CHANGE MANAGEMENT PROCESS - PLANNED AND UNPLANNED CHANGES ............................................................ 15
6.7 IT CHANGE MANAGEMENT PROCESS - PRE-AUTHORIZED CHANGES ........................................................................... 16
6.7.1 Pre-Authorized CRQ Implementation Instructions ................................................................................................. 17
6.7.2 Pre-Authorized CRQ Change tracking Method....................................................................................................... 26
6.7.3 General Category Changes Implementation Instructions ....................................................................................... 26
6.8 CHANGE MANAGEMENT – IMPLEMENTATION .............................................................................................................. 36
6.8.1 Raising a RFC .................................................................................................................................................... 36
6.8.2 Scheduled Downtime Notification ...................................................................................................................... 36
6.8.3 Changes that Require an RFC and their Change Status .................................................................................... 36
6.8.4 RFC Timings ...................................................................................................................................................... 36
6.8.5 RFC Work Instructions ...................................................................................................................................... 37
6.8.6 Support Group Names........................................................................................................................................ 37
6.9 SERVICE KNOWLEDGE MANAGEMENT ......................................................................................................................... 37
7 REMEDY TICKET INITIATION AND ASSIGNMENT .......................................................................................... 37
7.1 QUEUES NAMES ........................................................................................................................................................... 38
7.2 REMEDY SERVICE TAGGING ......................................................................................................................................... 38
8 SERVICE MANAGER ROLES & RESPONSIBILITIES ......................................................................................... 39
Page 4 of 58
8.1 UPDATING RESOURCE SCHEDULING CALENDAR .......................................................................................................... 39

9 FREQUENTLY USED LINKS ..................................................................................................................................... 41
10 VENDOR SUPPORT DETAIL ..................................................................................................................................... 42
TERADATA SUPPORT.............................................................................................................................................................. 42
11 TERADATA DATABASE STANDARDS ................................................................................................................... 42
11.1 SERVICE REQUEST MANAGEMENT AND ACCESS MANAGEMENT TOOL UNDERSTANDING .......................................... 43
11.2 USER’S ACCOUNT & GRANTS PROCESS................................................................................................................... 43
11.3 STANDARD PASSWORD COMPLEXITIES & REQUIREMENTS ...................................................................................... 43
12 DATABASE OPERATIONAL ACTIVITIES ............................................................................................................. 43
12.1 PDCR ...................................................................................................................................................................... 43
12.2 BACKUP AND RESTORE PROCESS............................................................................................................................. 44
12.3 PHYSICAL DATABASE HIERARCHY .......................................................................................................................... 44
12.4 HOW TO COLLECT STATISTICS FOR THE DATABASE .................................................................................................. 44
12.5 CREATING STANDARD ROLES, PROFILES, DATABASES, USERS, AND GRANTS FOR APPLICATION SETUP ..................... 45
12.6 HOW TO CREATE A STANDARD DATABASE HIERARCHY............................................................................................ 45
12.7 HOW TO IMPLEMENT APPLICATION SECURITY FOR A STANDARD APPLICATION ........................................................ 45
12.8 HOW TO CREATE A STANDARD ISID USER IN TERADATA......................................................................................... 46
12.9 HOW TO VIEW GRANTED PRIVILEGES ....................................................................................................................... 46
13 SCHEDULED ACTIVITIES......................................................................................................................................... 46
13.1 DAILY PROCEDURES ................................................................................................................................................ 46
13.1.1 Check for system Health ................................................................................................................................ 47
13.1.2 Check for Alert logs ...................................................................................................................................... 47
13.1.3 Resource Intensive Queries ........................................................................................................................... 48
13.2 WEEKLY PROCEDURES ............................................................................................................................................ 48
13.2.1 Backups ......................................................................................................................................................... 48
13.2.2 Performance tuning ....................................................................................................................................... 50
13.3 MONTHLY PROCEDURES.......................................................................................................................................... 50
13.3.1 Capacity planning ......................................................................................................................................... 50
13.3.2 Patch Upgrade / Maintenance Coordinate with Teradata GSC ................................................................... 50
13.4 QUARTERLY PROCEDURES ....................................................................................................................................... 50
14 SYSTEM MONITORING AND ALERTING ............................................................................................................. 51
14.1 VIEWPOINT ALERTS ................................................................................................................................................ 51
14.2 ALARM POINT ......................................................................................................................................................... 51
14.3 10 MINUTE OUTAGE NOTIFICATION PROCESS ......................................................................................................... 52
15 DATABASE TOOLS ..................................................................................................................................................... 52
15.1 VIEWPOINT .............................................................................................................................................................. 52
15.2 TERADATA TOOLS AND UTILITIES ........................................................................................................................... 53
15.3 CONFIGURE ODBC VIA DNS .................................................................................................................................. 53
15.4 ADDRESSING THE RELATIONSHIP BETWEEN DNS AND COP NAMES ........................................................................ 53
16 CRDB / REVISION AND MIGRATION DOCUMENT LINKS ............................................................................... 54
17 DISASTER RECOVERY SETUP DETAIL ................................................................................................................ 55
18 DATABASE SOX AUDITING & REPORTING ........................................................................................................ 55
19 SLA MANAGEMENT ................................................................................................................................................... 55
Page 5 of 58
20 DATABASE IN-FLIGHT PROJECTS ........................................................................................................................ 55

21 ESCALATION MATRIX .............................................................................................................................................. 55
22 GENERAL DOS AND DON’TS ................................................................................................................................... 56
23 TRAINING ..................................................................................................................................................................... 56
24 REFERENCES ............................................................................................................................................................... 57
25 RESOURCES ................................................................................................................................................................. 57
Page 6 of 58
1 Introduction
1.1 Purpose
The purpose of this document is to define the roles, responsibilities and procedures involved in managing the Teradata
Database Platform. This document describes the high-level Teradata Level 3 administration tasks performed within Merck.
The audience of this document is primarily HCL-Merck Teradata team, who would be performing Teradata Administration
from HCL Chennai facility.
This document defines the “Merck &Co., Inc.” procedures and policies, and provides guidelines for its implementation. All
the information captured during the knowledge acquisition phase during the workshops is held at Rahway New Jersey or
via WebEx.
1.2 Scope
The scope of this document includes the necessary information to manage the Teradata Database Services Platform. The
content of this document will provide engineering and operational guidelines and the basic principles that govern the
platform.
1.3 Amendments to the Support Plan

The content material contained within this document shall be in effective until the document is retired. Minimally, the
content of the document will be examined by platform subject matter experts and management on an annually basis to
ensure the content is reflective of current support processes and strategies.
Amendments to this support plan will be accomplished through the revision of this document with signature approval by all
original signatories, their delegates, or functional equivalents. A history of approved issues of this document will be
reflected in the Document Issue History section.
2 Support Strategy and Environment

2.1 Support Objective
The Platform Service provides server environments to host Merck applications. Specific support objectives are
identified following Operational Level Agreements.
 GISLM-OLA-GEN-001413 – Database Services Operational Level Agreement
2.2 Support Strategy
DATABASE SERVICES Platform support is divided into three levels. Each level is supported by the HCL
Managed Service.
The different levels of support allow the Teradata Database Platform to be supported 24x7.
Issues to the platform are reported as follow:

 Datacenter Technicians raise Remedy incidents cases about issues related to the
DATABASE_SERVICES_RESOURCES reported by various sources not limited to OVO events,
application groups, etc.
 Application groups submit helpdesk cases about issues they are experiencing with the platform.
Page 7 of 58
 Monitoring tools generate alerts based on some pre-defined threshold. These events are managed by
DATABASE_SERVICES_RESOURCES using the incident management process.
All requests for Teradata Database Support must first be initiated by a Remedy Work Order, Service Request, or
Incident. The Requestor should include a detailed description of the request or problem, the database name if
applicable, and the server’s name. The Remedy ticket should initially be submitted to the
“DATABASE_SERVICES_RESOURCES” Remedy Group for initial service management.
HCL actively monitors these queues on a 24x7 basis to respond, manage, and complete the request according to the
SLA for the requests’ priority.
2.3 Support Environment

There are four Teradata environments Development (2650), Test (2650), Pre-prod (2650) and Production (2960). All
are on version 13.10. The location of Development, Pre-prod and Production boxes is Charlotte, NC and Test is at
Whitehouse station, NJ. Teradata identifies (TD@YS) the Production machine by MERCKPROD2, Development
machine by MERCKDEV, Pre-prod/QA by MERCKQA and Test machine by MERCKTEST.
Inventory of Teradata databases can be found at the below link:
https://remedy-web.merck.com/
For searching a database in Remedy, follow below sequence:

Login to Remedy Applications  Asset Management  Asset Management Console
 Manage CI CI Type  logical entity  database  Enter CI name
All database related detail like criticality, associated application etc. can be found in the Remedy.
Please refer to the document path and link below for Teradata nodes and IP addresses in the below document.
“Teradata Nodes and IP Addresses"
G3A Core Infrastructure Transition  LibrariesProject Documents KA Documentation-Temp Database
Teradata  Documents  Teradata Nodes and IP Addresses.Doc
http://projectsext.merck.com/sites/core_infr_transition/_layouts/WordViewer.aspx?id=/sites/core_infr_transition/Pro
jectDocuments/KA%20Documentation%20-
Temp/Database/Teradata/Documents/Teradata%20Nodes%20and%20IP%20Addresses.docx&Source=http%3A%2
F%2Fprojectsext%2Emerck%2Ecom%2Fsites%2Fcore%5Finfr%5Ftransition%2FProjectDocuments%2FForms%2
FAllItems%2Easpx%3FRootFolder%3D%252Fsites%252Fcore%255Finfr%255Ftransition%252FProjectDocument
s%252FKA%2520Documentation%2520%252DTemp%252FDatabase%252FTeradata%252FDocuments%26Folder
CTID%3D0x0120005765ACD97B235541AEE16F2D43DF78F6%26View%3D%7B08ACF559%2D78EF%2D401
D%2DA481%2D98785CDBE59E%7D%26InitialTabId%3DRibbon%252EDocument%26VisibilityContext%3DW
SSTabPersistence&DefaultItemOpen=1&DefaultItemOpen=1
2.3.1 Contact Information

All support requests must have a support case. A Consumer can call the Help Desk or the Charlotte Technology
Center (CTC) Command Center and a case will be created. If they cannot provide immediate support, they will
Page 8 of 58
contact the Service Provider for assistance. Requests that are not initiated by the CTC Command Center will be
responded to by the Service Provider according to the support model detailed below.
If issues arise that cannot be resolved between the Client and the Support Lead, the following escalation procedure
should be followed.
 Contact the Help Desk and ask for your call to be routed to the Command Center.
 Ask the Command Center for details on escalation chains, which can be found in My Support Center
and AlarmPoint.
Automated alerts are generated by the different monitoring tools and reported through Alarmpoint. Alarmpoint is
also used by application groups and data center technicians to page a DBA about a problem or incident.
2.3.2 Scope of Database Services

Database services includes Administrative Services, Monitoring Services, Tier 2 and Tier 3 support services,
Operations Services, Backup & Recovery Services, Incident Management, Escalation Management, Outage
Notification, SLA Management, Business Continuity Plan and Disaster Recovery Services. Detailed Scope of Work
can be found in Agreement in effect.
Database Inventory
This would be probably the most frequently visited and updated document. It holds links to all the documents
referred to by Teradata DBAs. It is mandatory to keep all the documents updated and especially keep this inventory
sheet to quickly reach to the document you are looking for.
Please refer to the document path and link location below:

G3A Core Infrastructure Transition  LibrariesProject Documents KA Documentation-Temp Database
Teradata  Documents
http://projectsext.merck.com/sites/core_infr_transition/ProjectDocuments/Forms/AllItems.aspx?RootFolder=%2Fsit
es%2Fcore%5Finfr%5Ftransition%2FProjectDocuments%2FKA%20Documentation%20%2DTemp%2FDatabase%
2FTeradata%2FDocuments&FolderCTID=0x0120005765ACD97B235541AEE16F2D43DF78F6&View={08ACF5
59-78EF-401D-A481-
98785CDBE59E}&InitialTabId=Ribbon%2EDocument&VisibilityContext=WSSTabPersistence
3 Abbreviations and Acronyms
Acronym - Term Definition

AKA Also Known As
CM/IM Change Management/Incident Management
DBA Database Administrator
DC Data Centre
DEV Development Environment
Page 9 of 58
PPT Pre-Prod Environment
DR Disaster Recovery
DRP Disaster Recovery Plan
HA High Availability
LOB Line of Business
NFS Network File Sharing
OS Operating System
PPM Project and Portfolio Management
PRD Production Environment
QA Quality Assurance
RDBMS Relational Database Management Systems
SA System Administrator
SD Service Desk
SLA Service Level Agreement
SOP Standard Operating Procedure
TST Test Environment
WHS Whitehouse Station
4 Roles and Responsibilities
Role Role Responsibility
Database Administrator Build and Manage the Teradata Database Platform

Shared Services Team performs the infrastructure activities for the databases.
Shared Services (SS) DBA Mostly, they get requests from the Client Facing DBAs (CFBDA) for the activities
to be performed. Shared Service Team is made up of Engineering DBAs
Page 10 of 58
Client facing DBAs coordinate with application team and business users to
Client Facing DBA
understand the application requirement and their needs. CFDBA collects all the
(CFBDA)
information and provide the support.
Managed Services Operations DBAs that perform routine tasks and operational activities.
Database Team Structure
There are 2 Database teams that operate in the Merck Environment:
Shared Services (SS) DBAs & CFDBA’s

Shared Services team performs the infrastructure activities for the databases. Mostly they get requests from the
Client Facing DBAs (CFBDA) for the activities to be performed.
Client facing DBAs coordinate with application team and business users to understand the application requirement
and their needs. CFDBA collects the all information and provide the support.
Shared Services & CFDBA’s Activities include, but are not limited to:
Task Source of Task
Database Software Installations SSDBA
Database Creation SSDBA
Database Upgrade and Migrations SSDBA
Patching SSDBA
Tablespace and Space management CFDBAs
Backup Scheduling & Monitoring SSDBA
Database Decommissioning SSDBA /CFDBAs
Backup Troubleshooting& Reporting Backup Notification Mail
Application and user account setup CFDBA
Performance monitoring and alert monitoring of the database CFDBA
Merck – Current Teradata Support Team

Mike Tucker
Barbara Reilly
HCL –Current Database Support Team

Puneet Goomer
Kranthi Kiran Patil
Mushtaq Ahmed
Page 11 of 58
5 Tools, Techniques and Methods*

5.1 Support Tools
5.1.1 Alarm Point for escalation and notification

Alarm Point is an alerting tool that interacts with monitoring software to provide the following functions to Merck:
 Alerting relevant personnel to critical IT events, and
 Escalating and acknowledging the events as required throughout the business enterprise --
application management, network / system monitoring, business continuity and information security.
Most production database servers are being monitored using the HP Openview tool, SPI. The EMS team is
responsible to configure the tool on the servers. Whenever an incident happens, it sends an alert to corresponding
team (in case of database servers, goes to “Database Operations Resources”
(1datoperr@northamerica.msx.merck.com). The team is responsible to respond to the alert within 15 minutes failing
which it will get escalated to higher level.
5.1.2 Teradata Viewpoint

Teradata Viewpoint is advanced web-based systems management portal for Teradata performance and health
management. It provides a consistent interface via configurable portlets which allows the user to customize their
own systems management dashboard. With Viewpoint, DBAs can determine system status, trends and individual
query status. By observing trends in system capacity usage, they can plan new project implementations, batch jobs
and maintenance better around peak usage periods. Viewpoint users and programmers can watch their queries
running like a car dashboard. When something bad happens, the red light blinks on. If the query is stalled – a red
light – you can call the help desk. If the queries are running fast, users can submit a few more and get more work
done. Viewpoint's security is provided by a fully-featured, roles-based permissions engine that can function in
standalone mode or seamlessly integrate with your pre-existing enterprise authentication solution via LDAP, such as
Active Directory. DBAs appreciate the advanced authorization system that gives great flexibility in defining which
users can perform specific actions and access to specific data, systems, or portlets.
5.1.3 My Support Center (Remedy)

BMC Software’s Remedy application is an enterprise wide, web based ticketing system for Business Service
Management. The following Remedy modules support Merck’s ITIL-based Service Management Processes:
 Change Management
 Incident Management
 Problem Management
 Configuration Management
 Service Level Management
 Knowledge Management
 Service Request Management
5.2 Communication Method

Scheduled and unscheduled service disruptions will be communicated to clients by email notifications from My
Support Center. Scheduled downtime will be communicated at least one week in advance. Unscheduled downtime
will be communicated though the Command Center as soon as is practicable.
Page 12 of 58
6 IT Service Management Procedures

6.1 Service Request Management
Datacenter Technician submits a service request/ Case to “Database Services Resources " inbox. Tier2 SA monitors
the support Group “Database Services Resources “inbox 24x7. Tier 2 SA investigates the case and if within scope
resolve incident using KM articles. If not then escalate incident to Tier3 on-call DBA.
The procedure to support Service Request Management is as follows
SDLC-SOP-05-08: Global Services (GS) Service Request Fulfillment
6.2 Incident Management

Incident Management defines the roles, responsibilities, and processes for managing the Lifecycle of an Incident in the
entire environment wherever applicable. Please relate this with “Remedy Ticket Initiation and Assignation” for handling
the remedy tickets.
The processes identified within this section are as follows:
 Incident Registration – Whenever an incident occurs, an alert is generated through the monitoring tools. Once alert
is received, it is registered in remedy tool by user or support team.
 Incident Assignment – Once the incident is registered, a ticket is created and assigned to the concerned support
team (here Database Team). The person who is assigned the ticket to work upon is called assignee. Assignee will be
owner of the ticket until it is resolved or it is transferred to other person.
The remedy group name for Database Support team is “Database Services Resources”. The ticket is further
assigned to specific Database Team member who works on the incident ticket. The time between ticket assigned to
group and the assignation to assignee is measured as response time. Database Team must be careful about the ticket
response time to meet the agreed SLA.
 Incident Tracking – Once ticket is created or resolved, it can be tracked for its lifecycle. When support staff logs in
to Remedy and goes to overview screen, all the tickets that are open (assigned / pending) are listed on the screen
with little important information. You can select other column of your choice to display in remedy. Even after the
incident ticket is resolved / closed, you can track by its incident ID or other parameters available in remedy search.
 Incident Resolution – The owner of the ticket (assignee) works on the ticket and resolves the issue and checks and
confirms that the issue is resolved. If the incident was raised by some user and only user can confirm the resolution,
ticket owner needs to get confirmation from user before resolving the ticket. In case user does not confirm and the
SLA of the ticket is reaching towards expiration, Database Support Team will resolve the issue mentioning the case.
Alternately, the ticket can be kept in pending for few hours /days to get the confirmation whichever is suitable to the
Page 13 of 58
case. The time taken to resolve the incident (minus the time for which ticket was kept in pending due to a reason
beyond the control of HCL) is measured as Resolution time.
 Incident Closure – Once the incident is resolved and it is not re-opened, it automatically closed after 15 days. After
closure, ticket cannot be re-opened.
 Incident Reporting – Report is generated for the incidents of different types to check the SLA met/missed at the
end of every month. Remedy tool has the feature to provide the reports.
You can find detailed information about the Incident Management in the document path and link below mentioned SOP.
Teamspace SDLCSDLC Doc Operation  SDLC-SOP-05-01.pdf
http://teamspace.merck.com/sites/SDLC/SDLC%20Doc/Operation/SDLC-SOP-05-01.pdf
6.3 Problem Management

Problem Management defines the roles, responsibilities, and processes for managing the Lifecycle of an IT Problem in the
production / UAT environment. Remedy Tool is used to register and track the problem tickets.
The processes identified within this procedure are as follows:
 Registration of Problem ticket– There are two scenarios where we need to review the incident in more depth. 1)
When a major outage occurred and 2) when there is repetitive incident. In both these scenarios, incident should be
investigated in detail. For this purpose, a Problem ticket is created
 Root Cause Analysis – To resolve problem ticket, root cause of the incident needs to be identified. Generally 3
times “Why” is asked for the reason of incident and the root cause is identified. However, in some case, you may
need to dig it further.
 Problem Closure – Once the root cause is identified, it mostly comes with some recommendations / change in
environment which is then taken care of by Change Management system separately. The Problem ticket can be
closed once the root cause is identified.
 Problem Reporting – Report of open Problem tickets is prepared weekly basis and there is weekly meeting on
every Thursday at 9:00 AM EST to discuss the open Problem tickets.
For additional details about the Problem management, please refer the document path and link below mentioned SOP.
Teamspace SDLCSDLC Doc Operation DLC-SOP-05-02.pdf
http://teamspace.merck.com/sites/SDLC/SDLC%20Doc/Operation/SDLC-SOP-05-02.pdf
Page 14 of 58
6.4 Configuration Management

Configuration Management is the process that maintains information about Configuration Items required delivering
an IT Service, including their Relationships. Its primary objective is to provide accurate data to all IT Service
Management Processes when and where it is needed
The procedure to support Configuration Management is as follows:
 ITSM-SOP-GEN-000068 - GTS Configuration Management Process
6.5 Change Management
The objective of change management is to ensure that standardized methods and procedures are used for efficient and
prompt handling of all changes to control IT infrastructure, in order to minimize the number and impact of any related
incidents upon service. Changes in the IT infrastructure may arise reactively in response to problems or externally imposed
requirements, e.g. legislative changes, or proactively from seeking improved efficiency and effectiveness or to enable or
reflect business initiatives, or from programs, projects or service improvement initiatives. Change Management can ensure
standardized methods, processes and procedures which are used for all changes, facilitate efficient and prompt handling of
all changes, and maintain the proper balance between the need for change and the potential detrimental impact of changes.
6.6 IT Change Management Process - Planned and Unplanned Changes

The purpose of this procedure is to establish a disciplined process for introducing Planned and Unplanned Changes into the
Merck IT environment with minimal disruption to on-going operations. These changes have been classified in 4 Classes.
1. Normal Change
2. Emergency Change
3. Latent Change
4. No Impact Change
Page 15 of 58
For more details, please refer “IT Change Management Process - Planned and Unplanned Changes”
6.7 IT Change Management Process - Pre-Authorized Changes

The change management process ensures that standardized, consistent methods are used for efficient and prompt handling
of all changes to the Merck IT environment, in order to minimize the impact of change-related incidents upon service
quality, and consequently improve the day-to-day operations of the organization. IT Change Management processes must
not impact the regulatory business processes and products. This procedure defines the three classes of Pre-Authorized
Change sanctioned by the ETAS Change Management Authority and are contained in a Common Catalog.
1. Standard Change - A Pre-Authorized Change that is low risk, relatively common and may follow a Procedure or
Work Instruction. Pre-Approval and Post Approval are not required
2. Pre-Approved Change - A Pre-Authorized planned change that does not require Pre-Approval but requires Post
Approval
Page 16 of 58
3. Administrative Change - A Pre-Authorized Change that can be implemented without the creation of an RFC and
without any signature events
For more details please refer “IT Change Management Process - Pre-Authorized Changes”.
All Production and Validated or GXP Formal Test Environment non-operational changes require RFC (Request for
change) and CAB approvals prior to implementing the change. If you have any doubt about the Validated / GXP, please
refer the “Regulatory Indicator” in the CI of the Server/Instance.
Source of RFC (Request for Change): There are different sources for RFC such as Engineering team requesting to build a
Server / Database Instance, Application Team requesting a release / deployment consisting a new database, services or
Objects in existing database, patching and upgrades as per schedules. In addition, the root cause analysis also, sometimes
comes with Change recommendation.
6.7.1 Pre-Authorized CRQ Implementation Instructions
C.01.Create Viewpoint users

C.02.Modify/Delete Viewpoint users
C.03.Create Viewpoint roles and privileges
C.04.Add new Teradata System to Viewpoint
C.05.Run PENDINGOPS operation
C.06.Release lock - ARCMAIN or MLOAD lock
C.07.Recreate locked empty table due to fast load job failure
C.08.Create Database Hierarchy
C.09.Modify User SPOOL or TEMP space
C.10.Create / update Viewpoint notification methods and action sets
C.11.Run_PDCR_Load_Jobs
C.12.Update PDCR retention / configuration table - Update PDCR data
C.13.Run_Teradata diagnostic_database_utilities such as scandisk, checktable
C.14.Updated /etc/hosts entries
C.15.Create Application Query Capture Database
C.16.Modify Viewpoint roles and privileges
C.17.Create / update Viewpoint metrics, canary queries, and alerting thresholds
C.01. Create Viewpoint users

The User Manager Portlet allows the Teradata Viewpoint Administrator to manage Teradata Viewpoint
user accounts. Using this portlet, you can:
 Define or modify a user account.

 Reset forgotten or compromised passwords.
 Assign roles to users.
Page 17 of 58
 Set role precedence.

 Search for existing users.
Why and when do we need it?

User will raise the request to access the Teradata Viewpoint Portlet to monitor the queries which are
running in Teradata systems. User need to monitor the queries which need to understand the
performance of the application queries.
How does it work?

Add URL in internet explorer to login into Viewpoint servers
http://qlwsnct0110.merck.com/c
http://qlctccp0110.merck.com/c
1. Login into the viewpoint with username as ISID and network password.
2. To add user, click on Admin drop down select the user manager
3. Click on Add User In General tab the following
Portal Username - Add user ISID
First Name - Add First Name of user
Last Name - Add Last Name of User
Email - Add user Merck Mail ID
Externally Authenticated? - click the check box
4. Click Apply user will be added in Teradata Viewpoint.
C.02. Modify/Delete Viewpoint users

The user details view allows you to delete a user, edit Teradata Viewpoint user accounts.
How does it work?

1. Login into the viewpoint username as ISID and network password.
2. To Modify/Delete user, click on Admin drop down select the user manager
3. Find users sort by first name, last name, portal username, email etc.
4. Click on user which is need to Modify/Delete.
5. Click on Delete user tab to delete the user and then click apply.
C.03. Create Viewpoint roles and privileges

The Roles Manager portlet allows the Teradata Viewpoint Administrator to assign permissions
efficiently by creating classes of users called roles.
The Teradata Viewpoint Administrator can perform the following tasks:
• Add and configure new roles
Page 18 of 58
• Edit the configuration and settings of existing and default roles

• Copy roles, saving time in creating new roles
• Enable or disable portlets for a role.
• Delete roles that are no longer needed


2. To Create Viewpoint roles click on Admin drop down select the Roles Manager
3. Click Add to create role ,customize the role using the following tabs:
i. General
Name :- Role name
Desc :- Role name
Role state :- Enable (radio button)
Rewind Sate :- Enable (radio button)
check box Enable Systems For Role
ii.Users
Search for users and add them to, or remove them from, the list of users assigned to the selected role.
iii.Portlets
Enable or disable portlets for a role. This tab can also be used to configure default settings per portlet,
per role.
iii.Metrics
Select the metrics to display per metric, per system.
iv.Permissions
Configure permissions per system, per portlet, per role.
C.04. Add new Teradata System to Viewpoint
You can add a Teradata Database System to Teradata Viewpoint. The system must be added before you
can configure the data collectors to monitor the database.

Page 19 of 58
1. login into the viewpoint username as ISID and network password.

2. To add user, click on Admin drop down select the Teradata systems
3. Click next to SYSTEMS
4. Enter a SYSTEM NICKNAME (up to 8 characters).
5. [Optional] Select the System Enabled check box to activate the Teradata Database system for
monitoring.
6. Enter the TDPID of the Teradata Database system.
7. Enter a LOGIN name and password.
You also have the option to specify an account string, add more logins, and test that the login settings
are correct.
8. [Optional] In the CHARACTER SET section, select default character sets and enter a JDBC Flag
value.
• [Optional] Select a character set from the Session menu.
• [Optional] Select a character set from the Monitor menu.
• [Optional] Enter a JDBC Flag value.
9. Click Apply to apply the settings.
C.05. Run PENDINGOPS operation
Level-pendingop checking checks field 4 of a table header and returns warnings(not errors) if the table is
in any of the following states:
Pending FastLoad
Table check bypassed due to pending Fast Load.
Pending Restore
Pending Reconfig
Pending Rebuild
Pending Replicate copy
Pending MultiLoad
Steps for run check table PENDINGOP

1. login into teradata system as Root user
2. Run cnsterm command
3. Input Supervisor Command:
> start checkable
Started 'checkable' in window 1
4. press Ctl+z come out cnsterm window
5. run cnsterm 1 to run checktable
Enter a command, "QUIT;", "HELP;" or F7 for interactive help:
CHECK databasename.tablename at level pendingop; (single table)
or
CHECK ALL TABLES AT LEVEL PENDINGOP; (all tables)
Page 20 of 58
C.06. Release lock - ARCMAIN or MLOAD lock
1. Login as Teradata system as root user

2. check to see what Phase the MultiLoad job is in
# cnsterm 6
# start qrysessn
[break]
# cnsterm 1
>*
>*
>y
3.Login into SQL Assistance/bteq using CFDBA user
4. release a MultiLoad job in the ACQUISITION Phase
RELEASE MLOAD <db.table1>, <db.table2>;
OR
release a MultiLoad job in the Application Phase
RELEASE MLOAD <db.table1>, <db.table2> IN APPLY;
C.07. Recreate locked empty table due to fast load job failure
Teradata FastLoad is a command-driven utility which can be used to quickly load large
amounts of data in an empty table on a Teradata Database.
1. First check the tables whether tables are locked due to fast load by accessing the table (Ex: sel
count (*) from Db.Tab;).
It gives 7652 error, if it is not accessible
2. Take DDL back of the table i.e either by SHOW or CREATE TABLE
CREATE TABLE DB.TABNLE_NMAE_TMP AS DB.TABLE_NAME;
SHOW TABLE DB.TABLE_NAME;
3. Preserve the statistics on the table by using the statement
HELP STATS DB.TABLE_NAME;
4. Prepare the COLLECT STAT statement after taking the columns from step 3.
COLLECT STATISTICS ON DB.TABLE_NAME COLUMN COLUMN_NAME;
5. Drop the table
DROP TABLE DB.TABLE_NAME;
6. Finally CREATE/RENAME the table. This depends on SHOW or CREATE TABLE in step 2
CREATE TABLEDB.TABLE_NAME;
RENAME TABLE TABLEDB.TABLE_NAME_TMP AS TABLEDB.TABLE_NAME;
Page 21 of 58
7. Run the collect stat statement, that is prepared in step 4.
C.08 Create Database Hierarchy
The database hierarchy user is the primary database account which “owns” most of the database objects
for a specific application.
The hierarchy owner creates databases to store the objects. The Data Application Hierarchy defines the
collections of databases and users within each application.
Merck Databases Hierarchy structure:
<APPL>
<APPL>_ETL
<APPL>_WORK
<APPL>_RPT -- Depends on the flag that we pass through as we know some cases we need PUB and
some cases we need RPT.
<APPL>_PUB -- Depends on the flag that we pass through as we know some cases we need PUB and
some cases we need RPT.
<APPL>_STAGE
Profiles:
<APPL>_SPR_ADMIN_PROFILE
<APPL>_DEV_PROFILE
<APPL>_USER_PROFILE
1. login to SQL Assistance/bteq using CFDBA user

2. Call the procedure CFDBA.CREATE_HIER_PROC which creates standard hierarchy with users,
databases and profiles.
CALL CFDBA.CREATE_HIER_PROC ('Application Name',
'password',null,100,100,100,'Y','N',omsg,oresult);
C.09. Modify User SPOOL or TEMP space
Spool space holds intermediate query results or formatted answer sets to queries and volatile tables. The
system can use unassigned PERM space for spool space
Temp space defines the number of bytes the system will use to store data for global temporary tables.
The value you specify must not exceed the value of the immediate parent at the time of creation. If you
do not specify a value, the maximum value defaults to that of the parent.
Page 22 of 58
1. Login to Teradata Administrator with CFDBA

2. Select the required database/user
3. Select the Modify Database… in Tools bar menu
4. Enter the required/requested Temp Space/Spool Space and click on the Modify button
C.10. Create / update Viewpoint notification methods and action sets

The ALERT SETUP portlet allows Teradata Viewpoint Administrators to configure the Alert Service to
perform any of the following actions in response to alerts:
• Write to the alert log
• Send email notifications
• Send SNMP notifications
• Run BTEQ scripts
• Run programs

http://qlwsnct0110.merck.com/c - TEST
http://qlctccp0110.merck.com/c - DEVPPT/PROD
2. Click on Admin (Top right corner of the portal) and select Teradata Systems
3. Select the system name in SYSTEMS that you want to setup / update the alerts
4. Select the Alerts option in SETUP section
5. Select the alert type in ALERT TYPES section and proceed with the setup
6. Need to fill the following with the details you have or planned
7. Alert Name, Alert Action, Severity and Message
C.11. Run_PDCR_Load_Jobs
Steps: Generally PDCR jobs runs automatically through cron in Linux

1. Login Linux server with pdcr user.
2. Go to directory /home/pdcr
3. Display the the cron entries with the command: crontab –l, which gives job list i.e hourly, daily,
weekly etc.
4. Copy the corresponding cron entry of the failed job starting from “TDPID” to the end of the line,
paste the command at the prompt, and submit the job.
Page 23 of 58
C.12. Update PDCR retention / configuration table - Update PDCR data
The Weekly PDCR job automatically purges old data

1. Data Retention Rates controlled by PDCR_Table_Retention
2. Specifies number of days to keep historical data for each table
3. Optionally purges corresponding DBC table
4. Modify Retention Rates by manually updating values in table
C.13. Run_Teradata diagnostic_database_utilities such as scandisk, checktable
You can run all the listed utilities in REMOTE CONSOLE portlet of Teradata Viewpoint
1. Open a Teradata Viewpoint portal with admin credentials
http://qlwsnct0110.merck.com/c - TEST
http://qlctccp0110.merck.com/c - DEV/UAT/PROD
2. You need to have REMOTE CONSOLE portlet added. If not, add by selecting Add Content (
Top left corner of the portal)
3. In remote console, select the system
4. Select the Utility you want to run from the list displayed
5. Once it get connected, you can give help command and displays the list of commands
6. You can diagnosis the system for more info
C.14. Updated /etc/hosts entries

Teradata nodes and IP addresses
Add cop entries to the hosts entries for Teradata nodes and IP addresses
1. Login as root user in teradata system

2. vi /etc/hosts
3.Add the cop entries to IP addresses
ex:-54.50.28.31 GHHTDPRD GHHTDPRDCOP1
4.save and exit
C.15. Create Application Query Capture Database
To capture and visualize new query execution plans using Teradata Visual Explain, a QCD must be set
up
Page 24 of 58
1 From the menu bar, click Tools>Control Center.

The Control Center dialog box appears, and the Manage QCD tab is selected.
2 Click Setup QCD.
The Setup QCD dialog box appears.
3 Click Create all QCF Database objects.
4 In the QCD Name box, enter a name for the QCD.
5 In the Owner box, enter an owner name.
If the box is left blank, the owner defaults to the name of the logged‑on user.
6 In the Perm Space box:
a Enter a value.
b Click KB, MB, or GB.
7 In the Spool Space box:
a Enter a value.
b Click KB, MB, or GB.
The default is 0 MB.
8 [Optional] Select the Fallback check box to use fallback protection.
9 [Optional] Click View Schema to view the schema of the tables and macros created in the new QCD.
The View Schema dialog box appears and displays the DDL and macro definitions of the QCD. What is
displayed depends on the version of QCF that is running.
10 [Optional] Click Clear to clear all boxes and restore the default options.
11 [Optional] Click Command to open the Execute SQL window.
For more information, see “Executing an SQL Query”.
12 Click Create to set up the QCD
C.16.Modify Viewpoint roles and privileges

2.To Modify Viewpoint roles click on Admin drop down select the Roles Manager
3. Select the role , customize/Modify the role using the following tabs:
i. General
Enable or disable a role, and select the systems available to that role.
ii. Users
Search for users and add them to, or remove them from, the list of users assigned to the selected role.
iii. Portlets
Enable or disable portlets for a role. This tab can also be used to configure default settings per portlet,
per role.
Page 25 of 58
iv. Metrics
Select the metrics to display per metric, per system.
v. Permissions
Configure permissions per system, per portlet, per role.
C.17.Create / update Viewpoint metrics, canary queries, and alerting thresholds
2. Click on Admin (Top right corner of the portal) and select Teradata Systems
Select the system name in SYSTEMS that you want to setup / update the alerts
4. Select the Alerts option in SETUP section like Canary queries
5. Write the sql statement for Canary querirs.
6.7.2 Pre-Authorized CRQ Change tracking Method
Each Administrative change will be documented in the AMSU Change log.

http://uctvp615:7777/pls/apex/f?p=129:101:15111114570980
6.7.3 General Category Changes Implementation Instructions
1. Object level changes (e.g. tables, indexes, etc.)

2. Object level changes (e.g. tables, indexes, etc.) that are NOT covered by an Application change
control process.
3. Installation of Database monitoring agents/software on a server.
4. Installation of new database software on an OS Server
5. Addition_of_database_space
6. Change end user account passwords
7. Creating users (requires Remedy ticket and manager approval)
8. Modify users with no owned objects(requires Remedy ticket and manager approval)
10. Granting security, roles or profiles to users
11. Revoking security, roles or profiles from a users
12. Running backups - failed or non-existent
13.Create ssh equivalence
14.Update ssh equivalence
15. Create, add or Migrate an application to a shared/non-shared database
16. Create a new database as a clone of an existing database to non-production environment
17. Creation of a new schema, database to existing Instance
Page 26 of 58
18. Create Role or Profile

19. Modify role or profile
20. Password changes for privileged database accounts (ex: sys, system,DBC, Viewpoint, PDCR,
cfdba, app schema)
21.Modify Backup job/scripts- include/exclude databases
22.Modify Database Initialization Parameter (No Downtime Required
23.Administration of Database Management tools
24.Installation of database client tools
25.Installation of 3rd party tools/libraries on database server - no downtime
26.Update Application Statistics
27.Update Database or System Statistics
28.Adding new features to existing environment
29. A/B Database Switch
30.Create O/S user environmental configuration scripts
1.Object level changes (e.g. tables, indexes, etc.)
1. Logon to Teradata SQL Assistant as CFDBA on TDCPRD01

2.Implement the script related to the Migration Summary per Change Request process
3.Extract the log from the history
4.save the log in the Application’s script folder in J drive
2.Object level changes (e.g. tables, indexes, etc.) that are NOT covered by an Application change
control process.
1. Logon to Teradata SQL Assistant as CFDBA on TDCPRD01

2.Implement the script related to the Migration Summary per Change Request process
4.save the log in the Application’s script folder in J drive
3.Installation of Database monitoring agents/software on a server.
[This is for any monitoring agent]
1. Logon to the Linux box as root

2. Perform the installation and configuration of the monitoring software per the installation
instructions for the tool.
Note: This is always performed in a non-production environment. Each 3rd party tool
installation instructions will be documented and added to the Runbook
Installation of Teradata PDCR:

PDCR
Page 27 of 58
4.Installation of new database software on an OS Server
1. As request from the application team.

2.Engage the Teradata GSC team to install database software
3. Raise SR for the for install database software on OS server
***********
5.Addition_of_database_space
1. Login to Teradata Administrator with CFDBA

2.Select the required database/user
3.go to tools from menu bar
4. select move space
5. Move space from Available_perm to Database and add amount of space.
6. check the database perm space is moved.
****************
6.Change end user account passwords
1. Login into SQL Assistance/bteq using CFDBA user

2. MODIFY USER ISID AS PASSWORD = ***;
3. Password is communicated to end user by separate mail
7.Creating users (requires Remedy ticket and manager approval)
1.Login into SqlAssistance as CFDBA

2.execute the Standard procedure to create user
CALL CFDBA.CREATE_USER_PROC
('ISID',null,'password',null,null,'PROFILE',’Account for ISID Name,
Application’,RETURNMSG,OUTMSG);
8.Modify users with no owned objects(requires Remedy ticket and manager approval)
1.Logon to the Teradata Administrator as CFDBA on TDCPRD01

2.select the user which need to modify under application hierarchy
3.Go to Tools from menu bar click on modify user
4. Modify user as requested by the team
9.Delete users with no owned objects

2.check the objects in the users
3. Drop user username
Page 28 of 58
10.Granting security, roles or profiles to users
1.Logon to the Teradata Administrator as CFDBA on TDCPRD01.

2.Go to Tools – Grant/Revoke- Object rights
3.grant roles to the requested user for the requested database.
11.Revoking security, roles or profiles from a users
1.Logon to the Teradata Administrator as CFDBA on TDCPRD01.

2.Go to Tools – Grant/Revoke- Object rights
3.revoke roles to the user for the requested database.
12.Running backups - failed or non-existent
1.Logon as root user

2.go to the backup job script directory
3.Rerun the backup job script (.ksh)
4. Check the root alert backup job success.
13.Create ssh equivalence

2.go to the path where you need to .ssh file and mkdir
3.vi filename.ssh to edit the scirpt
4. save and exit
14.Update ssh equivalence

2.go to the path where .sh file to
3.Take the backup of the original .sh file
4.update using vi filename.sh to update the scirpt
5. save and exit
15.Create, add or Migrate an application to a shared/non-shared database
1.Login into SqlAssistance as CFDBA user

2.Run the standard procedure CALL CFDBA.CREATE_HIER_PROC ('IMDCI',
'adminpassword',null,100,100,100,'Y','N',omsg,oresult);
3. Once you run the above procedure standard application hierarchy created
4.implement the script at one go.
6.save it in the Application’s script folder in J drive
Page 29 of 58
16.Create a new database as a clone of an existing database to non-production environment
1.Login into SqlAssistance as CFDBA user into non-production environment

2.Run the standard procedure CALL CFDBA.CREATE_HIER_PROC ('IMDCI',
'adminpassword',null,100,100,100,'Y','N',omsg,oresult);
3. Once you run the above standard procedure application hierarchy created. 1. Login into SqlAssistance
as CFDBA user into non-production environment
2. Run the standard procedure to create the database hierarchy for target of clone CALL
CFDBA.CREATE_HIER_PROC ('IMDCI', 'adminpassword',null,100,100,100,'Y','N',omsg,oresult);
3. Once you run the above standard procedure application hierarchy created.
4. Log into Teradata Administrator and connect to production environment to copy the objects from
production to target clone hierarchy created in step 2
5. Highlight the database you want to clone and select file -> copy objects
6. In the left side panel Identify the Source system, database and tables
7. In the right side panel identify the To Data Source system and database. This is the database you
created in step 2
17.Creation of a new schema, database to existing Instance

2.execute the Standard procedure to create user
CALL CFDBA.CREATE_USER_PROC
('ISID',null,'password',null,null,'PROFILE',’Account for ISID Name,
Application’,RETURNMSG,OUTMSG);
4.CREATE TABLE new D.N.new T.N AS old D.N.old T.N WITH DATA;
5. Above command copy data from old table to new table from one datase to another.
18.Create Role or Profile

2.create profile
CREATE PROFILE "APPL_USER_PROFILE" AS
--Custom spool size for your application USER account
SPOOL=1073741824000
TEMPORARY=524288000
ACCOUNT=$M_+APPL_B&S&D&H
--Default Semantic or Access View database
DEFAULT DATABASE="APPL_PUB" or "APPL_RPT"
PASSWORD =
(EXPIRE=NULL,MINCHAR=NULL,MAXCHAR=NULL,MAXLOGONATTEMPTS=NULL,LOCKE
Page 30 of 58
DUSEREXPIRE=NULL,REUSE=NULL,DIGITS=NULL,RESTRICTWORDS=NULL,SPECCHAR=N
ULL)
3.Create Role
create role 'rolename'
19.Modify role or profile

2.Modify Profile
MODIFY PROFILE profile_name AS
parameter_setting [ ... ,parameter_setting ];
3.Modify role
modify user username as default role=all;
20.Password changes for privileged database accounts (ex: sys, system,DBC, Viewpoint, PDCR,
cfdba, app schema)
1. Login to root user

2. enter commanad su
3. Type the command passwd
4. Enter your new password and repeat it at the confirmation prompt.
DBC
1. Logon as Root (on to the gateway node).
2. Enter cnsterm 6 on the UNIX command line. This will display the
Database Window Supervisor screen.
3. To start a console utility, type 'start tstsql'). The display will tell you which partition tstsql was
started in (i.e., Window 1).
4. Press the Delete key (CTRL + D) and enter cnsterm 1. You will now see the
contents of Window 1 (i.e., the TSTSQL program)
TSTSQL here. Enter your logon or TSTSQL command:
>.logon dbc,
TSTSQL -- Enter DBC/SQL request or TSTSQL command:
> MODIFY USER dbc AS RELEASE PASSWORD LOCK;
TSTSQL -- Enter DBC/SQL request or TSTSQL command:
5. Press the DELETE key and enter cnsterm 6 to move back to the
Supervisor display.
6. Type stop 1 to terminate the utility that was running in Window 1 (i.e.,
TSTSQL).
cfdba
1. Login into SQL Assistance/bteq using CFDBA user
2. MODIFY USER cfdba AS PASSWORD = ***;
3. login into SQL Assistance as username = cfdba
4. Change the password once login into the system
Page 31 of 58
21.Modify Backup job/scripts- include/exclude databases

2.go to the existing backup script file path
3. edit using vi backupscriptfile Include/exclude database
4. save and exit
22.Modify Database Initialization Parameter (No Downtime Required)
1. login into teradata system as Root user

2. # Run cnsterm command
3.Input Supervisor Command:
> start dbscontrol
Started 'dbscontrol' in window 1
4. press Ctl+z come out cnsterm window
5. # run cnsterm 1 to run dbscontrol
The current DBS Control GDO has been read.
Enter a command, HELP, or QUIT:
6. Modify required change as requested
23.Administration of Database Management tools
1.Teradata Administrator - create user, roles, proile,move space etc

1.Roles Manager in Admin in Viewpoint – Adding users to roles, creating new roles etc
2.Alert setup in Admin in Viewpoint – Adding new Alerts, modifying existing Alerts etc
3.Teradata Systems in Admin in Viewpoint
24.Installation of database client tools
1. Download the client latest software from TAYS(Teradata At your Service).

2. Login Into system as Administrator
3. run client installation package file like (.exe file for windows), (.rpm for Linux) etc.
4. Complete the client installation package file successfully.
25.Installation of 3rd party tools/libraries on database server - no downtime

2. Download the 3rd tools like arcmain,bteq,sqlassistance etc.
3. run the packagename.rpm file on the database server
4.complete the installation successfully
26.Update Application Statistics
Page 32 of 58

2.Run the scripts to collect the tables for stats
SELECT a.DATABASENAME
, a.TABLENAME
, 'INSERT INTO
STATS_PRCSS.ADWStats_Table('||''''||TRIM(a.DATABASENAME)||''''||','||''''||TRIM(a.TABLENAME)|
|''''||','||''''||'10'||''''||');' AS XQL
FROM DBC.TABLES a
WHERE a.TABLEKIND IN ('T','O')
--and upper(databasename) not like '%_WORK'
and databasename = ${DATABASENAME}
and tablename not in (select tablename from STATS_PRCSS.ADWStats_Table where
databasename=a.databasename);
3.insert the tables in the ADWSTATS_TABLE.Run the insert statements from the above SQL.
4.Application team will run the stats table to update the stats
27.Update Database or System Statistics

2.Need to collect the DBC tables system statistics
3.runt the below scripts to
select 'COLLECT STATISTICS ON DBC.'||TABLENAME|| ' COLUMN '||COLUMNNAME||';' from
DBC.COLUMNS
where tablename in ('TVM',
'DBase' ,
'TVFields' ,
'AccessRights',
'Indexes' ,
'Profiles',
'Owners' ,
'Roles' ,
'RoleGrants',
'UDFInfo')
4. Copy and run the COLLECT STATISTICS ON DBC.'||TableName)||' COLUMN ') output
28. Adding new features to existing environment

2. Uncompress as follows:
# gunzip tdwalletxx__linux_indep.xxx.tar.gz
3. Untar as follows:
# tar xof tdwalletxx__linux_indep.xxx.tar
4. If applicable, remove previous software as follows:
Page 33 of 58
# rpm -e tdwalletxx
5. Install the software as follows:
# rpm -ivh tdwalletxx/tdwalletxxx1.noarch.rpm
29. A/B Database Switch
1. Generate the DDL from model.

2. Identify the ETL and RPT views Databasename.
--*** SET DATABASE TO BE THE SAME ONE THAT ETL SHOULD BE POINTING TO
EXEC GCRMDW_US_ETL.DATABASE_POINTER;
SELECT DATABASE AS GCRMDW_US_ETL;
--*** SET DATABASE TO BE THE SAME ONE THAT RPT SHOULD BE POINTING TO
EXEC GCRMDW_US_RPT.DATABASE_POINTER;
SELECT DATABASE AS GCRMDW_US_RPT;
3.Replace “GCRMDW_US.” To “GCRMDW_US_D1.”
4.Verify existing both ETL and RPT views definitions. In Test, All RPT views are using D1 database.
Replace “GCRMDW_US_ETLorRPT.” with “GCRMDW_US_RPT.”.
5.Save the DDL under script folder with naming standard as Rev<##>_D1_DDL.
6.Generate/compile the DDL on target DB.
7.D1 database changes are completed.
8.Open the existing D1 DDL and replace “GCRMDW_US_D1.” with “GCRMDW_US_D2.”
9.Replace “GCRMDW_US_RPT.” view with “GCRMDW_US_ETL.”
10. Save the DDL under script folder with naming standard as Rev<##>_D2_DDL.
11. Generate/Compile the DDL on target DB.
12. Execute both scripts in production.
Switch procedure for additional views
ETL point to D2
RPT point to D1
1. Execute below SQL
select databasename,tablename,requesttext
from dbc.tables
where databasename in ('GCRMDW_US_ETL')
and tablekind='V'
and lastaltertimestamp >= '2013/06/04 00:00:00'/*Time CR got implemented*/
2. Extract all the requesttext and replace “GCRMDW_US_ETL.” With “GCRMDW_US_RPT.D2”
3. Compile all the views.
4. Execute below SQL
select databasename,tablename,requesttext
from dbc.tables
where databasename in ('GCRMDW_US_RPT')
and tablekind='V'
and lastaltertimestamp >= '2013/06/04 00:00:00'
Page 34 of 58
30.Create O/S user environmental configuration scripts
A profile file is a start-up file of an UNIX user, like the autoexec.bat file of DOS. When a UNIX user tries to login to
his account, the operating system executes a lot of system files to set up the user account before returning the
prompt to the user.
In addition to the system settings, the user might wish to have some specific settings for his own account. To
achieve this in UNIX, at the end of the login process, the operating system executes a file at the user level, if
present. This file is called profile file.
The name of the profile file varies depending on the default shell of the user. The profile file, if present, should
always be in the home directory of the user. The following are the profile files of the commonly used shells:
Shell Profile File

Ksh .profile
Bourne .profile
Bash .bash_profile
Tcsh .login
Csh .login
The specific settings which an unix user usually does is:

•Setting of any environment variable
•Setting of any alias.(Though it is always recommended to keep the aliases in a separate file).
•Setting of PATH variable or any other path variables.
To create/edit a user profile vi the .profile file and add the variables as required. A .profile would look similar to
this:
#cat $HOME/.profile
export PATH=$PATH:~blogger/bin
export LIB_IND="1"
alias l="ls -lrt"
#
The content of the other profile will depend on the syntax of the respective shell.
Any changes made to the profile file will reflect by either of the following ways:
1. Login to the user account again.

2. Source the profile file. Sourcing the profile file means running the profile file at the command prompt.
The command to run the profile file, say .profile is:
. $HOME/.profile
Page 35 of 58
In case of sourcing the profile file in tcsh or csh, the command is:
source $HOME/.login
6.8 Change Management – Implementation
6.8.1 Raising a RFC

A RFC can be raised by a Remedy User who has permission for the Change Management module. Below are the important
points to follow when you raise an RFC that should be taken care of.
1. The tasks and team involved should be identified and their availability should be agreed to in advance.
2. For pre-authorized changes, ensure that the change is registered in the Common Catalog.
3. Tasks should be created and assigned to the respective team in the correct order based on the dependency in
Remedy. In case there is no dependency between two tasks, they (both tasks) should have the same sequence
number to enable both of the implementers to work in parallel, saving time.
4. If the implementation of a change requires down time, it (downtime) must be agreed to with the application
owner before submitting the RFC for approval.
5. The change request must be written clearly and should include any dependencies such as Pre-implementation
check, etc.
6. There should be a proper Test plan.
7. There should be a proper Rollback plan to fallback in case the test result fails.
8. Once submitted for approval, it goes through the approval process with the concerned service owner.
9. There is new feature of Future Approvers which will help you chase the approvers to approve if you are short
on time.
6.8.2 Scheduled Downtime Notification

In Remedy, scheduled down time is created to send the notification to all of the parties. To create it, go to Remedy >
Application >Quicklink> Scheduled Down Time.
6.8.3 Changes that Require an RFC and their Change Status

All Production and Validated or GXP Formal Test Environment non-operational changes require an RFC (Request for
change) and CAB approvals prior to implementing the change. If you have any doubt about the Validated / GXP, please
refer the “Regulatory Indicator” in the CI of the Server/Instance.
Page 36 of 58
6.8.4 RFC Timings

Productions changes are only permitted after 7 PM EST on Friday and must be completed by 6 PM EST on Sunday.
Exemption Requests to this procedure are submitted to Chuck Denecke.
Emergency RFCs:
Any exceptions to the above will require an Emergency Change Request. The Emergency Change Request must follow the
normal approval process with an additional approval — all Emergency Changes must be pre-approved by an ETAS
Leadership Team member.
1. All Major RFCs must be submitted by 12 AM EST Tuesday for changes to be implemented for the
upcoming weekend.
2. All Minor RFCs must be submitted by 2 PM EST Thursday for changes to be implemented for the
upcoming weekend.
6.8.5 RFC Work Instructions
Here are some pre-defined questionnaires that are addressed before the RFC is raised.
1. What is the full name of the application impacted by the change?

2. Does this change impact an Oss application? If so, have you attached approval from the Oss CAB in Work Info
History?
3. What is the environment impacted by the change (database name & server name)?
4. What is the test environment where this change was previously implemented (database name & server name) or
explanation if N/A?
5. What is the business driver or reason for the change?
6. What are the before and after details of the proposed change?
7. If downtime is required what are the details if they are not stated in the Summary field?
8. If the change is listed as an emergency, what is the reason and urgency for the change?
6.8.6 Support Group Names

The following Remedy groups are used for database related changes.
1. IS Change_Management (RFC Change Manager)
2. GTS Inf_Database_Services (RFC Change Coordinator)
6.9 Service Knowledge Management

Knowledge Management is the process that governs the gathering, analyzing, storing and sharing of knowledge
information. Its primary objective is to improve efficiency by reducing the need to rediscover knowledge. All operational
tasks should be entered into a KM article. This information in KM articles is used to manage the platform by each tier
group; therefore, it is important that such information is up to date and accurate.
The procedure to support Knowledge Management is as follows:
SDLC-SOP-05-07: Administration and Control of Service Knowledge Management Articles for Global Services (GS)
Page 37 of 58
7 Remedy Ticket Initiation and Assignment

All requests for Teradata database support must first be initiated by a Remedy Work Order, Service Request, or Incident.
The requestor should include a detailed description of the request or problem, the database name, and the server’s name.
The Remedy ticket should initially be submitted to the “DATABASE_SERVICES” Remedy Group for initial service
management.
The Service Manager monitors this queue, receives the request, and performs resource assessment. The Service Manager
routes the ticket to “DATABASE_SERVICES RESOURCES” Remedy Groups for HCL to manage.
HCL actively monitors these queues on a 24x7 basis to respond, manage, and complete the request according to the SLA
for the requests’ priority.
7.1 Queues Names

All requests/incidents are submitted through Remedy. The following groups are used for database related request, incidents
and changes.
1. Infrastructure DATABASE_SERVICES
2. Infrastructure DATABASE_SERVICES RESOURCES
Note: E-mails/IMs/Phone calls will be responded to for emergency requests, but a Remedy ticket must be submitted shortly
thereafter.
7.2 Remedy Service Tagging

At present there is no Remedy Service Tagging available for Teradata but going forward this would to be included. Once a
Remedy case is received by the offshore resource, a service tag needs to be assigned to the request.
Service Tag Status Service Description
Teradata Database Backups Request to backup an Teradata Database
Teradata Database Creation Request to create an Teradata Database
Teradata Database Migration Request to move a database or objects from one Teradata
Database to another with the application’s SLC
Teradata Performance Tuning Request to optimize the performance of the database and its
queries
Teradata Database Scheduled Request to create and/or execute an Teradata Database task or job
Activity
Page 38 of 58
Service Tag Status Service Description
Teradata Database Monitoring Request to monitor the database
Teradata Database Software Upgrade Request to apply software patch to the database
or Patch
Teradata Application Decommissioning of an Teradata Database Instance or database

Decommissioning
Teradata Database User Management Request for Teradata Database login or user creation or
modification
8 Service Manager Roles & Responsibilities

For Teradata activities there is no Service Manager. The HCL team needs to perform the Service Manager role for
Teradata.
8.1 Updating Resource Scheduling Calendar

Resource Scheduling Calendar is used to get the Service Manager, Escalation Manager and On-call resource names for
current week and near future weeks.
We need to make changes before 3PM EST (for the upcoming week) otherwise the changes will not be reflected in the
notification email.
Steps for adding names in the calendar:

1. Request Service Manager role access to Database Shared Services Portal http://ssdba-central-prd.merck.com
from Keith or any other relevant authority
2. Login to Database Shared Services Portal http://ssdba-central-prd.merck.com
3. Click on DBA Scheduling icon.
Page 39 of 58
4. Click on Event Management icon
5. Select Schedule Name as Database Shared Services and select Date Range
6. Select week in which you want to add/edit resource name
7. For adding resource click on green plus (+) symbol.
8. Select resource name, role and region and press Next button.
9. Press Ok button to confirm Service Manager Assignation or Cancel to cancel it.
Page 40 of 58
10. For Editing Calendar, select week in which you want to make changes. For editing existing resource click Edit
button under Action column and to add new resource click Green Plus icon.
11. Make changes and then press Apply and Return button.
12. To see existing Calendar, Schedule and Matrix report go to Report section
Page 41 of 58
9 Frequently Used Links

Description Link URL
Remedy https://remedy-web.merck.com
Documentum http://webtop.merck.com/
http://teamspace.merck.com/sites/edbmm/_layouts/viewlsts.aspx?BaseType
Teamsite
=1
Online Courses http://mylearning.merck.com
Egate https://egate.merck.com
Share Data Access http://sda.merck.com
Database Information e.g. http://teamspace.merck.com/sites/edbmm/Database%20Information/Forms/

Password files location AllItems.aspx
10 Vendor Support Detail

Teradata Support
Teradata support hours for P1 tickets are 365 days a year, 24 hours per day, 7 days per week. Teradata support hours for P2
tickets are 8am - 5pm ET Monday - Friday. P1 is for an unavailable production system. P2 is for a degraded production
system or for any problem on a non-production system. "
For any hardware related problem on Teradata, RFC is to be assigned to Linux team and for any patch upgrades etc RFC
would be with DBA.
Patrick Donnelly is the Technical contact from Teradata and Deborah Shylock is the account lead. The link to Teradata for
raising the incidents is https://tays.teradata.com/ . Also, refer to the document path and link for Teradata-
_Merck_KA_Document_Inventory as below:
G3A Core Infrastructure Transition  LibrariesProject Documents  KA Documentation -Temp Database
Teradata  Documents Teradata_Merck_KA_Document_Inventory.xls
http://projectsext.merck.com/sites/core_infr_transition/_layouts/xlviewer.aspx?id=/sites/core_infr_transition/ProjectDocum
ents/KA%20Documentation%20-
Temp/Database/Teradata/Documents/Teradata_Merck_KA_Document_Inventory.xlsx&Source=http%3A%2F%2Fprojects
ext%2Emerck%2Ecom%2Fsites%2Fcore%5Finfr%5Ftransition%2FProjectDocuments%2FForms%2FAllItems%2Easpx%
3FRootFolder%3D%252Fsites%252Fcore%255Finfr%255Ftransition%252FProjectDocuments%252FKA%2520Document
ation%2520%252DTemp%252FDatabase%252FTeradata%252FDocuments%26FolderCTID%3D0x0120005765ACD97B
235541AEE16F2D43DF78F6%26View%3D%7B08ACF559%2D78EF%2D401D%2DA481%2D98785CDBE59E%7D%2
6InitialTabId%3DRibbon%252EDocument%26VisibilityContext%3DWSSTabPersistence&DefaultItemOpen=1&DefaultIt
emOpen=1
Page 42 of 58
Each HCL Teradata resource will required his/her own TD@YS account. Merck's Master Customer Number is 31062. This
information is required when requesting an account. The reporting, ticket flow, and resolution of hardware incidents that
are discovered by Teradata need to follow the instructions on Remedy Knowledge Article KBA0001453.
11 Teradata Database Standards

Please go to the document path and link below for Teradata Engineering standards.
Teamspace  DatabaseShared Documents  Teradata  Teradata Engineering Standards V1.1
http://teamspace.merck.com/sites/InfSolEng/ISEDBEMS/ISEDB/Shared%20Documents/Teradata/Teradata%20Engineerin
g%20Standards%20V1.1.docx
11.1 Service request management and access management tool understanding

All the requests and work order should be managed by Remedy. There are in-house learning courses available to
understand the functionality of remedy.
Management or business unit approval is required on any Remedy work order requesting a new user account to be created
on Teradata.
11.2 User’s Account & Grants Process

Merck’s high level system hierarchy, user creation process, roles and profiles should be followed as per the process
described in the Teradata Account Administration document. Please find the below the document path and link to this
document.
Teamspace  DatabaseShared Documents  Teradata  Teradata Account Administration
http://teamspace.merck.com/sites/InfSolEng/ISEDBEMS/ISEDB/Shared%20Documents/Teradata/Teradata%20Account%2
0Administration.docx
11.3 Standard Password Complexities & Requirements

All passwords should be changed quarterly as per the Merck’s defined password changed guidelines. There are in-house
learning courses available for password changes.
CPOL 26: http://policy.merck.com/company/cpol26.html
12 Database Operational Activities

12.1 PDCR
5. All the performance and capacity reporting is done through PDCR. The PDCR TJC process utilizes Viewpoint
Alerts
 Job inserts rows into DBCMNGR.AlertRequest table on error
 Viewpoint monitors this table and distributes alerts based on the setting of the PDCR Action Set or Group
 Use Viewpoint Screen
– Admin -> Alert Setup -> Alert Presets -> Action Sets (or) Groups
Page 43 of 58
6. PDCR toolkit, Installation guide, operation and overview guide are available at Teamsite.
7. Please find below the document path and link to these docs
G3A Core Infrastructure Transition  LibrariesProject Documents KA Documentation -Temp
Database Teradata  PDCR
http://projectsext.merck.com/sites/core_infr_transition/ProjectDocuments/Forms/AllItems.aspx?RootFolder=%
2Fsites%2Fcore%5Finfr%5Ftransition%2FProjectDocuments%2FKA%20Documentation%20%2DTemp%2FD
atabase%2FTeradata%2FPDCR&FolderCTID=0x0120005765ACD97B235541AEE16F2D43DF78F6&View={
08ACF559-78EF-401D-A481-98785CDBE59E}
12.2 Backup and Restore Process

TDCPRD01 backups begin at midnight Wednesday night / Thursday morning. TDCDEV01 and TDWTST01 begin at 8pm
Wednesday night. TDCPPT01 begins at 7pm Wednesday night. Backups are Application Backup Utility (ABU) driven.
Currently, there are 4 types of backups for each Teradata system. They are DBC Data backup DBC NODATA backup
Application Data backup, DDL extraction script.
Emails are sent to the SSDBA for the success / failure of the backups. At present, in case of failure of backups this is fixed
by the SSDBA the next day but going forward this needs to be fixed immediately after getting the alert.
Restores are done on need basis. Generally we need to restore only the table structures and application teams restores the
data from the source system.
Please refer to the below document path and link for the complete process
Teamspace  DatabaseShared Documents  Teradata  Teradata_Backup_Guide
http://teamspace.merck.com/sites/InfSolEng/ISEDBEMS/ISEDB/Shared%20Documents/Teradata/Teradata_Backup_Guide
.docx
The installation and user guide for ABU is can be downloaded from the Teradata site for reference.
Note: These Teradata backups are to be integrated with the Dashboard for tracking.
12.3 Physical Database Hierarchy

Refer to the Merck Teradata client facing DBA below the document path and link for Physical database hierarchy.
Teradata  Documents  APPL DB Hierarchy Structures.ppt
http://projectsext.merck.com/sites/core_infr_transition/_layouts/PowerPoint.aspx?PowerPointView=ReadingView&Present
ationId=/sites/core_infr_transition/ProjectDocuments/KA%20Documentation%20-
Temp/Database/Teradata/Documents/APPL%20DB%20Hierarchy%20Structures.ppt&Source=http%3A%2F%2Fprojectse
xt%2Emerck%2Ecom%2Fsites%2Fcore%5Finfr%5Ftransition%2FProjectDocuments%2FForms%2FAllItems%2Easpx%3
FRootFolder%3D%252Fsites%252Fcore%255Finfr%255Ftransition%252FProjectDocuments%252FKA%2520Documenta
tion%2520%252DTemp%252FDatabase%252FTeradata%252FDocuments%26FolderCTID%3D0x0120005765ACD97B2
35541AEE16F2D43DF78F6%26View%3D%7B08ACF559%2D78EF%2D401D%2DA481%2D98785CDBE59E%7D&De
faultItemOpen=1&DefaultItemOpen=1
Page 44 of 58
12.4 How to collect statistics for the database

Go to section 10 of the EIM Teradata Database Engineering Guidelines.

Teradata  Documents EIM Teradata Database Engineering Guidelines.Doc
http://projectsext.merck.com/sites/core_infr_transition/_layouts/WordViewer.aspx?id=/sites/core_infr_transition/ProjectDo
cuments/KA%20Documentation%20-
Temp/Database/Teradata/Documents/EIM%20Teradata%20Database%20Engineering%20Guidelines.doc&Source=http%3
A%2F%2Fprojectsext%2Emerck%2Ecom%2Fsites%2Fcore%5Finfr%5Ftransition%2FProjectDocuments%2FForms%2FA
llItems%2Easpx%3FRootFolder%3D%252Fsites%252Fcore%255Finfr%255Ftransition%252FProjectDocuments%252FK
A%2520Documentation%2520%252DTemp%252FDatabase%252FTeradata%252FDocuments%26FolderCTID%3D0x012
0005765ACD97B235541AEE16F2D43DF78F6%26View%3D%7B08ACF559%2D78EF%2D401D%2DA481%2D98785
CDBE59E%7D&DefaultItemOpen=1&DefaultItemOpen=1
12.5 Creating standard roles, profiles, databases, users, and grants for application setup
Go to section 13.1 of EIM Teradata Database Engineering Guidelines.
G3A Core Infrastructure Transition LibrariesProject Documents  KA Documentation -Temp Database Teradata
 Documents EIM Teradata Database Engineering Guidelines.Doc
12.6 How to create a standard database hierarchy

Go to section 13.2 of the EIM Teradata Database Engineering Guidelines.
Page 45 of 58
12.7 How to implement application security for a standard application

Go to section 13.3 of the EIM Teradata Database Engineering Guidelines
12.8 How to create a standard ISID user in Teradata

12.9 How to view granted privileges

Page 46 of 58
13 Scheduled Activities
13.1 Daily Procedures
Following daily / weekly procedures /monthly activities need to be carried out and Teradata_Daily_log sheet needs to be
updated on daily basis. Go to the document path and link below for updating the Log sheet:
G3A Core Infrastructure TransitionLibrariesProject Documents  KA Documentation -Temp Database Teradata
 Documents Teradata_Daily_Log.xlxs
http://projectsext.merck.com/sites/core_infr_transition/_layouts/xlviewer.aspx?id=/sites/core_infr_transition/ProjectDocum
ents/KA%20Documentation%20-
Temp/Database/Teradata/Documents/Teradata_Daily_Log.xlsx&Source=http%3A%2F%2Fprojectsext%2Emerck%2Ecom
%2Fsites%2Fcore%5Finfr%5Ftransition%2FProjectDocuments%2FForms%2FAllItems%2Easpx%3FRootFolder%3D%25
2Fsites%252Fcore%255Finfr%255Ftransition%252FProjectDocuments%252FKA%2520Documentation%2520%252DTe
mp%252FDatabase%252FTeradata%252FDocuments%26FolderCTID%3D0x0120005765ACD97B235541AEE16F2D43
DF78F6%26View%3D%7B08ACF559%2D78EF%2D401D%2DA481%2D98785CDBE59E%7D&DefaultItemOpen=1&
DefaultItemOpen=1
13.1.1 Check for system Health

Login to Viewpoint to see the productivity portal for system health of all the environments i.e. Production, Pre-prod, and
Test & Development. There are also alerts defined for system health which are generated through Viewpoint as and when
the system health is critical or system health degraded.
1. Login in to viewpoint as and check the System health portal. It should be healthy /green on Production, Pre-
prod and Development systems.
2. For Test system login to viewpoint check the System health portal. It should be healthy/green.
3. Check the Database spaces on Production, Test, Pre-prod and Development systems from Space Usage portal.
Only databases using Perm Space > 97% need be investigated.
4. If the space falls above threshold in Production we need to add space and interact with application owner that
the space has been added. However, if the space falls above threshold in any other environments i.e. Pre-Prod,
Test or Development we need to first send the mail to the Application owner and then add space after receiving
the confirmation.
5. If Database space falls below threshold in Prod, add the space and inform the application team that we have
added the space. If the Database falls below threshold on Dev, Test & PProd need to inform the application
team and add space accordingly.
13.1.2 Check for Alert logs

Check for the daily PDCR Alert logs in mails generated through Viewpoint for all the systems i.e. Dev, Test, Pre-Prod and
Production. If failed, logon to Linux Viewpoint and fix them as per the instructions in the Merck PDCR operation
document.
[Alert] CTCPRD01 - TDCPRD01_PDCR_Daily (Source: Viewpoint, Type: ALERTREQUEST)

Page 47 of 58
[Alert] GHHITDev - TDCDEV01_PDCR_Daily (Source: Viewpoint, Type: ALERTREQUEST)

[Alert] GHHITTes - TDWTST01_PDCR_Daily (Source: Viewpoint, Type: ALERTREQUEST)
[Alert] GHHITPpd - TDCPPT01_PDCR_Daily (Source: Viewpoint, Type: ALERTREQUEST)
Merck-PDCR Operations the document path and link:

 Documents  Merck–PDCR operations.pptx
Temp/Database/Teradata/Documents/Merck%20-
%20PDCR%20Operations.pptx&Source=http%3A%2F%2Fprojectsext%2Emerck%2Ecom%2Fsites%2Fcore%5Finfr%5Ft
ransition%2FProjectDocuments%2FForms%2FAllItems%2Easpx%3FRootFolder%3D%252Fsites%252Fcore%255Finfr%
255Ftransition%252FProjectDocuments%252FKA%2520Documentation%2520%252DTemp%252FDatabase%252FTerad
ata%252FDocuments%26FolderCTID%3D0x0120005765ACD97B235541AEE16F2D43DF78F6%26View%3D%7B08A
CF559%2D78EF%2D401D%2DA481%2D98785CDBE59E%7D&DefaultItemOpen=1&DefaultItemOpen=1
Alerts are defined for PDCR data space, System AMP CPU Skew, System Node CPU Usage, Database Perm Space, and
Database Space also.
13.1.3 Resource Intensive Queries

Check for long running CPU intensive queries and inform the corresponding business owner.
1. From query monitor portal, click on active sessions and check for long running queries which are taking more
than 4hrs and consuming CPU usage more than 90%. We need to inform the application owner/user through
mail with attaching the SQL Explain.
2. Monitor blocked session/user from query monitor portal. If there are any blocking sessions click on block
session and check what type of lock are placed. Write a mail the application owner/team that there is blocked
session for long time and kill the session to release the blocks.
3. If it’s a host utility lock on the session then monitor though tools>remote console>start show locks utility. If
there are any HUT locks then release them with the following command through SQL Assistant. RELEASE
LOCK (database) ALL, OVERRIDE.
13.2 Weekly Procedures

Following Weekly procedures / activities need to be carried out.
13.2.1 Backups
Check for the Teradata backup mails generated for all the environments and fix the backups in case of failure. There are 4
backup mails from each environment for DBC Data, DBC NODATA, App Data and DDL’s as follows:
Page 48 of 58
Teradata Backup - DDL

Teradata Backup - App Data
Teradata Backup - DBC NODATA
Teradata Backup - DBC Data
After checking the mail for backup errors, verify the jir files and purge scripts completed successfully.
DEVELOPMENT:
TO CHECK JIR FILES COPIED SUCCESSFULLY
ls -ltr /var/opt/teradata/abu_service/service/jir - Local Storage
ls -ltr /var/abu_storage/backup/jir/DEV/ - Off Node Storage i.e. the data domain box
TO CHECK PURGE RAN OK
cat /var/opt/teradata/scripts/log/teradata_purge_backups.ksh.DEV.out
TEST
ls -ltr /var/opt/teradata/abu_service/service/jir - Local storage
ls -ltr /var/abu_storage/backup/jir/TST/ - Off Node Storage i.e. the data domain box
cat /var/opt/teradata/scripts/log/teradata_purge_backups.ksh.TEST.out
PRE-PROD
ls -ltr /var/abu_storage/backup/jir/PPT/ - Off Node Storage i.e. the data domain box
cat /var/opt/teradata/scripts/log/teradata_purge_backups.ksh.PPT.out
PROD
ls -ltr /var/abu_storage/backup/jir/PROD/ - Off Node Storage i.e. the data domain box
cat /var/opt/teradata/scripts/log/teradata_purge_backups.ksh.PROD.out
As also mentioned in the Teradata backup guide.

the document path and Link to Teradata backup guide:
Teamspace  DatabaseShared Documents  Teradata  Teradata_Backup_Guide
http://teamspace.merck.com/sites/InfSolEng/ISEDBEMS/ISEDB/Shared%20Documents/Teradata/Teradata_Backup_Guide
.docx
Page 49 of 58
The exit code -1 means the job did not finish in the hour allowed by the foreground process. If this is the case, the following
resolution process needs to be followed:
1. Check the ABU log file to determine the exit code of the job. If the job finished successfully, then the issue is
resolved. If it failed or is still running, proceed.
2. Log into Viewpoint and determine whether the DBC backup job is blocked by another session. If so, proceed.
3. Abort the session for the DBC user running the backup. Do not abort the session of the application user.
4. Notify the application owner blocking the backup, inform him of the backup window and ask him not to
schedule any backups during that window. If the same application consistently blocks the backup, escalate the
issue to management.
13.2.2 Performance tuning

Look for long running queries in Viewpoint through query monitor and tune them.
13.3 Monthly Procedures

Following Monthly procedures / activities need to be carried out.
13.3.1 Capacity planning

Monitor capacity of all the environments and get inflight project details from the Application teams so that current and
predicted growth rate can be generated. Capacity related reports needs to be sent monthly which would include each
Application team’s currentperm usage. This needs to be compared with last year’s currentperm of the Applications to
predict the future growth.
13.3.2 Patch Upgrade / Maintenance Coordinate with Teradata GSC

Recommend patch upgrade / maintenance in co-ordination with Teradata GSC.
Monthly report should be generated containing Activity to be carried out, server name, when it is planned, who would
perform the activity from the Teradata GSC. and the same needs to be published to Merck for their intimation /
confirmation. Also, this needs to be coordinated / communicated with the Application teams for the Teradata box outage.
13.4 Quarterly procedures

All administrative passwords viz O/S user root on the Teradata database servers and the Viewpoint servers, Database user
DBC, Database user CFDBA should be changed quarterly as per the Merck’s defined password changed guidelines. There
are in-house learning courses available for password changes.
Passwords that needs to be changed are as follows:
Teradata System Name Environment User

TDCDEV01 Development DBC / SYSDBA/root
TDWTST01 Test DBC / SYSDBA/root
Page 50 of 58
TDCTCPPT01 Pre-Production DBC / SYSDBA/root
TDCPRD01 Production DBC / SYSDBA/root
Viewpoint
8. CTC – Root Password for server.
9. WHS – Root Password for server.
For Merck’s defined password changed guidelines. Please refer to below link.
CPOL 26: http://policy.merck.com/company/cpol26.html/
14 System Monitoring and Alerting

14.1 Viewpoint Alerts
All the alerts related to system health, node, database space, session & PDCR job failures etc. are set up through Viewpoint.
Emails and Alarm Point notifications are to CFDBA & SSDBA accordingly as per the defined actions
14.2 Alarm Point

Critical production events detected by Viewpoint alert via SNMP to the Alarm Point tool. Whenever an incident happens,
it sends an alert to corresponding team. The team is responsible to respond to the alert within 15 minutes failing which it
will get escalated to higher level. For escalation matrix, it basically starts with sending e-mail to the DBA team. Then after
each delay period, it moves on to the next Manager.
The Escalation Matrix for Alarm Point is as below:
Teradata Name Phone Time Interval (Min.)

Offshore Pager +1 (979) 092-6110
+(91) 44- 61050000 (Desk)

Vijay Sargunar
+1 (950) 000-4035 (Mobile)
30
+(91) 44- 61050000 (Desk)
Narayan Ramakrishna
+1 (956) 619-1392 (Mobile)
+1 (408) 328-7777 (Desk)

Kranthi Reddy 45
+1 (678) 900-8401 (Mobile)
+1 (408) 328-7777 (Desk)

Srini Damera 60
+1 (408) 334-5555 (Mobile)
Barbara A Reily +1 (215) 652-3055 (Desk)

75
Mike Tucker +1 (901) 320-5144 (Desk)
Page 51 of 58
+1 (215) 652-9553 (Desk)

Jim Fletcher 90
+1 (215) 264-9345 (Mobile)
+1 (901) 320-2714 (Desk)
Gene Phillips 120
+1 (901) 288-6358 (Mobile)
Based on your access, you can get information about Alarm Point and its configuration at below link.
http://alerting.merck.com/alarmpoint/signOn.do/
Once Database Team has resolved the issue, Database Team sends an email to “Database Operations Resources”
(1datoperr@northamerica.msx.merck.com).
It is desired to create an incident ticket (currently it not being created) so any investigation, root cause and resolution
provided can be recorded.
Note: Here we use different login and password (not the isid). Check with team for further information.
All the SNMP alerts for critical System Health metrics are sent to Alarm Point where a notification is sent out and an action
is required or else the issue escalates to management.
14.3 10 Minute Outage Notification Process

In case outage was caused by the incident, it is mandatory for the support team to notify the Merck Database Management
Team (Distribution list is not available at this time). Following are the recipients:
james_fletcher@merck.com; gene.phillips@merck.com; michael.tucker@merck.com; michael_lamagna@merck.com;
scott_hillegass@merck.com; keith.santulli@merck.com; surjeet.singh2@merck.com; charles_denecke@merck.com
Please find attached the e-mail format. The WI has to be created and uploaded to the SharePoint location later.
Server / service down alerts are received through Alarm Point, Bridge Call or Case.
a. If it is Alarm Point, we need to accept it and start work on it.
Page 52 of 58
b. If it is Bridge Call, we need to accept it and join the call.

c. If it is remedy case, we need to assign it ourselves and start work on it.
15 Database Tools
15.1 Viewpoint
All the Production and non-production Teradata systems are getting monitored through Teradata Viewpoint. Viz query
monitoring, space usage, CPU utilization, capacity heatmap, lock viewer etc.
Viewpoint access links for Production, Pre-prod and Development systems is http://qlctccp0110.merck.com/c. (CTC) and
for Test system is
http://qlwsnct0110.merck.com (WHS).
15.2 Teradata Tools and Utilities

All of the TTU 14 client software for Windows, HP and Linux plus the Teradata document path and link is located on
usdata003  Teradata Teradata Installs Teradata Client 14.0.1 TTU 14.0.1 Teradata Tools and Utilities
\\usdata003\Teradata\Teradata Installs\Teradata Client 14.0.1\TTU 14.0.1 Teradata Tools and Utilities
Windows_Linux_HP
Teradata tools and utilities for windows client are packaged and can be installed from http://shopping.merck.com also.
15.3 Configure ODBC via DNS

ODBC is configured via DNS. The instructions to configure ODBC via DNS are the Teradata document path and link
located:
usdata003  Teradata Teradata Installs ODBC and CLI Configuration\Configure ODBC and CLI through DNS.docx
\\usdata003\Teradata\Teradata Installs\ODBC and CLI Configuration\Configure ODBC and CLI through DNS.docx
15.4 Addressing the relationship between DNS and COP names

ODBC or CLI access
Network client utilities access the Teradata nodes via ODBC or via CLI (Call Level Interface). Some utilities support
ODBC, some support CLI, and some support both.
COP Aliases
If the client utility connects via CLI, then the hosts file or DNS (Domain Name System) must contain “COP aliases” for
accessing the Teradata system.
A COP alias consists of two parts: the TDPID and the COPx extension. You need to add the <tdpid>cop<n> alias to the
hosts file on the client machine or to the DNS used by the client machine
In the below hosts file or DNS example, the TDPID is “TDCDEV01” and the eighteen nodes have IP addresses 54.50.28.1
thru 54.50.28.39:
Page 53 of 58
54.50.28.1 TDCDEV01 TDCDEV01COP1

Configuring CLI on the client machine:

When configuring CLI access from a Teradata client utility, the user must specify the TDPID without the COPx extension,
e.g. “TDCDEV01”. CLI will choose a random COP alias to access the Teradata system thus ensuring load balancing and
node fault tolerance.
Configuring ODBC Data Sources on the client machine:

When configuring an ODBC Data Source (DSN) in the Teradata ODBC driver on a Windows client machine the user has
two options:
1. The ODBC Data source can be defined using the TDPID part of the cop alias similar to the way it is done
with CLI. ODBC will round-robin through the COP aliases to access the Teradata system thus ensuring
load balancing and node fault tolerance.
2. The ODBC Data source can be defined using one or more IP addresses or hosts/DNS network names. If the
system has more nodes, e.g. four, then the user should specify at least two IP addresses or hosts/DNS
names. That way the ODBC driver will be able to connect to the system even if one node is down. The
ODBC driver will select between the IP addresses or hosts/DNS names at random, thus ensuring load
balancing and node fault tolerance.
16 CRDB / Revision and Migration Document Links

Please find below the document links for CRDB install, Revision and Migration Process.
The Teradata document path and link to Process Flow ppt:

Teradata  Documents ChangeRequestProcess.pptx
Temp/Database/Teradata/Documents/ChangeRequestProcess.ppt&Source=http%3A%2F%2Fprojectsext%2Emerck%2Eco
m%2Fsites%2Fcore%5Finfr%5Ftransition%2FProjectDocuments%2FForms%2FAllItems%2Easpx%3FRootFolder%3D%
252Fsites%252Fcore%255Finfr%255Ftransition%252FProjectDocuments%252FKA%2520Documentation%2520%252DT
emp%252FDatabase%252FTeradata%252FDocuments%26FolderCTID%3D0x0120005765ACD97B235541AEE16F2D43
DF78F6%26View%3D%7B08ACF559%2D78EF%2D401D%2DA481%2D98785CDBE59E%7D%26InitialTabId%3DRib
bon%252EDocument%26VisibilityContext%3DWSSTabPersistence&DefaultItemOpen=1&DefaultItemOpen=1
Page 54 of 58
The Teradata Document path and Link to Revision Checklist:

Teradata  Documents revised-RevisionChecklist-2012-02-27.xlxs
http://projectsext.merck.com/sites/core_infr_transition/ProjectDocuments/KA%20Documentation%20-
Temp/Database/Teradata/Documents/revised-RevisionChecklist-2012-02-27.xls
The Teradata Document path and Link to Data Administration Manual (see ‘Data Administration Process: Data
Model Revision’ Chapter):
Teradata  Documents revised-DataAdministrationManual-2012-08-09.Docx
cuments/KA%20Documentation%20-Temp/Database/Teradata/Documents/revised-DataAdministrationManual-2012-08-
09.doc&Source=http%3A%2F%2Fprojectsext%2Emerck%2Ecom%2Fsites%2Fcore%5Finfr%5Ftransition%2FProjectDoc
uments%2FForms%2FAllItems%2Easpx%3FRootFolder%3D%252Fsites%252Fcore%255Finfr%255Ftransition%252FPro
jectDocuments%252FKA%2520Documentation%2520%252DTemp%252FDatabase%252FTeradata%252FDocuments%2
6FolderCTID%3D0x0120005765ACD97B235541AEE16F2D43DF78F6%26View%3D%7B08ACF559%2D78EF%2D401
D%2DA481%2D98785CDBE59E%7D%26InitialTabId%3DRibbon%252EDocument%26VisibilityContext%3DWSSTabP
ersistence&DefaultItemOpen=1&DefaultItemOpen=1
17 Disaster Recovery Setup Detail

Teradata System Recovery was tested on QA box before. However, there is no separate DR box available for testing.
Please refer to the Teradata document path and Link below for the process to be followed for Total Teradata system
recovery.
Teamspace  DatabaseShared Documents  Teradata  Teradata System Restore - Total System.Docx
http://teamspace.merck.com/sites/InfSolEng/ISEDBEMS/ISEDB/Shared%20Documents/Teradata/Teradata%20System%2
0Restore%20-%20Total%20System.docx
18 Database SOX Auditing & Reporting

SOX compliance is not a requirement for Teradata systems.
19 SLA Management
Below are the SLA’s defined by MERCK for resolving the support request or incidents.
Type of Tickets Response Time Resolution Time

Priority 1 15 Min 4 Hr.
Priority 2 15 Min 8 Hrs.
Page 55 of 58
20 Database In-flight Projects

Currently, there are no inflight project details available from the Application teams. Going forward this detail would be
required to so that current and predicted growth rate can be generated.
21 Escalation Matrix
This team must be engaged for all the Major/ Critical Incidents (any unplanned disruption to an application, core network
outage, major infrastructure outage resulting in significant business impact, etc.) when the page is sent to the on call technical
team.
Teradata Contacts
Name Number Time Interval
Offshore Pager +1(979) 092-6110

Vijay/Narayan +1(950) 000-4035 30
+1(/956) 619-1392
Kranthi R Patil +1(443) 280-7714 45
Srini Damera +1(408) 334-5555 60
Jim Fletcher +1(215) 264-9345 90
Escalation Timelines: This timeline should be maintained for escalating to the next level if the previous level has not
acknowledged/ acted on the issue.
22 General DOs and DON’Ts

DOs:
10. Get a remedy case for all kind of requests
11. Get a remedy case with Merck Manager Approval for DML or DDL tasks for all environments
12. Get Merck Manager approval for Account Administration in Production Environment
13. Spool SQL output for all your DML or DDL activities
14. Ensure to take backup of approved data deletion in all environments
15. Make sure RFC has been approved before making production changes
16. Make sure the Application Manager Approval is mandatory to create the user in the PRODUCTION Databases.
DON’Ts:
17. Don’t work on any request without a Remedy Case
18. Don’t run any DML or DDL simultaneously in multiple environments
Page 56 of 58
19. Don’t work any DML activity without Merck Manager’s approval
20. Don’t’ delete or truncate data from any table without manager approval (Production Environment)
23 Training
MyLearning provide the list of required training based on the curriculum assigned to the individual’s role. HCL has
identified Non Merck Persons to play the myLearning Report Runner role across 3A Group. These individuals are
responsible to run myLearning reports & track course completions / overdue items.
Merck’s required training curriculum by role is maintain and updated by Merck Training Lead and is found at below link:
IT Managed Services Partner Training Requirements
24 References
Reference Identification / Description Source / Location
SDLC-SOP-05-08: Global Services (GS) Service Request Fulfillment https://remedy-web.merck.com
http://teamspace.merck.com/sites/InfSolE
ng/ISEDBEMS/ISEDB/Shared%20Docu
Teradata Technical Security Standards ments/Teradata/Teradata%20Engineering
%20Standards%20V1.1.docx
SDLC-SOP-05-01: IT Incident Management https://sdlc.merck.com

SDLC-SOP-05-02: IT Problem Management https://sdlc.merck.com
SDLC-SOP-05-07: Administration and Control of Service Knowledge
https://sdlc.merck.com
Management Articles for Global Services (GS)
ITSM-SOP-GEN-000068 - GTS Configuration Management Process http://edms.merck.com
ITSOP-0006 - Qualification of Infrastructure Systems and Components http://edms.merck.com
ITSOP-0011: System Administration and Use of Enterprise Backup and
http://edms.merck.com
Recovery Systems
ITSOP-0014 - Account and Access Management for IRM&C Supported
Applications and Systems
ITSOP-0015 – IT Change Management Process – Pre-Authorized
Changes
ITSOP-0020 - IT Change Management Process – Planned and Unplanned
Changes
ITSOP-0018 - Information Security Threat and Vulnerability
Management
ITSOP-0019 - Information Security Investigation Management http://edms.merck.com
ITSOP-0021 - Information Security Incident Management http://edms.merck.com
Page 57 of 58
25 Resources
Resource Identification /
Description
Source / Location
Teamsite http://teamspace.merck.com/sites/edbmm/_layouts/viewlsts.aspx?BaseType=1
Prod/ Pre-Prod/Dev Viewpoint http://qlctccp0110.merck.com/c
Test Viewpoint http://qlwsnct0110.merck.com/c
Database Information e.g. http://teamspace.merck.com/sites/edbmm/Database%20Information/Forms/AllItems.
Password files location aspx
Teamsite http://teamspace.merck.com/sites/edbmm/_layouts/viewlsts.aspx?BaseType=1
Online Courses http://mylearning.merck.com
Egate https://egate.merck.com
Share Data Access http://sda.merck.com
Page 58 of 58

Database Teradata Support Plan

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Database Teradata Support Plan

Hochgeladen von

Copyright:

Verfügbare Formate

Platform Support Plan

System Name: Database Teradata Support Plan

Document Revision: 1.0

TECHNICAL UNIT APPROVAL:

BUSINESS UNIT APPROVAL:

Document created to establish support plan for Enterprise Database Services

8.1 UPDATING RESOURCE SCHEDULING CALENDAR .......................................................................................................... 39

20 DATABASE IN-FLIGHT PROJECTS ........................................................................................................................ 55

1.3 Amendments to the Support Plan

2 Support Strategy and Environment

Issues to the platform are reported as follow:

2.3 Support Environment

For searching a database in Remedy, follow below sequence:

2.3.1 Contact Information

2.3.2 Scope of Database Services

Please refer to the document path and link location below:

3 Abbreviations and Acronyms

Acronym - Term Definition

CM/IM Change Management/Incident Management

DBA Database Administrator

DEV Development Environment

PPT Pre-Prod Environment

DRP Disaster Recovery Plan

LOB Line of Business

NFS Network File Sharing

PPM Project and Portfolio Management

PRD Production Environment

RDBMS Relational Database Management Systems

SLA Service Level Agreement

SOP Standard Operating Procedure

TST Test Environment

WHS Whitehouse Station

4 Roles and Responsibilities

Role Role Responsibility

Database Administrator Build and Manage the Teradata Database Platform

Database Team Structure

There are 2 Database teams that operate in the Merck Environment:

Shared Services (SS) DBAs & CFDBA’s

Task Source of Task

Database Software Installations SSDBA

Database Creation SSDBA

Database Upgrade and Migrations SSDBA

Tablespace and Space management CFDBAs

Backup Scheduling & Monitoring SSDBA

Database Decommissioning SSDBA /CFDBAs

Backup Troubleshooting& Reporting Backup Notification Mail

Application and user account setup CFDBA

Performance monitoring and alert monitoring of the database CFDBA

Merck – Current Teradata Support Team

HCL –Current Database Support Team

5 Tools, Techniques and Methods*

5.1.1 Alarm Point for escalation and notification

5.1.2 Teradata Viewpoint

5.1.3 My Support Center (Remedy)

5.2 Communication Method

6 IT Service Management Procedures

6.2 Incident Management

The processes identified within this section are as follows:

6.3 Problem Management

6.4 Configuration Management

6.6 IT Change Management Process - Planned and Unplanned Changes

6.7 IT Change Management Process - Pre-Authorized Changes

6.7.1 Pre-Authorized CRQ Implementation Instructions

C.01.Create Viewpoint users