IBMHD0000001925539 Maximo Angola Incident..

IBMHD0000001927102 Maximo (Angola Instance) (E&P)

IBMHD0000001935183 Telecoms & Scada
INC000000071385 Maximo (Angola Instance) (E&P)
INC000000076690 eLogbook (Angola Greater Plutonio Instance)

INC000000095801 Sun Virtual Environment - Alaska - Prod (E&P)

INC000000112370 Internet Gateway - North Slope Proxy - North Slope (Alaska)

eLogbook (Angola Greater Plutonio Instance)

600-02-12056673 Nt id:njamkf
Name:NZAMBI, KIDY
location:Fpso (Office)
Contact:+244222694215
Section 1 - Contact Information
Incident Reporter Name: AYALA, GEORGE
Incident Reporter Phone Number: +1 (409) 949-3136
Site Address: Sterling-Texas City
Alternate Contact Person:
Alternate Contact Number:
Site Address:
Section 2 - Incident Information
Full Description of Incident: USer called in to inform Our E-logbook application
is down, The application is located on BP1GPLAP003 server.
Can you please create a ticket for us.
Date and Time Incident was Reported: 4th February 2011 @ 0:16 GMT+05:30
Time Incident became a Major incident (if different) 03/2/11
Last time the service was working: 03/2/11 4 pm angola time
Region Impacted: NA
Country Impacted: NA
Segment(s)/Stream(s) of site impacted: upstream
SPU(s)/BU(s) of site impacted: application
Site(s)/Location(s): NA
Number of Users impacted: 150
Please provide the NT IDs of 3 additional users that is affected:
Reputational/business impact: (high, medium, low) Medium
RESULTS FOR SERVER BP1GPLAP003
Criticality of affected Application or Service:
eLogbook Application - Web Server
eLogbook (Angola Greater Plutonio Instance) (E&P) (CRITICALITY : Normal)
***AFFECTED*** BY INCIDENT - Service hosted in following locations :
FPSO Greater Plutonio Production
Error Message (if any):
Server name: BP1GPLAP003
CMDB name: BP1GPLAP003
Alias: 149.193.188.138
Server Owner: Francis, Clem A
Web site URL:
Print Server and Print Queue name:
Path, Folder, or Shared Drive name:
Does Server or Website Ping:
Pinging BP1GPLAP003.bp1.ad.bp.com [149.193.188.138] with 32 bytes of data:
Reply from 149.193.188.138: bytes=32 time=623ms TTL=113
Reply from 149.193.188.138: bytes=32 time=627ms TTL=113
Reply from 149.193.188.138: bytes=32 time=625ms TTL=113
Reply from 149.193.188.138: bytes=32 time=621ms TTL=113
Ping statistics for 149.193.188.138:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 621ms, Maximum = 627ms, Average = 624ms
Is there currently an open RFC affecting this server/application: Yes
CHG000000116125 - Summary : OH - Greater Plutonio eLogbook upgrade (part 2 - pro
duction eLogbook upgrade)
CHG000000115186 - Summary : OH - Greater Plutonio eLogbook upgrade
Support Information from CMDB:
Operating System Support : EMDC Best Shore Wintel
Facilities Support : DCT GOI Angola
Hardware Support : DCT GOI Angola
Backup Support : 3rd Party - HP DSMC Backups
Status*: Production - Live - Operationally Supported

Section 3 - Computer Information

Specify where the user is calling from (home, office, hotel, etc.): n/a
Specify which of these are NOT working (Internet/email/shared drives/network pri
nter): n/a
Build Type: (Passport/Explorer) n/a
How are the users connected to the BP1 Network: (wired, iRAS, iLink) n/a
Are they connecting to the Vanilla network: (wired or wireless?) n/a
Run IPCONFIG on user s machine (copy and paste): n/a
IP Address: n/a
Physical Address (MAC): n/a
DNS Servers: n/a
Informed the incendent managent team regarding the ticket

CHG000000115186 (eLogbook (Angola Greater Plutonio Instance)) - Pending Approval

OH - Greater Plutonio eLogbook upgrade
CHG000000102937 (BP1GPLAP003) - Closed - Successful - CAM 93902: <OOH> Quarterly
Release Patching - QR2 - of EDS servers, Altiris and Exchange servers

================================================
Desktop Team is IBM
Service accounts there is multiconfiguration account
Regional Operations team : HP team
Exchnage Team
Desktop Configuration

==========================================
Quarterly Release Patching - QR8
what was the root cause ?
Brian Lowe, ashutosh...
2-3 Question..
==========================================================
Application team were working on RFC # 115789
they has installed the application and found the services are in stopped state
and the CPU is in 100% utilization.
Apps team has asked us to reboot the server . We got the approval from Mark Rams
ey to reboot the server .
Before reboot we could find multiple instances of w3wp.exe and is taking high CP
U utilization , As per approvals we are rebooting the server
Rebooted the server but we found the Monitoringhost.exe. is utilizing the 50% C
PU . After we killed the process CPU came to normal. And the application team st
arted the the applications
High cpu occurred while the application team were deploying their application.

monitoringhost.exe is not a Windows OS process/application. it's related to SCOM

2007.
Confirmed with Application team to close the case

BP1XTXIS134 has been hitting more than 100% server utilisation while BP1XTXIS133
was normal