Beruflich Dokumente
Kultur Dokumente
Description
Current
Aug 27: CR 01614853 has been raised for the UMU not
Summary
also
State
Outage
No
Severity
Priority
Defect
Sub-Type
Software
Internal
Yes
Assignment Events
Ask Alcatel-Lucent
Product
Product
Version
LR14.2.W
Instance
Site Company
City
Abu Dhabi
Country
Contact
Name
Ayman Wagdy
Company
Phone
20 1005511876
Ayman.Wagdy@alcatel-lucent.com
Request Method
Email-Eng
Additional
+971-506722947
Contact Info
Dates
Occurred
(GMT+4)
05-Aug-2015 06:44
Time now
29-Aug-2015 09:38
Reported
05-Aug-2015 06:44
AR Created
05-Aug-2015 06:48
Service Start
05-Aug-2015 06:44
Next Customer Contact
03-Sep-2015 10:30
Responded
05-Aug-2015 07:03
SA - Calculated
Respond Target
05-Aug-2015 07:44
Restored
05-Aug-2015 07:03
SA - Calculated
Restore Target
09-Aug-2015 19:42
Resolved
-- --- ---SA - Calculated
Resolve Target
04-Oct-2015 19:42
-- --- ----
Last Modified
28-Aug-2015 13:17
Modified By
mohmadi
Entitlement
Agreement
People
Owner
TSCd-WLS-WCDMA-IN : mohmadi
Assignee
TEC-WLS-WCDMA-RNC : rameezk
Referred 1
NorP-WLS-WCDMA-vGlobal : mohmadi
Referred 2
TEC-WLS-WCDMA-RNC : rameezk
Submitter
eoniszcz
Description
Attachments
[empty]
2. 05-Aug-2015 07:05
mohmadi
Show
3. 05-Aug-2015 08:59
mohmadi
ALCATEL-LUCENT PROPRIETARY
Hello Ayman,
Tried to download the logs files but its giving error as file broken, can
you
upload the logs again.
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Email ID: mohmadi.cutlerywala@alcatel-lucent.com
4. 05-Aug-2015 16:48
mohmadi
ALCATEL-LUCENT PROPRIETARY
Hello Mohamadi,
Best Regards,
Ayman Wagdy
Senior WCDMA NPI-EMEA-RBC
Alcatel-Lucent U.A.E
Cellular: +971 50 672 2947
E-mail: Ayman.Wagdy@alcatel-lucent.com
Hello Mohamdi,
All logs have been uploaded completely at 10 am local time please confirm
Best Regards,
Ayman Wagdy
Senior WCDMA NPI-EMEA-RBC
Alcatel-Lucent U.A.E
Cellular: +971 50 672 2947
E-mail: Ayman.Wagdy@alcatel-lucent.com
Hello Ayman,
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Email ID: mohmadi.cutlerywala@alcatel-lucent.com
Hello Ayman,
Do you have logs of 2nd Aug when the issue happened as from the logs
provided I
cannot find any logs during that time.
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Email ID: mohmadi.cutlerywala@alcatel-lucent.com
5. 05-Aug-2015 17:53
mohmadi
ALCATEL-LUCENT PROPRIETARY
Hello Mohamdai,
Karim will upload the logs collected before rebooting the VMs
Best Regards,
Ayman Wagdy
Senior WCDMA NPI-EMEA-RBC
Alcatel-Lucent U.A.E
Cellular: +971 50 672 2947
E-mail: Ayman.Wagdy@alcatel-lucent.com
6. 05-Aug-2015 19:51
mohmadi
ALCATEL-LUCENT PROPRIETARY
Hello Mohamdi
Thanks,
BR,
Karim Ibrahim
LTE/3G Integration Professional
Alcatel-Lucent, Dubai, Etisalat Program
M: +971(0)563544568| ONNET: 22341798
Every Success has its network
7. 05-Aug-2015 19:56
mohmadi
ALCATEL-LUCENT PROPRIETARY
Product:
SW Version:
Detailed Description
VM
RTE
ID
ID
CPU
UTILIZATION
----------------------
1 6
2 5
3 5
1 10
2 10
3 9
1 7
2 11
3 unknown
1 6
2 8
3 7
3 6
22
1 9
2 unknown
3 13
2.
As above.
3.
All the steps taken before the issue observed by the engineer, who
reported the
AR
N/A
4.
5.
Current Status
Required solution and RCA.
6.
Major
7.
Attachments
Escalation Note.
8.
Thanks,
Mohmadi
8. 05-Aug-2015 20:03
mohmadi
9. 05-Aug-2015 20:05
mohmadi
ALCATEL-LUCENT PROPRIETARY
+ Escalation Note
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Email ID: mohmadi.cutlerywala@alcatel-lucent.com
Hello Paul/Sunil,
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Email ID: mohmadi.cutlerywala@alcatel-lucent.com
ALCATEL-LUCENT PROPRIETARY
Dear Karim,
Just to update that we have involved TEC for further support on this issue
and
will keep you updated on it.
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Email ID: mohmadi.cutlerywala@alcatel-lucent.com
ALCATEL-LUCENT PROPRIETARY
CR 01614853 has been raised for the UMU not initialzation issue.
Best Regards,
Rameez
Update to Current Summary: Aug 6: CR 01614853 has been raised for the UMU
not
initialzation issue.
ALCATEL-LUCENT PROPRIETARY
Hi Rameez,
Thanks,
Sudhakar,
Mobile Phone: +91 9342582713; Email Id: Sudhakar.babu@alcatel-lucent.com
From: owner-india-wcdma-utran-be@LIST.ALCATEL-LUCENT.COM
[mailto:owner-india-wcdma-utran-be@LIST.ALCATEL-LUCENT.COM] On Behalf Of
KHAN,
Rameez (Rameez)** CTR **
Sent: Thursday, August 06, 2015 12:25 AM
To: BABU, Sudhakar (Sudhakar)** CTR **
Cc: Leveille, Gerald (Gerry); Secchi, Michael (Michael); ROY, Paul (Paul)**
CTR
**; ARORA, Gaurav (Gaurav)** CTR **; AGRAWAL, Ritesh (Ritesh)** CTR **;
INDIA-WCDMA-UTRAN-BE; SERISE, Ghislain (Ghislain)** CTR **
Subject: RE: AR1-5843064//Critical//UAE- ETISALAT-ABU DHABI//9771 WCE
RNC//LR14.2// cnode internal sync error on RNCWCE242 which has cause WO to
stop on
the RNC
Hi Sudhakar,
I have raised CR 01614853 for the UMU not initialization issue and when that
issue
is fixed then we shouldnt run into cnode internal snyc error message. For
now
there is no workaround except RNC MIB rebuild. But in case we see the
notInit
(there should be a VM degraded alarm @WMS for it), we can reset just one VM
and
avoid the rebuild.
Thanks,
Rameez
ALCATEL-LUCENT PROPRIETARY
Hello Karim,
TEC has raised CR 01614853 for the UMU not initialization issue and when
that
issue is fixed then we shouldnt run into cnode internal snyc error
message. For
now there is no workaround except RNC MIB rebuild. But in case we see the
notInit
(there should be a VM degraded alarm @WMS for it), we can reset just one VM
and
avoid the rebuild.
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Email ID: mohmadi.cutlerywala@alcatel-lucent.com
ALCATEL-LUCENT PROPRIETARY
From: owner-india-wcdma-utran-be@LIST.ALCATEL-LUCENT.COM
[mailto:owner-india-wcdma-utran-be@LIST.ALCATEL-LUCENT.COM] On Behalf Of
MERAD,
ABDESSAMAD (ABDESSAMAD)
Sent: Thursday, August 06, 2015 1:32 PM
To: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **; IBRAHIM ABD EL NABY, Karim
(Karim)**
CTR **; WAGDY MANSOUR, AYMAN (AYMAN)** CTR **
Cc: INDIA-WCDMA-UTRAN-BE
Subject: RE: AR 1-5859176/Major/Emirates Telecommunications Corporation/9771
WCE
RNC/LR14.2/UMU not Initialized Error
Importance: High
Hello Mohamdi,
What is the severity of this CR? If we got in the problem and only solution
is RNC
rebuild then impact is huge!
Please ensue we receive wa and/or final solution asap
thanks
Best Regards
ALCATEL-LUCENT PROPRIETARY
Hi Rameez,
Thanks,
Mohmadi.
ALCATEL-LUCENT PROPRIETARY
To avoid the RNC MIB rebuild, we can also check which UMU's are not
initialized
and only reboot those VM's.
Final AP will be provided to the customer later today.
Best Regards,
Rameez
ALCATEL-LUCENT PROPRIETARY
a.
b.
c.
if any of the RTE is in Initialization Status:Not Initialized then
reset the
RTE, otherwise do nothing and go to step 4
d.
Example:
RTE
INITILIZATION
ID
ID
STATUS
------------------------0
initialized
notInitialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
2.
a.
b.
request module vm Umu <Umu Id> (e.g. if UMU 0: request module vm Umu 0
reset)
c.
3.
a.
b.
c.
d.
Example:
RTE
INITILIZATION
ID
ID
STATUS
------------------------0
initialized
4.
initialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
Apply your WO
5.
If alarm "Cnode Internal Synchronization error" is already present:
schedule a
maintenance window to perform a MIB rebuild. Avoid applying WO in the mean
time.
Update to Current Summary: Aug 7: CR 01614853 has been raised for the UMU
not
initialzation issue and WA action provided to site team, awaiting update.
ALCATEL-LUCENT PROPRIETARY
To: MERAD, ABDESSAMAD (ABDESSAMAD); IBRAHIM ABD EL NABY, Karim (Karim)** CTR
**;
WAGDY MANSOUR, AYMAN (AYMAN)** CTR **
Cc: INDIA-WCDMA-UTRAN-BE
Subject: RE: AR 1-5859176/Major/Emirates Telecommunications Corporation/9771
WCE
RNC/LR14.2/UMU not Initialized Error
Hello Abdessamad,
Action Plan:
a.
b.
c.
then
Example:
RTE
INITILIZATION
ID
ID
STATUS
------------------------0
initialized
notInitialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
2.
a.
b.
vm
Umu 0 reset)
c.
3.
a.
b.
c.
d.
Example:
Module Vm Umu
VM
RTE
INITILIZATION
ID
ID
STATUS
------------------------0
initialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
initialized
4.
Apply your WO
5.
present:
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Email ID: mohmadi.cutlerywala@alcatel-lucent.com
ALCATEL-LUCENT PROPRIETARY
Hi Rameez,
Just wanted to know is there any alert generated on this issue with the
recommendations of the procedures to be followed
if not
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Email ID: mohmadi.cutlerywala@alcatel-lucent.com
ALCATEL-LUCENT PROPRIETARY
Hi Mohmadi ,
Kindly
of
ALCATEL-LUCENT PROPRIETARY
ALCATEL-LUCENT PROPRIETARY
Update to Current Summary: Aug 10: CR 01614853 has been raised for the UMU
not
initialzation issue and WA action provided to site team, awaiting update.
ALCATEL-LUCENT PROPRIETARY
Hi Mohmadi,
We should have seen alarms on the WMS complaining about the UMUs are
degraded but
we dont see any. Thats why the local team didnt know/notice about the UMU
postmortems before running the WO. The UMU postmortems are very critical and
a CR
should be raised as soon as we see it. For now we can apply the AP and once
we
have a permanent fix then we shouldnt see the UMU not initialized issue
anymore.
The UMU not init is happening randomly on different UMUs and the OMU is not
aware
of it. Thats why the OMU is not sending an alarm to the WMS complaining
that some
of the UMUs are degraded and not initialized. This is an issue that needs
to be
fixed as well.
Thanks,
Rameez
Update to Current Summary: Aug 11: CR 01614853 has been raised for the UMU
not
initialzation issue and WA action provided to site team, awaiting update.
ALCATEL-LUCENT PROPRIETARY
Hello Abdessamad,
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Email ID: mohmadi.cutlerywala@alcatel-lucent.com
ALCATEL-LUCENT PROPRIETARY
Hi Mohamdi,
AP is understood and OK, however we need urgently reason for UMU state and
the
planned fix.
The AP is requiring SYSTEMATIC check before any WO activation, which is not
convenient at all in operation mode.
Thanks
Best Regards
ALCATEL-LUCENT PROPRIETARY
Hi Rameez,
Please find the site team update, as the AP is requiring SYSTEMATIC check
before
any WO activation, which is not convenient at all in operation mode.
And they need urgently reason for UMU state and the planned fix.
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Email ID: mohmadi.cutlerywala@alcatel-lucent.com
Update to Current Summary: Aug 12: CR 01614853 has been raised for the UMU
not
initialzation issue and WA action provided to site team, awaiting update.
ALCATEL-LUCENT PROPRIETARY
Hi Mohmadi,
Here is the reason why the UMUs were in bad state and not initialized.
UMU Reset happened due to Integer Divided by zero Exception. At the crash
point we
observed a zero value for a data member numOfTb (Number of Transport
blocks in
the frame) which is not expected in this flow.
Uplane is trying to process the filler/silent frame to power control
mechanism,
where the data member numOfTb coming as zero is used for calculation, which
is
causing the UMU Reset.
We have the fix available for it. It is being tested in our lab first and
when
its confirmed that the issue is fixed. Then we will provide it to the
customer.
Best Regards,
Rameez
Hello Abdessamad,
TEC has checked and found below reason why the UMUs were in bad state and
not
initialized.
UMU Reset happened due to Integer Divided by zero Exception. At the crash
point it
is observed a zero value for a data member numOfTb (Number of Transport
blocks
in the frame) which is not expected in this flow.
Uplane is trying to process the filler/silent frame to power control
mechanism,
where the data member numOfTb coming as zero is used for calculation, which
is
causing the UMU Reset.
For this fix is available and testing is ongoing in the TEC lab, will keep
you
updated on the progress of it.
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Email ID: mohmadi.cutlerywala@alcatel-lucent.com
Update to Current Summary: Aug 12: CR 01614853 has been raised for the UMU
not
initialzation issue and WA action provided to site team, also TEC testing
fix for
it. awaiting update.
Update to Current Summary: Aug 13: CR 01614853 has been raised for the UMU
not
initialzation issue and WA action provided to site team, also TEC testing
fix for
it. awaiting update.
ALCATEL-LUCENT PROPRIETARY
Thanks,
Please try to have all feebbacks on all issues so that we can plan for an
implementation in customer preise asapin LR14 of course 
Thanks
Best Regards
ALCATEL-LUCENT PROPRIETARY
Update to Current Summary: Aug 14: CR 01614853 has been raised for the UMU
not
initialzation issue and WA action provided to site team, also TEC testing
fix for
it. awaiting update.
ALCATEL-LUCENT PROPRIETARY
Hi Rameez ,
Any
further
Lab testing.
Update to Current Summary: Aug 17: CR 01614853 has been raised for the UMU
not
initialzation issue and WA action provided to site team, also TEC testing
fix for
it. awaiting update.
ALCATEL-LUCENT PROPRIETARY
Hi Gaurav,
I am attaching the email with the update from the Laverne test run
yesterday.
Thanks,
Rameez
ALCATEL-LUCENT PROPRIETARY
Hi ,
few
tests
a continuous discussion
on
this .
fix
which
were executed
and
today if the
the
can be applied .
Time Tracking Entry Added - IR01-TSA1
Update to Current Summary: Aug 27: CR 01614853 has been raised for the UMU
not
initialzation issue and WA action provided to site team, also TEC testing
fix for
it. awaiting update.
Hello Rameez,
ALCATEL-LUCENT PROPRIETARY
Thanks,
Mohmadi
ALCATEL-LUCENT PROPRIETARY