Assistance Request 1-5545833: New AR

View AR | AR without Proprietary
New AR:
History | States | Assignment | Timetracking
AR Text Search | More | CARES Home
Assistance Request 1-5545833
Customer Ticket N/A
Short Etisalat UAE/CTB trace on RNC235 stopped with error while
Description activation
Current Jan 28 : Issue resolved after MIB rebuild.
Summary
State Closed : Solution/Service Provided
Outage No
Severity 3
Priority 3
Service Request Remote Technical Support
Request Type Support
Sub-Type Configuration Assistance
Category Other
Sub-Category Customer Dedicated Support
Internal Yes
Assignment Events
Product
Product 9370 RNC
Model 9370 RNC 1500

Version LR13.3.W
No Change Request is linked to this AR
Product Location
Site ETISALAT ABU DHABI
Instance 9370RNC-Etisalat : In Service
Site Company Emirates Telecommunications Corporation
City Abu Dhabi
Country United Arab Emirates
Contact
Name Karim IBRAHIM ABD EL NABY
Company Alcatel-Lucent : Open on behalf of Customer
Phone 20 127 109 0616 Email

Karim.Ibrahim@alcatel-lucent.com
Request Method Email-Eng
Dates
Occurred 20-Jan-2015 22:09 Time now 29-Aug-2015 13:06
(GMT+4)
Reported 20-Jan-2015 22:09 AR Created 20-Jan-2015 22:37
Service Start 20-Jan-2015 22:09
Next Customer Contact 05-Feb-2015 10:30
Responded 20-Jan-2015 22:36 Respond Target 21-Jan-2015 22:09

SA - Calculated
Restored 20-Jan-2015 22:38 Restore Target -- --- ----

SA - None
Resolved 25-Jan-2015 11:36 Resolve Target 02-Apr-2015 08:00

SA - Calculated
Targets for Restore & Resolve were extended by 11 days
to account for Pending time
Closed 04-Feb-2015 09:20
Last Modified 04-Jun-2015 16:02 Modified By archive
Entitlement
Agreement 242895 (OXIA 627085)
Covered Service TS 24x7 (Gold, Wireless Mobile Access)
Script This request is entitled to service.
People
Owner TSCd-WLS-WCDMA-IN : pthota
Assignee CloQ-WLS-WCDMA-vGlobal : pthota
Referred 1 NorP-WLS-WCDMA-vGlobal : pthota
Referred 2 TEC-MoA-WCDMA_RNC-FR : lrigault
Resolve Group TEC-MoA-WCDMA_RNC-FR
Submitter glipski
Description
Attachments
Etisalat UAE/CTB trace on RNC235 stopped with error while activation
Showing Investigation and Proprietary logs together in time order Show

separately
Investigation Log and Proprietary Log

1. 20-Jan-2015 22:46
pthota
Update to Current Summary: Jan 21 : Under Investigation
2. 20-Jan-2015 22:49
pthota
Update to Current Summary: Jan 21 : Awaiting Receive Log Files / Technical

Info
3. 20-Jan-2015 23:04 ALCATEL-LUCENT PROPRIETARY

pthota
Karim (Karim)** CTR ** IBRAHIM ABD EL NABY
Workflow Status: Handled by an agent
Subject: Etisalat UAE/CTB trace on RNC235 stopped with error while

activation
Contact: Karim (Karim)** CTR ** IBRAHIM ABD EL NABY
From: "IBRAHIM ABD EL NABY, Karim (Karim)** CTR **"
<karim.ibrahim@alcatel-lucent.com>
To: ALU Support <support@alcatel-lucent.com>
Cc: "OKASHA, TAHER (TAHER)" <Taher.Okasha@alcatel-lucent.com>, "ABDEL-

HALIM,
SAYED (SAYED)" <Sayed.Abdel-Halim@alcatel-lucent.com>, "WAGDY MANSOUR,
AYMAN
(AYMAN)** CTR **" <Ayman.Wagdy@alcatel-lucent.com>, "ZAHABI, RAMSEY
(RAMSEY)" <ramsey.zahabi@alcatel-lucent.com>, "EL-MIDANY, AHMED (AHMED)"
<Ahmed.El-Midany@alcatel-lucent.com>
Sent on: January 20, 2015 7:08:46 PM CET
Hello,
Short Description : CTB trace on RNC235 stopped with error while
activation,
Current Summary : CTB trace on RNC235 stopped with error while
activation, we performed RNC MIB audit and completed successfully with

attached
warning, below the error appeared while activation,

Contact Surname : Karim
Contact Given Name : Ibrahim
Contact Phone :+971 56 354 4568
Contact Company : Alcatel-Lucent
Company : Emirates Telecommunications Corporation
Contract Number : OXIA 627085
Region : EMEA
Country : United Arab Emirates
City : Abu Dhabi
Site : ETISALAT ABU DHABI
Product Instance : RNC 9370
Severity : 3
Priority : 3
Product Version : LR13.3W

Outage : No
Internal/External: :Internal
--
Best Regards,
Karim Ibrahim
LTE/WCDMA Integration Professional
META WLS Access Network (GNE EMEA)
M(UAE): +971-56-354-4568
M(Egy): +2010-668-44-810
ONNET: 2205 5534
image001.png (15 KB)
image002.gif (1.78 KB)
RNC_RNC235_20150120113837_NC.HTML (9.74 KB)
4. 22-Jan-2015 11:09
pthota
Update to Current Summary: Jan 22 : Suspecting issue with MIB audit
inconsistencies, escalating the AR to next Level to take expert support as

there
is already CR for MIB audit fail on this RNC.

pthota
From: THOTA, Praveen-K (Praveen-K)** CTR **
Sent: Wednesday, January 21, 2015 12:17 AM
To: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **
Cc: INDIA-WCDMA-CdTSC-TEAM; OKASHA, TAHER (TAHER); ABDEL-HALIM, SAYED

(SAYED);
WAGDY MANSOUR, AYMAN (AYMAN)** CTR **; ZAHABI, RAMSEY (RAMSEY); EL-MIDANY,
AHMED
(AHMED)
Subject: AR 1-5545833//Minor//9370 RNC -235//Emirates Telecommunications
Corporation//UAE-Abudhabi//CTB trace on RNC235 stopped with error while

activation
Hi Karim,
To check the reported problem please provide the below logs.
Please provide below files from WMS
" Files from /opt/nortel/logs/wom: wom_configurationManager-00110.*
" Files from /opt/nortel/logs/wtk : JobHistory-00114_<YYYMMDD>.log and
WTK_JOBSERVER-00114.*
" Files from /opt/nortel/data/cmXML/SystemWO/ct_*
" Files from /opt/nortel/logs/utran: cp_utran-20000.*
wms_cm_mediationserver-20001.* and dump/_RNC.userLabel_<RNC name>/xge.dump
" Files from /opt/nortel/logs/utran: WOM_COMM_SEPE_RNC-20010.*
" Files from /opt/nortel/logs/utran/RncProxy :
build_log_xxx_xxxx_xx_xx_xx_xx_xx.log
Please provide below files from RNC
" Files from /CPx/OMUx/local/log/bos/sc (X is Active CP and Active OMU)

:
blogOMUx.xxxxxxxxTxxxxxx.tmp file with NACK Error time stamp.
Thanks and Regards,
Praveen Thota
+91 96899 18050
From: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **
To: THOTA, Praveen-K (Praveen-K)** CTR **

(SAYED);
AHMED
(AHMED)
Subject: RE: AR 1-5545833//Minor//9370 RNC -235//Emirates

Telecommunications

activation
Hi Praveen,
Kindly find all required logs
@/srv/data202256/data/server/Traces/from_france/ETISALAT_UAE/AR 1-5545833 ,
server
details below,
Please note that we already have Audit fail on RNC235 because the
parameter "the
parameter amountRep1A under RNC235 RadioAccessService/0 DedicatedConf/0
MeasurementConfClass/0 FullEventRepCritShoMgt/0
FullEventRepCritShoMgtEvent1A/0
needs to be changed from infinity to 16 for the Audit to pass" but it's
known
issue for us respect to old ticket opened before,
When yesterday we modified the parameter and did audit to the RNC, the
audit
completed with the attached warning which all related to CTB traces,
I just mention the audit issue as a note for you but it's not the main
target of
the AR, we need to know the RC why the CTB stopped with error while
activation& the corrective action of this issue,
Waiting your feedback , thanks to treat it as urgent as we need to solve

the issue
before receiving any escalation from the customer,
Server details:
SFTP://172.25.80.47
user: ftraces
password: Elv24trf
Thanks,
--
BR,
Karim

(SAYED);
AHMED
(AHMED)

Telecommunications

activation
Hi Karim,
I am checking for the NACK message sent by RNC when the trace was stopped
with
Errors.
"NACK happened when RNC does not like something in the MIB and does not
know what
to do with it so RNC send a NACK message to OAM when mib build fail"
As per the Image the CTB trace was started on 18-01-2015 03:48AM. But I am
not
finding the logs for that date in RNC or WMS. So please check for the
corresponding date logs or start the ctb trace again and request you to
provide
the required all logs again. Thanks.

Thanks and Regards,
Praveen Thota
+91 96899 18050

(SAYED);
AHMED
(AHMED)

Telecommunications

activation
Hi Karim,
Apart from Logs I may need some more information to understand the issue
better.
1) Please confirm whether this is the first time CTB trace was started on
this RNC
on 18th Jan?
No , the CTB trace was working fine before, also a week before we
launched a CTB
trace on the RNC normally but the output trace files was empty!.
2) CTB trace was started on 18th Jan and it stopped with errors? Whether
it has
been tried again recently on this RNC?

Yes I tried to it more times with different IMSIs but I received the
same error,
3) Any changes done on RNC before trace start time or preceding days?
@ 18th Jan we had activity to add 2 new MSC to the RNC, but we already
applied
the WOs before this date about 3 weeks, Also I performed @ 18th Lp/0
switchover as
the new MSCs Assoc was up but the signaling down (CS new components were
down), so
switchover done the components come Up,
4) As per below the parameter change happened on 20th Jan and after that
Audit
completed with warnings, before it was okay and there was no issue in audit
when
CTB trace was started?
The Audit failure is exiting on the RNC after LR13.3 intra upgrade and
it was
recommend to change the mentioned parameter below to this value to enhance

the
KPIs, so if we need to make Audit anytime it will fail except of we change

the
parameter , so I changed ot to try the MIB audit and see if it will give
any new
errors or warning then I reverted the parameter again to the recommend

value,
5) Please provide any other details if you have related to this issue.
Waiting your feedback.
Thanks and Regards,
Praveen Thota
+91 96899 18050

Sent: Wednesday, January 21, 2015 2:32 PM

(SAYED);
AHMED
(AHMED)

Telecommunications

activation
Hello Praveen,
Kindly find my comments highlighted below,
Waiting your feedback,
Thanks,
--
BR,
Karim

(SAYED);
AHMED
(AHMED)
Telecommunications

activation
Hi Karim,
As per below there is some traces exist in WMS, Please confirm all traces
are
deleted and there is no trace active from WMS NSP.
>: utran list TraceSession
{TraceSession RNC235_UMAINSERV1} {TraceSession trial_UMAINSERV1}
Thanks and Regards,
Praveen Thota
+91 96899 18050

(SAYED);
AHMED
(AHMED)

Telecommunications

activation
Hello Praveen,
Do you mean that there are a hanging trace session? From NSP no active CTB
traces
on RNC235.
Thanks,
--
BR,
Karim

(SAYED);
AHMED
(AHMED)

Telecommunications

activation
Hi Karim,
Yes, there is a trace session hanging for RNC235 (TraceSession

RNC235_UMAINSERV1).
Please try to delete it through WICL and launch MIB audit again and share
the
results.
If TraceSession trial_UMAINSERV1 is completed then please delete this as

well from
NSP or WICL.
Thanks and Regards,
Praveen Thota
+91 96899 18050

(SAYED);
AHMED
(AHMED)

Telecommunications

activation
Hello Praveen,
Please note that I performed list check using WICL and only TraceSession
trial_UMAINSERV1 is the existing one, I tried to delete it as below but the
command failed, can you check,

....wait please
Wireless Internet Command Language (WICL)
{TraceSession trial_UMAINSERV1}
>: delete TraceSession >trial
wrong # args: should be "::wisci::manipulationServices::delete domain

server mod
ule internalDN args"
>: delete TraceSession> trial
wrong # args: should be "::wisci::manipulationServices::delete domain

server
module internalDN args"
Thanks,
--
BR,
Karim

(SAYED);
AHMED
(AHMED)

Telecommunications

activation
Hi Karim,
As per MOC conversation, Now there is no TraceSessions. Please perform MIB

audit
and share the results. Also provide logs when you run CTB again in this
RNC.
Thanks and Regards,
Praveen Thota
+91 96899 18050

(SAYED);
AHMED
(AHMED)

Telecommunications

activation
Hi Praveen,
Yes I deleted the TraceSession trial_UMAINSERV1 and I didn't find the

TraceSession
RNC235_UMAINSERV1 , as discussed we performed the parameter change and

pefromed
Audit, the same warnings still exist as attached, also we tried to do

another CTB
trace and stopped with error.
We terminated the session but as below the session still exist on

utrantrace
session, Please advise, attached the required files,
{TraceSession Test_UMAINSERV1} {TraceSession test_UMAINSERV1}
Thanks,
--
BR,
Karim
Sent: Thursday, January 22, 2015 11:12 AM

(SAYED);
AHMED
(AHMED)

Telecommunications

activation
Hi Praveen,
The RNClogcollection uploaded to the same mentioned path below,
Thanks,
--
BR,
Karim

pthota
Escalating this AR to TEC for support, suspecting issue with MIB audit
inconsistencies.
1 - DESCRIPTION TEMPLATE
SUMMARY: RNC235 CTB trace stopped with Errors with message "RNC/RNC235'
rejects
the command with a NACK : TgeAckEvent ( neId=183422173, refPe=35021)
Cause: CAUS_NS"
Concerned RNC: RNC235

Concerned component (card, LP, OMU...): None
Type of alarm: CTB trace stopped with Errors
Time of the issue: 2015/01/21-17:03
Impact (partial, full outage on RNC...): No Outage, Not able to run CTB
trace.
Product release: LR 13.3
Patch release:
Reproducible: Yes
Suggested solution (workaround...): Suggested to remove the warnings in MIB

Audit.
PROBLEM DETAILS:
1. Action performed (feature activation, upgrade, work-order

application...): None
2. System behavior (high-level + detailed description if relevant): No

Impact
other than MIB audit warnings and CTB trace issue.
3. TSC/TIS analysis: When verified through wicl, found some trace sessions
which
were not present in NSP. deleted them manually with Wicl and performed
MIB audit, but again condition was same. In MIB audit we have observed
warnings
related to Access session but when we verified in snapshot those parameters
are not exist in RNC. So unable to correct them. we suspect CTB trace
stopped with
Errors due to the warnings of MIB audit. these warnings are pointed to
Access session and issue is with CTB trace only. CTG is not having any
issue as
per AR requestor.
4. Reason for priority/severity choice: Customer want CTB trace to be

enabled for
some other issue and hence customer pressure is there.

5. Contact information: Praveen Kumar Thota +91 9689918050
CONFIGURATION:
1. Hardware configuration: RNC 9370 with 12 DCPS
2. Software configuration: LR 13.3
IMPACT:
1. Frequency of the problem (once, x times, sporadic, systematic...): None
2. Features impacted (iMCTA, CAC, Rate Adaptation, HSxPA over IUR,

other...): None
3. Impact (call drop, board reset, KPI degradation, redundancy...): Unable

to run
CTB trace.
HISTORY:
1. Which & how many NEs impacted: None
2. How the issue was detected by customer or local ALU support (via KPIs
degradation, customer complaint...): Local ALU team.
3. When the issue has started: Not Sure
4. What actions were performed before or in time when the issue begun
(feature
activation, upgrade, 3rd party actions...) and is the issue
still present: No actions as per requestor and now issue is present.
5. What recovery actions have been done & with what results: removed the
trace
sessions through wicl and tried again with no success.
MIB rebuild not tried since there is inconsistencies in MIB audit. we need
TEC
confirmation to proceed with MIB rebuild after checking the logs.

2 - LOGS AND TRACES are under path :
/srv/data202256/data/server/Traces/from_france/ETISALAT_UAE/AR 1-5545833
1. RNCLogCollection => collected and uploaded on server :
LogCollection_RNC235_full_20150121_194536.tar.gz.
2. MIB audit result is uploaded : AR_1-5545833_Logs_21012015.rar.
3. network snapshot.
2. Below logs are collected and uploaded on France server : file name -
AR_1-5545833_Logs_21012015.rar.
files from WMS
o Files from /opt/nortel/logs/wom: wom_configurationManager-

00110.*
o Files from /opt/nortel/logs/wtk : JobHistory-

00114_<YYYMMDD>.log
and WTK_JOBSERVER-00114.*
o Files from /opt/nortel/data/cmXML/SystemWO/ct_*
o Files from /opt/nortel/logs/utran: cp_utran-20000.*
wms_cm_mediationserver-20001.* and dump/_RNC.userLabel_<RNC name>/xge.dump
o Files from /opt/nortel/logs/utran: WOM_COMM_SEPE_RNC-20010.*
o Files from /opt/nortel/logs/utran/RncProxy :
build_log_xxx_xxxx_xx_xx_xx_xx_xx.log
files from RNC
o Files from /CPx/OMUx/local/log/bos/sc (X is Active CP and

Active OMU)
: blogOMUx.xxxxxxxxTxxxxxx.tmp file
History : There is already one AR related to MIB audit failed for this
RNC235
(1-5351273), Later CR created for the same., GPS provided WA as one WO to
run
before starting the Audit
to make it successful. attached that WO as well.
Please let us know if any other Logs required. Thanks.
Time Tracking Entry Added - IR01-TSA3

lrigault
Lab investigation result:
Futher lab retest on AR 1-5351273, provided another work-around to solve

the MIB
audit failure for "the parameter amountRep1A under RNC235

RadioAccessService/0
DedicatedConf/0 MeasurementConfClass/0 FullEventRepCritShoMgt/0
FullEventRepCritShoMgtEvent1A/0 needs to be changed from infinity to 16 for

the
Audit to pass". Currently the parameter value is set to infinity in WMS

and to
propagate to the RNC then MIB rebuild can be used to update this parameter
with
"change not implemented ON LINE" (CR 1356516).
We tested CTB creation with such MIB audit failure in our lab and CTB IMSI
creation was successful.
The issue of CTB creation is not related to this MIB audit failure.
Current logs analysis from the customer site

++++++++++
It is found OMU log warning for each occurence of CTB IMSI creation failure
"Object Trca_SessionCTb_inst already created" and such MIB audit execution

result
with 6 inconsistencies.
File: "utran/AuditResult/RNC_RNC235_20150120113837_NC.xml"
target="XML_AUDIT
Will clearly report object diffMissingOmcData on RNC/0 CallTrace/0

AccessSession/0
|-logs
| +-log[0]
| | |-id -> 0
| | |-severity -> 0
| | |-msg -> Begin Audit on 'RNC/RNC235'
| +-log[1]
| | |-id -> 1
| | |-severity -> 1
| | |-msg -> Verdict NOT consistent, found 6 inconsistencies
| +-log[2]
| | |-id -> 2
| | |-severity -> 0
| | |-msg -> <a
href="utran/AuditResult/RNC_RNC235_20150120113837_NC.xml"
target="XML_AUDIT">
| +-log[3]
| |-id -> 3
| |-severity -> 0
| |-msg -> End Audit on 'RNC/RNC235' succeeded

|-startTime -> 2015/01/20-11:29:42,851+04:00
|-endTime -> 2015/01/20-11:38:38,543+04:00
2015/01/20-11:13:52,442+04:00 | ItMedServUtilities.cc:255 | Thr:23 | EXT |
component-name | class-name | CId:13473713.9268
extern: DN impacted for TGE 0 = {
, RNC/RNC235 CallTrace/0 AccessSession/0, }(number of elements=1)
18/01/2015 03:48:03.284 num:0000117b adm_umts_trca_p.c.766 [0x6c458000]

CM_CA @
OMU_1(Lp/5,Ap/5) (PERM): Adm:Object Trca_SessionCTb_inst already created.
18/01/2015 04:48:59.612 num:000015e5 adm_umts_trca_p.c.766 [0x6c458000]

CM_CA @
20/01/2015 11:13:52.426 num:0000c3ec adm_umts_trca_p.c.766 [0x6c458000]

CM_CA @
20/01/2015 11:28:18.328 num:0000c4e6 adm_umts_trca_p.c.766 [0x6c458000]

CM_CA @

CM_CA @
18/01/2015 03:48:03.284 num:0000117c adm_umts_trca_p.c.767 [0x6c458000]

CM_CA @
OMU_1(Lp/5,Ap/5) (PERM): Adm:Can't perform CREATE operation.
18/01/2015 03:48:03.284 num:0000117d adm_std_mo_p.c.662 [0x6c458000] CM_CA

@
OMU_1(Lp/5,Ap/5) (PERM): Adm -> ao update cds funct ko
processing_sessionctb_trca_Trca_SessionCTb_inst returns : 1
+++++++++++
WMS log for MIB audit execution report this error in file
wms_cm_mediationserver-20001.tra
<diffMissingOmcData on cd> -object:"RNC/0 CallTrace/0 AccessSession/0"
2015/01/20-11:35:54,164+04:00 | WMS_CM_ItAuditManager_i.cc:269 | Thr:23 |

EXT |
component-name | class-name | CId:13474310
end input: End Traitment: #AUDITMIB# -DN:???
<WMS_CM_ItAuditManager_i::computeNextTge> Succeeded

EXT |
begin input: Start Traitment: #AUDITMIB# -DN:???
<WMS_CM_ItAuditManager_i::computeNextTge>

EXT |
begin input: WMS_CM_ItAuditManager_i::calculateNextTge
2015/01/20-11:35:55,159+04:00 | Container.cc:49 | Thr:20 | EXT | component-

name |
class-name | CId:13474310
extern: no such dn in tree "RNC/0 CallTrace/0 AccessSession/0"
2015/01/20-11:35:55,161+04:00 | WMS_CM_AuditMediation.cc:40 | Thr:20 | EXT

|
extern: Traitment: #AUDITMIB# -DN:??? <diffMissingOmcData on cd> -

object:"RNC/0
CallTrace/0 AccessSession/0"
-cd:
<administrativeState>
<locked/>
</administrativeState>
<duration>240</duration>
<datafileUploadSize>524288</datafileUploadSize>
<datafileUploadPeriod>15</datafileUploadPeriod>
<externalSessionId>378</externalSessionId>
2015/01/20-11:35:55,164+04:00 | AuditMed.cc:442 | Thr:20 | EXT | component-

name |
extern: Adding InconsistencyOnSMO in the diff tree: RNC/0 CallTrace/0
AccessSession/0 TraceFilter/0

name |
AccessSession/0

name |
extern: Adding missing OMC Object in the diff tree: RNC/0 CallTrace/0
AccessSession/0 NbapDedicatedMeasConfigForCallTrace/0
name |
AccessSession/0 RrcDedicatedMeasConfigForCallTrace/0

name |
AccessSession/0 UePositionMeasConfigForCallTrace/0

name |
AccessSession/0
It is suspected such parameter was changed to make the audit successful and
report
the inconsistencies.
"the parameter amountRep1A under RNC235 RadioAccessService/0

DedicatedConf/0
needs to be changed from infinity to 16 for the Audit to pass"
| |-registTime -> 2015/01/20-11:24:22
| |-userData
| | |-neDn -> RNC/RNC235
| | |-objectDn -> RNC/RNC235
| | |-sessionName -> Online Commands
"cmXML/CMSessions/20150120/Online+Commands/RNC_RNC235_validate_201501201124
24_0.ht
ml"
And again ON LINE command was executed a second time to set back the value
which
fail the MIB audit
| |-registTime -> 2015/01/20-11:48:42
| |-userData
45_0.ht
ml"
++++++++++
==========
Action Plan for this AR 1-5545833 on CTB creation issue.
1) Check and provide MIB audit result file showing the 6 inconsistencies
For last MIB audit execution on RNC235 it was completed
| +-log[2]
| | |-id -> 2
| | |-severity -> 0
| | |-msg -> <a
target="XML_AUDIT">
| +-log[3]
| |-id -> 3
| |-severity -> 0
|-startTime -> 2015/01/20-11:29:42,851+04:00
|-endTime -> 2015/01/20-11:38:38,543+04:00
2) Backup RNC 235 MIB
Record RNC commit provisioning name with telnet session "d pr"
Select RNC, right click configuration / RNC backup
This MIB backup command execution takes several minutes.
This MIB backup could be studied afterwards if MIB rebuild was not going to
fix
the IMSI CTB creation failure.
3) Check if CTB IMSI session are already created for that RNC in WMS
database
WICL
>: RNC/RNC235 CallTrace/0 list AccessSession
On all listed session executed display command, for instance here

"AccessSession
0" instance.
>: RNC RNC235 CallTrace 0 AccessSession 0 display
If RNC RNC235 CallTrace 0 AccessSession 0 exist in WMS then delete it with

the
call trace wizzard then you should be able to create the new one you
expect.
++++++++
If RNC RNC235 CallTrace 0 AccessSession 0 does NOT exist in WMS then

execute RNC
MIB rebuild to clean the already created instance in the RNC.
You could temporary change change back the parameter value to 16 to get the
MIB
audit not failed anymore. This might indicate with the MIB completion the
inconsistency on object "RNC RNC235 CallTrace 0 AccessSession 0 " being

present in
RNC MIB but not WMS database from the MIB audit result.
This seems what was already executed on site on 20th Jan 2015 (two ON
LINE
command execution previously listed)
++++++++
4) Execute impacted RNC 235 MIB rebuild from WMS
5) Run again MIB and check it is successful
6) Create CTB IMSI session and check it is successful.
7) In case this action is not succesful , please provide:
- this RNC backup (from /opt/nortel/data/utran/neBackup)
- jobHistory file from /opt/nortel/wtk
- RncFullLogCollection.
Here example for lab RNC73
/opt/nortel/data/utran/NEImage/RNC/RNC73
RNC73_20150123_101244.tar.gz
==========
8. 25-Jan-2015 13:17
pthota
Update to Current Summary: Jan 25 : provided TEC update to the site team,
waiting
for their feedback.
9. 28-Jan-2015 09:55
pthota
Update to Current Summary: Jan 28 : provided TEC update to the site team,
waiting
for their feedback.
10. 04-Feb-2015 09:19

pthota
Update to Current Summary: Jan 28 : Issue resolved after MIB rebuild.
11. 04-Feb-2015 09:19 ALCATEL-LUCENT PROPRIETARY

pthota
Sent: Sunday, January 25, 2015 1:16 PM


(SAYED);
AHMED
(AHMED)

Telecommunications

activation
Hi Karim,
Please find below TEC update on this issue.
Lab investigation result:
Futher lab retest on AR 1-5351273, provided another work-around to solve

the MIB
audit failure for "the parameter amountRep1A under RNC235

RadioAccessService/0

the
Audit to pass". Currently the parameter value is set to infinity in WMS

and to
propagate to the RNC then MIB rebuild can be used to update this parameter
with
"change not implemented ON LINE" (CR 1356516).
We tested CTB creation with such MIB audit failure in our lab and CTB IMSI
creation was successful.
The issue of CTB creation is not related to this MIB audit failure.
Current logs analysis from the customer site
++++++++++
It is found OMU log warning for each occurence of CTB IMSI creation failure
"Object Trca_SessionCTb_inst already created" and such MIB audit execution

result
with 6 inconsistencies.
File: "utran/AuditResult/RNC_RNC235_20150120113837_NC.xml"
target="XML_AUDIT
Will clearly report object diffMissingOmcData on RNC/0 CallTrace/0

AccessSession/0
|-logs
| +-log[0]
| | |-id -> 0
| | |-severity -> 0
| | |-msg -> Begin Audit on 'RNC/RNC235'
| +-log[1]
| | |-id -> 1
| | |-severity -> 1
| +-log[2]
| | |-id -> 2
| | |-severity -> 0
| | |-msg -> <a
target="XML_AUDIT">
| +-log[3]
| |-id -> 3
| |-severity -> 0
|-startTime -> 2015/01/20-11:29:42,851+04:00
|-endTime -> 2015/01/20-11:38:38,543+04:00
2015/01/20-11:13:52,442+04:00 | ItMedServUtilities.cc:255 | Thr:23 | EXT |
component-name | class-name | CId:13473713.9268
extern: DN impacted for TGE 0 = {
, RNC/RNC235 CallTrace/0 AccessSession/0, }(number of elements=1)

CM_CA @
18/01/2015 04:48:59.612 num:000015e5 adm_umts_trca_p.c.766 [0x6c458000]

CM_CA @
20/01/2015 11:13:52.426 num:0000c3ec adm_umts_trca_p.c.766 [0x6c458000]

CM_CA @
20/01/2015 11:28:18.328 num:0000c4e6 adm_umts_trca_p.c.766 [0x6c458000]

CM_CA @

CM_CA @
18/01/2015 03:48:03.284 num:0000117c adm_umts_trca_p.c.767 [0x6c458000]

CM_CA @
OMU_1(Lp/5,Ap/5) (PERM): Adm:Can't perform CREATE operation.
18/01/2015 03:48:03.284 num:0000117d adm_std_mo_p.c.662 [0x6c458000] CM_CA

@
OMU_1(Lp/5,Ap/5) (PERM): Adm -> ao update cds funct ko
processing_sessionctb_trca_Trca_SessionCTb_inst returns : 1
+++++++++++
WMS log for MIB audit execution report this error in file
wms_cm_mediationserver-20001.tra
<diffMissingOmcData on cd> -object:"RNC/0 CallTrace/0 AccessSession/0"

EXT |
end input: End Traitment: #AUDITMIB# -DN:???
<WMS_CM_ItAuditManager_i::computeNextTge> Succeeded

EXT |
begin input: Start Traitment: #AUDITMIB# -DN:???
<WMS_CM_ItAuditManager_i::computeNextTge>

EXT |
begin input: WMS_CM_ItAuditManager_i::calculateNextTge
2015/01/20-11:35:55,159+04:00 | Container.cc:49 | Thr:20 | EXT | component-

name |
extern: no such dn in tree "RNC/0 CallTrace/0 AccessSession/0"

2015/01/20-11:35:55,161+04:00 | WMS_CM_AuditMediation.cc:40 | Thr:20 | EXT
|
extern: Traitment: #AUDITMIB# -DN:??? <diffMissingOmcData on cd> -

object:"RNC/0
CallTrace/0 AccessSession/0"
-cd:
<administrativeState>
<locked/>
</administrativeState>
<duration>240</duration>
<datafileUploadSize>524288</datafileUploadSize>
<datafileUploadPeriod>15</datafileUploadPeriod>
<externalSessionId>378</externalSessionId>

name |
AccessSession/0 TraceFilter/0

name |
AccessSession/0

name |
AccessSession/0 NbapDedicatedMeasConfigForCallTrace/0

name |
AccessSession/0 RrcDedicatedMeasConfigForCallTrace/0

name |
AccessSession/0 UePositionMeasConfigForCallTrace/0

name |
AccessSession/0
It is suspected such parameter was changed to make the audit successful and
report
the inconsistencies.

DedicatedConf/0
needs to be changed from infinity to 16 for the Audit to pass"

| |-registTime -> 2015/01/20-11:24:22
| |-userData
24_0.ht
ml"
And again ON LINE command was executed a second time to set back the value
which
fail the MIB audit
| |-registTime -> 2015/01/20-11:48:42
| |-userData
45_0.ht
ml"
++++++++++
==========
Action Plan for this AR 1-5545833 on CTB creation issue.
1) Check and provide MIB audit result file showing the 6 inconsistencies
For last MIB audit execution on RNC235 it was completed

| +-log[2]
| | |-id -> 2
| | |-severity -> 0
| | |-msg -> <a
target="XML_AUDIT">
| +-log[3]
| |-id -> 3
| |-severity -> 0
|-startTime -> 2015/01/20-11:29:42,851+04:00
|-endTime -> 2015/01/20-11:38:38,543+04:00
2) Backup RNC 235 MIB
Record RNC commit provisioning name with telnet session "d pr"
Select RNC, right click configuration / RNC backup
This MIB backup command execution takes several minutes.
This MIB backup could be studied afterwards if MIB rebuild was not going to
fix
the IMSI CTB creation failure.
3) Check if CTB IMSI session are already created for that RNC in WMS
database
WICL
>: RNC/RNC235 CallTrace/0 list AccessSession

?
On all listed session executed display command, for instance here

"AccessSession
0" instance.
>: RNC RNC235 CallTrace 0 AccessSession 0 display
If RNC RNC235 CallTrace 0 AccessSession 0 exist in WMS then delete it with

the
call trace wizzard then you should be able to create the new one you
expect.
++++++++
If RNC RNC235 CallTrace 0 AccessSession 0 does NOT exist in WMS then

execute RNC
MIB rebuild to clean the already created instance in the RNC.
You could temporary change change back the parameter value to 16 to get the
MIB
audit not failed anymore. This might indicate with the MIB completion the
inconsistency on object "RNC RNC235 CallTrace 0 AccessSession 0 " being

present in
RNC MIB but not WMS database from the MIB audit result.
This seems what was already executed on site on 20th Jan 2015 (two ON
LINE
command execution previously listed)
++++++++
4) Execute impacted RNC 235 MIB rebuild from WMS
5) Run again MIB and check it is successful
6) Create CTB IMSI session and check it is successful.
7) In case this action is not succesful , please provide:

- this RNC backup (from /opt/nortel/data/utran/neBackup)
- jobHistory file from /opt/nortel/wtk
- RncFullLogCollection.
Here example for lab RNC73
/opt/nortel/data/utran/NEImage/RNC/RNC73
RNC73_20150123_101244.tar.gz
==========
Thanks and Regards,
Praveen Thota
+91 96899 18050
Sent: Sunday, January 25, 2015 3:21 PM

(SAYED);
AHMED
(AHMED)

Telecommunications

activation
Hello Praveen,
Thanks for your feedback, So finally it's recommended to perform RNC
Rebuild to
solve this issue but from below we don't receive the RC , Why the CTB fail?
, also
for next we should change the parameter with MIB Rebuild not online?.
Thanks,
--
BR,
Karim
Sent: Monday, January 26, 2015 8:29 PM

(SAYED);
AHMED
(AHMED)

Telecommunications

activation
Hi Karim,
Please find below update for your queries........
===========================================================================
=======
=============================================
Yes, please execute MIB rebuild operation on this RNC235.
About the RCA on the CTB inconsistency with object not existing in the WMS
but
being present in RNC, it will be difficult to find but you can collect all
jobHistory files (JobHistory-00114-yyyymmdd) from /opt/nortel/logs/wtk

(grep
command on callTrace object could be used to check what might have

happened). If
issue occurred more than one one month ago then we will not have logs for
that. It
is advised to perform at least on successful MIB audit (followed by

dbbackup +
image backup) every two weeks or every month (but every month is a limit
with risk
of lising jobHistory files information limited to one month backlog).
Now about the CR 1356516 and old AR 1-5351273 (2 options are proposed).
It was advised to get the MIB audit successful (as initially reported by
one site
team).

the
Audit to pass".
As on site team seems to insist on using the parameter value "infinity"

intead of
value "16", i checked in our lab what kind of additional work-around could
be
used. CR owner reported the SET ONLINE command is not implemented until
LR13.3.B.
But this parameter can be updated with MIB rebuild as verified in our lab.
This is
provide full answer to on site team wish of using "infinity" parameter

value and
get the MIB audit successful. It is not advised at all to keep the MIB
audit
always failed because MIB audit inconsistency will not be detected

immediately.
===========================================================================
=======
=============================================
Thanks and Regards,
Praveen Thota
+91 96899 18050
Sent: Thursday, January 29, 2015 5:17 AM

(SAYED);
AHMED
(AHMED); BERIDY, AHMED (AHMED)

Telecommunications

activation
Hello Praveen,
Kindly note that the RNC235 rebuild done , Audit after is OK without
warning or
erros as attached also CTB trace is working fine.
Please note we back the parameter again to infinity in order don't have any
KPIs
degradation, we will plan for RNC audit every 2 weeks,
Will keep you updated if we face any issue related to this issue.
Thanks,
--
BR,
Karim
Sent: Tuesday, February 03, 2015 12:01 PM

(SAYED);
AHMED
(AHMED); BERIDY, AHMED (AHMED); MERAD, ABDESSAMAD (ABDESSAMAD)

Telecommunications

activation
Hello Praveen,
Thanks to close the AR, the issue solved as I mentioned below, and I tested
again
the Audit &CTB and both OK.
Thanks,
--
BR,
Karim
M: +971 (0)56-354-4568
Sent: Wednesday, February 04, 2015 10:36 AM

(SAYED);
AHMED
(AHMED); BERIDY, AHMED (AHMED); MERAD, ABDESSAMAD (ABDESSAMAD)

Telecommunications

activation
Hi Karim,
Thanks for AR closure confirmation. We are closing this AR.
Thanks and Regards,
Praveen Thota
+91 96899 18050
Time Tracking Entry Added - IR01-TSA1

Resolution
issue solved after MIB rebuild of the RNC.
End of Assistance Request 1-5545833

View AR | AR without Proprietary | OLCS
New AR:
History | States | Assignment | Timetracking
AR Text Search | More | CARES Home

Assistance Request 1-5545833: New AR

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Assistance Request 1-5545833: New AR

Hochgeladen von

Copyright:

Verfügbare Formate

View AR | AR without Proprietary

Short Etisalat UAE/CTB trace on RNC235 stopped with error while

Current Jan 28 : Issue resolved after MIB rebuild.

State Closed : Solution/Service Provided

Service Request Remote Technical Support

Request Type Support

Sub-Type Configuration Assistance

Sub-Category Customer Dedicated Support

Model 9370 RNC 1500

No Change Request is linked to this AR

Instance 9370RNC-Etisalat : In Service

Site Company Emirates Telecommunications Corporation

City Abu Dhabi

Country United Arab Emirates

Company Alcatel-Lucent : Open on behalf of Customer

Phone 20 127 109 0616 Email

Request Method Email-Eng

Reported 20-Jan-2015 22:09 AR Created 20-Jan-2015 22:37

Service Start 20-Jan-2015 22:09

Next Customer Contact 05-Feb-2015 10:30

Responded 20-Jan-2015 22:36 Respond Target 21-Jan-2015 22:09

Restored 20-Jan-2015 22:38 Restore Target -- --- ----

Resolved 25-Jan-2015 11:36 Resolve Target 02-Apr-2015 08:00

Targets for Restore & Resolve were extended by 11 days

to account for Pending time

Closed 04-Feb-2015 09:20

Last Modified 04-Jun-2015 16:02 Modified By archive

Covered Service TS 24x7 (Gold, Wireless Mobile Access)

Script This request is entitled to service.

Assignee CloQ-WLS-WCDMA-vGlobal : pthota

Referred 1 NorP-WLS-WCDMA-vGlobal : pthota

Referred 2 TEC-MoA-WCDMA_RNC-FR : lrigault

Resolve Group TEC-MoA-WCDMA_RNC-FR

Etisalat UAE/CTB trace on RNC235 stopped with error while activation

Showing Investigation and Proprietary logs together in time order Show

Investigation Log and Proprietary Log

Update to Current Summary: Jan 21 : Under Investigation

Update to Current Summary: Jan 21 : Awaiting Receive Log Files / Technical

3. 20-Jan-2015 23:04 ALCATEL-LUCENT PROPRIETARY

Workflow Status: Handled by an agent

Subject: Etisalat UAE/CTB trace on RNC235 stopped with error while

Contact: Karim (Karim)** CTR ** IBRAHIM ABD EL NABY

From: "IBRAHIM ABD EL NABY, Karim (Karim)** CTR **"

To: ALU Support <support@alcatel-lucent.com>

Cc: "OKASHA, TAHER (TAHER)" <Taher.Okasha@alcatel-lucent.com>, "ABDEL-

SAYED (SAYED)" <Sayed.Abdel-Halim@alcatel-lucent.com>, "WAGDY MANSOUR,

(AYMAN)** CTR **" <Ayman.Wagdy@alcatel-lucent.com>, "ZAHABI, RAMSEY

(RAMSEY)" <ramsey.zahabi@alcatel-lucent.com>, "EL-MIDANY, AHMED (AHMED)"

Sent on: January 20, 2015 7:08:46 PM CET

Short Description : CTB trace on RNC235 stopped with error while

Current Summary : CTB trace on RNC235 stopped with error while

activation, we performed RNC MIB audit and completed successfully with

warning, below the error appeared while activation,

Contact Given Name : Ibrahim

Contact Phone :+971 56 354 4568

Contact Company : Alcatel-Lucent

Company : Emirates Telecommunications Corporation

Contract Number : OXIA 627085

Country : United Arab Emirates

City : Abu Dhabi

Site : ETISALAT ABU DHABI

Product Instance : RNC 9370

Product Version : LR13.3W

Contact: Karim (Karim) CTR IBRAHIM ABD EL NABY

From: "IBRAHIM ABD EL NABY, Karim (Karim) CTR "

(AYMAN) CTR " <Ayman.Wagdy@alcatel-lucent.com>, "ZAHABI, RAMSEY

From: THOTA, Praveen-K (Praveen-K) CTR

To: IBRAHIM ABD EL NABY, Karim (Karim) CTR

From: IBRAHIM ABD EL NABY, Karim (Karim) CTR

To: THOTA, Praveen-K (Praveen-K) CTR

From: THOTA, Praveen-K (Praveen-K) CTR

To: IBRAHIM ABD EL NABY, Karim (Karim) CTR

From: THOTA, Praveen-K (Praveen-K) CTR

To: IBRAHIM ABD EL NABY, Karim (Karim) CTR

From: IBRAHIM ABD EL NABY, Karim (Karim) CTR

To: THOTA, Praveen-K (Praveen-K) CTR