Customer Experience Test

Date :1-4-2015
Number Test :
Location :JL ASIA AFRIKA BCA
Longitude Latitude :107.61285 -6.922370 -77
PIC : DEAN Contact Number :

Status
No Level Function Basic Service Functionali Attempt
ty

1 Mandatory Location Update Success 10x

Mandatory Moving from 3G to 4G (UE IDLE

2 Success 3x
MODE)

Mandatory Moving from 4G to 3G (UE IDLE

3 Success 3x
MODE)

Mandatory Browsing until fullpage (signal

4 Success 10x
mode 4G/LTE)

5 Additional Chat apps (signal mode 4G/LTE) Delivered 10x

Mandatory

Streaming static (signal mode

6 enable 3x
4G/LTE)

7 Mandatory Streaming Mobile (4G/LTE 3G) Success 3x

Mandatory Streaming Mobile (signal mode 3G

8 Success without
3x package
4G/LTE) without package
hotrod 4G
hotrod 4G
 without package
without package
hotrod 4G
hotrod 4G

9 Additional Video Call Success 3x

10 Mandatory Video Confrence on 4G/LTE Pass 5x

SMS on 4G/LTE
Local
Mandatory
Onnet Success 5x
11 Offnet Success 5x
international
Additional
Any prefix (destination number) Success 3x

Voice Call (CSFB)

Local
Mandatory
Onnet Success 5x
12 Offnet Success 5x
international
Additional
Any prefix (destination number) Success 5x

[Voice] after fall back to 3G will be

13 Mandatory directly reselection to 4G/LTE Success 10x
automatically

14 Mandatory Voice Call (VOIP) Success 10x

MMS
Local

Onnet unable 3x

15 Additional
Offnet unable 3x

international

Any prefix (destination number) unable 3x

 [USSD/UMB] after fall back to 3G
16 Mandatory will be directly reselection to Success 5x
4G/LTE automatically

Consuming Quota/allowence
17 Additional Quota is consumed 1x
packet 4G/LTE

18 Additional Stoppage Success 1x

19 Mandatory APN Redirection Success 5x

20 Mandatory Download Speed Test Success 5x

21 Upload Speed Test Success 5x

22 PING/ Latency Test Success 5x

23 Additional Post call notification (PCN) Success 1x

24 Mandatory DL (FTP, max throughput) Success 1x

25 Mandatory UL (FTP, max throughput) Success 1x

Mandatory Game online (UE mobile 4G/LTE

26 Success 1x
3G

Mandatory Game online (UE mobile

27 3G4G/LTE Success 1x
 Methode Measurement Unit

success rate & avg

Turn on devices/data services, 1st signal shown 4G/LTE performance from total second
attempt

try UE in the border Area success rate & avg

UE should go to 4G spot & measure how many dBm performance from total dBm
on 4G/LTE that UE receivced attempt

success rate & avg

UE should leave 4G spot & measure how many dBM performance from total dBm
on 4G/LTE UE moved to 3G attempt

success rate & avg

UE browsing to local URL. E.g. www.google.co.id or performance from total second
www.xl.co.id attempt

success rate & avg

Using apps BBM/ Whats App/ Path etc. Sending and performance from total second
delivered at destination attempt
delay video second
User test 3 times of watching video (HD) of 3 minutes count of stall/buffering during numbers
duration video played
aplication reference use Youtube
success/not
success rate success

during streaming UE move from 4G/LTE to 3G spot

video will stop or continue. success rate total attempt numbers
Continue = success
stop = not success

during streaming UE move from 3G to 4G/LTE spot

video will stop or continue.
ithout package success rate total attempt numbers
ithout4G
Continue = success
package
otrod stop = not success
otrod 4G
ithout package
ithout4G
otrod package
otrod 4G

* UE attached in LTE then performs traditional video

call
* CS fallback will happen success rate total attempt second
* UE should be support fitur video call (CS Fallback)

Clear without lagging : 2

Using Skype/Google Hang out, etc Clear with lagging : 1 point
connected duration is 3 minutes Unclear/blur or lagging :0

A & B number same operator (deliverd to b#) second

A & B number different operator (deliverd to b#) success rate & avg second
performance from total
attempt
XL number sent to B number (out roamer) second

A & B number same operator second

A & B number different operator success rate & avg second
performance from total
attempt
XL number call to B number (out roamer) second

do out going calls, after dial tone hearing, then end call setup (< 7 sec), success
call and count how many second back to 4G/LTE, you rate, performance from total second
can get sample from testcase no 12 also Voice call

success/not
Minimum 9 calls are successful success

*using apps : e.g. line, wechat, kakao talk,viber etc <7 sec second
*select features phone call on apps
voice clarity
voice clear

count of unclear voice during numbers

connected

success/not
A & B number same operator success
success/not
A & B number different operator success
must fail
success/not
XL number sent to B number (out roamer) success
 must fail

Initiate 5 times of UMB/USSD attempts

UMB/USSD must be successful, and users reselect second
back to LTE in max 5 sec

- Check current quota status success/not

- Attempt internet data services (e.g. streaming) success rate total attempt success
- Re-check again the remaining quota
once Quota/allowence run out, UE will be redirect to success/not
success rate total attempt
stop page layer success
success/not
Use random APN name and try internet services success rate total attempt success
* using Ookla apps, find the best server > 21.9 Mbps (3 MHz)
* attempts 10x > 73 Mbps (10 MHz) Kbps
* capture log > 109.5 Mbps (15 MHz)

* using Ookla apps, find the best server > 8.1 Mbps (3 MHz)
* attempts 10x > 27 Mbps (10 MHz) Kbps
* capture log > 40.5 Mbps (15 MHz)

* using Ookla apps, find the best server

* attempts 10x <50 ms
* capture log

when UE on 4G/LTE, UE make outgoing call minimum

6 second duration calls then UE end call. success/not
success rate total attempt
UE will get notification (USSD) of remaining balance success
after used.

> 21.9 Mbps (3 MHz)

> 73 Mbps (10 MHz) Kbps
* instal application net per sec to capture statistic > 109.5 Mbps (15 MHz)
throughput
* file size 10 G
*capture maximum throughput > 8.1 Mbps (3 MHz)
> 27 Mbps (10 MHz) Kbps
> 40.5 Mbps (15 MHz)
success/not
during online UE moves from 4G/LTE to 3G spot connected/lost connection success
success/not
during online UE moves from 4G/LTE to 3G spot connected/lost connection success
1st test 2nd test 3rd test 4th test 5th test 6th test

2 1 1 2 3

-90 -91 -94

-82 -84 -113

2 1 2 2 2 1

3 3 1 1 1 1

2 3 2

0 0 0

SUCCESS SUCCESS SUCCESS

SUCCESS SUCCESS SUCCESS

SUCCESS SUCCESS SUCCESS

handset not support handset not support handset not support

2 1 0 1 2

3 3 6 7 5
3 3 3 3 3

not test not test not test not test not test

12 9 6 6 6
6 7 6 6 7

not test not test not test not test not test

12 9 6 6 6 6

SUCCESS SUCCESS SUCCESS SUCCESS SUCCESS SUCCESS

1 1 2 2 1 1

CLEAR CLEAR CLEAR CLEAR CLEAR CLEAR

0 0 0 0 0 0

not success not success not success

not success not success not success

not test not test not test

 2 3 1 2 2

SUCCESS

SUCCESS

SUCCESS SUCCESS SUCCESS SUCCESS SUCCESS

302,2 177,5 219,7 224,8 205,4

208,9 180,7 208,3 208,3 165,6

46 43 42 46 45

SUCCESS

not test

not test

SUCCESS

SUCCESS
7th test 8th test 9th test 10th test Remarks

1 1 1 1

1 1 2 1

1 3 3 3
 12

7 6 6 7

SUCCESS SUCCESS SUCCESS SUCCESS

choose one

1 2 1 2 clear

CLEAR CLEAR CLEAR CLEAR delay

0 0 0 0 robotic
any
TEST 1 TEST 2 TEST 3

MAF & GNETTRACK

TEST 4 TEST 5