LTE

Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE

Benchmarking measurements &

technical analysis

Dominique Dubois, ElectroRent

9/Oct/2013 v1.0

Stefan Engels, Commsquare

Commsquare
Mission & facts
Commsquare helps mobile
operators excel in terms of the
Coverage LTE
quality of service they deliver
to
customers.
Data
3G/their
LTE
Introduction

Data LTE

Network doctors
Voice LTE
Technology excellence
Focus on new technologies
End-to-end view
Independent partner

9/Oct/2013 v1.0

Founded in 2002
HQ in Belgium
Offices in GR & UK
40 engineers &
consultants
30 customers, in BE, NL,
DE, LUX, CH, GR, ROM,
BUL, TR, etc.

Commsquare
Services & Products
Competence areas:
Coverage LTE
RAT system
Data 3G/ LTE
End-to-end
Data LTE SW & IT
Introduction

Voice LTE

Benchmark approach:
In-depth technical
analysis

Services

Products

Hands-on
training

DAP
(data robot)

Advanced
consultancy

DataMon
(Gn/S11 probe)

Benchmarking

Standard reporting
Measurements
9/Oct/2013 v1.0

DAP = active monitoring, test robot for data tests

DataMon = passive monitoring system
Both products are developed by Commsquare
Both products support 2G/3G/LTE

Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE

9/Oct/2013 v1.0

Introduction

Disclaimer
Introduction
Coverage LTE
Data 3G/ LTE

The results in the presentation should not be generalised; the

results are just a snapshot (moment-opname).
Hence, based on these measurements one cant claim operator X
has the best LTE network.

Data LTE

The results presented are a snapshot based on:

Voice LTE

Three networks in roll-out undergoing fast changes

A very limited geographical area
A short measurement duration (2 days)

The main goal of this presentation is to demonstrate LTE

measurement and analysis capabilities.

9/Oct/2013 v1.0

Measurement setup
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE

Drive test in Rotterdam & roads

26 & 27 Sep 2013
SwissQual Diversity
Huawei E392 & Win7 (data)
Samsung S3 (voice)
PcTel scanner

Voice LTE

9/Oct/2013 v1.0

Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE

9/Oct/2013 v1.0

LTE coverage

LTE coverage

Total coverage (800+1800MHz)

Introduction
Coverage LTE

KPN
Data 3G/ LTE

VOD

KPN coverage
extended,
contiguous &
indoor

Data LTE

VOD coverage
gaps: earlystage roll-out

Voice LTE

9/Oct/2013 v1.0

Deep Indoor: RSRP -90 dBm

Indoor: -90 dBm > RSRP -100 dBm
Coverage: -100 dBm > RSRP -116 dBm
No Coverage: RSRP < -116 dBm

LTE coverage

Coverage strategy (KPN)

Introduction
Coverage LTE

KPN

Data 3G/ LTE

800MHz
Data LTE
Voice LTE

9/Oct/2013 v1.0

KPN
1800MHz

KPN focus on
roll-out in
800MHz band
Very limited
1800
coverage

LTE coverage

Coverage strategy (VOD)

Introduction

Both 800 and 1800-band roll-out in Rotterdam

Coverage LTE
Data 3G/ LTE

VOD
Data LTE

800MHz

VOD
1800MHz

Voice LTE

800Mhz band: 71 cells

1800Mhz band: 102 cells

More cells, worse coverage
than 800-band

9/Oct/2013 v1.0

10

Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE

9/Oct/2013 v1.0

2G/3G/LTE data

11

2G/3G/LTE data

Summary KPIs
Introduction

Data Summary

KPN

T-Mobile

Vodafone

Coverage LTE

FTP download (Mbps)

average

20.3

3.9

11.7

Data 3G/ LTE

FTP upload (Mbps)

average

13.8

1.3

7.3

Ping 40 bytes (ms)

median

31

79

49

Web Kepler (s)

median

1.67

5.31

1.85

Kepler success rate

98.63%

97.60%

96.60%

Dominant Technology

LTE

HSPA

LTE

Data LTE
Voice LTE

This is a snapshot for networks in fast change

Limited to Rotterdam & roads during only 2 days
Results are not network-wide relevant

9/Oct/2013 v1.0

12

2G/3G/LTE data

Technology breakdown
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE

KPN: 100% use of LTE

T-Mobile: HSPA (14Mbps) is dominant technology
VOD: 82% LTE usage

9/Oct/2013 v1.0

13

Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE

9/Oct/2013 v1.0

LTE data

14

LTE data

Summary KPIs
Introduction

Data Summary

KPN NL

Vodafone NL

Coverage LTE

FTP download (Mbps)

average

20.3

13.0

Data 3G/ LTE

FTP upload (Mbps)

average

13.8

8.6

Ping 40 bytes (ms)

median

31

46

Web Kepler (s)

median

1.67

1.79

98.63%

98.20%

Data LTE
Voice LTE

Kepler success rate

This is a snapshot for 2 networks in fast change

Limited to Rotterdam & roads during only 2 days
Results are not network-wide relevant

9/Oct/2013 v1.0

15

LTE data

Technology & band usage

KPN

Introduction
Coverage LTE

KPN

Data 3G/ LTE

VOD

LTE800 service
dominant
LTE1800 only in
Delft

Data LTE

VOD

Voice LTE

LTE1800 service
in Rotterdam
although LTE800
coverage better
Still UMTS2100
on roads
Each dot on the map represents an FTP download or upload test

9/Oct/2013 v1.0

16

LTE data

Technology breakdown
Introduction

KPN

Coverage LTE
Data 3G/ LTE
Data LTE

VOD

Voice LTE

Band and Bandwidth

100%

97%
74%

80%

60%
20%

Mainly 10MHz BW
LTE800

VOD: 82% LTE usage

26%

40%

KPN: 100% use of LTE

Mainly 15MHz BW
LTE1800

3%

0%

10MHz 20MHz 10Mhz 15MHz

800
9/Oct/2013 v1.0

1800

KPN NL

800

1800

Vodafone NL

17

LTE data

Download performance
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE

VOD 38% below 10Mbps

KPN only 7% below 10Mbps
9/Oct/2013 v1.0

18

LTE data

Download investigation (VOD)

Introduction

Vodafone NL 7.8 Mbps (avg)

Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE

VOD: typical case download <10Mbps

Ample resource blocks (50-70)
IP throughput flat below 10Mpbs

Backhaul limitation?
Resolving backhaul capacity will improve user experience

9/Oct/2013 v1.0

19

LTE data

Upload performance
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE

VOD 38% below 5Mbps

KPN only 5% below 5Mbps
9/Oct/2013 v1.0

20

LTE data

Web browsing
Median page download time - Kepler
Introduction
Coverage LTE

1.67s

1.79s

Data 3G/ LTE

Data LTE
Voice LTE

KPN
95% in 2s
VOD
80% in 2s
95% in 2.5s
9/Oct/2013 v1.0

VOD: all browsing sessions >2s have download <10Mpbs

21

LTE data

Ping performance
Introduction
Coverage LTE
Data 3G/ LTE

31ms

46ms

Ping tested with

data stick on Win-7
computer
(SwissQual test)

Data LTE
Voice LTE

WARNING: ping is
not the best delay
test method
TCP SYN-ACK is
VOD has wide
distribution

9/Oct/2013 v1.0

22

LTE data

Security
Introduction

KPN NL

Vodafone NL

Authentication

Yes

No

AS Security

Yes

Yes

NAS Security

Yes

No

Data LTE

PDN Address

Public, IPv4

Private, IPv4

Voice LTE

Max bit-rate for downl (Mbps)

156

102

Max bit-rate for upl (Mbps)

78

51

Coverage LTE
Data 3G/ LTE

KPN
More security but public IP address

VOD
Less security but private IP address
9/Oct/2013 v1.0

23

LTE data

Conclusion
Introduction
Coverage LTE

The data results mainly demonstrate 2 networks in a

different stage in their roll-out (in the area tested)

Data 3G/ LTE

Data LTE
Voice LTE

KPN network in a more mature stage (in tested area)

More & contiguous coverage (based on LTE800)
Better throughput & web browsing experience

VOD at early commercial stage (in tested area)

Improvement possible in coverage & parameter tuning
Some limitations observed (backhaul?)
Bandwidth of 15MHz (LTE1800) has higher potential

9/Oct/2013 v1.0

24

LTE data

Benelux benchmark
Comparison with good & more mature network in
Coverage LTEBenelux, also LTE1800
Introduction

Data 3G/ LTE

Data Summary

KPN NL

Vodafone NL

Benelux

Data LTE FTP download (Mbps)

average

20.3

13.0

21.6

Voice LTE FTP upload (Mbps)

average

13.8

8.6

12.5

Ping 40 bytes (ms)

median

31

46

20

Web Kepler (s)

median

1.67

1.76

1.36

98.63%

98.20%

99.50%

Kepler success rate

9/Oct/2013 v1.0

25

Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE

9/Oct/2013 v1.0

LTE voice

26

LTE voice

Test setup
Introduction
Coverage LTE

Limited drive tests in Rotterdam (no roads)

During 1.5 hours on 27/Sep
100% LTE coverage for both KPN & VOD

Data 3G/ LTE

Data LTE
Voice LTE

Test setup
SwissQual Diversity rel 13, Samsung Galaxy S3
Only calls from mobile to fixed line (speaking clock)

Test script
Call setup (dial)
Disconnect (stop) call after 5 sec
Wait 30s before start of next call
9/Oct/2013 v1.0

27

LTE voice

CS-fall-back & KPIs

Introduction
Coverage LTE
Data 3G/ LTE

CS-fall-back implementation
During call setup, mobile is pushed to 3G or 2G network
At call end, mobile returns to LTE

Redirection to 3G maintains data; redirection to 2G stops data

Voice LTE

User
dial
Idle / UE on LTE

CS-Fall-Back

Data LTE

Ringing
tone

Idle / UE on LTE

UE in call

UE on 2G or 3G

Call setup time

9/Oct/2013 v1.0

End of
call

Return to LTE
28

LTE voice

Summary
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE

CS-FB Summary

KPN NL

Vodafone NL

Call setup time (s)

average

5.63

4.28

CSFB setup success

average

100%

100%

Time to return to LTE (s)

average

7.7

0.8

Voice LTE

9/Oct/2013 v1.0

29

LTE voice

Call setup success rate

Introduction

Excellent call setup

Coverage LTE

Both networks achieve

Data 3G/ LTE
100% success for roughly
150 tests
Data LTE
Voice LTE

Most calls set-up on LTE

97% (KPN) and 95%
(VOD) respectively

9/Oct/2013 v1.0

30

LTE voice

CS-FB strategy
Introduction
Coverage LTE
Data 3G/ LTE

Different strategy
KPN: fall-back to 3G (UTRAN)
VOD: fall-back to 2G (GERAN)
And 1 call to 3G (UTRAN)

Data LTE
Voice LTE

KPN

VOD
VOD

9/Oct/2013 v1.0

31

LTE voice

Call setup time

Introduction
Coverage LTE
Data 3G/ LTE

KPN
KPN
5.68s

VOD average setup time

4.28s
KPN average setup time
5.68s

Data LTE

Different strategy

Voice LTE

VOD
VOD
4.28s

9/Oct/2013 v1.0

KPN redirects to 1st

carrier on 3G
VOD redirects to
explicit list of ARFCNs
on 2G (=a few cells)

32

LTE voice

Time to return to LTE

KPN: typical performance

Introduction
Coverage LTE

KPN

Data 3G/ LTE

Majority returns in 7s
90% returns within 10s
Technical: KPN mobile returns to
idle mode in UTRAN; UE then
returns to LTE

Data LTE
Voice LTE

VOD: very fast return

VOD

Fastest in Benelux
99% returns in 1s
Technical: VOD includes redirection
info to LTE in the GERAN release
message

9/Oct/2013 v1.0

If KPN implements same feature,

then it can possibly achieve same
performance

33

LTE voice

Conclusions
Introduction
Coverage LTE
Data 3G/ LTE
Data LTE

Call setup on LTE is excellent on both networks

KPN has slower setup & return times
Fall-back to 3G network
Data sessions can be maintained during voice call
Faster setup & return possible when introducing features

Voice LTE

VOD has fastest call setup time & return to LTE

Fall-back to 2G
Fast thanks to features on 2G
At the expense of breaking data sessions during voice call

9/Oct/2013 v1.0

34

LTE voice

Benelux benchmark
Comparison with more mature network in Benelux,
Coverage LTEalso LTE1800
Introduction

Data 3G/ LTE

Data LTE

CS-FB Summary

KPN NL

Vodafone NL

Benelux

Voice LTE

Call setup time (s)

average

5.63

4.28

6-8

CSFB setup success

average

100%

100%

~98%

Time to return to LTE (s)

average

7.7

0.8

8-10

9/Oct/2013 v1.0

35

Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE

9/Oct/2013 v1.0

Conclusion

36

Overall conclusion
Introduction
Coverage LTE

We have
Demonstrated typical LTE benchmarking measurements
Scratched the surface of technical analyses

Data 3G/ LTE

Data LTE
Voice LTE

REMINDER: results are a snapshot and shouldnt be

generalised
LTE networks in Rotterdam and surroundings are in fast
roll-out & change:
Data performance is close to international benchmarks for KPN
but not for VOD
Voice performance is very good for both, and in particular good
for VOD

9/Oct/2013 v1.0

37

Introduction
Coverage LTE
Data 3G/ LTE
Data LTE
Voice LTE

9/Oct/2013 v1.0

Contact details
Dominique Dubois

Stefan Engels (presenter)

ddubois@electrorent.com
www.electrorent-europe.com

stefan@commsquare.com
www.commsquare.com

38