Beruflich Dokumente
Kultur Dokumente
Objective
Develop QoS framework and requirements to support Service Oriented Network in rural environment
Network Characteristics Ad hoc type networks scattered groups of users Lossy links Fluctuating traffic demand Low bandwidth Network heterogeneity (e.g., LTE and WiFi) User mobility pattern (e.g., cell mobility, subnet mobility) Application characteristics Current Application Cellular voice, File transfer (e.g., non-real-time), SMS Future NGN Application Real-time streaming (e.g., IPTV) Interactive (e.g., VoIP) Delay sensitive Distance learning, conferencing, Mobile banking (Priority) Services need QoS reservation
GISFI
Slide 2
QoE Metrics
QoE => Perceived Quality of Service Network Metrics
Bit rate, delay, jitter, packet loss rate Power consumption
Application Metrics
Call setup delay Failed calls, dropped calls, retransmission MOS (Mean Opinion Score)
Slide 3
P-CSCF
GETSApplication Server
SIP DNS
I-CSCF
Monitoring Agent
Network/ Application Feedback
PCRF
IMS
PCRF
Layer 2 control
(EPC)
GW
GW
GETS Call
RAN
Caller (UE1)
IP CAN
RAN
RAN RAN
IP CAN
Called (UE2)
Operator control, non-operator control Availability, Retainability, Accessibility, Integrity, Mobility IRAT success rate Global Metrics
GISFI
Slide 5
3G KPI/KQI benchmarking
Call setup time Call setup success rate Call completion success rate MOS Retainability Accessibility Latency Jitter Throughput
GISFI
Slide 6
3G Services
FTP, Context, Performance benchmarking
GISFI
Slide 7
QoE Requirements
QoE is driven by measurements Which layer, which application, how often Require Multi-timescale Measurements Require Multi-Layer Measurements Network layer, application layer Require Session-based Measurements E.g., SIP-based multimedia application Require DPI for Application Classification Classification and Marking Policing and Markdown Queuing and Dropping
GISFI
Slide 8
QoE Requirements
Require mechanisms to provide priority services for signaling and transport of voice services in case of emergency.
e.g., SIP signaling for VoIP use Resource Priority Header (RPH) e.g., Diffserv mechanisms to take care of associated media (e.g., RTP) traffic.
Slide 9
QoE Requirements
Future NGN application requires cross-layer feedback in order to provide desired QoS during handover and due to bandwidth fluctuation IEEE 802.21 Interoperability with Mobility protocols SIP, Mobile IP, Mobile IPv6 Tightly coupled or loosely coupled Require correlation between signaling and media
(e.g., EPC and IMS)
Require Open API to dynamically control the network element in the network
GISFI
Slide 10
KPIs
callDistribution networkVolume Abnormally Terminated Calls
Remarks
Total of all VoIP calls per given time period Total of all data traffic on the network Total of Abnormally Terminated Calls per time period
Initiated vs. Rejected Total of Initiated vs. Rejected calls per time period Calls Codec Distribution Total Duration Time List of all codecs utilized by VoIP calls Total aggregated Call Duration Time of VoIP Calls per given time period The number of simultaneous calls on the network Total aggregated Setup Time per given time period Error Distribution for VoIP calls Top N VoIP protocols on the network, i.e. SIP, H.323, MGCP/NCS, MEGACO, SCCP, T.38, etc.
KPIs
mediaStream faxStream bounceFlow packetTree callMos
Remarks
List of media streams utilized by a VoIP session List of media streams utilized by a Fax(T.38) session Message Flow(bounce diagram contents) for a given VoIP session List of all packets in the context of a VoIP session Average MOS score for a given VoIP session
Performance
callJitter
Performance
Performance
KPI NAMES List of all SIP methods Aggregate IMS signaling packets
DESCRIPTION SIP message flows with same Call-ID Sums all the methods and responses.
LOGICAL FORMULA Shows the list all SIP messages. Ratio of total count and all IMS messages Sum of all SIP Registers from all UE to PCSCF
WHY INTERFAC INTERESTING? E Basic functionality for session view. Dos Attack Gm
Gm
Aggregate total Total number of attempted initial registrations registration b/w UEs and PCSCF Aggregate successful initial registration b/w UE & CSCF Count the successful registrations of UEs.
Gm
Informative
Gm
Aggregate failed initial authenticatio n b/w UE & CSCF Aggregate failed initial registration b/w UE & CSCF Mean IMS initial registration set-up time Aggregate unauthenticat ed initial registration b/w UE & PCSCF
Count the number of failed authenticatio n of UEs. Count the number of failed registrations of UEs. Time taken between register and 200 OK Identify the unauthenticat ed registrations
Count SIP 401 errors from PCSCF in response to Register from PCSCF. Count SIP 4XX (except 401) errors from PCSCF in response to Register. Total time of getting an response after the registration request is sent. Count Registers without nonces that get 200(2XX) responses without 401(4XX) challenges.
Gm
Informative
Gm
System Gm Performanc e indication Security Gm risk, allows impersonati on attack, VZW req.
Aggregate total attempted reregistration between UE & P-CSCF Aggregate successful reregistration between UE & P-CSCF Aggregate Failed reregistration between UE & P-CSCF Mean IMS reregistration setup time.
Gm
Informative
Gm
SIP 4xx errors in response to registration request. Time to set-up the registration
Gm
Gm
Aggregate Failed re-authentication between UE & PCSCF Aggregate unauthenticated re-registration between UE & PCSCF Aggregate successful deregistration between UE & PCSCF Aggregate Failed de-registration between UE &PCSCF
Count SIP 401 errors in response to Register Count re-register msgs without nonces that get 200 without 401 challenges Count Register with expiry==0 or contact: expiry==0
Total unauthenticated REGs Security risk, allows impersonation attack System load indication
Gm
Gm
Successful deregistrations.
Gm
Informative
Gm
Aggregate total attempted deregistration b/w UE & PCSCF Mean IMS deregistration time Aggregate failed authentication for de-registration Aggregate unauthenticated de-registration Attempted session establishments
Total of attempted deregistration Mean of deregistration time Total number of failed authentication de-registrations Total number of unauthenticated de-registration How many attempts have been made to establish the session.
SIP
Gm
Gm
Count SIP 401 errors in response to register Count de register msgs with 200 and without 401 challenges. #initial Invite sessions with #INV==1
Gm
Gm
Tracking all successful sessions we have. Tracking all successful answered sessions we have. All the failed session establishments. Time taken to have a success in establishment. Number of attempts to have re-established session
Gm
Count #2XX System load ringing, should be indication 200 but 2XX is fine when #INV==1 Count 4XX messages Time b/w INVITE and 180 response. #INVITE where a 180 has already been seen, or #INV > 1 System performance indication System performance indication System tuning aid
Gm
Failed session establishments Mean time for successful session establishment Attempted session re-establishment.
Gm
Gm
Gm
Aggregate session deestablishments Number of attempted subscription procedures Number of successful subscription procedures Number of failed subscription procedures Number of unauthenticated subscribers
Total deestablishments.
Count BYEs.
Security risk, allows impersonation attack, VZW req. System load indication, VZW Req. System performance indication System performance indication
Gm
Number of attempted procedures requests. Number of successful procedures for subscription. Number of failed subscription procedures Total number of unauthenticated subscribers
Gm
Gm
Gm
Number of emitted notify procedures Number of successful notify procedures Number of failed notify procedures Number of unauthenticated Notifys Number of spoofed notify procedures
Total count of SIP notify messages. Total count of SIP notify messages with OK response. Total count of SIP notify messages. Total count of SIP notify messages. Total count of Spoofed notify messages.
Gm
Gm
PCSCF receives a SIP 4XX Number of Notify without a 401 Challenge Count notifys from src!=PCSCF
Gm
Security risk, allows Gm impersonation attack, VZW req. System load indication, VZW req. Gm
Attempted immediate Messaging procedures Successful immediate Messaging procedures Failed immediate Messaging procedures Number of unauthenticated messages NAIs of top 5 originators of IMS signaling traffic
Procedures attempted to have immediate messaging. Successful procedures attempted to have immediate messaging. Failed procedures attempted to have immediate messaging. Number of all unauthenticated messages . Display associated packets, bytes, exclude CSCF as originator.
PCSCF System load receives a indication SIP message PCSCF sends a SIP 200 OK message System load indication
Gm
Gm
PCSCF System load sends a SIP indication 4XX message in response Messages without 401 challenges From: and #bytes Security risk , allows impersonation attack System load indication, Dos attack
Gm
Gm
Gm
NAIs of top 5 receivers of IMS signaling traffic NAIs of top 5 senders of Instant messages NAIs of top 5 receivers of 4XX errors Top 5 visited n/w IDs in IMS signaling traffic Top 5 access n/w Infos in IMS signaling traffic
Display associated packets, bytes, exclude CSCF as receiver. Display associated packets, bytes, exclude CSCF as originator. Display associated packets, bytes, exclude CSCF as receiver. Display associated packets and bytes Display associated packets and bytes
Gm
#bytes
Gm
#bytes
Gm
#bytes
Gm
#bytes
Gm
Top 5 sender domains in IMS signaling traffic Receiver domains in IMS signaling traffic NAI of top 5 originators of register request
Display associated packets and bytes Display associated packets and bytes Display associated packets ,bytes exclude CSCF as originator(PCSCF callee) Display associated packets ,bytes exclude CSCF as originator Display associated packets ,bytes exclude CSCF as originator(DUP)
#bytes
Gm
#bytes
Gm
#bytes
Gm
NAI of top 5 originators of INVITE request NAI of top 5 originators of INSTANT message request
#bytes
Gm
#bytes
Gm
NAI of top 5 originators of subscribe requests IP Addresses of top 5 originators of register requests IP Addresses of top 5 originators of INVITE requests IP Addresses of top 5 originators of INSTANT message requests IP Addresses of top 5 originators of subscribe requests Top methods & response types sent. Drill down to send UE.
Display associated packets ,bytes exclude CSCF as originator Display associated packets, bytes exclude PCSCF as originator. Display associated packets, bytes exclude PCSCF as originator. Display associated packets, bytes exclude PCSCF as originator. Display associated packets, bytes exclude PCSCF as originator. Calculate the number of UEs
#bytes
System load indication System load indication System load indication System load indication System load indication Informative
Gm
#bytes
Gm
#bytes
Gm
#bytes
Gm
#bytes
Gm
Gm
KPI Names
Description
Logical Formulae
KPI object
Why interesting?
Interfa ce
Provides the number of attempted EPS attach procedure within the MME. Number of successfully performed EPS attach within MME.
Valid for PS
Receipt of ATTACH REQUEST message with ATTACH TYPE information element indicating EPS attach from the MS.
Valid for PS
Transmission of a "ATTACH ACCEPT" message to the MS but retransmission doesnt increase the counter. Transmission by the SGSN of the ATTACH REJECT message to the MS, in response to a "ATTACH REQUEST"
Number of failed EPS attach procedures . It is split into sub counters per the reject cause.
Valid for PS
KPI Names
Description
Logical Formulae
KPI object
Interf ace
Number of attempted EPS detach procedures initiated by UE within the MME. Number of successful EPS detach procedures initiated by UE within the MME. Number of attempted detach procedures by MME.
Receipt of DETACH Valid for REQUEST PS message with detach type information element from UE Transmission of DETACH ACCEPT from MME
Valid for Gives the total PS number successfully of attempted detaches. Valid for All the attempts PS of detach to calculate success
KPI Names
Description
Logical Formulae
KPI object
Why interesting?
Interf ace
Receipt of DETACH Valid for Calculate ACCEPT message PS success rate by UE from the MME.
Receipt of CANCEL LOCATION message by MME from the HSS with cancel type information element indicating delete user. Receipt of CANCEL LOCATION ACK message from MME by HSS.
Valid for All the attempts PS of detach by HSS and uses RADIUS.
KPI Names
Description
Logical Formulae
KPI object
Why interesting?
Interf ace
Number of attempted tracking area update procedure with SGW change initiated with MME. Successful number of attempted tracking area update procedure with SGW change initiated with MME.
Receipt of TRACKING Valid for Calculate AREA AREA UPDATE PS UPDATE REQUEST message attempts. from the MS, with last registered TAI information element indicating to the MME that it wishes to be served by new SGW. Transmission of TRACKING AREA UPDATE ACCEPT message from the MS, with last registered TAI information element indicating to the MME that it wishes to be served by new SGW Valid for Success rate of PS Area updates.
KPI Names
Description
Logical Formulae
Interf ace
Number of failed tracking area update procedure with SGW change initiated with MME.
Transmission of TRACKING AREA UPDATE REJECT message from the MS, with last registered TAI information element indicating to the MME that it wishes to be served by new SGW. Causes are also defined in the message.
Number of attempted tracking area update procedure without SGW change initiated with MME.
Valid for Receipt of TRACKING AREA UPDATE REQUEST PS message from the MS, with last registered TAI information element indicating to the MME that it wishes to be served by same SGW.
KPI Names
Description
Logical Formulae
KPI object
Interf ace
Successful number of attempted tracking area update procedure without SGW change initiated with MME. Number of failed tracking area update procedure without SGW change initiated with MME.
Transmission of TRACKING Valid for AREA UPDATE ACCEPT PS message from the MS, with last registered TAI information element indicating to the MME that it wishes to be served by same SGW
Transmission of TRACKING Valid for AREA UPDATE REJECT PS message from the MS, with last registered TAI information element indicating to the MME that it wishes to be served by same SGW. Causes are also defined in the message.
Description
Logical Formulae
KPI object
Interf ace
Incremented when an EPS paging procedure is started i.e. at the transmission of first Paging message from MME to the eNodeB, which are counted when paging area is smaller than or equal to one TA. Receipt of SERVICE REQUEST message with Service Type = Paging Response from the UE.
Failed EPS Number of paging failed PS procedures paging procedures initiated at the MME.
Valid for Increments when an EPS PS paging procedure times out, PS which are counted when paging area is smaller than or equal to one TA.
KPI Names
Description
Logical Formulae
KPI object
Why interesting?
Interfa ce
Sampling at a pre-defined interval, sum of dedicated EPS bearer established by MME in active mode and then taking the arithmetic mean. Sampling at a pre-defined interval, sum of dedicated EPS bearer established by MME in active mode and then taking the maximum.
Valid for PS
The total active mode bearers would give the load. The total active mode bearers would give the maximum load.
Valid for PS
KPI Names
Description
Logical Formulae
Why interestin g? This would let us calculate the mean of set-up time.
Interfa ce
Obtained by accumulating the time intervals for every successful dedicated bearer setup by MME B/W ACTIVATE DEDICATED EPS BEARER CONTEXT REQUEST and corresponding ACTIVATE DEDICATED EPS BEARER CONTEXT ACCEPT message receipt by the MME. This time when divide by the total number of successful dedicated bearer gives the mean. Transmission of ACTIVATE DEDICATED EPS BEARER CONTEXT REQUEST message by MME.
Valid FOR PS
KPI Names
Description
Logical Formulae
KPI object
Interfa ce
Valid FOR PS
Failed dedicated bearer activation procedure by MME Attempted dedicated bearer de activation procedure by MME
Transmission of ACTIVATE DEDICATED EPS BEARER CONTEXT REJECT message by MME from UE with ESM cause indicating the cause of failure.. Transmission of DEACTIVATE EPS BEARER CONTEXT REQUEST message by MME.
Valid FOR PS
Failure detection.
Valid FOR PS
KPI Names
Description
Logical Formulae
Interf ace
Successful dedicated bearer de activation procedure by MME Attempted EPS bearer modification procedure by MME Successful EPS bearer modification procedure by MME
Valid FOR PS
Valid FOR PS
KPI Names
Description
Logical Formulae
Interf ace
Receipt of MODIFY EPS BEARER CONTEXT REJECT message by MME from UE. With ESM Cause. Total Number of PDNGW created.
Attempted number of PDNGW initiated dedicated bearer creation Successful number of PDNGW initiated dedicated bearer creation
Number of attempted PDN-GW containing multiple bearer IDs, each cumulated to the counter. Number of successful PDN-GW containing multiple bearer IDs, each cumulated to the counter.
Valid for PS
Valid for PS
KPI Names
Description
Logical Formulae
Inte rfac e
Failed PDN-GW containing multiple bearer IDs, each cumulated to the counter. Number of attempted PDNGW initiated Dedicated Bearer Deletion. Number of successfully PDNGW initiated Dedicated bearer deletion.
Transmission of Delete Bearer REQUEST message may contain multiple bearer ID, each bearer shall be cumulated to the counter. Receipt of Delete Bearer Response message by PDNGW where cause and receives a n ACCEPTNCE RESPONSE
Valid for PS
Valid for PS
KPI Names
Description
Logical Formulae
KPI objec t
Why interestin g?
Inte rfac e
Failed number of PDNGW initiated dedicated bearer deletion Attempted number of PDNGW initiated modification with QoS update. Successful number of PDNGW Dedicated bearer modification with QoS update
Failed PDN-GW initiated dedicated bearer deletion. Split into sub counters per reject cause. Number of attempted PDNGW initiated Dedicated Bearer modification with QoS update Successfully PDN-GW initiated Dedicated bearer modification with QoS update.
Transmission of Update Bearer Request message from PDNGW with Bearer Level QoS containing all bearer IDs. Response to the request sent.
Valid Attempt s of for PS number of QoS updation . Valid Success for rate PS calculati on.
KPI Names
Description
Logical Formulae
KPI objec t
Why interestin g?
Inte rfac e
Failed PDNGW initiated dedicated bearer with QoS update Attempted number of PDN-GW initiated Dedicated Bearer Modification without QoS update.
Failed PDN-GW initiated dedicated bearer modification with QoS. Split into sub counters per reject cause. Number of attempted PDN-GW initiated Dedicated Bearer Modification without QoS update.
Receipt of Update Bearer response MESSAGE where cause is indicated. Transmission of "Update Bearer REQUEST" message From PDN-GW without Bearer Level QoS IE, this message may contains multiple Bearer IDs
KPI Names
Description
Logical Formulae
Inte rfac e
Successful number of PDN-GW initiated Dedicated Bearer Modification without QoS update.
Number of successfully performed PDN-GW initiated Dedicated Bearer Modification without QoS update.
Failed number of PDN-GW initiated Dedicated Bearer Modification without QoS update
Number of failed PDN-GW initiated Dedicated Bearer Modification without QoS update. The measurement is split into sub counters per the reject cause
Receipt of Update Bearer Response message by PDN-GW without Bearer Level QoS IE in the Update Bearer Request message which contains the same EPS Bearer ID and where Cause IE identifies a failed bearer handling with Rejection Response
Valid for PS
EF
BE NetDetector
controller
match DSCP 46 assured minimum bandwidth - Best effort match DSCP 0 set maximum bandwidth usage
Commands can be read in the form of CLI and are encapsulated in XML format when issuing edit-config operation of NETCONF
- /ngen/srvc/dashboardData?recorder=<recorder>&iface=<iface>&startTime=2%20min&endTime=now&layer=Application&dataType=time,bitRate&customDT=dataField s&window=10 - Trigger router configuration when cross traffic exceeds upper threshold
After marking
Packet Classification
Before marking
After marking
50
Deliver streaming media and generate cross traffic VLC media player works as both streaming media server and client
Server: RTP unicast is delivered to destination host address using port 1234 (MPEG2) Client: receive the RTP stream and save it as MPEG2 file in local host
MOS
No cross traffic
Packet loss
time
Future work
Integrating the controlling program into NetDetector Reducing the response time
Depends on polling interval and time spent in configuring the router Using interruption/alarms rather than polling can further reduce response time
QoE Metrics
Slide 57
What To Measure?
Which Layer? Which Application? Which User? Hop-by-hop or End-to-end? Etc?
GISFI
Slide 58
QoE Metrics
Slide 59
QoE Metrics
Slide 60
GISFI
Slide 61
QoE Metrics
Slide 62
IP Layer
UDP Layer
GISFI
Slide 63
Application SSH
GISFI
Slide 64
User 2
GISFI
Slide 65
Facebook Only
GISFI
Slide 66
QoE Metrics
Slide 67
QoE Metrics
Require DPI for Application Classification Classification and Marking Policing and Markdown Queuing and Dropping
GISFI
Slide 68
DSCP classification and marking at the edge, and class-based priority queuing and assured bandwidth scheduling at the edge and core routers EF Treatment: For Emergency SIP signaling and SIP Register transport (Strict-priority forwarding for bounded bandwidth) AF Treatment: For Emergency VoIP and Video/IP RTP media (Weighted shared bandwidth scheduling of remaining bandwidth among AF classes) BE Treatment: Best-effort traffic forwarding only when bandwidth not in use by higher priority traffic; Drop otherwise
69
70
Non Priority VoIP 10 Mb/s cross traffic (packet size 1440 bytes)
Non Priority VoIP 10 Mb/s cross traffic (packet size 400 bytes)
71
Measurements - IMS
Top N Reports Clients based on SIP Registrations/Re-Registrations/De-Registrations Number of INVITE/NOTIFY/ SUBSCRIBE messages Failed/Successful authentications at P-CSCF/S-CSCF/I-CSCF depending on
Authentication type (SIP Digest, IMS AKA) Message type (INVITE, NOTIFY, BYE)
Malformed messages (SIP, RTP, H.248) identified using protocol grammar, rules Unauthenticated/Un-authorized messages Spoofed messages Session minutes (service usage)
Slide 72
Number of authentication challenges/failure messages Malformed signaling/media messages Key exchanges during IMS AKA authentication
Ability to drill down into data sessions - e.g., for given P-CSCF, find all the messages from client X between time T1 to T2.
Slide 73
Measurements - LTE
Insight into signaling and user data volumes Data volume distribution by
Top SGW Top MME Top eNodeB Top IMSI
Minimize the interruption in QoS during handover Interoperability with Mobility protocols
GISFI
Slide 75
QoE Metrics
GISFI
Slide 76
Slide 77
Next Steps
Baseline Document (June ) Define a set of requirements that is application specific and applicable to Indian rural environment Define a set of methodologies to define end-toend QoS parameters Multi-layer QOS Network, Application Layer Multi-time-scale measurements Granularity of measurement Define a set of APIs to control the network based on measurement Slide 78
GISFI
80