Node Controller

Traffic overview of the master

module

 GGSN-C Node Controller/master module

Load balancing - distributes traffic over GGSN-C, GGSN-U/I

and GGSN-T
Supervises the GGSN PICs and takes action if there is a
failure.
Admission control
Imsi/Msisdn tree
Local IP pool
MS Route handling
O&M handling
SBR
Charging
QoS implementation
etc

?
GTPCD

Application
MASTER

SACC

PDP

CHARGING

LI

TDM

NCT

LIH

SAC

GTP

DHCP

SBCC

ANP Mgr

FSAD
RADIUS

Services

PRSI

PPSI

DBP

DAPP

Data

DATA

GTP Basic
OAM

GCCP

RE Proxy

Basic

GUCP

IPC

Core

Framework
APP

SrvLib

junos_lib
gtpcd libraries

RPD

XMLParse
XMLTok

SSA/
SNMP

librpd

libirsd

JunOS libraries

pwc

Support Layer

Components in Node Controller

Node controller thread view
Commo
n data

Legacy
Coreloop
thread
Get Write
lock

Get Read lock

GCCP SC, U/T PIC etc.

GTP
v0/v1
threads

Get Read lock

GNSP
threads

Admission
control
threads.

Producer
Consumer
Queues
M20/120 M120 only

SGSN

GTPCD_MULTICORE set a compilation time

Core loop

Gccp
Configuration
Statistics
U/T/SC handling
Etc

SC

1 thread on each CPU core on M120

Threads and locks

GTP
v97

v99

MtIpc
Send

From SGSN

MT_IPC

UDP

To/from
SC

Adm. Ctrl.

TCP
callback

MtIpc
Send

Core

GTP

Adm Ctrl

MT_IPC

Traffic flow in PDP create

SGSN

FSAD

GTP Create PDP

Request

GTP-C/NC

GTP-C/SC

GTP-U/I

GCCP_FWD_PDP_
CREATE_IND
GCCP_ADMCTL_
ACQUIRE_REQ
rpd_rt_add (ifl,rtid,MS slice)
rpd_cb_rt_add ()
GCCP_ADMCTL_
ACQUIRE_RSP
GUCP_SESSION_
OPEN_REQ
GUCP_SESSION_
OPEN_RSP

GTP Create PDP Response

RPD

SGSN calling
GTP-C/NC

GTP Create PDP

Request

master_gtpthread.c:

Running on thread
GTP v1 or v0

NcGtpThread_Run(..) 1.
2.
3.

ProcessCreatePdpRequest(..)

Reads GTP packet from socket

Basic packet sanity check
Determines if create or echo request
or drop the packet

1.
2.
3.
4.
5.

SendFwdPdpCreateInd(..)

1.
2.

Decodes apn, imsi, msisdn, nsapi and

pdp_address
Adds imsi in table (only shadow entry at the
moment) and msisdn
Check for max pdp per apn (avoid Radius
overflow)
Gets a C-Pic to run on (least loaded)
Gets a charging id

Build a GNSP message

Send to designated C-Pic

Admission acquire
GTP-C/NC
GCCP_ADMCTL_
ACQUIRE_REQ

adm_ctrl.c:
AdmCtrl_ReqCb(..)

3 threads handling IPC, packet FWD

on M120

AdmCtrl_ScheduleWorker(..)
Running on 2 threads
on M120

AdmCtrl_Req(..)

1.
2.
3.
4.
5.
6.
7.
8.

Update load from slave (used

previously)
Use of T-pic check -> get one.
Check max contexts on node
Check max context per APN
Handle QoS
Check IP address
Get a U-pic
Sends back to calling C-pic.

QoS i.e.
5

octet 1

Length of quality of service IE

Octet 2

Delay
class

Peak
throughput

0
spa
re

0
spare

Quality of service IEI

spare

Reliability
class

octet 3

Precedence
class

octet 4

Mean
throughput

Traffic ClassSGSNDSCP

Delivery order

octet 5

Delivery of erroneous SDU

Maximum SDU size

Octet 7

Maximum bit rate for uplink SGSN

Octet 8

Maximum bit rate for downlink SGSN

Octet 9

Residual BER

Guaranteed bit rate for uplink

SGSN

Guaranteed bit rate for downlink

0
spare

DSCP

Octet 10

Traffic Handling priority

Octet 11

SDU error ratio

Transfer delay SGSN DSCP

Octet 6

Signal-ling
Indicat-ion

SGSN

Source Statistics Descriptor

Octet 12
Octet 13

Oct
et
14

Maximum bit rate for downlink (extended)

Octet 15

Guaranteed bit rate for downlink (extended)

Octet 16

Maximum bit rate for uplink (extended)

Octet 17

Guaranteed bit rate for uplink (extended)

Octet 18

QoS DSCP (Diff Serv Code Point)

Used on both uplink and downlink IP packets
Dependent upon neqotiated QoS:
Traffic class
Transfer delay
SDU error ratio

and the APN configuration. DSCP bits are set

in the Gi and Gn IP headers.

DSCP cont.
DiffServ is a coarse-grained, class-based
mechanism for traffic management.
DiffServ operates on the principle of traffic
classification, where each data packet is
placed into a limited number of traffic classes.
DiffServ relies on a mechanism to classify and
mark packets as belonging to a specific class.

DSCP uplink packet marking

Admission release
GTP-C/NC
GCCP_ADMCTL_
RELEASE_REQ

adm_ctrl.c:
AdmCtrl_ReqCb(..)

3 threads handling IPC, packet FWD

on M120

AdmCtrl_ScheduleWorker(..)
Running on 2 threads
on M120

AdmCtrl_Rel(..)

1.
2.
3.
4.
5.
6.

Update load from slave

Update load on DSCP.
Decrease the number of PDP in slice.
Decreate PDP counter on U-pic
statistic , APN and total GGSN
Release PDP address
Sends release resp back to calling Cpic.

Local pool
Support for fast ip address allocation from
subnets for the apn
Clean up ip addresses for a specific slave
No reuse of ip addresses before for a
configured time
At U pic limit response, flag not reuse the
address

Soak timer
released

used

Soak timer
Adm Create request

Soak timer

Adm release request

Used
Released
Soaked

0 1 1 1 0
1 -> 1 -> 0 -> 1 -> 1
1 1 1 0 1

Config options, functions used

Soak timer

Available

Three bitfields used to secure this (per IP address)

soaked

Master_Wa.Config.Pdp.bIpReuseTimeout no-address-reuse-quarantine
address-reuse-timeout 10 - 1800s.
Master_Wa.Config.Pdp.noReleaseAddrOnFullGtpu pdp1 no-address-release-on-exhaustedgtpu

pReleaseInfo->wNoSoakOnPdpIp
pReleaseInfo->noResourcesOnGtpu

Local pool structure

MasterAddrPool_t
IdSource_t

Hash table of APNs

Address pool workarea

List of address ranges

Released batch bitset list

List of GTP-SCs

subSource_t

Hash table of APNs

IdSubSource_IdSource_t

Available slice
tree
Full slice tree

Slice tree

Batches of 128 bits

IdSubSource_IdRange_t

Batches of 128 bits

IdRange_t
Bit 1: used Id
Bit 2: Release Id
Bit 3: Soaking Id

IdSubSource_idSlice_t

idSlice_t

Batch 128 bits (addresses)

Slice 1024 addresses

IdSoakBatch_t
IdSubSource_IdBatch_t -> available Id

Calculate address and id

Id (in relation to slice) = ((slice no) *
(SliceSize)) + (BatchNum * BatchSize) +
BatchId
Address = Subnet.StartAddr + Id (ipv4)
To find batch =>
bitPos = Id ((slice no) * (SliceSize))
BatchNum = bitPos / BatchSize
BatchId = bitPos BatchNum * BatchSize

Release / Failure scenarios

Adm Ctrl Release
adt_toggleAddr (setOff = TRUE, noSoak?, NoCb = TRUE)

Adm Ctrl Request failure (U limit, other error)

adt_toggleAddr (setOff = TRUE, noSoak?, NoCb = TRUE)

Session controller failure

toggleId called directly (setOff = TRUE, noSoak = FALSE,
NoCb = FALSE)

Node Controller failure session controller takeover

adt_toggleAddr (setOff = FALSE, noSoak = FALSE, NoCb =
TRUE)

Important implementation details

Find and create an address - Master_adtAllocAddr( U32 ApnIndex, Slave_Info_t *pSlave,

InAddr2_t *pAddr, U8* reqSubnetName )

Remove an address - Master_adtFreeAddr -> adt_toggleAddr -> ap_toggleId

The actual toggle of the id for the address - ap_toggleId(Apnmgt_ApnEntry_t *pApn, U8

slaveIndex, U32 id, U8 masterRangeIndex, U8 setOff, U8 noSoak, bool bNoCb)

Background task every soak timer call, next soak timer start after the previous is done ap_BTSoakTimerAction -> ap_loopSoakBatch( IdSoakBatch_t * pIdSB, idSlice_t * pSlice)

MAP_MASTER_INDEX_TO_APNMGT_INDEX(Master_index, pApn)
Master_Wa.pApnTable[pApn->ApnIndex]->Master_to_Apnmgt_Mapping[Master_index]
Master_Wa.pApnTable[pApn->ApnIndex]->Range[index]

adt_convertIdToAddr

Mutex_Lock(Master_Wa.pIdPoolMtx);

GTP-U load balancing and slice

handing

Steps

IP address selected
Find range from apn
Find slice in slice tree from apn range - RwMutex_LockRead(pRangeInfo-

>pRange->SliceTreeMtx);
If slice is not found

Create a new slice

Find least loaded Upic - Master_GetLeastLoadedGtpu
admCtrl_AssignSliceToGtpu
Assign slice to Upic
Add Upic index to slice
Add slice to SliceTree on range
Add route for slice - Master_AddSliceRoute -> Rpd_AddRoute

If existing just find the U pic

Check that U still is active and okay

Respond back to session controller

Conclusions and
Questions