Construction of Provider-Independent Overlay Networks with High Resilience

Xian Zhang
Queen Mary University of London, Mile End Road, London, United Kingdom xian.zhang@elec.qmul.ac.uk

Chris Phillips
Queen Mary University of London Mile End Road, London, United Kingdom chris.phillips@elec.qmul.ac.uk proposed method performs the best and can achieve the comparatively good performance in various failure models with limited overhead. Although the proposed algorithm is verified on AS-level physical networks, the definition and verification can be easily extended to router-level networks. This paper is organized as follows: Section II describes related work; Section III presents the proposed algorithm in detail; Section IV provides the simulation results as well as analysis; and Section VI includes conclusions and indicates areas for future consideration. II. RELATED WORK

AbstractIt is difficult to change the Internet infrastructure in support of new services because of its distributed and autonomous features. In this context overlay networks provide a promising means of offering a supplement to the existing best efforts delivery mechanism. This paper discusses one of the key issues: topology construction in the context of providerindependent overlay networks. A heuristic method is proposed to build a physical-aware overlay topology which can provide reasonable resilience while incurring little overhead. Simulation results show that this scheme performs the best among the three comparable methods. Keywords- Network Provider Independence; Overlay Network; Resilience

I.

INTRODUCTION

Overlay networks have attracted much attention in the context of Internet evolution [1]. Overlay networks are usually mapped on top of a physical network (e.g. the multi-AS Internet or a single Internet Service Provider (ISP) network). The nodes in overlay networks connect with each other through virtual links, which are in turn composed of possibly multi-hop physical paths. The health of these virtual links is usually monitored by periodically sending probes. Thus, the monitoring overhead increases proportionally with the incremental deployment of virtual links (i.e. the average connectivity of overlay nodes). The design of overlay network topology has been shown to have a direct impact on the performance of overlay networks in previous works [4, 11 and 12]. Therefore, it is critical to construct an efficient overlay topology (i.e. the connection relationship between overlay nodes and the placement of overlay nodes) whilst taking into account the characteristics of the physical network it is deployed upon [13]. Most of the existing work on overlay topology construction focuses on network provider dependent overlays [5 12, 13]. To be more specific, there is no constraint on the placement of overlay nodes. Generally, routers have greater utility by providing alternative paths with better performance such as lower latency for customers, as shown in [5, 14]. However, it requires the necessity of information sharing among different ISPs administering their network independently with their own strategies and policies. Therefore, it is difficult, although not impossible, to host the overlay nodes across a multiAutonomous System (AS) infrastructure at present. In this paper, our aim is to construct an efficient overlay topology to provide better resilience under different failure models in the context of network-provider independent overlay networks such as ROMCA (Resilient Overlay for MissionCritical Applications) [15]. Simulation results show that the

Currently the Internet only provides best effort packet transport. Furthermore, the Border Gateway Protocol (BGP), used for routing across ASes, is characterized by reconvergence times of several minutes or longer [7]. Overlay networks, for example RON [3], are proposed to provide better performance by actively monitoring the network among a group of participating nodes. The nodes in RON form a full mesh topology and use active probing to monitor the health of Internet paths included in the overlay. In contrast to the high convergence time of the BGP protocol, RON can achieve recovery time in the order of tens of seconds based on test-bed experiments [3]. However, the overhead increases in the order of O(n2), where n is the number of overlay nodes. In [8], the authors discuss the relationship between the effectiveness of the overlay and the overhead consumption. The conclusion is that with a lower overlay node degree, a comparable Quality of Service (QoS) performance to that of full connectivity can be achieved. Overlay topology construction has been researched from various aspects. For example, in [4, 13], different types of overlay topologies are analyzed given overlay node locations. On the other hand, there remains discussion on where to place the overlay nodes [5, 12]. For instance, [12] considers the question of how many ISPs and the number of routers inside one ISP network is enough for overlay routing. The correlations between direct and indirect paths for source and destination pairs using different available nodes are calculated. Then, the intermediate nodes are categorized into different performance clusters for overlay construction use. Both of them show that the diversity of virtual links is indispensable for providing better performance in overlays. However, they are both categorized as provider-dependent overlay architectures. Moreover, customers only use the overlay services in the case of direct Internet paths failures, which means they should be equipped with the ability to diagnose the health of the original path in a timely manner.

QoSMap [6] showed that it can provide high overlay resilience in a provider-independent overlay. It maps an overlay topology with specific QoS requirements onto a physical network topology by sequentially selecting the PlanetLab nodes that can provide the best QoS performance. However, it strategically chooses the physical network to provide a diversified set of paths for the overlay construction. The possibility of overlapping between the virtual links that may result in concurrent failures still needs to be discussed. Most existing work on overlays focuses on improving QoS performance using a single intermediate node for detouring, there is no work addressing network provider independent overlay networks resilience under different failure models. In [9, 11], two availability models are proposed to define an overlay that can still be fully connected in case of no more than three physical node failures. They assume that all the physical nodes are overlay candidates and physical nodes with a lower degree than three are not considered. Conversely, our overlay node candidates are constrained to ASes with low connectivity as would be expected of smaller tier-3 stub networks. Moreover, they focus on proving the NP-completeness of the two models [11] and how to construct an overlay with high availability in a scalable distributed way [9]. Our work considers finding a near optimal solution of overlay topology using heuristic methods and verification is carried out under different failure models. According to their definition, overlay nodes can route through all the possible physical paths between a pair of overlay nodes. However, we assume the paths between the overlay nodes are determined by the physical layer. It is based on the observation that the nodes in the stub areas generally do not have control over which intermediate nodes (i.e. physical routers or ASes) they employ for routing purposes unless with the help of other overlay nodes in the overlay layer. III. OVERLAY TOPOLOGY CONSTRUCTION

B. Problem Description The overlay topology construction problem can be stated as: given a physical network topology G p (V p , E p ) and the overlay topology requirements include: (1) The provider independence requirement: only physical ASes with lower connectivity can be selected to host overlay nodes, namely, ND physical (ON ) PNDmax ; (2) The overlay node degree constraint

NDoverlay (ON ) ONDmax ; the objective is to find an

to

reduce

the

overhead,

namely,

overlay topology Go (Vo , Eo ) that will have maximum resilience under various physical failure(s) scenarios.

C (ON i , ON j ) = 1 ; otherwise it equals to 0. In this paper,

resilience () of the overlay is defined as follows:

If there exists a route between a pair of overlay nodes, then

afterfailure i{Vo } j{Vo }, i j nofailure i{Vo } j{Vo }, i j

C C

(ON i , ON j )
(2)

(ON i , ON j )

The main issue of overlay construction is that physical failures may result in concurrent overlay link failures, especially when the overlay nodes only reside in those areas with low connectivity (e.g. stub areas). The objective of the overlay topology design is to construct an overlay topology that has maximal AS-disjoint virtual links. In other words, the virtual links chosen should overlap the least in the physical layer. C. Proposed Algorithm The problem stated in the above sub-section can be formulated as to find a Go (Vo , Eo ) so as to minimize:
j{ Eo } i{ E o },i j

A. Definition Node Degree (ND): There are two types of node degree defined in this paper. Physical ND (PND) means the number of neighbours an AS has whilst Overlay ND (OND) means how many other overlay nodes (ON) an ON has virtual links with.

O (l , l
i

(3) (4)

s.t.

OND(Vo ) ONDmax

l i : The virtual link between two overlay nodes,

which is generally composed of a physical path. It is be notated as P (li ) , which is an ordered physical AS list. Each virtual link must be probed to monitor its status.

It can be solved in a two-step manner: (1) to find a topology in which all the nodes have the same node degree (i.e. a regular graph) and then (2) to map it onto the selected physical node set {V p ' } so as to obtain an overlay topology with minimum overlap as defined in equation (3). The second step can be rewritten as:
i , j ,k ,l{Vo } m , p , s ,t{V p '}

O (l i , l j ) : The overlap between two virtual links.

b(i, j )b(k , l )O( P(m, p), P( s, t ))x

ij

im

x jp x ks xlt
(5)

O (li , l j ) = LO (li , l j ) Li L j
Where

(1) s.t.:

LO (li , l j ) =| P(l i ) I P(l j ) | and means

i{Vo }

=1,

j{V p '}

ij

= 1 , xij {0,1}

(6)

the number of AS hops two virtual links share in common, and Li ( L j ) represents the number of AS hops

li ( l j ) has. Usually the longer a virtual link is,

the higher the probability it will overlap with other virtual links.

Where b(i , j ) equals to 1 if there is a connection between ON i and j, otherwise 0. It is a variant of BiQuadratic Assignment Problem (BiQAP), which is a generalization of the NP-hard QAP problem [17]. As there is no known way to find the optimal solution of this problem in polynomial time in a

large scale network, simulated annealing is chosen to find a near optimal solution in an efficient way. IV. SIMULATION RESULTS

include both covered and uncovered physical nodes, so the first failure is chosen from the whole set. D. Results and Analysis The proposed algorithm is verified considering different topologies, overlay node number, overlay node degree and failure models. The results are averaged over 300 trials. Due to space limitations, only some of the results are shown here. However, all the s results are consistent. The overlay node number, overlay node degree and failure model chosen are 30, 5 and multiple random failures unless stated otherwise. Except for grid topologies where PNDmax equals to 3, all the other physical topologies have constraint of PNDmax equals 1. 1) Physical Topology Impact The performance difference among the four topological construction methods (i.e. M1-M4) show similar trends irrespective of the size and type of the underlying physical topology. Moreover, M3 performs the best among the three comparable methods. Nevertheless, as shown in Fig. 1, the resilience difference is much larger (as high as 30%) in the grid topology than that (max 10%) of the other two. The reason behind this is because there is less physical degree variation in the grid topology. So the higher the overlay node degree is, more resilient the overlay topology will be against multiple random network failures. Furthermore, as depicted in the figure, for given a maximum overlay node degree constraint, careful design of the overlay topology is needed and can improve the overlay network availability even when the number of network failures is small and the improvement can be more than 10% and 30% in scale-free and grid topologies.
1

A. Assumptions The virtual link between adjacent overlay nodes follows a series of physical hops that are determined using a least-cost routing algorithm. Only symmetric routes are considered here for simplicity. It is assumed that the physical topology is known to the overlay. All the overlay nodes are assumed to have the ability to detect the performance degradation and/or the failures in the physical network. They get the up-to-date routing information based on link-state routing that operates across the overlay at the time of making overlay routing decisions.

B. Simulation Scenarios In order to evaluate the performance of the proposed scheme (notated as M3), three methods are also included: (1) M1: Full Mesh, which provides the upper bound of the resilience metric; (2) M2: randomly mapping of regular graph; (3) M4: the random method of selecting overlay virtual links proposed in [8]. The average node degree of M4 is maintained to be the same as our proposed method for fair comparison. A variety of physical networks are chosen including (1) CN05 AS-level network obtained using traceroute [5]; (2) 200, 500 and 2000-node scale-free topologies generated using Pajek [16]; (3) 200 and 500-node random topologies generated using Pajek (average PND equals to 4); (4) 200 and 400-node grid topologies. Here, the nodes in the physical topology represent ASes. C. Failure Models Only AS failures in physical network are considered within this paper. A failure does not necessarily mean the physical breakdown but can also represent performance degradation (e.g. delay or loss rate) below an acceptable threshold as perceived by overlay probing and monitoring. There are three failure models employed in simulations. They are the Random Single Failure model, Random Multiple Failure model similar to that of [10] and Accumulative Focused Failure model similar to large-scale failure scenarios in [2] (the geographical information is not considered here), respectively. The first two are self-explanatory. With the third approach the failure starts from a single AS and propagates to its neighbours. All the neighbours of the previous failure set will be viewed as failed by turns. For the resilience evaluation of the overlay network, failure of ASes that do not convey overlay virtual links are not considered. Therefore in the simulations, the failures are randomly selected from the subset of ASes that ON virtual links traverse. We therefore introduce the term ON Supporting AS Failures to represent the number of failed ASes that are covered by the ON. As we do not consider AS failures beyond these (as they have no impact on the performance of the ON), the reported failure figures represent values that would be typically much higher than would be witnessed if the whole Internet were to be considered. However, for the third failure model, the radiating effect will

500-Node Scale Free Topology

0.9

0.8 0.7 0

M1 M2 M3 M4
2 4 6 8 10

ON-Supporting AS Failures(%)
500-Node Random Topology

1 0.9

0.8 0.7 0

M1 M2 M3 M4
2 4 6 8 10

ON-Supporting AS Failures(%)
1 0.8

400-Node (20*20) Grid Topology

0.6 0.4 0.2 0 0 2 M1 M2 M3 M4

ON-Supporting AS Failures(%)

10

Figure 1. Physical Topologies Impact

2) Overlay Degree and Node Number Impact The impact of overlay node degree (i.e. OND ranges from 4-10) and overlay node number (i.e. the number varies from 10 to 50.) on overlay resilience is verified in scale-free topologies. Although, only some of the simulation results are presented in Fig. 2 and 3 (in 500-node scale-free network), the following conclusions can be drawn: (1) as the overlay node degree increases, the performance of the three methods will become better in terms of resilience and closer to that of the full mesh. (2) M3 performs the best, but the performance difference will diminish as the overlay node degree increases. (3) M3 performs the best irrespectively of the number of overlay nodes. 3) Failure Model Impact In Fig. 4, the x-axis represents the Failure Radius of the accumulative-focused failure model. Specifically, 0 represents the failure of a single AS and x (i.e. 1-5) means all the nodes that are x AS hops away from this point are deemed to have malfunctioned too. As illustrated in the figure, all the four schemes perform similarly with no significant difference under this failure model, except for that of grid network. M3 performs closest to that of full mesh method. And the gap between M3 and M4 in grid network can reach as high as 8%. This is probably because there is a lower overlap between the overlay nodes in grid network than that of the other types of physical networks. V. CONCLUSION

1 0.8

500-Node Scale Free Topology

1 0.8

500-Node Random Topology

0.6 0.4 0.2 0 0 M1 M2 M3 M4

0.6 0.4 0.2 0

M1 M2 M3 M4

Failure Radius(AS Hops)

1

Failure Radius(AS Hops)

400-node (20*20) Grid Topology

0.8

0.6

M1 M2 M3 M4

0.4
0 1 2

Failure Radius(AS Hops)

Figure 4. Accumulative-Focused Failures Impact

REFERENCES
[1] N. M. K. Chowdhury and R. Boutaba, David R. Cheriton, A Survey of Network Virtualization,, School of Computer Science, University of Waterloo, Tech. Rep., Oct. 2008, [Online]: Available: http://www.cs.uwaterloo.ca/research/tr/2008/CS-2008-25.pdf; Bijan Bassiri, Shahram Shah Heydari, Network survivability in largescale regional failure scenarios,C3S2E, Montreal, Canada, 2009; David G. Anderson, Hari Balakrishnan, Frans kaashoek, Robert Morris, Resilient Overlay Networks, In Symposium on Operating Systems Principles, page(s): 131-145, 2001; Li, Z., Mohapaira, P., The impact of topology on overlay routing service, INFOCOM 2004, page(s): 408- 418, March 2004; Bin Yuan, Guoqiang Zhang, Yanjun Li, Guoqing Zhang, Zhongcheng Li. Improving Chinese Internets Resilience through Degree Rank Based Overlay Relays Placement, ICC 2008, page(s): 5823-5827,2008; Jawwad Shamsi, Monica Brockmeyer, QoSMap: Achieving Quality and Resilience through Overlay Construction, Proceedings of the 2009 Fourth International Conference on Internet and Web Applications and Services,page(s): 58-67, 2009; Amit Sahoo, Krishna Kant, P. Mohapatra, BGP convergence delay after multiple simultaneous router failures: characterization and solutions, Computer Communications, Vol. 23(2009), page(s): 1207-1218; Sushant Rewaskar, Jasleen Kaur, Testing the Scalability of Overlay Routing Infrastructures, PAM 2004, page(s): 3342, 2004; M. Kumar S.D., Umesh Bellur, A distributed algorithm for underlay aware and available overlay formation in Event Broker Networks for publish/subscribe systems, DEPSA 07, Toronto, Canada, June 2007; Justin R. Rohrer, Abdul Jabar, James P.G. Sterbenz, Path diversification: a multipath resilience mechanism, DRCN 2009, Washington DC, America, October 2009; Madhu Kumar S. D, U.Bllur, Availability Models for Underlay Aware Overlay Networks, DEBS08, Rome, Italy, page(s):169-180, July 2008; Junghee Han, David Watsonb, Farnam Jahanianc, Enhancing end-toend availability and performance via topology-aware overlay networks, Computer Networks, Volume 52, Issue 16, page(s): 3029-3046, 2008; Zhi Li, Prasant Mohapatra, On Investigating Overlay Service Topologies, Computer Networks, Vol. 51, page(s): pages 54-68, 2007; L. Tang, Y. Huai, J. Zhou, H. Yin, Z. Chen, J. Li, A Measurement Study on the Benefits of Open Routers for Overlay Routing, Journal of Communications, Vol. 4, No 9, page(s):714-723, Oct 2009; Xian Zhang, Chris Phillips, Network Operator Independent Resilient Overlay for Mission Critical Applications (ROMCA), ChinaCom 2009, Xian, China, August 26-28, 2009; Pajek, available online: http://vlado.fmf.uni-lj.si/pub/networks/pajek/; E. Cela, The Quadratic Assignment Problem: Theory and Algorithms, Kluwer Academic Publishers, 1998;

[2] [3]

[4]

An efficient physical aware topology construction algorithm is proposed and verified under various network and failure models. It is shown according to the results that in scale-free networks, the proposed method can achieve better resilience by as much as 10%, compared with random construction methods whilst maintaining a relatively low overhead by constraining the overlay connectivity. In networks where physical node degree is similar, such as grid-like topologies, the performance benefit is even greater. Estimating the virtual link overlap exploiting physical topology inference techniques is now under investigation.
1

[5]

[6]

[7]

[8] [9]

Overlay Degree = 4

Overlay Degree = 8

[10]
0.9

0.8

0.7

M1 M2 M3 M4

0.9 M1 M2 M3 M4

[11] [12]
4 6 8 10

0.8

0.6 0

0.7

ON-Supporting AS Failures (%)

10

ON-Supporting AS Failures(%)

[13] [14]

Figure 2. Overlay Node Degree Impact

1 0.9 0.8 0.7 0.6 0.5 0 3 6 9 12 15 M1 M2 M3 M4 0.5 0 3

Overlay Node Number = 20

1 0.9 0.8

Overlay Node Number = 50

[15]
M1 M2 M3 M4 6 9 12 15

0.7 0.6

[16] [17]

ON-Supporting AS Failures(%)

ON-Supporting AS Failures(%)

Figure 3. Overlay Node Number Impact