A Symmetric Load Balancing Algorithm with Performance Guarantees for Distributed Hash Tables

Abstract Peers participating in a distributed hash table (DHT) may host different numbers of virtual servers and are enabled to balance their loads in the reallocation of virtual servers. Most decentralized load balance algorithms designed for DHTs based on virtual servers require the participating peers to be asymmetric, where some serve as the rendezvous nodes to pair virtual servers and participating peers, thereby introducing another load imbalance problem.While state-of-the-art studies intend to present symmetric load balancing algorithms, they introduce significant algorithmic overheads and guarantee no rigorous performance metrics. In this paper, a novel symmetric load balancing algorithm for DHTs is presented by having the participating peers approximate the system state with histograms and cooperatively implement a global index. Each peer independently reallocates in our proposal its locally hosted virtual servers by publishing and inquiring the global index based on their histograms. Unlike competitive algorithms, our proposal exhibits analytical performance guarantees in terms of the load balance factor and the algorithmic convergence rate, and introduces no load imbalance problem due to the algorithmic workload. Existing system : To tackle the load imbalance problem in DHTs, prior proposals have presented centralized algorithms that rely on a few rendezvous nodes to balance the loads of peers in a DHT. In contrast to the centralized approach, some studies, e.g., [9], [10], suggest organizing rendezvous nodes in a hierarchical manner. Virtual servers are first

matched with peers through the rendezvous nodes in the lower layer of the hierarchy; for unpaired virtual servers, the rendezvous peers relay them to the rendezvous in the upper layer to seek reallocation. This process repeats iteratively until an unpaired virtual server reaches a rendezvous in the highest layer. Demerits : considering large-scale and dynamic DHT networks, the centralized algorithms may introduce the performance bottleneck and the single point of failure. Consequently, the rendezvous nodes may experience skewed workloads, introducing another load imbalance problem. They may also become the performance bottleneck and the single point of failure. Moreover, the hierarchical networks (e.g., the tree-shaped network in ) facilitating the load balancing algorithms are prone to node/communication failure, thus demanding sophisticated maintenance for the networks. Proposed system : In this paper, we present a fully decentralized load balancing algorithm for DHTs. Our proposal, is essentially different from the previous rendezvous-based solutions in that each peer estimates and represents the system state with histograms. Based on the histograms, each peer reallocates its load independently without introducing an asymmetric rendezvous in collecting the system state as well as matching virtual servers and participating peers.