Dynamic load balancing of data parallel applications on a distributed network

Cluster-based computing, which exploits the aggregate power of networked collections of workstations, has drawn increasing attention from the parallel processing community. The main problem with this computing environment is the permanently changing workload of individual workstations which makes the execution time of parallel applications unpredictable. In this paper, we introduce a load balancing scheme which aims at dynamically balancing the workload of data parallel applications. Simulation and experimental studies of our load balancing strategy are performed under various load situations and it is shown that it can effectively balance the workload among the workstations involved. Further, it was shown that a significant improvement in performance can be achieved when compared to the case where no load balancing is employed. The main limiting factor in our computing environment is the bandwidth of the network. Thus, with emerging high speed networks, computing on networks of workstations can be an attractive alternative to traditional parallel computers.