Scheduling And Load Balancing In Parallel And Distributed Systems

Advances in hardware and software technologies have led to an increased interest in the use of large-scale parallel and distributed systems for database, real-time, defense, and large-scale commercial applications. One of the biggest system issues is developing effective techniques for the distribution of multiple program processes on multiple processors. This book discusses how to schedule the processes among processing elements to achieve the expected performance goals, such as minimizing execution time, minimizing communication delays, or maximizing resource utilization.

This book focuses on the future directions of the static scheduling and dynamic load balancing methods in parallel and distributed systems. It provides an overview and a detailed discussion on a wide range of topics from theoretical background to practical, state-of-the-art scheduling and load balancing techniques.

The book will be a useful guide to industry professionals, academic professors, and students who are interested in these important aspects of parallel and distributed systems. Also, it will be helpful to those working on research and development in parallel processing applications, compilers and operating systems, system design, and software tools for parallel program development.

Preface.

Acknowledgments.

CHAPTER 1: INTRODUCTION TO SCHEDULING AND LOAD BALANCING.

A Taxonomy of Scheduling in General-Purpose Distributed Computing Systems (T.L. Casavant and J.G. Kuhl)
(IEEE Transactions on Software Engineering, February 1988, pages 141-154).

Multiprocessor Scheduling with the Aid of Network Flow Algorithms (H.S. Stone)
(IEEE Transactions on Software Engineering, January 1977, pages 85-93).

Load Distributing for Locally Distributed Systems (N.G. Shivaratri, P. Krueger, and M. Singhal)
(Computer, December 1992, pages 33-44).

CHAPTER 2: STATIC SCHEDULING.

Analysis and Evaluation of Heuristic Methods for Static Task Scheduling (B. Shirazi, M. Wang, and G. Pathak)
(Journal of Parallel and Distributed Computing, Volume 10, 1990, pages 222-232).

Compile-time Partitioning and Scheduling of Parallel Programs (V. Sarkar and J. Hennessy)
(Proceedings of the SIGPLAN '88 Symposium on Compiler Construction, 1986, pages 17-26).

Guided Self-Scheduling: A Practical Scheduling Scheme for Parallel Supercomputers CD (Polychronopoulos and D.J. Kuck)
(IEEE Transactions on Computers, December 1987, pages 1425-1439).

Multiprocessor Scheduling with Communication Delays (B. Veltman, B.J. Lageweg, and J.K. Lenstra)
(Parallel Computing, Volume 16, 1990, pages 173-182).

Heuristic Algorithms for Task Assignment in Distributed Systems (V.M. Lo)
(IEEE Transactions on Computers, November 1988, pages 1384-1397).

Allocating Modules to Processors in a Distributed System (D. Ferndndez-Baca)
(IEEE Transactions on Software Engineering, November 1989, pages 1427-1436).

Compile-Time Scheduling and Assignment of Data-Flow Program Graphs with Data-Dependent Iteration (S. Ha and E.A. Lee)
(IEEE Transactions on Computers, November 1991, pages 1225-1238).

A Program Allocation Scheme for Data Flow Computers (A.R. Hurson, et al.)
(Prodeedings 1990 International Conference on Parallel Processing, pages 1-415 -1-423).

CHAPTER 3: TASK GRANULARITY AND PARTITIONING 143

On the Granularity and Clustering of Directed Acyclic Task Graphs (A. Gerasoulis and T. Yang)
(IEEE Transactions on Parallel and Distributed Systems, June, 1993, pages 686-701).

The Effects of Problem Partitioning, Allocation, and Granularity on the Performance of Multiple-Processor Systems (Z. Cvetanovic)
(IEEE Transactions on Computers, April 1987, pages 421-432).

Grain Size Determination for Parallel Processing (B. Kruatrachue and T. Lewis)
(IEEE Software, January 1988, pages 23-32).

Lazy Task Creation: A Technique for Increasing the Granularity of Parallel Programs (E. Mohr, D.A. Kranz, and R. H. Halstead, Jr.)
(IEEE Transactions on Parallel and Distributed Systems, July 1991, pages 264-280).

CHAPTER 4: SCHEDULING TOOLS.

PARSA: A Parallel Program Software Development Tool (B. Shirazi, et al.)
(Proceedings 1994 Symposium on Assessment of Quality Software Development Tools, 1994, pages 96-111).

The TOPSYS Architecture (T. Bemmerl)
(Proceedings ofCONPAR '90 VAPPIV: LNCS, Volume 457,1990).

Parallax: A Tool for Parallel Program Scheduling (T. Lewis and H. El-Rewini)
(IEEE Parallel & Distributed Technology, May 1993, pages 62-72).

Parafrase-2: An Environment for Parallelizing, Partitioning, Synchronizing, and Scheduling Programs on Multiprocessors (CD. Polychronopoulos, et al.)

(Proceedings 1989 International Conference on Parallel Processing, Volume II, 1989, pages II-39-II-48.).

Mapping Function-Parallel Programs with the Prep.-P Automatic Mapping Preprocessed (F. Berman and B. Stramm)
(Technical Report Number CS 94-397, December 1994, pages 1-28).

OREGAMI: Tools for Mapping Parallel Computations to Parallel Architectures (V.M. Lo, et al.)
(International Journal of Parallel Programming, 1991, pages 237-270).

CHAPTER 5: LOAD BALANCING.

Load Sharing in Distributed Systems (Y.-T. Wang and RJ.T. Moms)
(IEEE Transactions on Computers, March 1985, pages 204-217).

The Probability of Load Balancing Success in a Homogeneous Network (C.G. Rommel)
(IEEE Transactions on Software Engineering, September 1991, pages 922-933).

Adaptive Load Sharing in Homogeneous Distributed Systems (D.L. Eager, E.D. Lazowska, and J. Zahorjan)
(IEEE Transactions on Software Engineering, May 1986, pages 662-675).

Imbedding Gradient Estimators in Load Balancing Algorithms (S. Pulidas, D. Towsley, and J.A. Stankovic.)
(Proceedings IEEE 8th International Conference on Distributed Computing Systems, 1988, pages 482-490).

The Diverse Objectives of Distributed Scheduling Policies (P. Krueger and M. Livny)
(Proceedings IEEE 7th International Conference on Distributed Computing Systems, 1987, pages 242-249).

Effects of Response and Stability on Scheduling in Distributed Computing Systems (T.L. Casavant and J.G. Kuhl)
(IEEE Transactions on Software Engineering, November 1988, pages 1578-1588).

Condor-A Hunter of Idle Workstations (M. J. Litzkow, M. Livny, and M.W. Mutka)
(Proceedings IEEE 8th International Conference on Distributed Computing Systems, 1988, pages 104-111).

GAMMON: A Load Balancing Strategy for Local Computer Systems with Multiaccess Networks (K.M. Baumgartner and B.W. Wah)
(IEEE Transactions on Computers, August 1989, pages 1098-1109).

The Stealth Distributed Scheduler (P. Krueger and R. Chawla)
(Proceedings IEEE 11th International Conference on Distributed Computing Systems, 1991, pages 336-343).

CHAPTER 6: MECHANISMS FOR PROCESS MIGRATION.

Design Issues of Process Migration Facilities in Distributed Systems (M.R. Eskicioglu)
(IEEE Technical Committee on Operating Systems Newsletter, Winter 1989, pages 3-13).

Heterogeneous Process Migration by Recompilation (M.M. Theimer and B. Hayes)
(Proceedings IEEE 11th International Conference on Distributed Computing Systems, 1991, pages 18-25).

Attacking the Process Migration Bottleneck (E.R. Zayas)
(Proceedings of the 11th ACM Symposium on Operating Systems Principles, 1987, pages 13-24).

Designing a Process Migration Facility (Y. Artsy and R. Finkel)
(Computer, September 1989, pages 47-56).

Process Migration in DEMOS/MP (M.L. Powell and B.P. Miller)
(Proceedings of the 9th ACM Symposium on Operating Systems Principles, 1983, pages 110-119).

Preemptable Remote Execution Facilities for the V-System (M.M. Theimer, K.A. Lantz, and D.R. Cheriton)
(Proceedings of the 10th ACM Symposium on Operating Systems Principles, 1985, pages 2-12).

CHAPTER 7: LOAD INDICES.

Predictability of Process Resource Usage: A Measurement-Based Study on UNIX (M.V. Devarakonda an R.K. Iyer)
(IEEE Transactions on Software Engineering, December 1989, pages 1579-1586).

An Empirical Investigation of Load Indices for Load Balancing Applications (D. Ferrari and S. Zhou)
(Proceedings Performance '87, The 12th Annual International Symposium on Computer Performance Modeling, Measurement and Evaluation, 1987, pages 515-528).

The Influence of Different Workload Descriptions on a Heuristic Load Balancing Scheme (T. Kunz)
(IEEE Transactions on Software Engineering, July 1991, pages 725-730).

About the Authors.