AN EFFICIENT PROCESSOR ALLOCATION FOR NESTED PARALLEL LOOPS ON DISTRIBUTED MEMORY HYPERCUBE

2.1. Perfectly-nested DOALL Loops
A perfectly-nested DOALL loop of nest depth k is a loop of the form of gure 1 (a), where stat(T) is an assignment statement block and T is its computing time. The loop partitioning scheme we proposed for a nested DOALL loop is carried out establishing a one-to-one correspondence between the nested loop and a partition of the hypercube dimensions. A de nition of such a partition follows: De nition 1. Given Hq , a k-partition of its dimensions is a vector pk = q P? (qk?1; qk?2; : : :; q0), such that qi 0 (i = 0; 1; : : :; k ? 1) and k=01 qi = q. i This partition of the dimensions allows us to decompose the number of processors of the hypercube into Q = 2q = Qk?1Qk?2 : : :Q0 , with Qi = 2qi . This can be interpreted as the hypercube has been subdivided into k subcubes, Hqk?1 , Hqk?2 , 2
1. Introduction
One of the phases in the scheduling is the processor allocation, which is the assignment of the number of processors to each task of the parallelized program. In this paper, we concentrate on the problem of static processor allocation for arbitrarily nested parallel loops on distributed memory, message-passing hypercubes. The programming model is SPMD (Single Program, Multiple Data). Using the minimization of the completion time as performance criterion, we solve the processor allocation in two consecutive stages 1] 2]: loop partitioning and loop distributing. The former calculates an upper bound for the number of iterations each processor will execute, out of the total speci ed by the loops. The latter distributes such sets of iterations across the corresponding processors. These stages are interdependent through the performance criterion, and they mutually depend on the data distribution scheme. In order to break these interdependencies, we follow the next sequence
This work has been partially supported by grants TIC90-0407 and TIC92-0942 of the CICYT and XUGA20604A90 of the Xunta de Galicia
1
of actions: (i) Design of a loop partitioning scheme that minimizes the parallel computing time of the loops. (ii) Design of a loop distributing scheme that, on one hand, optimizes the workload balance and, on the other hand, minimizes the communication time. (iii) Design of a data distributing scheme which exploit the data locality (i.e., it minimizes the communication time), as well as maintain the data redundance at a reasonable level (as a function of the capacities of the local memories). With this three-step scheme, the structure of the nested loop is preserved, as we only partition the loops at the individual level. The e ect of the partition is to reduce the value of the upper bounds of the loops. Therefore, the inter-loop dependencies are not a ected by action (i). As action (ii) is under the partition performed by action (i), any type of loop distribution is going to verify this dependencies. This means that we only have to worry about the intra-loop dependencies, which are going to be the ones responsible for the possible interprocessor communication instructions we must incorporate to our nal parallel algorithm. Next, we propose HYPAL (HYpercube Partitioning ALgorithm) as an e cient processor allocation algorithm for arbitrarily nested parallel loops. In principle, HYPAL does not consider the communication cost because it is not known until the data distribution has been concluded. However, we can include the communication cost in the overall parallel time with no restructuration in HYPAL.
AN EFFICIENT PROCESSOR ALLOCATION FOR NESTED PARALLEL LOOPS ON DISTRIBUTED MEMORY HYPERCUBES
OSCAR PLATA, TOMAS F. PENA, FRANCISCO F. RIVERA Dept. Electronica y Computacion. University of Santiago 15706 Santiago de Compostela, SPAIN and EMILIO L. ZAPATA Dept. Arquitectura de Computadores. University of Malaga Plaza El Ejido, s/n, 29013 Malaga, SPAIN May 10, 1993 ABSTRACT We consider the static processor allocation problem for arbitrarily nested parallel loops on distributed memory, message-passing hypercubes. We present HYPAL (HYpercube Partitioning ALgorithm) as an e cient algorithm to solve this problem. HYPAL calculatesan optimalset of partitionsof the dimensionof the hypercube,and assigns them to the set of iterations of the nested loop. Some considerations about the in uence of the communicationoverhead in order to get a more realistic approach are considered. The main problem at this point is to obtain the communication pattern associated to the parallel program because it depends on scheduling and data distribution. Keywords : Distributed memory hypercube multiprocessor, parallelizing compiler, processor allocation, loop schedulocessor Allocation Algorithm