Slides2

Creating Concurrent Processes
• FORK-JOIN construct:
Ptheads: Creation and termination
pointer to ID
pthread_create ( pthread_t *thread_handle, const pthread_attr_t *attribute, void * (*thread_function) (void * ), void *arg) pthread_join ( pthread_t thread void **ptr)
Latency Hiding
While one thread is waiting for a communication operation, others can utilize the CPU.
Scheduling and Load Balancing
Allow the programmer to specify a large number of concurrent tasks and support dynamic mapping of tasks to processors with minimal idling overheads.
forall Construct:
forall (i = 0; i < n; i++) { S1; S2; . . Sm; }
Example:
forall (i = 0; i < 5; i++) { a[i] = 0;
Cruces for Parallel Programming with Shared Memory multiprocessor system How do processes share information? How do processes synchronize? In shared-memory model, both accomplished through shared variables Creating Concurrent Processes Dealing with the Sharing Data Synchronization Problems
Several Alternatives for Programming
Using a new programming language Modifying an existing sequential language Using library routines with an existing sequential language Using a sequential programming language and ask a parallelizing compiler to convert it into parallel executable code. Threads (Pthreads, Java, ..) OpenMP
BTB & I-TLB Decoder
BTB & I-TLB Decoder
BTB
uCode ROM
BTB
L2 L2 Cache andControl Cache and Control
L2 L2 Cache andControl Cache and Control
uCode ROM
Trace Cache
• Compositional C++ • Fortran M • High performance Fortran
Language Constructs for Parallelism
Par Construct: Shared Data: shared int x;
par { S1; S2; . . Sn; }
Process state, Program counter, Values of CPU registers and Memory management information
Threads: “A unit of control within a process”
Threads have own program counter, copy of CPU, registers, and stack of activation records Threads share process’s data, code, address space, and other resources
Domain Decomposition Using Threads
Thread 0 f()
Shared Memory
Thread 2 f ( )b
Thread 1 f()
Pipelining Using Threads
Thread 0 Data e ( ) set 4 Thread 1 f() Data set 3 Data set 2 Thread 2 g() Data set 1 Output Shared Memory
•
•
• •
Why Threads?
Software Portability
Applications can be developed on serial machines and run on parallel machines. Ease of Programming, Widespread Use.
Programming with Pthreads
Threads vs Process
Process: A program in some state of execution
Include: Code, Data, Logical address space
Information about a process includes:
3
Trace Cache
3
Rename/Alloc
3
Rename/Alloc
3
uop Queues Schedulers
uop Queues Schedulers
Floating Point
Integer
Floating Point
L1 D-Cache and D-TLB
Integer
L1 D-Cache and D-TLB
BTB & I-TLB Decoder
BTB
L2L2 Cache andControl Cache and Control
uCode ROM
Trace Cache
3
Rename/Alloc uop Queues Schedulers
3
Floating Point
Integer
L1 D-Cache and D-TLB Cache D
Threads vs Process
•
“heavyweight” process –completely separate program with its own variables, stack, and memory allocation. Threads - shares the same memory space and global variables between routines. Creation of a thread can take three orders of magnitude less time than process creation. A thread can immediately access to shared global variables. Synchronization of threads can be done much more efficiently than synchronization of processes.
Parallel Programming Language
•
With special parallel programming constructs and statements that allow shared variables and parallel code sections to be declared. Then the compiler is responsible for producing the final executable code. Language Concurrent Pascal Ada Modula-P C* Concurrent C Fortran D Originator/date Brinch Hansen, 1975 U.S. Dept. of Defense, 1979 Bräunl, 1986 Thinking Machines, 1987 Gehani and Roome, 1989 Fox et al., 1990 Comments Extension to Pascal Completely new language Extension to Modula 2 Extension to C for SIMD systems Extension to C Extension to Fortran for data parallel programming • Some Early Parallel Programming Languages:
pointer to the argument to function
Executing a Pthread Thread
Detached Threads
pthread_exit (0)
Creating Shared Data
For UNIX heavyweight processes additional shared memory system calls are necessary. The shared memory segment is created using the shmget() system call which return a shared memory identifier. The shared segment can be attached to the data segment of the calling process using the shmat() system call which returns the starting address of the data segment. For threads it is not necessary to create shared data items explicitly, variables declared at the top of the main program (main thread) are global and are available to all threads.