1

Lahore Garrison University

Parallel and Distributed
Computing
Session Spring 2023

Lecture – 15
INSTRUCTOR: MUHAMMAD ARSALAN RAZA
 2

Lahore Garrison University

NAME: EMAIL: PHONE NUMBER: VISITING HOURS:
MUHAMMAD ARSALAN RAZA [email protected] 0333-7843180 (PRIMARY) MON-FRIDAY:(10:00AM - 1:00PM)
K 0321-4069289 Thursday: (11:00Am - 12:00pm)
 3
Preamble

 Memory Models

 Message Passing Interface (MPI)

 Making Sockets Easier to Work With

 MPI Primitives and Meanings

Lahore Garrison University

 4
Lesson Plan

 OpenMP

 OpenMP Essentials

 What is OpenMP and What’s Not?

 Goals of OpenMP

 OpenMP Programming Model

 Programming

Lahore Garrison University

 5
OpenMP

 OpenMP is an Application Program Interface (API), jointly defined by a group of major computer hardware
and software vendors.
 OpenMP provides a portable, scalable model for developers of shared memory parallel applications.
OpenMP Essentials
 The API supports C/C++ and Fortran on a wide variety of architectures.
 This lecture covers most of the major features of OpenMP 3.1, including its various constructs and
directives for specifying parallel regions, work sharing, synchronization, and data environment.
 Runtime library functions and environment variables are also covered.

Lahore Garrison University

 6
What is OpenMP?

 An Application Program Interface (API) that may be used to explicitly direct multi-threaded, shared
memory parallelism.

 Comprised of three primary API components:

 Compiler Directives*

 Runtime Library Routines**

 Environment Variables

 An abbreviation for: Open Multi-Processing

*See References on the last slide.
Lahore Garrison University
 7
OpenMP is Not:

 Necessarily implemented identically by all vendors

 Guaranteed to make the most efficient use of shared memory

 Required to check for data dependencies, data conflicts, race conditions, deadlocks, or code sequences

that cause a program to be classified as non-conforming

 Designed to handle parallel I/O. The programmer is responsible for synchronizing input and output.

Lahore Garrison University

 Goals of OpenMP 8

 Standardization:
 Provide a standard among a variety of shared memory architectures/platforms
 Jointly defined and endorsed by a group of major computer hardware and software vendors
 Lean and Mean:
 Establish a simple and limited set of directives for programming shared memory machines.
 Significant parallelism can be implemented by using just 3 or 4 directives.
 This goal is becoming less meaningful with each new release, apparently.
 Ease of Use:
 Provide capability to incrementally parallelize a serial program, unlike message-passing libraries which typically require an all or nothing
approach
 Provide the capability to implement both coarse-grain and fine-grain parallelism
 Portability:
 The API is specified for C/C++ and Fortran
 Public forum for API and membership
 Most major platforms have been implemented including Unix/Linux platforms and Windows

Lahore Garrison University

 9

OpenMP Programming Model

Lahore Garrison University
 10
Hybrid Programming

 Hierarchical parallel architectures are gaining more importance.

 Uniform programming interface is possible but might not be most efficient.
 MPI requires optimized library for intranode communication.
 OpenMP requires global address space implemented in software.
 Hybrid programming allows to optimize both levels of the hierarchy.
 MPI across nodes
 OpenMP within a node
 In addition to increased programming complexity, it requires additional performance-relevant design decisions
prior to coding.

Lahore Garrison University

 11

Lahore Garrison University

Programming Interfaces
OpenMP

www.openmp.org
 12
OpenMP: Description-based SM
Parallelization
 OpenMP is a standard shared memory programming interface (1997)
 Directives for Fortran 77 and C/C++
 Fork-join model resulting in a global program
 It includes:
 Parallel loops
 Parallel sections
 Parallel regions
 Shared and private data
 Synchronization primitives
 barrier
 critical region
Lahore Garrison University
 13

OpenMP
Example

Lahore Garrison University

 14
C Language Example
// OpenMP program to print Hello World
// using C language

// OpenMP header For CMD

#include <omp.h>
#include <stdio.h> gcc -o hello -fopenmp hello.c
#include <stdlib.h>
int main(int argc, char* argv[]) ./hello
{
// Beginning of parallel region
#pragma omp parallel
{
printf("Hello World... from thread = %d\n",
omp_get_thread_num());
}
// Ending of parallel region
}

Lahore Garrison University Tutorial Link: https://www.geeksforgeeks.org/openmp-hello-world-program/

 15
Notes & Restrictions

 The dynamic threads mechanism modifies the effect of this routine.

 Enabled: specifies the maximum number of threads that can be used for any parallel region by the dynamic

thread's mechanism.

 Disabled: specifies exact number of threads to use until next call to this routine.

Lahore Garrison University

 16
Lesson Review

 Tools and Development

 OpenMP

 OpenMP Essentials

 What is OpenMP and What’s Not?

 Goals of OpenMP

 OpenMP Programming Model

 Notes & Restrictions

Lahore Garrison University

 17
Next Lesson Preview

 Tools and Development

 MPI using C#

Lahore Garrison University

 18
References

 To cover this topic, different reference material has been used for consultation.

 Textbook:
Distributed Systems: Principles and Paradigms, A. S. Tanenbaum and M. V. Steen, Prentice Hall, 2 nd Edition, 2007.
Distributed and Cloud Computing: Clusters, Grids, Clouds, and the Future Internet, K. Hwang, J. Dongarra and GC. C.
Fox, Elsevier, 1st Ed.

 Google Search Engine

*https://docs.microsoft.com/en-us/cpp/parallel/openmp/reference/openmp-directives?view=msvc-170
**https://www.cita.utoronto.ca/~merz/intel_f10b/main_for/mergedProjects/optaps_for/common/optaps_par_libr.htm
Lahore Garrison University