OS Support for Building

Distributed Applications:
Multithreaded Programming
using Java Threads

Dr. Rajkumar Buyya

Cloud Computing and Distributed Systems (CLOUDS) Laboratory
School of Computing and Information Systems
The University of Melbourne, Australia
http://www.buyya.com

1
 Outline

◼ Introduction to Middleware
◼ Thread Applications
◼ Defining Threads
◼ Java Threads and States
◼ Architecture of Multithreaded servers
◼ Threads Synchronization
◼ Summary

2
 Introduction

◼ Middleware is a layer of software Applications

(system) between Applications and Middleware
Operating System (OS) powering Distributed
Nodes with OS
the nodes of a distributed system.
◼ The OS facilitates:
◼ Encapsulation and protection of
resources inside servers;
◼ Invocation of mechanisms required to
access those resources including
concurrent access/processing.

3
Middleware and Network Operating
System (NOS)
◼ Many DOS (Distributed OS) have been investigated, but
there are none in general/wide use. But NOS are in
wide use for various reasons both technical and non-
technical.
◼ Users have much invested in their application software; they
will not adopt a new OS that will not run their applications.
◼ Users tend to prefer to have a degree of autonomy of their
machines, even in a closely knit organisation.
◼ A combination of middleware and NOSs provides an
acceptable balance between the requirement of
autonomy and network transparency.
◼ NOS allows users to run their favorite word processor.
◼ Middleware enables users to take advantage of services that
become available in their distributed systems.

4
 Introducing a middleware
◼ Building Distributed Systems
◼ DOS or NOS are not enough to build a DS!
◼ NOS are a good starting point but ….
◼ … we need an additional layer “gluing” all together

Middleware

NOS
Building Distributed Systems
◼ Middleware
◼ High-level features for DS
◼ Communication
◼ Management
◼ Application specific
◼ Uniform layer where to build DS services
◼ Runtime environment of applications
◼ Operating System
◼ Low / medium level (core) features
◼ Process / threads management
◼ Local hardware (CPU, disk, memory)
◼ Security (users, groups, domain, ACLs)
◼ Basic networking
 Operating system layers and
Middleware
Ap plicatio ns, se rvic es

Midd le wa re

OS: ke rn el, OS1 OS2

libraries & Proc ess es, thread s, Proc ess es, thread s,
se rv ers co mmun ica tion, ... co mmun ica tion, ...
Platform

Computer & Computer &

ne twork hard wa re ne twork hard wa re

Node 1 Node 2
◼ Unix and Windows are two examples of Network Operating Systems – have a
networking capability built into them and so can be used to access remote
resources using basic services such as rlogin, telnet.
7
Core OS components and
functionality
Proces s manager

Comm unicat ion

manager

T hread manager Memor y manager

Superv isor

8
 Threaded Applications

◼ Modern Applications and Systems

◼ Operating System Level
◼ Multitasking: multiple applications running at
once
◼ Application Level
◼ Multithreading: multiple operations performed at
the same time within an application.
◼ Bottom Line:
◼ Illusion of concurrency
 Threaded Applications

◼ Modern Systems
◼ Multiple applications run concurrently!
◼ This means that… there are multiple processes on
your computer

games

web & email

office automation

multimedia
pictures

Multitasking
 A single threaded program

class ABC
{
….
public void main(..) begin

{
… body

..
end
}
}

11
 Threaded Applications

◼ Modern Systems
◼ Applications perform many tasks at once!
◼ This means that… there are multiple threads within
a single process.
Background printing

GUI rendering

Application core logic

A Multithreaded Program

Main Thread

start
start start

Thread A Thread B Thread C

Threads may switch or exchange data/results

13
 Single and Multithreaded
Processes
threads are light-weight processes within a process

Single-threaded Process Multi-threaded Process

Threads of
Execution

Single instruction stream Common Multiple instruction stream

Address Space

14
Multithreaded Server: For Serving
Multiple Clients Concurrently
◼ Modern Applications
◼ Example: Multithreaded Web ServerProcess Request Client 1
Web/FTP
server

Process Request Client 2

Client 1

Process Request Client N

Client 2

Client N
 Threaded Applications

◼ Modern Applications
◼ Example: Internet Browser + Youtube

Video Streaming

Favorities, Share,
Comments Posting
 Modern Applications need Threads (ex1):
Editing and Printing documents in background.
Printing Thread

Editing Thread

17
Multithreaded/Parallel File Copy

reader()
{ writer()
- - - - - - - - - buff[0] {
- - - - - - - - - - -
lock(buff[i]); lock(buff[i]);
read(src,buff[i]); buff[1]
write(src,buff[i]);
unlock(buff[i]); unlock(buff[i]);
- - - - - - - - - - - - - - - - - - -
- }
}

Cooperative Parallel Synchronized

Threads
18
 Defining Threads

◼ Applications – Threads are used to

perform:
◼ Parallelism and concurrent execution of
independent tasks / operations.
◼ Implementation of reactive user interfaces.
◼ Non blocking I/O operations.
◼ Asynchronous behavior.
◼ Timer and alarms implementation.
 Defining Threads

◼ Example: Web/FTP Server

Main Thread
while <running>
{
<wait for request>
<create a new worker thread>
<start the thread>
Execution Timeline
} <request 1>
Web/FTP
server Worker
Thread <request 2>

<request N>
Worker
Thread
Worker
Thread
 Defining Threads
◼ A Thread is a piece of code that runs in
concurrent with other threads.
◼ Each thread is a statically ordered sequence
of instructions.
◼ Threads are used to express concurrency on
both single and multiprocessors machines.
◼ Programming a task having multiple threads
of control is called: Multithreading or
Multithreaded Programming.
 Java Threads

◼ Java has built in support for Multithreading

◼ Synchronization
◼ Thread Scheduling
◼ Inter-Thread Communication:
◼ currentThread start setPriority
◼ yield run getPriority
◼ sleep stop suspend
◼ resume
◼ Java Garbage Collector is a low-priority
thread.
22
 Threading Mechanisms...
◼ Create a class that extends the Thread class
◼ Create a class that implements the Runnable
interface

Thread Runnable Thread

MyThread MyClass

(objects are threads) (objects with run() body)

[a] [b]

23
 1st method: Extending Thread
class
◼ Create a class by extending Thread class and override
run() method:
class MyThread extends Thread
{
public void run()
{
// thread body of execution
}
}
◼ Create a thread:
MyThread thr1 = new MyThread();
◼ Start Execution of threads:
thr1.start();
◼ Create and Execute:
new MyThread().start();

24
 An example
class MyThread extends Thread {
public void run() {
System.out.println(" this thread is running ... ");
}
}

class ThreadEx1 {
public static void main(String [] args ) {
MyThread t = new MyThread();
t.start();
}
}

25
 2nd method: Threads by
implementing Runnable interface
◼ Create a class that implements the interface Runnable and
override run() method:
class MyThread implements Runnable
{
.....
public void run()
{
// thread body of execution
}
}
◼ Creating Object:
MyThread myObject = new MyThread();
◼ Creating Thread Object:
Thread thr1 = new Thread( myObject );
◼ Start Execution:
thr1.start();

26
 An example
class MyThread implements Runnable {
public void run() {
System.out.println(" this thread is running ... ");
}
}

class ThreadEx2 {
public static void main(String [] args ) {
Thread t = new Thread(new MyThread());
t.start();
}
}

27
 Life Cycle of Thread
new
start()
I/O completed

ready
Time expired/ resume()
notify() interrupted

sleeping blocked
waiting
dispatch
sleep()
wait() suspend()
running Block on I/O
completion

stop() dead
28
A Program with Three Java Threads

◼ Write a program that creates 3 threads

29
 Three threads example
◼ class A extends Thread
◼ {
◼ public void run()
◼ {
◼ for(int i=1;i<=5;i++)
◼ {
◼ System.out.println("\t From ThreadA: i= "+i);
◼ }
◼ System.out.println("Exit from A");
◼ }
◼ }

◼ class B extends Thread

◼ {
◼ public void run()
◼ {
◼ for(int j=1;j<=5;j++)
◼ {
◼ System.out.println("\t From ThreadB: j= "+j);
◼ }
◼ System.out.println("Exit from B");
◼ }
◼ }

30
 Three threads example
◼ class C extends Thread
◼ {
◼ public void run()
◼ {
◼ for(int k=1;k<=5;k++)
◼ {
◼ System.out.println("\t From ThreadC: k= "+k);
◼ }

◼ System.out.println("Exit from C");

◼ }
◼ }

◼ class ThreadTest
◼ {
◼ public static void main(String args[])
◼ {
◼ new A().start();
◼ new B().start();
◼ new C().start();
◼ }
◼ }

31
 Run 1
◼ [raj@mundroo] threads [1:76] java ThreadTest
From ThreadA: i= 1
From ThreadA: i= 2
From ThreadA: i= 3
From ThreadA: i= 4
From ThreadA: i= 5
Exit from A
From ThreadC: k= 1
From ThreadC: k= 2
From ThreadC: k= 3
From ThreadC: k= 4
From ThreadC: k= 5
Exit from C
From ThreadB: j= 1
From ThreadB: j= 2
From ThreadB: j= 3
From ThreadB: j= 4
From ThreadB: j= 5
Exit from B

32
 Run 2
◼ [raj@mundroo] threads [1:77] java ThreadTest
From ThreadA: i= 1
From ThreadA: i= 2
From ThreadA: i= 3
From ThreadA: i= 4
From ThreadA: i= 5
From ThreadC: k= 1
From ThreadC: k= 2
From ThreadC: k= 3
From ThreadC: k= 4
From ThreadC: k= 5
Exit from C
From ThreadB: j= 1
From ThreadB: j= 2
From ThreadB: j= 3
From ThreadB: j= 4
From ThreadB: j= 5
Exit from B
Exit from A

33
 Thread Priority

◼ In Java, each thread is assigned a priority,

which affects the order in which it is scheduled
for execution. The threads so far had the same
default priority (NORM_PRIORITY) and they are
served using FCFS (First Come and First
Served) policy.
◼ Java allows users to change priority:
◼ ThreadName.setPriority(intNumber)
◼ MIN_PRIORITY = 1
◼ NORM_PRIORITY=5
◼ MAX_PRIORITY=10

34
 Thread Priority Example
class A extends Thread
{
public void run()
{
System.out.println("Thread A started");
for(int i=1;i<=4;i++)
{
System.out.println("\t From ThreadA: i= "+i);
}
System.out.println("Exit from A");
}
}
class B extends Thread
{
public void run()
{
System.out.println("Thread B started");
for(int j=1;j<=4;j++)
{
System.out.println("\t From ThreadB: j= "+j);
}
System.out.println("Exit from B");
}
}

35
 Thread Priority Example
class C extends Thread
{
public void run()
{
System.out.println("Thread C started");
for(int k=1;k<=4;k++)
{
System.out.println("\t From ThreadC: k= "+k);
}
System.out.println("Exit from C");
}
}
class ThreadPriority
{
public static void main(String args[])
{
A threadA=new A();
B threadB=new B();
C threadC=new C();
threadC.setPriority(Thread.MAX_PRIORITY);
threadB.setPriority(threadA.getPriority()+1);
threadA.setPriority(Thread.MIN_PRIORITY);
System.out.println("Started Thread A");
threadA.start();
System.out.println("Started Thread B");
threadB.start();
System.out.println("Started Thread C");
threadC.start();
System.out.println("End of main thread");
}
}

36
 Assignment 1 at a Glance: Multithreaded
Dictionary Server – Using Sockets and Threads

A Client Program Meaning (“guru”)

Meaning(“guru”)? Multithreaded
Dictionary Server
“master or teacher”

A Client Program
Meaning(“love”)?

A Client
Program in “C” A Client
Meaning(“java”)? Program in “C++”
Meaning(“channel”)?

37
Use of Sockets in Assignment 1:
A Suggestion
◼ First time:
◼ Use TCP Sockets
◼ Prior background in TCP Sockets:
◼ Use UDP Sockets
◼ Full details –released as part of:
◼ “Assignment 1 Guideline” Document

38
Accessing Shared Resources

◼ Applications access to shared resources need to

be coordinated.
◼ Printer (two person jobs cannot be printed at the
same time)
◼ Simultaneous operations on your bank account.
◼ Can the following operations be done at the same
time on the same account?
◼ Deposit()
◼ Withdraw()
◼ Enquire()

39
 Online Bank: Serving Many
Customers and Operations

Internet
Bank Server
Client 1 Bank Operator 1

Bank Local Area Network

Client 2

Bank Operator M

Client N Bank Database

 Shared Resources

◼ If one thread tries to read the data and other

thread tries to update the same data, it leads to
inconsistent state.
◼ This can be prevented by synchronising access
to the data.
◼ Use “synchronized” method:
◼ public synchronized void update()
◼ {
◼ …
◼ }

41
 the driver: 3 Threads sharing the
same object
class InternetBankingSystem {
public static void main(String [] args ) {
Account accountObject = new Account ();
Thread t1 = new Thread(new MyThread(accountObject));
Thread t2 = new Thread(new YourThread(accountObject));
Thread t3 = new Thread(new HerThread(accountObject));
t1.start();
t2.start();
t3.start();
// DO some other operation
} // end main()
}

42
 Shared account object
between 3 threads
class MyThread implements Runnable {
Account account;
public MyThread (Account s) { account = s;}
public void run() { account.deposit(); }
} // end class MyThread

class YourThread implements Runnable { account

Account account;
public YourThread (Account s) { account = s;}
(shared
public void run() { account.withdraw(); } object)
} // end class YourThread

class HerThread implements Runnable {

Account account;
public HerThread (Account s) { account = s; }
public void run() {account.enquire(); }
} // end class HerThread
43
 Monitor (shared object access):
serializes operation on shared objects
class Account { // the 'monitor'
int balance;

// if 'synchronized' is removed, the outcome is unpredictable

public synchronized void deposit( ) {
// METHOD BODY : balance += deposit_amount;
}

public synchronized void withdraw( ) {

// METHOD BODY: balance -= deposit_amount;
}
public synchronized void enquire( ) {
// METHOD BODY: display balance.
}
}

44
Architecture for Multithread Servers

◼ Multithreading enables servers to maximize

their throughput, measured as the number of
requests processed per second.
◼ Threads may need to treat requests with
varying priorities:
◼ A corporate server could prioritize request
processing according to class of customers.
◼ Architectures:
◼ Worker pool
◼ Thread-per-request
◼ Thread-per-connection
◼ Thread-per-object

45
 Client and server with threads
(worker-pool architecture)

Thread 2 makes
requests to server
Receipt & Input-output
queuing
Thread 1
generates
results
T1
Requests
N threads
Client
Server

◼ In worker-pool architectures, the server creates a fixed pool of worker

threads to process requests.
◼ The module “receipt and queuing” receives requests from sockets/ports
and places them on a shared request queue for retrieval by the workers.

46
 Alternative server threading
architectures

per-c onnect ion threads per-obj ec t threads

work ers

I/O remote I/O remote

remote objects
objects objects

a. Thread-per-reques t b. Thread-per-c onnecti on c. Thread-per-object

Server associates a Thread Associates Thread with each

IO Thread creates a with each connection and object. An IO thread
new worker thread for destroys when client closes receives request and queues
each request and the connection. them for workers, but this
worker thread Client may make many time there is a per-object
destroys itself after requests over the queue.
serving the request. connection.
47
Invocations between address
spaces
(a ) Sys te m ca ll Thr ead Contr ol tr ansfer v ia
trap ins tr uction

Contr ol tr ansfer v ia
pr iv ileg ed in struc tions
User Ker nel

Pr otection domain
(b ) RPC/RMI ( within one c omputer ) bou ndar y

Thr ead 1 Thr ead 2

User 1 Ker nel User 2

(c) RPC/RM I (be twee n co mput ers)

Thr ead 1 Net work Thr ead 2

Use r 1 Use r 2
Ker nel 1 Ker nel 2

49
 Summary
◼ Operating system provides various types of facilities to support
middleware for distributed system:
◼ encapsulation, protection, and concurrent access and management of
node resources.
◼ Multithreading enables servers to maximize their throughput,
measured as the number of requests processed per second.
◼ Threads support treating of requests with varying priorities.
◼ Various types of architectures can be used in concurrent
processing:
◼ Worker pool
◼ Thread-per-request
◼ Thread-per-connection
◼ Thread-per-object
◼ Threads need to be synchronized when accessing and
manipulating shared resources.
◼ New OS designs provide flexibility in terms of separating
mechanisms from policies.
52
 References

◼ CDK Book (Textbook)

◼ Chapter 7 – “Operating System Support”
◼ Chapter 14: Multithread Programming
◼ R. Buyya, S. Selvi, X. Chu, “Object
Oriented Programming with Java:
Essentials and Applications”, McGraw
Hill, New Delhi, India.

53