Queuing models in an Operating System (OS) are mathematical models that help manage and optimize the way processes are scheduled, resources are allocated, and input/output (I/O) requests are handled. These models play an important role in ensuring efficient CPU scheduling algorithms, process scheduling, and resource management in OS. By organizing tasks in queues, the OS can prioritize and schedule them for execution, minimizing latency and maximizing throughput.
Components of Queuing System
A queuing system typically includes the following elements:
- Arrival process: The arrival process describes how customers enter the system.
- Server: The server is the entity which provides the service to the customers.
- Queue: Customers who are waiting for service are held in a queue.
- Service discipline: The order in which customers are served is determined by service discipline.
- Service time distribution: The amount of time required to serve a customer is described as service time distribution.
- Departure process: The departure process describes how customers exit the system once they have been served.
- System performance measures: These are used to analyze and evaluate the system's performance.
Number of Servers
The number of servers in a queuing system can vary depending on the application and the level of service desired. In some cases, a single server may suffice, whereas, in others, multiple servers may be required to meet demand.
- Single-server queuing systems: These systems involve customers queuing for service from a single server, commonly seen in simple applications like retail stores or fast-food restaurants.
- Multi-server queuing systems: These systems are used in high-demand scenarios with multiple servers handling the workload, such as a call center with several agents managing incoming calls.
Notation for Queues
In queueing theory, Kendall's notation and A/S/c/K/N/D notation are widely used to describe queues and their characteristics.
1. Kendall's Notation
This notation is a three-letter code that represents a queue's key components: the arrival process, the service distribution, and the number of servers.
For example, an M/M/1 queue describes a system with Markovian arrivals (Poisson process), exponential service time, and a single server.
2. A/S/c/K/N/D Notation
This notation provides more flexibility. A represents the arrival process (like deterministic or Markovian arrivals), S stands for the service distribution (such as exponential or Erlang service), and c indicates the number of servers. K and N refer to the system capacity and population size, while D stands for the service discipline
For example, FIFO, LIFO, priority queues, preemptive queues, or non-preemptive queues)
Queuing Models
Following are 4 examples of common queuing models. There are many other variations and extensions that can be used to analyze more complex systems.
1. M/M/1 Queue
In an M/M/1 queue, customers arrive following a Poisson distribution (random arrival times), and the server provides exponential service times. There is one server and a First-Come-First-Served (FCFS) service rule. This type of queue is also known as an M/M/1/FCFS queue or an M/M/1/FIFO (first-in-first-out) queue.
It is one of the most basic and widely studied queueing models in queuing theory, and it is frequently used as a starting point for understanding the performance of more complex queueing systems. To analyze and evaluate an M/M/1 queue, several performance measures are commonly used. Among them most important measures are:
- The average number of customers in the system.
- The average waiting time in the queue.
- The system utilization.
- The probability of a customer finding the server busy.
Example:
The arrival rate of customers is 2 per minute, and the service rate of each server is 3 per minute.
The probability of the system being empty is calculated using the Erlang-C formula: P0 = (2/(2+3)) = 0.4
The average number of customers in the system is calculated as: L = (2/3) / (1-(2/(2+3))) = 0.8
The average time a customer spends in the system is calculated as: W = 1 / (3 - 2) = 1 minute
The probability of there being n customers in the system is calculated using the Poisson distribution with lambda = 2.
This is just a high-level example and parameters such as arrival, service rate or utilization are often used to solve this in real-world scenarios.
2. M/M/1 with Limited Queue (M/M/1/N)
This model is similar to M/M/1, but with a limited queue length (N). If the queue reaches its maximum, new customers are blocked or rejected. It’s called Balking or Reneging when customers leave if the queue is full. Balking means when customer refuses to join a queue upon arrival if the queue is too long or the wait time is unacceptable. Reneging is when a customer joins a queue but leaves prematurely before being served due to impatience or other reasons. An M/M/1/FCFS/N or M/M/1/FIFO/N queue is another name for this type of queuing system.
It is a variant of the basic M/M/1 queue with a limited buffer capacity, which means that the number of customers in the system is limited by N. When the buffer is full, additional customers may be blocked or rejected, complicating the analysis and necessitating the modification of some performance measures to include the blocked/rejected customers. Some of the most important performance measures that can be calculated, similar to an M/M/1 system, are:
- The average number of customers in the system (including those blocked or rejected).
- The average waiting time in the queue.
- The system utilization.
- The probability of a customer finding the server busy.
- The probability of customers being blocked/reneged.
Example:
The arrival rate of customers is 2 per minute, the service rate of each server is 3 per minute and the maximum queue length is N=5.
The probability of the system being empty is calculated using the Erlang-C formula: P0 = (2/(2+3)) = 0.4
The average number of customers in the system is calculated as: L = (2/3) / (1-(2/(2+3))) = 0.8
The average time a customer spends in the system is calculated as: W = 1 / (3 - 2) = 1 minute
The probability of there being n customers in the queue is calculated using the Poisson distribution with lambda = 2, But with the condition that once the queue reaches the limit (n=N), the additional customers will be lost or rejected.
The probability of rejection P_R = P(n > N) = probability of customers waiting in the queue when n > N = P(n= N+1) + P(n= N+2) + ... (by using Poisson formula)
Note that when the queue length is limited, and if the arrival rate exceeds the service rate, it will eventually leads to a high rejection rate, and customers will experience long waiting time before being served.
3. M/D/1 Queue
In an M/D/1 queue, the arrival process is Poisson, but service times are deterministic (constant). This means customers are always served in exactly the same time. This is also known as an M/D/1/FCFS or M/D/1/FIFO queue, where FCFS or FIFO denotes the first-come-first-served service discipline. This type of queuing system is useful for simulating situations where customer service times are known in advance and are consistent, such as a carwash service.
The queue will be stable if the arrival rate is less than the service rate, otherwise, it will be unstable due to the deterministic service time. The performance measures for this system are similar to those for the M/M/1 queue, but because the service time is deterministic, closed-form solutions for these measures are frequently easier to obtain. Among the most important measures are:
- The average number of customers in the system.
- The average waiting time in the queue.
- The system utilization.
- The probability of a customer finding the server busy.
Example:
The arrival rate of customers is 2 per minute and the service rate of each server is 3 per minute.
The utilization of the server is calculated as: U = 2 / (2 + 3) = 0.4
The probability of the system being empty is calculated using the Erlang-B formula: P0 = (2 / 3) / (1 + (2 / 3)) = 0.4
The average number of customers in the system is calculated as: L = 2 * (2 / (2 + 3)) / (1 - (2 / (2 + 3))) = 0.8
The average time a customer spends in the system is calculated as: W = 1 / (3 - 2) = 1 minute
This is just a high-level example, in practice there are other parameters that can be used in real-world scenarios such as the average service time, variance of service time. Also, one important aspect to consider is that M/D/1 is a specific case of a more general queueing system called G/D/1 where the service time is general and not necessarily follows a Markov Process.
4. M/M/c Queue
The M/M/c queue is a queuing system where customer arrivals follow a Poisson distribution (M) and service times are exponentially distributed (M). This model is also known as M/M/c/FCFS or M/M/c/FIFO, where FCFS (First-Come-First-Served) denotes the service discipline. The system has c servers, and customers are served concurrently by these servers.
It is also known as the Erlang-c queue. This type of queuing system is useful for simulating situations in which multiple servers provide service. This system's performance metrics are similar to those of the M/M/1 queue. However, some measures are more difficult to calculate because the number of servers c influences the queue's behavior. The most important measures are as follows:
- The average number of customers in the system.
- The average waiting time in the queue.
- The system utilization.
- The probability of a customer finding the server busy.
- The probability of customers waiting in the queue.
- The probability of customers being blocked.
Example:
The arrival rate of customers is λ per minute, the service rate of each server is μ per minute, and there are c=3 servers.
The utilization of the servers is calculated as U = λ / (c*μ)
The probability of the system being empty is calculated using Erlang-C formula: P0 = (λ/(cμ))^c / (c! * (1-(λ/(cμ))) ), where c! = c*(c-1)(c-2)...1
The average number of customers in the system is calculated as: L = (λ/(c*(μ-λ)))
The average time a customer spends in the system is calculated as: W = 1 / (c*(μ-λ))
In practice there are other parameters that can be used in real-world scenarios such as utilization, number of customer in queue, in service, probability of n customer in the system, and etc.
It's worth mentioning that for large value of c the system is considered as an Erlang-C queue and this approximation is becoming accurate. Also, this formula is only valid when the arrival rate is less than the service rate (λ < μ), so the system is stable and the queue length remains finite.
Queue Discipline
The order in which customers are served in a queuing system is referred to as queue discipline. In practice, there are several queue disciplines that are used, including:
- First-In-First-Out (FIFO): Customers are served in the order in which they arrive (first-in, first-out). This is the most commonly used queue discipline in retail stores, fast-food restaurants, and other similar establishments.
- Last-In-First-Out (LIFO): Customers are served in reverse order of arrival (last-in-first-out, or LIFO). This discipline is less commonly used than FIFO, but it can be found in some applications, such as a stack of plates in a cafeteria.
- Priority: Customers are served in accordance with their priority level. Customers with the highest priority are served first, followed by customers with lower priority. This discipline is used in situations where certain customers, such as in an emergency room or a customer service call center, must be served before others.
- Random: Customers are served at random.
- Shortest Job first (SJF): Customers are served based on the time required to complete their service, with the shortest jobs served first.
- Processor sharing: Processor sharing means that all customer requests are treated equally and receive an equal share of the server's time.
Each queue discipline has advantages and disadvantages, so the discipline chosen will be determined by the system requirements and performance objectives.
Similar Reads
Microkernel in Operating Systems
A MicroKernel is an approach to designing an Operating System (OS). The microkernel provides very fundamental services required to run the OS like basic memory management, task scheduling, etc. Microkernels use Inter-Process Communication (IPC) for communication. In this article, we will discuss Ker
8 min read
Kernel in Operating System
A kernel is the core part of an operating system. It acts as a bridge between software applications and the hardware of a computer. The kernel manages system resources, such as the CPU, memory, and devices, ensuring everything works together smoothly and efficiently. It handles tasks like running pr
10 min read
Six-State Process Model in Operating System
In this article, we are going to discuss the Six-State Process Model in Operating Systems and we will also understand what was the need for introducing these process models, what are the states present in these process models, and all the possible transitions that can occur in these models. Need for
4 min read
5 State Process Model in Operating System
In an operating system (OS), managing how programs run and interact with system resources is crucial for efficient performance. The 5-state process model is a fundamental framework used by OSs to categorize and control the behavior of processes, which are individual instances of programs running on
6 min read
Multi Processing Operating System
The operating system functions like a manager of all the available resources. Therefore operating system is defined as an interface between the system and the user. There are various types of operating systems such as Batch Operating Systems, Multi-programming Operating Systems, distributed operatin
4 min read
Handheld Operating System
An operating system is a program whose job is to manage a computer's hardware. Its other use is that it also provides a basis for application programs and acts as an intermediary between the computer user and the computer hardware. An amazing feature of operating systems is how they vary in accompli
5 min read
Two State Process Model in Operating System
The process in an operating system passes from different states starting from its formation to its completion. A process consists of program data and its associated data and a process control block (PCB). A process may change its state because of the following events like I/O requests, interrupt rou
5 min read
IoT Operating Systems
An Operating System is a software program software that manages and controls all the resources in a computer, together with the hardware and the software program. A running system is chargeable for controlling, dealing with, and coordinating all the sports and resources in a computer. An operating m
13 min read
Process in Operating System
A process is a program in execution. For example, when we write a program in C or C++ and compile it, the compiler creates binary code. The original code and binary code are both programs. When we actually run the binary code, it becomes a process. A process is an 'active' entity instead of a progra
3 min read
Resource Management in Operating System
Resource Management in Operating System is the process to manage all  the resources efficiently like CPU, memory, input/output devices, and other hardware resources among the various programs and processes running in the computer. Resource management is an important thing because resources of a comp
3 min read