Os Notes
Os Notes
Chapter-2
Process and Thread
2.1.Introduction
2.2 PROCESSES
A process is basically a program in execution. The execution of a process must
progress in a sequential fashion. A process is defined as an entity which represents the
basic unit of work to be implemented in the system. o put it in simple terms, we write
our computer programs in a text file and when we execute this program, it becomes a
process which performs all the tasks mentioned in the program. When a program is
loaded into the memory and it becomes a process, it can be divided into four sections ─
stack, heap, text and data. The following image shows in fig.2.2. a simplified layout of a
process inside main memory.
void main()
{
printf("Welcome to Operating System By Er.Manoj Kavedia! \n");
getch();
}
A computer program is a collection of instructions that performs a specific task when
executed by a computer. When we compare a program with a process, we can conclude
that a process is a dynamic instance of a computer program. We emphasize that a
program by itself is not a process; a program is a passive entity, such as a file containing a
list of instructions stored on disk (often called an executable file), whereas a process is an
active entity, with a program counter specifying the next instruction to execute and a set
running at that time. It eventually is scheduled when it starts executing. At that time its
state is changed to running.
Running - Instruction Being Executed:
Once the process has been assigned to a processor by the OS scheduler, the process
state is set to running and the processor executes its instructions. Is the only process
which is executed by the CPU at any given moment. In multiprocessor systems with
multiple CPUs however, them will be many running processes and the Operating
System will have to keep track of all of them.
Blocked / Waiting - The process is waiting for some event to occur:
When a process is waiting for an external event such as an I/0 operation, the process
is said to be in a blocked state. Process moves into the waiting state if it needs to wait for
a resource, such as waiting for user input, or waiting for a file to become available. The
major difference between a blocked and a ready process is that a blocked process cannot
be directly scheduled even if the CPU is free, whereas, a ready process can be scheduled
if the CPU is free.
Terminated or Exit or Release:
Once the process finishes its execution, or it is terminated by the operating system, it
is moved to the terminated state where it waits to be removed from main memory.
Similarly after the process terminates, the Operating System can put it in the halted
state before actually removing all details about it. In UNIX, this state is called the
Zombie state
2.2.2 Process Control Block:
The Operating System maintains the information about each process in a record or a
data structure called Process Control Block (PCB) as shown in Fig.2.4. Each user process
has a PCB. It is created when user creates a process and it is removed from the system
when the process is killed. All these PCBs art kept in the memory reserved for the
Operating System.
Process management is an integral part of any modern day operating system (OS).
The OS must allocate resources to processes, enable processes to share and exchange
information, protect the resources of each process from other processes and enable
synchronisation among processes. To meet these requirements, the OS must maintain a
data structure for each process, which describes the state and resource ownership of that
process, and which enables the OS to exert control over each process.
Different process management models are
(1) Two State Process management model
(2) Three state Process management model
(3) Five State Process management model
2.3 THREADS
A thread is a flow of execution through the process code, with its own program
counter that keeps track of which instruction to execute next, system registers which
hold its current working variables, and a stack which contains the execution history.
A thread shares with its peer threads few information like code segment, data
segment and open files. When one thread alters a code segment memory item, all other
threads see that.
A thread is also called a lightweight process. Threads provide a way to improve
application performance through parallelism. Threads represent a software approach to
improving performance of operating system by reducing the overhead thread is
equivalent to a classical process.
Each thread belongs to exactly one process and no thread can exist outside a process.
Each thread represents a separate flow of control. Threads have been successfully used
in implementing network servers and web server. They also provide a suitable
foundation for parallel execution of applications on shared memory multiprocessors.
The following figure shows the working of a single-threaded and a multithreaded
process.
Advantages:
The most obvious advantage of this technique is that a user-level threads package can
be implemented on an Operating System that does not support threads. Some other
advantages are
User-level threads does not require modification to operating systems.
Simple Representation: Each thread is represented simply by a PC, registers, stack
and a small control block, all stored in the user process address space.
Simple Management: This simply means that creating a thread, switching between
threads and synchronization between threads can all be done without intervention
of the kernel.
Fast and Efficient: Thread switching is not much more expensive than a procedure
call.
Disadvantages:
There is a lack of coordination between threads and operating system kernel.
Therefore, process as whole gets one time slice irrespect of whether process has one
thread or 1000 threads within. It is up to each thread to relinquish control to other
threads.
User-level threads requires non-blocking systems call i.e., a multithreaded kernel.
Otherwise, entire process will blocked in the kernel, even if there are runable
threads left in the processes. For example, if one thread causes a page fault, the
process blocks.
Kernel Level Threads:
In this case, thread management is done by the Kernel. There is no thread
management code in the application area. Kernel threads are supported directly by the
operating system. Any application can be programmed to be multithreaded. All of the
threads within an application are supported within a single process.
The Kernel maintains context information for the process as a whole and for
individuals threads within the process. Scheduling by the Kernel is done on a thread
basis. The Kernel performs thread creation, scheduling and management in Kernel
space. Kernel threads are generally slower to create and manage than the user threads.
Advantages:
Kernel can simultaneously schedule multiple threads from the same process on
multiple processes.
If one thread in a process is blocked, the Kernel can schedule another thread of the
same process.
Kernel routines themselves can be multithreaded.
Disadvantages:
Kernel threads are generally slower to create and manage than the user threads.
Transfer of control from one thread to another within the same process requires a
mode switch to the Kernel.
Advantages of Threads over Multiple Processes:
Context Switching Threads are very inexpensive to create and destroy, and they
are inexpensive to represent. For example, they require space to store, the PC, the
SP, and the general-purpose registers, but they do not require space to share
memory information, Information about open files of I/O devices in use, etc. With
so little context, it is much faster to switch between threads. In other words, it is
relatively easier for a context switch using threads.
Sharing Treads allow the sharing of a lot resources that cannot be shared in
process, for example, sharing code section, data section, Operating System resources
like open file etc.
Disadvantages of Threads over Multiprocesses:
Blocking The major disadvantage if that if the kernel is single threaded, a system
call of one thread will block the whole process and CPU may be idle during the
blocking period.
Security Since there is, an extensive sharing among threads there is a potential
problem of security. It is quite possible that one thread over writes the stack of
another thread (or damaged shared data) although it is very unlikely since threads
are meant to cooperate on a single task.
Multithreading Models:
Some operating system provide a combined user level thread and Kernel level
thread facility. Solaris is a good example of this combined approach. In a combined
system, multiple threads within the same application can run in parallel on multiple
processors and a blocking system call need not block the entire process. Multithreading
models are three types
Many to many relationship.
Many to one relationship.
One to one relationship.
Many to Many Model:
The many-to-many model as shown fig.2.11. multiplexes any number of user
threads onto an equal or smaller number of kernel threads.
The following diagram shows the many-to-many threading model where 6 user level
threads are multiplexing with 6 kernel level threads. In this model, developers can create
as many user threads as necessary and the corresponding Kernel threads can run in
parallel on a multiprocessor machine. This model provides the best accuracy on
concurrency and when a thread performs a blocking system call, the kernel can schedule
another thread for execution. IRIX, HP-UX, and Tru64 UNIX use the two-tier model, as
did Solaris prior to Solaris 9.
Thread Libraries:
Thread libraries provide programmers with an API for creating and managing
threads.
Thread libraries may be implemented either in user space or in kernel space. The
former involves API functions implemented solely within user space, with no
kernel support. The latter involves system calls, and requires a kernel with thread
library support.
There are three main thread libraries in use today:
(1) POSIX Pthreads - may be provided as either a user or kernel library, as an
extension to the POSIX standard.
(2) Win32 threads - provided as a kernel-level library on Windows systems.
(3) Java threads - Since Java generally runs on a Java Virtual Machine, the
implementation of threads is based upon whatever OS and hardware the JVM
is running on, i.e. either Pthreads or Win32 threads depending on the system.
Difference between Thread and Process:
No. Process Thread
1 Process is heavy weight or resource Thread is light weight, taking lesser
intensive. resources than a process.
2 Process switching needs interaction Thread switching does not need to interact
with operating system. with operating system.
3 In multiple processing All threads can share same set of open files,
environments, each process executes child processes.
the same code but has its own
memory and file resources.
4 If one process is blocked, then no While one thread is blocked and waiting, a
other process can execute until the second thread in the same task can run.
first process is unblocked.
5 Multiple processes without using Multiple threaded processes use fewer
threads use more resources. resources.
6 In multiple processes each process One thread can read, write or change
operates independently of the others another thread's data.
Difference Between User Level and Kernel level Thread:
No. User Level Thread Kernel Thread
1 User-level threads are faster to Kernel-level threads are slower to create
create and manage and manage.
2 Implementation is by a thread Operating system supports creation of
library at the user level. Kernel threads.
3 User-level thread is generic and can Kernel-level thread is specific to the
run on any operating system. operating system.
4 Multi-threaded applications cannot Kernel routines themselves can be
take advantage of multiprocessing. multithreaded.
Benefits of Thread:
(1) Responsiveness:
Multithreading an interactive application may allow a program to continue running
even if part of it is blocked or is performing a lengthy operation, thereby increasing
responsiveness to the user. For instance, a multithreaded web browser could still allow
user interaction in one thread while an image is being loaded in another thread.
(2) Resource sharing:
By default, threads share the memory and the resources of the process to which they
belong. The benefit of code sharing is that it allows an application to have several
different threads of activity all within the same address space.
(3) Economy:
It is much more time consuming to create and manage process than threads, therefore
allocating memory and resources for process creation is costly. Because threads share
resources of the process to which they belong, it is more economical to create and
context switch threads. In Solaris 2, creating a process is about 30 times slower than is
creating a thread, and context switching is about five times slower.
(4) Utilization of multiprocessor architectures:
The benefits of multithreading can be greatly increased in a multiprocessor
architecture, where each thread may be running in parallel on a different processor. A
single-threaded process can only run on one CPU, no matter how many are available.
Multithreading on a multi-CPU machine increases concurrency. In a single processor
architecture, the CPU generally moves between each thread so quickly as to create an
illusion of parallelism, but in reality only one thread is running at a time.
(5) Shared Memory: This allows the interchange of data through a defined area of
memory. Semaphore value has to be obtained before data can get access to shared
memory.
(6) Sockets: This method is mostly used to communicate over a network, between a
client and a server. It allows for a standard connection which I computer and
operating system independent.
Facility Provided by IPC
(1) IPC provides a mechanism to allow Processes to communicate and to synchronize
their actions without sharing the same address space.
(2) IPC is particularly useful in a distributed environment where the communicating
processes may reside on different computers connected with a network. An
example is a chat program used on the World Wide Web.
IPC is best provided by a message-passing system, and message systems can be
defined in many ways.
Message Passing:
The function of a message system is to allow processes to communicate with one
another without the need to resort to shared data. We have already seen message
passing used as a method of communication in microkernel‘s. In this scheme, services
are provided as ordinary user processes. That is the services operate outside of the
kernel. Communication among the user process is accomplished through the passing of
messages. An IPC facility provides at least the two operations:
send(message) and
receive(message).
Messages sent by a process can be of either
fixed or
variable size.
Fixed Sized Message:
When fixed-sized messages are sent, the system-level implementation is
straightforward ie easy. But makes the task of programming more difficult.
Variable Sized Message:
If variable-sized messages needs a more complex system-level implementation, but
the programming task becomes simpler.
Example:
If processes X and Y want to communicate, they must send messages to and receive
messages from each other, there exist a link between two process . There are variety of
ways to implement this link. Here physical implementation is not of much concern (such
as shared memory, hardware bus, or network), but logical implementation is more
important. There are several methods for logically implementing a link and the
send/receive operations:
(1) Direct or indirect communication
(2) Symmetric or asymmetric communication
(3) Automatic or explicit buffering
(4) Send by copy or send by reference
(5) Fixed-sized or variable-sized messages
2.5 SCHEDULING
Operating systems may feature up to three distinct types of schedulers:
(1) A long-term scheduler (also known as an admission scheduler or high-level
scheduler),
(2) A mid-term or medium-term scheduler and
(3) A short-term scheduler (also known as a dispatcher). The names suggest the
relative frequency with which these functions are performed.
Long-term Scheduler:
The long-term, or admission, scheduler decides which jobs or processes are to be
admitted to the ready queue; that is, when an attempt is made to execute a program, its
admission to the set of currently executing processes is either authorized or delayed by
the long-term scheduler.
Thus this scheduler dictates what processes are to run on a system, and the degree of
concurrency to be supported at any one time - ie whether a high or low amount of
processes are to be executed concurrently, and how the split between IO intensive and
CPU intensive processes is to be handled. In modern OS's, this is used to make sure that
real time processes get enough CPU time to finish their tasks. Without proper real time
scheduling, modern GUI interfaces would seem sluggish.
Long-term scheduling is also important in large-scale systems such as batch
processing systems, computer clusters, supercomputers and render farms. In these cases,
special purpose job scheduler software is typically used to assist these functions, in
addition to any underlying admission scheduling support in the operating system.
Mid-term Scheduler:
The mid-term scheduler, present in all systems with virtual memory, temporarily
removes processes from main memory and places them on secondary memory (such as
a disk drive) or vice versa. This is commonly referred to as "swapping out" or "swapping
in" (also incorrectly as "paging out" or "paging in"). The mid-term scheduler may decide
to swap out a process for example
(1) Process which has not been active for some time, or
(2) A process which has a low priority, or
(3) A process which is page faulting frequently, or
(4) A process which is taking up a large amount of memory in order to free up main
memory for other processes, swapping the process back in later when more
memory is available, or when the process has been unblocked and is no longer
waiting for a resource.
In many systems today (those that support mapping virtual address space to
secondary storage other than the swap file), the mid-term scheduler may actually
perform the role of the long-term scheduler, by treating binaries as "swapped out
processes" upon their execution. In this way, when a segment of the binary is required it
can be swapped in on demand, or "lazy loaded".
Short-term Scheduler:
The short-term scheduler (also known as the dispatcher) decides which of the ready,
in-memory processes are to be executed (allocated a CPU) next following a clock
interrupt, an IO interrupt, an operating system call or another form of signal.
Thus the short-term scheduler makes scheduling decisions much more frequently
than the long-term or mid-term schedulers - a scheduling decision will at a minimum
have to be made after every time slice, and these are very short. This scheduler can be
preemptive, implying that it is capable of forcibly removing processes from a CPU when
it decides to allocate that CPU to another process, or non-preemptive (also known as
"voluntary" or "co-operative"), in which case the scheduler is unable to "force" processes
off the CPU.
The scheduler selects from among the processes in memory that are ready to execute,
and allocates the CPU to one of them. The ready queue is not necessarily a first-in, first-
out (FIFO) queue.
As we shall see when we consider the various scheduling algorithms, a ready queue
may be implemented as a FIFO queue, a priority queue, a tree, or simply an unordered
linked list.
Conceptually, however, all the processes in the ready queue are lined up waiting for
a chance to run on the CPU. The records in the queues are generally process control
blocks (PCBs) of the processes.
Types of Scheduling:
Below given are the circumstances under which CPU Scheduling takes place.
(1) When a process switches from the running state to the waiting state (for
example, I/O request, or invocation of wait for the termination of one of the
child processes).
(2) When a process switches from the running state to the ready state (for example,
when an interrupt occurs).
(3) When a process switches from the waiting state to the ready state (for example,
completion of I/O).
(4) When a process terminates.
Hence there are two types of scheduling:
Non preemptive Scheduling
Preemptive Scheduling
Non preemptive Scheduling:
In circumstances 1 and 4, there is no choice in terms of scheduling. When scheduling
takes place only under circumstances 1 and 4, we say the scheduling scheme is
nonpreemptive; Under nonpreemptive scheduling, once the CPU has been allocated to a
process, the process keeps the CPU until it releases the CPU either by terminating or by
switching to the waiting state. This scheduling method is used by the Microsoft
Windows 3.1 and by the Apple Macintosh operating systems. It is the only method that
can be used on certain hardware platforms, because it does not require the special
hardware (for example, a timer) needed for preemptive scheduling
Example of Non Preemptive Scheduling Algorithm:
(1) First-come-first served
Preemptive Scheduling:
A new process (if one exists in the ready queue) must be selected for execution. There
is a choice, however, in circumstances 2 and 3. otherwise, the scheduling scheme is
preemptive. But preemptive scheduling incurs a cost. Consider the case of two
processes sharing data. One may be in the midst of updating the data when it is
preempted and the second process is run. The second process may try to read the data,
which are currently in an inconsistent state. New mechanisms thus are needed to
coordinate access to shared data; Preemption also has an effect on the design of the
operating-system kernel. Shown in fig.4.2.
During the processing of a system call, the kernel may be busy with an activity on
behalf of a process. Such activities may involve changing important kernel data (for
instance, I/O queues). What happens if the process is preempted in the middle of these
changes, and the kernel (or the device driver) needs to read or modify the same
structure.
Preemptive scheduling:
(1) Shortest-job-first
(2) Round Robin.
write and understand. The average waiting time under the FCFS policy, however, is
often quite
long. Consider the following set of processes that arrive at time 0, with the length of
the CPU-burst time given in milliseconds:
Process Burst Waiting Time
P1 24 0
P2 3 24
P3 7 27
P4 13 34
P5 21 47
If the processes arrive in the order PI, P2, P3,P4,P5 and are served in FCFS order, we
get the result shown in the following Gantt chart:
P1 /24 P2 / 3 P3 / 7 P4 / 13 P5 /21
0 24 27 34 47
The waiting time is
0 milliseconds for process P1
24 milliseconds for process P2
27 milliseconds for process P3
34 milliseconds for process P3
47 milliseconds for process P3
Thus when process arrives as P1,P2, P3,P4,P5 the average waiting time is
= (0 + 24 + 27 + 34 + 47 )/5
= 132 /5 milliseconds.
= 26.4 millisecond.
If the processes arrive in the order P3, P4,P1,P5,P2 the average waiting time is
= (0+7+20+44+65)/5
=136/5 milliseconds
= 27.2 millisecond.
From above two calculation it is clear that one process need 26.4msec and other need
27.2 msec. The average waiting time under a FCFS policy is generally not minimal, and
may vary substantially if the process CPU-burst times vary greatly.
Disadvantage of FCFS:
(1) Average waiting time is very large.
(2) FCFS is not an attractive alternative on its own for a single processor system.
Another difficulty with FCFS tends to favor processor bound, processes over I/O
bound processes and may result in inefficient use of both the processor and the I/O
devices.
Short Job First Scheduling (SJF):
Shortest-job-first (SJF) another scheduling algorithm. In this algorithm as soon as the
CPU is available, it is assigned to the process that has the smallest next CPU burst. If two
processes have the same length next CPU burst, FCFS scheduling is used to server the
process which came first in the queue.
Note that a more appropriate term would be the shortest next CPU burst, because the
scheduling is done by examining the length of the next CPU burst of a process, rather
than its total length.
Consider the following set of Five processes, with the length of the CPU-burst time
given in milliseconds
Process Burst
P1 24
P2 3
P3 7
P4 13
P5 21
Now the processes will arrive in the order P2, P3, P4,P5,P1 and are served in SJP
order, we get the result shown in the following Gantt chart:
P2 /3 P3 / 7 P4/ 13 P5 / 21 P1 /24
0 3 10 23 44
The waiting time is
0 milliseconds for process P2
3 milliseconds for process P3
10 milliseconds for process P4
23 milliseconds for process P5
44 milliseconds for process P1
Thus when process arrives as P2,P3, P4,P5,P1 the average waiting time is
= (0 + 3 +10 + 23 + 44 )/5
= 80 /5 milliseconds.
= 16 millisecond.
Conclusion:
The SJF scheduling algorithm is optimal because, it gives the minimum average
waiting time for a given set of processes. By moving a short process before a long one,
the waiting time of the short process decreases more than it increases the waiting time of
the long process. Consequently, the average waiting time decreases.
The SJF algorithm may be either preemptive or nonpreemptive. The choice arises when a
new process arrives at the ready queue while a previous process is executing. The new
process may have a shorter next CPU burst than what is left of the currently executing
process.
A preemptive SJF algorithm will preempt the currently executing process, whereas a
nonpreemptive SJF algorithm will allow the currently running process to finish its CPU
burst. Preemptive SJF scheduling is sometimes called shortest-remaining-time-first
scheduling.
Advantage and Disadvantage:
(3) otherwise, if the CPU burst of the currently running process is longer than 1 time
quantum, the timer will go off and will cause an interrupt to the operating
system.
A context switch will be executed, and the process will be put at the tail of the ready
queue. The CPU scheduler will then select the next process in the ready queue.
The average waiting time under the RR policy, however, is often quite long. Consider
the following set of processes that arrive at time 0, with the length of the CPU-burst time
given in milliseconds:
Process Burst
P1 24
P2 3
P3 3
Le the time quantum of 4 milliseconds , then how takes place
(1) Process P1 gets the first 4 milliseconds. Since it requires another 20 milliseconds,
it is preempted after the first time quantum, and
(2) the CPU is given to the next process in the queue, process P2. Since process P2
does not need 4 milliseconds, it quits before its time quantum expires.
(3) The CPU is then given to the next process, process P3. Once each process has
received 1 time quantum, the CPU is returned to process P1 for an additional
time quantum. The resulting RR schedule is
P1 P2 P3 P4 P1 P1 P1 P1 P1
0 4 7 10 14 18 22 26 30
The average waiting time is
= 17/3
= 5.66 milliseconds.
In the RR scheduling algorithm, no process is allocated the CPU for more than 1 time
quantum in a row. If a process' CPU burst exceeds 1 time quantum, that process is
preempted and is put back in the ready queue. Hence the RR scheduling algorithm is
preemptive.
If there are n processes in the ready queue and the time quantum is q, then each
process gets 1/n of the CPU time in chunks of at most q time units. Each process must
wait no longer than (n - 1) x q time units until its next time quantum.
For example, if there are five processes, with a time quantum of 20 milliseconds, then
each process will get up to 20 milliseconds every 100 milliseconds.
Advantages and Disadvantages of RR scheduling
(1) Round robin is particularly effective In a general purpose time sharing system or
transaction processing systems.
(2) If there is a mix of processor bound and 1/0 bound processes, then the processor
bound processes tend to receive an unfair portion of processor time, which result In
poor performance for I/O bound processes, inefficient use of I/O devices and an
increase in the variance of response time.
Chapter-3
Memory Management
3.1 INTRODUCTION
Main memory (RAM) is an important resource that must be very carefully managed..
Today‘s need for a programmer is private, infinitely large, infinitely fast memory that
is also nonvolatile, that is, does not lose its contents when the electric power is switched
off. But today‘s technology cannot provide such memory in bulk at less cost ie such
memories are expensive.
Hence the concept of a memory hierarchy shown in fig.3.1. is discovered, in which
computers have a
few megabytes of very fast,
expensive, volatile cache memory,
a few gigabytes of medium-speed, medium-priced, volatile main memory, and
a few terabytes of slow, cheap, nonvolatile magnetic or solid-state disk storage,
not to mention removable storage, such as DVDs and USB sticks.
The part of the operating system that manages (part of) the memory hierarchy is
called the memory manager.
Job of Memory Manager
is to efficiently manage memory:
keep track of which parts of memory are in use,
allocate memory to processes when they need it, and
deallocate it when they are done.
3.2 NO MEMORY ABSTRACTION
Early mainframe computers (before 1960), early minicomputers (before 1970), and
early personal computers (before 1980) had no memory abstraction. Only physical
memory was used , no mapping , nothing. When a program executed an instruction like
MOV R4,1000 the computer just moved the contents of physical memory location
1000 to REGISTER4.
Thus, the model of memory presented to the programmer was simply physical
memory, a set of addresses from 0 to some maximum, each address corresponding to a
cell containing some number of bits, commonly byte or group of 8 bits. Hence running
of multiple program was not possible at time.
If the first program wrote a new value to, say, location 000, this would erase whatever
value the second program was storing there. By any means if two program share same
memory then nothing would work and both programs would crash almost immediately.
Models (a) and (c) have the disadvantage that a bug in the user program can wipe out
the operating system, possibly with disastrous results. When the system is organized in
this way, generally only one process at a time can be running. As soon as the user types
a command,
The operating system copies the requested program from disk to memory and
executes it.
When the process finishes, the operating system displays a prompt character and
waits for a user new command.
When the operating system receives the command, it loads a new program into
memory, overwriting the first one.
One way to get some parallelism in a system with no memory abstraction is to
program with multiple threads. Since all threads in a process are supposed to see the
same memory image, the fact that they are forced to is not a problem. While this idea
works, it is of limited use since what people often want is unrelated programs to be
running at the same time, something the threads abstraction does not provide.
3.2.1 Running Multiple Programs without a Memory Abstraction
It is possible to run multiple programs with no memory abstraction at the same time.
Here the operating system save the entire contents of memory to a disk file, then bring
one program in and execute the next program. As long as there is only one program at a
time in memory, there are no conflicts. Brining in one program and executing and then
sending back to disk is called as swapping.
3.2.2 Running Multiple Program without Swapping
It is possible to run multiple programs concurrently, even without swapping with
some additional hardware. This was implemented in IBM 370 Model.
Here memory was divided into 2-KB blocks
Each was assigned a 4-bit protection key held in special registers inside the CPU.
A machine with a 1-MB memory needed only 512 of these 4-bit registers for a
total of 256 bytes of key storage.
This technique has major drawback, shown in Fig. 3-3. Here we have two programs,
each 16 KB in size, as shown in Fig. 3-3(a) and (b). The first is shaded to indicate that it
has a different memory key than the second. The first program starts out by jumping to
address 24, which contains a MOV instruction.
the number 28 to REGISTER1 must not be relocated. The loader needs some way to tell
what is an address and what is a constant.
IPv4 addresses are 32-bit numbers, so their address space runs from 0 to 2 32 -1 (again,
with some reserved numbers).Address spaces do not have to be numeric. The set of .com
Internet domains is also an address space. This address space consists of all the strings
of length 2 to 63 characters that can be made using letters, numbers, and hyphens,
followed by .com.
Types of address space:
The concept of a logical address space that is bound to a separate physical address
space is central to proper memory management
Logical address: Generated by the CPU; also referred to as virtual address
Physical address: Address seen by the memory unit
Logical and physical addresses are the same in compile: Time and load-time
address-binding schemes; logical (virtual) and physical addresses differ in
execution-time address-binding scheme
3.4.1 Implementation:
Base and Limit Registers:
Fig. 3.5 Base and limit registers can be used to give each process a separate address
space
Dynamic relocation technique is used to map each process‘ address space onto a
different part of physical memory in a simple way. This technique is used on machines
ranging from the CDC 6600 (the world‘s first supercomputer) to the Intel 8088 (the heart
of the original IBM PC).
To achieve these each CPU has two special registers, usually called the base and
limit registers. When these registers are used, programs are loaded into consecutive
memory locations wherever there is space in memory and without relocation during
loading, as shown in Fig. 3-4.
register contents added to it before being sent to memory. In many implementations, the
base and limit registers are protected in such a way that only the operating system can
modify them. Intel 8088/86 have multiple base registers, allowing program text and
data, for example, to be independently relocated, but offered no protection from out-of-
range memory references. A disadvantage of relocation using base and limit registers is
the need to perform an addition and a comparison on every memory reference.
Comparisons can be done fast, but additions are slow due to carry-propagation time
unless special addition circuits are used.
3.4.2 Swapping:
Swapping is a mechanism in which a process can be swapped temporarily out of
main memory (or move) to secondary storage (disk) and make that memory available
to other processes. At some later time, the system swaps back the process from the
secondary storage to main memory.
Though performance is usually affected by swapping process but it helps in running
multiple and big processes in parallel and that's the reason Swapping is also known as
a technique for memory compaction.
= 200 milliseconds
Now considering in and out time, it will take complete 400 milliseconds plus other
overhead where the process competes to regain main memory.
When swapping creates multiple holes in memory, it is possible to combine them all
into one big one by moving all the processes downward as far as possible. This
technique is known as memory compaction. It is usually not done because it requires a
lot of CPU time. For example, on a 16-GB machine that can copy 8 bytes in 8 nsec, it
would take about 16 sec to compact all of memory.
3.4.3 Managing Free Memory:
Since there is only a limited amount of disk space, it is necessary to reuse the space
from deleted files for new files. To keep track of free disk space, the system maintains a
free-space list. The free-space list records all disk blocks that are free (i.e., are not
allocated to some file). To create a file, the free-space list has to be searched for the
required amount of space, and allocate that space to a new file. This space is then
removed from the free-space list. When a file is deleted, its disk space is added to the
free-space list.
When memory allocation takes place dynamically, it is responsibility of an operating
system to manage it properly. In free space management techniques mainly two
methods are used as follows:
(1) Bitmap
(2) Linked List
(3) Grouping
(4) Counting
3.4.3.1. Bit Map technique:
Frequently, the free-space list is implemented as a bit map or bit vector shown in
fig.3.8. Each block is represented by a 1 bit. If the block is free, the bit is 0; if the block is
allocated, the bit is 1.
For example, consider a disk where blocks 2, 3, 4, 5, 8, 9, 10, 11, 12, 13, 17, 18, 25, 26,
and 27 are free, and the rest of the blocks are allocated. The free-space bit map would be:
11000011000000111001111110001111…
There is one problem associated with bitmap that when k unit processes runs in the
memory, the memory manager has to search a ‗0‘ bitmap which are consecutive ‗n‘
units.
The main argument against bitmap is that the searching a bitmap for a run of a given
length is a slow operation.
3.4.3.2. Linked Lists Technique:
Linked list is another way using which track of free and used memory can be tracked.
Link list is collection of allocated memory or hole between two processes. As shown in
figure P indicates the process and H indicates the hole(free memory). All the entries
present in the list specifies a hole (H) process (P) and address at which it starts, the
length and a pointer to the next entry.
From the example shown in fig.3.9. This list is always kept sorted. One of the major
advantages of sorting is that when a process terminates or when a process terminates or
when a process is swapped out the updating list is straight forward.
There could be for combinations as shown in the Fig. (a) shows modifying the list
requires replacing a P by H. Fig.(b) and (c)also two entries and combined into one and
the list becomes one entry shorter. Doubly link list is useful to find out most recent entry
and to see if merge is possible. When processes or even holes are kept sorted in the list,
then different algorithms could be used to allocate the memory. These algorithm are
called as
(i) First Fit
(ii) Best Fit
(iii) Next Fit
(iv) Worst Fit
3.4.3.3. Grouping:
A modification of the free-list approach is to store the addresses of n free blocks in the
first free block. The first n-1 of these are actually free. The last one is the disk address of
another block containing addresses of another n free blocks. The importance of this
implementation is that addresses of a large number of free blocks can be found quickly.
3.4.3.4. Counting:
Another approach is to take advantage of the fact that, generally, several contiguous
blocks may be allocated or freed simultaneously, particularly when contiguous
allocation is used. Thus, rather than keeping a list of free disk addresses, the address of
the first free block is kept and the number n of free contiguous blocks that follow the
first block.
Each entry in the free-space list then consists of a disk address and a count. Although
each entry requires more space than would a simple disk address, the overall list will be
shorter, as long as the count is generally greater than 1.
When we want to execute the process, we swap it into memory. A lazy swapper is
used to swap the page, which is needed, instead of swapping the whole process. The
page is swapped strictly if it is needed.
We can view the process as a sequence of pages rather than a large address space
Instead of swapper we use pager, which is concern with the individual pages of the
process.
When a process is to be swapped in, the pager guesses which pages will be used
before the process is swapped out again. Instead of swapping in a whole process, the
pager brings only those necessary pages into memory. It avoids reading into memory
pages that will not be used anyway, decreasing the swap time and amount of physical
memory needed.
The hardware to support demand paging is same as that is required for paging and
swapping as follows
(1) Page table
(2) Secondary memory
(3) Software needed to solve page fault problem
In demand paging if we guess right and demand for only those pages that are
actually needed, then process will rim exactly as per our expectation. But if in case,
process tries to access a page that was not brought into memory, then access to this page
is called as a page fault.
Advantages:
(1) Demand paging, as opposed to loading all pages immediately.
(2) Only loads pages that are demanded by the executing process.
(3) When a process is swapped out (context switch) of memory, only those pages
loaded in main memory need to be swapped out from main memory to secondary
storage.
(4) As there is more space in main memory, more processes can be loaded reducing
context switching time which utilizes large amounts of resources.
(5) Less loading latency occurs at program startup, as less information is accessed from
secondary storage and less information is brought into main memory.
(6) Does not need extra hardware support than what paging needs, since protection
fault can be used to get page fault.
Disadvantages:
(1) Individual programs face extra latency when they access a page for the first time. So
demand paging may have lower performance than anticipatory paging algorithms
such as prepaging, a method of remembering which pages a process used when it
last executed and preloading a few of them, is used to improve performance.
(2) Programs running on low-cost, low-power embedded systems may not have a
memory management unit that supports page replacement.
(3) Memory management with page replacement algorithms becomes slightly more
complex.
(4) Possible security risks, including vulnerability to timing attacks;
3.5.6 Page Table:
A page table is the data structure used by a virtual memory system in a computer
operating system to store the mapping between virtual addresses and physical
addresses. Virtual addresses are those unique to the accessing process. Physical
addresses are those unique to the CPU, i.e., RAM.
(A)
(B)
Fig. 3.19 Translation Process
The operating system will have an interrupt handler to deal with such page faults.
The handler will typically look up the address mapping in the page table to see whether
a mapping exists. If one exists, it is written back to the TLB (this must be done, as the
hardware accesses memory through the TLB in a virtual memory system), and the
faulting instruction is restarted. This subsequent translation will find a TLB hit, and the
memory access will continue.
3.5.8 Valid Valid-Invalid Bit Invalid Bit:
With each page table entry a valid–invalid bit is associated.(1 - in-memory, 0 - not-in-
memory) Initially valid–invalid but is set to 0 on all entries shown in fig.3.20.
During address translation, if valid–invalid bit in page table entry is 0 - page fault
not milliseconds as with page replacement). The reason for the shorter time scale is
that cache block misses are satisfied from main memory, which has no seek time
and no rotational latency.
(2) A second example is in a Web server. The server can keep a certain number of
heavily used Web pages in its memory cache. However, when the memory cache is
full and a new page is referenced, a decision has to be made which Web page to
evict. The considerations are similar to pages of virtual memory, except for the fact
that the Web pages are never modified in the cache, so there is always a fresh copy
on disk. In a virtual memory system, pages in main memory may be either clean or
dirty.
3.6.1 Different Page Replacement Algorithms
When total memory requirement in demand paging exceed the physical memory,
then there is need to replace pages from memory to free frames for new pages. For
replacement of pages various techniques are used.
(1) FIFO Page Replacement
(2) Optimal Page Replacement
(3) LRU Page Replacement
(4) NRU Not recently used
(5) Clock Page Replacement Algorithm
3.6.1.1. First-In, First-Out (FIFO) Page Replacement Algorithm:
Oldest page in main memory is the one which will be selected for replacement. Easy
to implement, keep a list, replace pages from the tail and add new pages at the head.
FIFO algorithm low-overhead paging algorithm. The operating system maintains a list
of all pages currently in memory, with the page at the head of the list the oldest one and
the page at the tail the most recent arrival. On a page fault, the page at the head is
removed and the new page added to the tail of the list.
Example of FIFO:
In all our examples, shown in Fig.3.23. the reference string is
2,3,2,1,5,2,4,5,3,2,5,2
.
Fig. 3.23 FIFO Page Replacement Algorithm
Example-1
In all our examples, Shown in Fig.3.24. the reference string is
7,0,1,2,0,3,0,4,2,3,0,3,2,1,2,0,1,7,0,1.
Drawback of FIFO:
Process execution is slow
3.7 SEGMENTATION
Segmentation is a memory management technique in which each job is divided into
several segments of different sizes, one for each module that contains pieces that
perform related functions. Each segment is actually a different logical address space of
the program.
When a process is to be executed, its corresponding segmentation are loaded into
non-contiguous memory though every segment is loaded into a contiguous block of
available memory.
Segmentation memory management works very similar to paging but here segments
are of variable-length where as in paging pages are of fixed size.
Segmented memory allocation. Job 1 includes a main program, subroutine A, and subroutine B, so it’s divided into three segments.
Chapter-4
File System
4.1 INTRODUCTION
A file can be described as a container that stores data that is accessible by a computer
system - it is basically a container holding a set of related information that will be stored
on some form of secondary storage. A data file may consist of for example, a set of
computer program instructions, text or another form of data. A text file may contain just
a short letter, or contain a complete book. In other words a file is basically a defined set
of named data.
A file is a named collection of related information that is recorded on secondary
storage. Also, a file is the smallest allotment of logical secondary storage; that is, data
cannot be written to secondary storage unless they are within a file.
Information may be stored in are:
file-source programs,
object programs,
executable programs,
numeric data,
text,
payroll records,
graphic images,
Sound recordings.
4.1.1 File Naming:
The rules for file naming vary somewhat from system to system, but all current
operating systems allow strings of one to eight letters as legal file names (in MSDOS).
Many file systems support names as long as 255 characters(in Windows). Some file
systems distinguish between upper- and lowercase letters, whereas others do not. Unix
considers upper and lower case as different while MSDOS consider both as same. Hence
vipul.txt and VIPUL.txt will be different file in unix but in MSDOS it 2 will considered
as same.
Many operating systems support two-part file names, with the two parts separated
by a period, as in vipul.txt. The part following the period is called the file extension and
usually indicates something about the file. In MS-DOS, for example, file names are 1 to 8
characters, plus an optional extension of 1 to 3 characters.
In UNIX, the size of the extension, if any, is up to the user, and a file may even have
two or more extensions, as in vipul.html.zip, where .html indicates a Web page in HTML
and .zip indicates that the file (vipul.html) has been compressed using the zip program.
Some of the more common file extensions and their meanings are shown in Fig. 4-1.
Extension Meaning
.bak Backup file
.c C source program
.gif Compuserve Graphical Interchange Format Image
.hlp Help file
3. Tree
Stream of Bytes:
For operating system regard files is just sequences of bytes and provides the
maximum amount of flexibility. User programs can put any type of data they want in
their files and accordingly. All versions of UNIX (including Linux and OS X) and
Windows use this file model. The main advantage of such file structuring is that it
simplifies file management for the operating system and Applications can have their
own structure.
file is clearly quite different from the unstructured byte streams used in UNIX and
Windows and is used on some large mainframe computers for commercial data
processing.
4.1.3 File Type:
The types of files recognized by the system are either regular, directory, or special.
However, the operating system uses many variations of these basic types. The following
basic types of files exist:
1 Regular Contain user information, Stores data (text, binary, and executable).All
files are regular file.
2 Directory Contains information used to access other files. for maintaining the
structure of the file system.
3 Special Character special files are related to input/output and used to model
serial I/O devices, such as terminals, printers, and networks. Block
special files are used to model disks.
Regular files:
Regular files are the most common files and are used to contain data. Regular files are
in the form of text files or binary files:
Regular files are generally either ASCII files or binary files. ASCII files consist of lines
of text. In some systems each line is terminated by a carriage return character. In others,
the line feed character is used. Some systems (e.g., Windows) use both. Lines need not
all be of the same length.
The great advantage of ASCII files is that they can be displayed and printed as is, and
they can be edited with any text editor.
Text files:
Text files are regular files that contain information stored in ASCII format text and
are readable by the user. You can display and print these files. The lines of a text file
must not contain NUL characters, and none can exceed {LINE_MAX} bytes in length,
including the new line character.
The term text file does not prevent the inclusion of control or other nonprintable
characters (other than NUL).
Binary files:
Binary files are regular files that contain information readable by the computer.
Binary files might be executable files that instruct the system to accomplish a job.
Commands and programs are stored in executable, binary files. Special compiling
programs translate ASCII text into binary code.
Directory files:
Directory files contain information that the system needs to access all types of files,
but directory files do not contain the actual file data. As a result, directories occupy less
space than a regular file and give the file system structure flexibility and depth. Each
directory entry represents either a file or a subdirectory. Each entry contains the name of
the file and the file's index node reference number (i-node number). The i-node number
points to the unique index node assigned to the file. The i-node number describes the
location of the data associated with the file. Directories are created and controlled by a
separate set of commands.
Special files:
Special files define devices for the system or are temporary files created by processes.
These files are also known as device files. These files represent physical device like
disks, terminals, printers, networks, tape drive etc.
These files are of two types:
Character special files: Data is handled character by character as in case of
terminals or printers.
Block special files: Data is handled in blocks as in the case of disks and tapes.
4.1.4 File Access:
Access Methods:
Purpose of file is to store information. This information must be accessed and read
into computer memory. The information in the file can be accessed in several ways.
Some systems provide only one access method for files and some provide many
methods of access.
For example IBM support many access methods, and But the problem is selection of
the right method for a particular application. Different access methods are
Sequential access method
Direct access method
Indexed access method
Sequential access method:
The simplest access method is sequential access. Information in the file is processed
in order, one record after the other. This mode of access is the most common method; for
example, editors and compilers usually access files in this fashion.
Given a logical record length L, a request for record N is turned into an I/O request
for L bytes starting at location L * (N -1) within the file (assuming first record is N = 1).
Since logical records are of a fixed size, it is also easy to read, write, or delete a record.
Indexed Access Method:
These methods generally involve the construction of an index for the file. The index,
similar to an index in the back of a book, contains pointers to the various blocks. To find
a record in the file, user has to first search the index, and then use the pointer to access
the file directly and to find the desired record.
First Name Logical
Record
Number
Manoj
Kaushal Asha Manager 37 KYN
Rishabh
Asha
Vimla
A file is named, for the convenience of its human users, and is referred to by its name.
A name is usually a string of characters, such as examp1e.java or .vbp. Some systems
differentiate between upper- and lowercase characters in names, whereas other systems
consider the two cases to be equivalent.
A file has certain other attributes, which vary from one operating system to another,
but typically consist of these:
Name
Identifier
Type
Location
Size
Protection
Time of creation
Date of creation, and
user identification
Name:
The symbolic file name is the only information kept in human readable form.
Identifier:
This unique tag, usually a number, identifies the file within the file system; it is the
non-human-readable name for the file.
Type:
This information is needed for those systems that support different types.
Location:
This information is a pointer to a device and to the location of the file on that device.
Size:
The current size of the file (in bytes, words, or blocks), and possibly the maximum
allowed size are included in this attribute.
Protection:
Access-control information determines who can do reading, writing, executing, and
so on.
Time, date of creation and user identification:
This information may be kept for creation, last modification, and last use. These data
can be useful for protection, security, and usage monitoring.
The information about all files is kept in the directory structure that also resides on
secondary storage. Typically, the directory entry consists of the file's name and its
unique identifier. The identifier in turn locates the other file attributes.
It may take more than a kilobyte to record this information for each file. In a system
with many files, the size of the directory itself may be megabytes.
Attribute Meaning
Protection Who can access the file and in what way
Password Password needed to access the file
Creator ID of the person who created the file
location in the file where the next write is to take place. The write pointer must be
updated whenever a write occurs.
Reading a file:
To read from a file, we use a system call that specifies the name of the file and where
(in memory) the next block of the file should be put. Again, the directory is searched for
the associated directory entry, and the system needs to keep a read pointer to the
location in the file where the next read is to take place.
Once the read has taken place, the read pointer is updated. A given process is usually
only reading or writing a given file, and the current operation location is kept as a per-
process current-file-position pointer. Both the read and write operations use this same
pointer, saving space and reducing the system complexity.
Repositioning within a file:
The directory is searched for the appropriate entry, and the current-file-position is set
to a given value. Repositioning within a file does not need to involve any actual I/O.
This file operation is also known as a file seek.
Deleting a file:
To delete a file, we search the directory for the named file. Having found the
associated directory entry, we release all file space, so that it can be reused by other files,
and erase the directory entry.
Truncating a file:
The user may want to erase the contents of a file but keep its attributes. Rather than
forcing the user to delete the file and then recreate it, this function allows all attributes to
remain unchanged-except for file length-but lets the file be reset to length zero and its
file space released.
Other operation which are performed on file:
Other common operations include
(1) Appending a file
(2) Renaming a file
(3) Creating a Copy
(4) Searching
Appending a File:
An appending new information to the end of an existing file . This call is a restricted
form of write. It can add data only to the end of the file. Systems that provide a minimal
set of system calls rarely have append, but many systems provide multiple ways of
doing the same thing, and these systems sometimes have append.
Renaming a File:
Renaming an existing file ie changing the name of the file for example vipul.txt is
renamed to new name vipul.pub.
Creating a copy of File:
Basic operations may then be combined to perform other file operations. For instance,
creating a copy of a file, or copying the file to another I/O device, such as a printer or a
display, may be accomplished by creating a new file, and reading from the old and
writing to the new.
We also want to have operations that allow a user to get and set the various attributes
of a file. For example, we may want to have operations that allow a user to determine
the status of a file, such as the file's length, and allow a user to set file attributes, such as
the file's owner.
Searching:
The file operations mentioned involve searching the directory for the entry associated
with the named file. To avoid this constant searching, many systems require that an
open system call be used before that file is first used actively. The operating system
keeps a small table containing information about all open files (the open-file table).
When a file operation is requested, the file is specified via an index into this table, so no
searching is required. When the file is no longer actively used, it is closed by the process
and the operating system removes its entry in the open-file table.
4.1.7 Information Associated with an Open File:
Several pieces of information are associated with an open file.
(1) File Pointer
(2) File open count
(3) Disk location of the file
(4) Access rights
File pointer:
On systems that do not include a file offset as part of the read and write system calls,
the system must track the last read-write location as a current-file-position pointer. This
pointer is unique to each process operating on the file, and therefore must be kept
separate from the on-disk file attributes.
File open count:
As files are closed, the operating system must reuse its open-file table entries, or it
could run out of space in the table. Because multiple processes may open a file, the
system must wait for the last file to close before removing the open-file table entry. This
counter tracks the number of opens and closes and reaches zero on the last close. The
system can then remove the entry.
Disk location of the file:
Most file operations require the system to modify data within the file. The
information needed to locate the file on disk is kept in memory to avoid having to read it
from disk for each operation.
Access rights:
Each process opens a file in an access mode. This information is stored on the per-
process table so the operating system can allow or deny subsequent I/O requests.
4.2 DIRECTORIES
The file systems of computers can be extensive. Some systems store Megabytes of
files on terabytes of disk. To manage all these data, we need to organize them. When
there are many file searching time increases hence in order to save that time
organization of data can be done on disk.
Directory structure
Organizes and provides
Prof.Manoj S. Kavedia (www.kavedasir.yolasite.com)(9324258878/9860174297)
information (e.g., name, size6
location, and type) on all the7
files in the system
Operating System Notes
partitions, or slices 2
Partition C
Di
sk
1
Both the directory Di
structure and the Directory sk
files reside on disk 3
files
Partition B
space for files. Hence for this reason partitions can be thought of as virtual
disks.
(c) Partitions can also store multiple operating systems, allowing a system to
boot and run more than one.
(2) Each partition contains information about files within it. This information is kept
in entries in a device directory or volume table of contents.
The device directory (more commonly known simply as a directory) records
information-such as name, location, size, and type-for all files on that partition. Figure
4.9 shows the typical file-system organization.
The directory can be viewed as a symbol table that translates file names into their
directory entries. If we take such a view, we see that the directory itself can be organized
in many ways. The most common schemes for defining the logical structure of a
directory.
Single Level Directory
Two-Level Directory
Tree Directory Structure
4.2.1 Single Level Directory:
The simplest directory structure is the single-level directory. All files are contained in
the same directory, which is easy to support and understand shown in Figure.4.11.
A single-level directory has significant limitations, however when the number of files
increases or when the system has more than one user. Since all files are in the same
directory, they must have unique names. If two users call their data file test, then the
unique-name rule is violated.
user job starts or a user logs in, the system's Master File Directory (MFD) is searched.
The MFD is indexed by user name or account number, and each entry points to the UFD
for that user. When a user refers to a particular file, only his own UFD is searched. Thus,
different users may have files with the same name, as long as all the file names within
each UFD are unique.
Tree-Structured
Directories
Subdirectories
Efficient searching
Grouping
Capability
Current directory
(working directory)
In the tree-structured directory, the directory themselves are files. This leads to the
possibility of having sub-directories that can contain files and sub-subdirectories shown
in fig.4.12.
An interesting policy decision in a tree-structured directory structure is how to
handle the deletion of a directory. If a directory is empty, its entry in its containing
directory can simply be deleted. However, suppose the directory to be deleted id not
empty, but contains several files, or possibly sub-directories. Some systems will not
delete a directory unless it is empty.
Thus, to delete a directory, someone must first delete all the files in that directory. If
these are any sub-directories, this procedure must be applied recursively to them, so that
they can be deleted also. This approach may result in a insubstantial amount of work.
An alternative approach is just to assume that, when a request is made to delete a
directory, all of that directory's files and sub-directories are also to be deleted.
The Microsoft Windows family of operating systems (95,95, NT, 2000) maintains an
extended two-level directory structure, with devices and partitions assigned a drive
letter.
4.2.4 Acyclic-Graph Directories:
Nevertheless, typing
cp /usr/lib/dictionary dictionar y
also works fine, as does
cp /usr/lib/dictionary /usr/ast/dictionar y
All of these do exactly the same thing.
Other Examples:
If a file name begins with only a disk designator but not the backslash after the colon,
it is interpreted as a relative path to the current directory on the drive with the specified
letter. Note that the current directory may or may not be the root directory depending
on what it was set to during the most recent "change directory" operation on that disk.
Examples of this format are as follows:
"C:tmp.txt" refers to a file named "tmp.txt" in the current directory on drive C.
"C:tempdir\tmp.txt" refers to a file in a subdirectory to the current directory on
drive C.
A path is also said to be relative if it contains "double-dots"; that is, two periods
together in one component of the path. This special specifier is used to denote the
directory above the current directory, otherwise known as the "parent directory".
Examples of this format are as follows:
"..\tmp.txt" specifies a file named tmp.txt located in the parent of the current
directory.
"..\..\tmp.txt" specifies a file that is two directories above the current directory.
"..\tempdir\tmp.txt" specifies a file named tmp.txt located in a directory named
tempdir that is a peer directory to the current directory
4.2.6 Directory Operation:
System call for operations of directories varies from system to system. Following are
the some system calls for managing directory used in unix oepating system
Create: A directory is created. It is empty except for dot and dotdot, which are
put there automatically by the system.
Delete: A directory is deleted. Only an empty directory can be deleted. A
directory containing only dot and dotdot is considered empty as these cannot be
deleted.
Opendir: Directories can be read. For example, to list all the files in a directory, a
listing program opens the directory to read out the names of all the files it
contains.
Closedir: When a directory has been read, it should be closed to free up internal
table space.
Readdir: This call returns the next entry in an open directory. Formerly, it was
possible to read directories using the usual read system call, but that approach
has the disadvantage of forcing the programmer to know and deal with the
internal structure of directories.
Rename: It renames the directory.
Link: Linking is a technique that allows a file to appear in more than one
directory. This system call specifies an existing file and a path name, and creates
a link from the existing file to the name specified by the path. In this way, the
same file may appear in multiple directories. A link of this kind, which
increments the counter in the file‘s i-node (to keep track of the number of
directory entries containing the file), is sometimes called a hard link.
Unlink: A directory entry is removed. If the file being unlinked is only present in
one directory (the normal case), it is removed from the file system. If it is present
in multiple directories, only the path name specified is removed.
4.3 FILE SYSTEM IMPLEMENTATION
4.3.1 File System Layout:
Disk on which operating system can be single or multiple partition. File system is
stored on this Disk. Each partition has independent File System. Sector 0 of disk is called
as Master Boot Record, Booting information is stored in this Sector. The end of the MBR
contains the partition table. This table gives the starting and ending addresses of each
partition. Out of Many partition One of the partitions in the table is marked as active.
This active partition help computer in booting. Control from BIOS is transferred to this
MBR by boot strap loader. The first block of MBR is also called as Boot Block.
.
Fig. 4.17 Linked File Allocation
be considered as an overhead of the method. In indexed allocation, each file has its own
index block, which is an array of disk sector of addresses.
addresses of the file‘s blocks. A simple example is shown in Fig. 4-20. Given the i-node,
it is then possible to find all the blocks of the file.
The main advantage of this scheme over linked files using an in-memory table is that
the i-node need be in memory only when the corresponding file is open. If each inode
occupies n bytes and a maximum of k files may be open at once, the total memory
occupied by the array holding the i-nodes for the open files is only kn bytes. But this
space need be reserved in advance.
The attributes of file system are generally stored with directory entry as shown in
Fig. 4-21(a). following are different ways of implementing
In this simple design, a directory consists of a list of fixed-size entries, one per
file, containing a (fixed-length) file name, a structure of the file attributes,
One or more disk addresses (up to some maximum) telling where the disk blocks
are.
For systems that use i-nodes, another possibility for storing the attributes is in
the i-nodes, rather than in the directory entries.
In that case, the directory entry can be shorter: just a file name and an i-node
number. This approach is illustrated in Fig. 4-21(b).
Linear Search
Hashing
Caching
Linear Search:
Generally directories are searched linearly from beginning to end when a file name
has to be looked up. For extremely long directories, linear searching can be slow. Hence
to speed up search is to use a hash table in each directory. For the size of the table n. To
enter a file name, the name is hashed onto a value between 0 and n-1,
Hashing:
The table entry corresponding to the hash code is inspected. If it is unused, a pointer
is placed there to the file entry. File entries follow the hash table. If that slot is already in
use, a linked list is constructed, headed at the table entry and threading through all
entries with the same hash value.
While searching the file name is hashed to select a hash-table entry. All the entries on
the chain headed at that slot are checked to see if the file name is present. If the name is
not on the chain, the file is not present in the directory. The disadvantage of more
complex administration but searching is faster.
Caching:
Yet another way to speed up searching large directories is to cache the results of
searches. Before starting a search, a check is first made to see if the file name is in the
cache. If so, it can be located immediately.
4.3.4 Virtual File System:
Modern operating systems now a days concurrently support multiple types of file
systems. The method of implementing multiple types of file systems is to write directory
and file routines for each type. Instead, however, most operating systems, including
UNIX, use object-oriented techniques to simplify, organize, and modularize the
implementation. The use of these methods allows very dissimilar file-system types to be
implemented within the same structure, including network file systems, such as NFS.
Users can access files that are contained within multiple file systems on the local disk or
even on file systems available across the network.
Data structures and procedures are used to isolate the basic system call functionality
from the implementation details. Thus, the file-system implementation consists of three
major layers, as depicted schematically in Figure 4.26.
(1) The first layer is the file-system interface, based on the open(), read(), write(), and
close() calls and on file descriptors.
(2) The second layer is called the virual File System (VFS) layer. The VFS layer
serves two important functions:
It separates file-system-generic operations from their implementation by
defining a clean VFS interface. Several implementations for the VFS interface
may coexist on the same machine, allowing transparent access to different
types of file systems mounted locally.
It provides a mechanism for uniquely representing a file throughout a
network. The VFS is based on a file-representation structure, called a vnode
that contains a numerical designator for a network-wide unique file. (UNIX
inodes are unique within only a single file system.) This network-wide
blocks. To store all these addresses at 255 per block requires about 4 million blocks.
Generally, free blocks are used to hold the free list, so the storage is essentially free.
Another approach is to link all the free disk blocks together, keeping a pointer to the
first free block. This block contains a pointer to the next free disk block, and so on. For
example, consider a disk where blocks 2, 3, 4, 5, 8, 9, 10, 11, 12, 13, 17, 18, 25, 26, and 27
are free, and the rest of the blocks are allocated, a pointer could be kept to block 2, as the
first free block. Block 2 would contain a pointer to block 3, which would point to block 4,
which would point to block 5, which would point to block 8, and so on. This scheme is
not efficient; to traverse the list, each block must be read, which requires substantial I/O
time.
11000011000000111001111110001111…
The main advantage of this approach is that it is relatively simple and efficient to find n
consecutive free blocks on the disk. Unfortunately, bit vectors are inefficient unless the
entire vector is kept in memory for most accesses. Keeping it main memory is possible
for smaller disks such as on microcomputers, but not for larger ones
Disk Quotas:
A disk quota is a limit set by a system administrator that restricts certain aspects
of file system usage on modern operating systems operating systems. The function of
using disk quotas is to allocate limited disk space in a reasonable way.
There are two basic types of disk quotas. The first, known as a usage quota or block
quota, limits the amount of disk space that can be used. The second, known as a file
quota or inode quota, limits the number of files and directories that can be created.
In addition, administrators usually define a warning level, or soft quota, at which
users are informed they are nearing their limit, that is less than the effective limit, orhard
quota. There may also be a small grace interval, which allows users to temporarily violate
their quotas by certain amounts if necessary.
Disk quotas are typically implemented on a per-user or per-group basis. That is,
a system administrator defines a usage or file quota specific to a certain user or group.
In doing so, an administrator can prevent one user from consuming an entire file
system's resources, or create a system of tiered access, whereby users can have different
levels of restriction. This is used, for example, by web hosting companies to provide
different levels of service based upon the needs and means of individual clients.
In most cases, quotas are also specific to individual file systems. Should an
administrator want to limit the usage of a specific user on all file systems, a separate
quota would have to be specified on each.
When a soft quota is violated, the system normally sends the user (and sometimes the
administrator as well) some sort of message. No further action is typically taken.
Disk quotas are supported by most modern operating systems, including Unix-
like systems, such as AIX (using JFS or JFS2 file system), Linux (using ext3, ext4, ext2,
xfs (integrated support) among other file systems), Solaris (using UFS or ZFS), Microsoft
Windows starting with Windows 2000, Novell NetWare, VMS, and others. The method
of administration for disk quotas varies between each of these operating systems.
4.4.2 Defragmentation:
Fragmentation:
In order to understand why defragging works, it is important to understand how
data is stored on your hard drive. When data, such as a file, is stored on a hard drive the
operating system attempts to store that file in one section of contiguous, locations that
are connecting without a break, space. When you have a new hard drive, storing data in
contiguous spaces is not a problem. As you use the hard drive, though, files will be
deleted from it and small pockets of space will be created on your hard drive. These
small pockets of space on your hard drive is called fragmentation.
When a hard drive is fragmented, and the operating system wants to store a file on
the hard drive, it attempts to store it in a section of contiguous space that will be large
enough to accommodate the file. If the hard drive is heavily fragmented, there is the
possibility that there will not be enough contiguous space available to store the file, and
therefore the file will be broken up and stored in multiple locations on the hard drive.
This causes the file to become fragmented. This is especially bad when installing new
software on your computer because the program will now be installed over multiple
locations on your hard drive. Now when you run this particular application its
performance will be degraded because it has to be loaded from multiple locations on the
hard drive.
Fig.4.29. below shows an example of a fragmented file. Notice how File1 is stored in
two locations which are not contiguous.
system. Thus the number of electronic devices using the MS-DOS file system is vastly
larger now than at any time in the past, and certainly much larger than the number
using the more modern NTFS file system.
To read a file, an MS-DOS program must first make an open system call to get a
handle for it. Although MS-DOS directories are variable sized, they use a fixed-size 32-
byte directory entry. The format of an MS-DOS directory entry is shown in Fig. 4-30. It
contains the file name, attributes, creation date and time, starting block, and exact file
size.
MS-DOS uses the FAT to keep track of free disk blocks. Any block that is not
currently allocated is marked with a special code. When MS-DOS needs a new disk
block, it searches the FAT for an entry containing this code. Thus no bitmap or free list is
required.
It is self contained. There are no dependencies between one file system and any
other.
The directories have specific purposes and generally hold the same types of
information for easily locating files. Following are the directories that exist on the major
versions of Unix:
Directory Description
/ This is the root directory which should contain only the directories needed
at the top level of the file structure.
/bin This is where the executable files are located. They are available to all user.
/dev These are device drivers.
/etc Supervisor directory commands, configuration files, disk configuration files,
valid user lists, groups, ethernet, hosts, where to send critical messages.
/lib Contains shared library files and sometimes other kernel-related files.
/boot Contains files for booting the system.
/home Contains the home directory for users and other accounts.
/mnt Used to mount other temporary file systems, such as cdrom and floppy for
the CD-ROM drive and floppy diskette drive, respectively
/proc Contains all processes marked as a file by process number or other
information that is dynamic to the system.
/tmp Holds temporary files used between system boots
/usr Used for miscellaneous purposes, or can be used by many users. Includes
administrative commands, shared files, library files, and others
/var Typically contains variable-length files such as log and print files and any
other type of file that may contain a variable amount of data
/sbin Contains binary (executable) files, usually for system administration. For
example fdisk and ifconfig utlities.
/kernel Contains kernel files
File Types:
The UNIX filesystem contains several different types of files:
Ordinary Files:
(1) Used to store your information, such as some text you have written or an image you
have drawn. This is the type of file that you usually work with.
(2) Always located within/under a directory file
(3) Do not contain other files
Directories:
Branching points in the hierarchical tree
Used to organize groups of files
May contain ordinary files, special files or other directories
Never contain "real" information which you would work with (such as text).
Basically, just used for organizing files.
All files are descendants of the root directory, ( named / ) located at the top of the
tree.
Special Files:
Used to represent a real physical device such as a printer, tape drive or terminal,
used for Input/Output (I/O) operations
Unix considers any device attached to the system to be a file - including your
terminal:
By default, a command treats your terminal as the standard input file (stdin) from
which to read its input
Your terminal is also treated as the standard output file (stdout) to which a
command's output is sent
Stdin and stdout will be discussed in more detail later
Two types of I/O: character and block
Usually only found under directories named /dev
Pipes:
(1) UNIX allows you to link commands together using a pipe. The pipe acts a
temporary file which only exists to hold data from one command until it is read by
another
(2) For example, to pipe the output from one command into another command:
who | wc -l
This command will tell you how many users are currently logged into the system.
The standard output from the who command is a list of all the users currently logged
into the system. This output is piped into the wc command as its standard input. Used
with the -l option this command counts the numbers of lines in the standard input and
displays the result on its standard output - your terminal.
The base standard defines three levels of compliance. Level 1 limits file names to MS-
DOS conventions -- 8 chars for the name, 3 chars for the extension. Levels 2 and 3 allow
up to 31 characters. In practice however, most CDs use Level 1 plus one of the various
extensions (covered in the following sections). That way, MS-DOS gets file names it can
handle, and most other systems get the full file names from the extension records.
ISO9660 puts its first superblock 32K into the device, i.e. in sector 16. (CDs always
have 2K sectors.) That allows it to co-exist with certain native file systems or partition
tables. ISO9660 actually uses a list of superblocks, or "Volume Descriptors", with
different roles. The "Primary" volume descriptor, which must be present and first on all
volumes, uses type 1.
The Joliet Extension:
The Joliet extension is favored in the MS Windows world. It allows Unicode
characters to be used for all text fields, which includes file names and the volume name.
A "Secondary" volume descriptor with type 2 contains the same information as the
Primary one, but in 16-bit encoded Unicode. As a result of this, the volume name is
limited to 16 characters.
disktype prints the Unicode volume name from the Joliet volume descriptor if
present.
The RockRidge Extension
The RockRidge extension is favored in the Unix world. It lifts file name restrictions,
but also allows Unix-style permissions and special files to be stored on the CD.
Since RockRidge does not affect the volume descriptors, disktype does not detect or
report it.
Hybrid PC/Macintosh CDs:
Apple has its own set of ISO9660 extensions to store Mac OS metadata for files, but
they are seldom used. Instead, a full HFS or HFS Plus file system is put on the CD
together with an ISO9660 file system. This works well, since both file systems are flexible
in placing their data structures, and the (fixed-position) superblocks don't overlap. Some
mastering programs use an Apple-style partition table with a single entry, for no reason
in particular. File data can be shared between both file systems simply by pointing at the
same blocks in the appropriate structures.
Hybrid CDs contain two valid file systems, and disktype will report them as such.
The El Torito Boot Specification:
The El Torito standard defines a flexible format to store boot code on ISO9660 file
systems. It is implemented on most PCs today. A volume descriptor with type 0
(actually called a "Boot Record") points to another sector containing the actual "boot
catalog". That catalog lists a default boot image, plus an optional list of boot images for
specific hardware.
A boot image can be an image of a floppy disk (1.2, 1.44, or 2.88 MB), which will be
loaded into RAM by the BIOS and used to emulate a bootable floppy. Hard disk
emulation works likewise, but is not as useful. Finally, a boot image can be 'non-
emulated', i.e. just a piece of boot code that knows how to access the CD without
emulation. Whether the boot image is also available as a file on the CD is up to the CD
mastering software.
disktype will report the kind and parameters of all boot images found. Their contents
are analyzed for file systems and boot code in the usual way.
UDF:
UDF (Universal Disk Format) is a new cross-platform file format for CDs. It offers the
capability to create the file system incrementally on write-once (i.e. CD-R) and read-
write (i.e. CD-RW) media. That allows users to use standard file manager operations to
create a disk (instead of specialized CD mastering programs). UDF is designed to co-
exist with an ISO9660 structure, and most UDF implementations write both structures
(for the same set of files) by default. The DVD-Video and DVD-Audio formats are based
on UDF.
disktype supports UDF and reports all interesting information it finds. However, it
may not detect some exotic variations. One reason for this is that the sector size is
variable, but must be guessed -- the "anchor volume descriptor" is located at sector 256,
not at a certain byte offset. Most UDF disks also carry an ISO9660 file system, which is
detected and reported separately.
Sega Dreamcast:
The Sega Dreamcast game console uses a special physical format called GD-ROM (?),
but a standard ISO 9660 file system on top of that. There is an additional signature at the
start of the first sector, which is detected by disktype. It is unclear if this is part of boot
code of just a general signature.
Xbox DVD file system:
The Microsoft Xbox game console uses a special file system for DVDs,
which disktype tries to recognize.
3DO file system:
The 3DO game console system (?) uses standard Mode 1 CD-ROM disks, but with a
custom file system. The signature in the first sector is detected by disktype.