LECTURE 7: OPERATING SYSTEMS SECURITY

Operating system security (OS security) is the process of ensuring OS integrity, confidentiality and
availability.
OS security refers to specified steps or measures used to protect the OS from threats, viruses, worms,
malware or remote hacker intrusions. OS security encompasses all preventive-control techniques, which
safeguard any computer assets capable of being stolen, edited or deleted if OS security is compromised.

Operating system consists of a collection of objects, hardware or software

 Each object has a unique name and can be accessed through a well-defined set of operations
(hopefully)
 Protection and security problem - ensure that each object is accessed correctly and only by those
processes of authorized users that are allowed to do so
 OS designer faces challenge of creating a protection scheme that cannot be bypassed by any software
that may be created in the future
 Networking adds to the problem as it allows access to a computer and its resources without being in
the same physical location

OS security may be approached in many ways, including adherence to the following:

 Performing regular OS patch updates
 Installing updated antivirus engines and software
 Scrutinizing all incoming and outgoing network traffic through a firewall
 Creating secure accounts with required privileges only (i.e., user management)

Authentication
Authentication refers to identifying the each user of the system and associating the executing programs
with those users. It is the responsibility of the Operating System to create a protection system which
ensures that a user who is running a particular program is authentic. Operating Systems generally
identifies/ authenticates users using following three ways:
 Username / Password - User need to enter a registered username and password with Operating
system to login into the system.
 User card/key - User need to punch card in card slot, or enter key generated by key generator in
option provided by operating system to login into the system.
 User attribute - fingerprint/ eye retina pattern/ signature - User need to pass his/her attribute
via designated input device used by operating system to login into the system.
One Time passwords
One time passwords provides additional security along with normal authentication. In One-Time
Password system, a unique password is required every time user tries to login into the system. Once a
one-time password is used then it can not be used again. One time password are implemented in various
ways.
 Random numbers - Users are provided cards having numbers printed along with corresponding
alphabets. System asks for numbers corresponding to few alphabets randomly chosen.
 Secret key - User are provided a hardware device which can create a secret id mapped with user
id. System asks for such secret id which is to be generated every time prior to login.
 Network password - Some commercial applications send one time password to user on
registered mobile/ email which is required to be entered prior to login.
Threats in securing operating systems
Some of the most common types of violations include:
 Breach of Confidentiality - Theft of private or confidential information, such as credit-card
numbers, trade secrets, patents, secret formulas, manufacturing procedures, medical information,
financial information, etc.
  Breach of Integrity - Unauthorized modification of data, which may have serious indirect
consequences. For example a popular game or other program's source code could be modified to
open up security holes on users systems before being released to the public.
 Breach of Availability - Unauthorized destruction of data, often just for the "fun" of causing
havoc and for bragging rites. Vandalism of web sites is a common form of this violation.
 Theft of Service - Unauthorized use of resources, such as theft of CPU cycles, installation of
daemons running an unauthorized file server, or tapping into the target's telephone or networking
services.
 Denial of Service, DOS - Preventing legitimate users from using the system, often by overloading
and overwhelming the system with an excess of requests for service.

There are four levels at which a system must be protected:

1. Physical - The easiest way to steal data is to pocket the backup tapes. Also, access to the root
console will often give the user special privileges, such as rebooting the system as root from
removable media. Even general access to terminals in a computer room offers some opportunities
for an attacker, although today's modern high-speed networking environment provides more and
more opportunities for remote attacks.
2. Human - There is some concern that the humans who are allowed access to a system be
trustworthy, and that they cannot be coerced into breaching security. However more and more
attacks today are made via social engineering, which basically means fooling trustworthy people
into accidentally breaching security.
 Phishing involves sending an innocent-looking e-mail or web site designed to fool people
into revealing confidential information. E.g. spam e-mails pretending to be from e-Bay,
PayPal, or any of a number of banks or credit-card companies.
 Dumpster Diving involves searching the trash or other locations for passwords that are
written down. ( Note: Passwords that are too hard to remember, or which must be
changed frequently are more likely to be written down somewhere close to the user's
station. )
 Password Cracking involves divining users passwords, either by watching them type in
their passwords, knowing something about them like their pet's names, or simply trying
all words in common dictionaries.
3. Operating System - The OS must protect itself from security breaches, such as runaway
processes ( denial of service ), memory-access violations, stack overflow violations, the launching
of programs with excessive privileges, and many others.
4. Network - As network communications become ever more important and pervasive in modern
computing environments, it becomes ever more important to protect this area of the system.
( Both protecting the network itself from attack, and protecting the local system from attacks
coming in through the network. ) This is a growing area of concern as wireless communications
and portable devices become more and more prevalent.

Program Threats
Operating system's processes and kernel do the designated task as instructed. If a user program made
these process do malicious tasks then it is known as Program Threats.
Following is the list of some well known program threats.
 Trojan Horse - Such program traps user login credentials and stores them to send to malicious
user who can later on login to computer and can access system resources.
 Trap Door - If a program which is designed to work as required, have a security hole in its code
and perform illegal action without knowledge of user then it is called to have a trap door.
 Logic Bomb - Logic bomb is a situation when a program misbehaves only when certain
conditions met otherwise it works as a genuine program. It is harder to detect.
  Virus - Virus as name suggest can replicate themselves on computer system .They are highly
dangerous and can modify/delete user files, crash systems. A virus is generatlly a small code
embedded in a program. As user accesses the program, the virus starts getting embedded in other
files/ programs and can make system unusable for user.
 Spyware is a version of a Trojan Horse that is often included in "free" software downloaded off
the Internet. Spyware programs generate pop-up browser windows, and may also accumulate
information about the user and deliver it to some central site. ( This is an example of covert
channels, in which surreptitious communications occur. ) Another common task of spyware is to
send out spam e-mail messages, which then purportedly come from the infected user.

System Threats
System threats refers to misuse of system services and network connections to put user in trouble. System
threats can be used to launch program threats on a complete network called as program attack. System
threats creates such an environment that operating system resources/ user files are mis-used. Following is
the list of some well known system threats.
 Worm -Worm is a process which can choked down a system performance by using system
resources to extreme levels.A Worm process generates its multiple copies where each copy uses
system resources, prevents all other processes to get required resources. Worms processes can
even shut down an entire network.
 Port Scanning - Port scanning is a mechanism or means by which a hacker can detects system
vulnerabilities to make an attack on the system. Port Scanning is technically not an attack, but
rather a search for vulnerabilities to attack. The basic idea is to systematically attempt to connect
to every known ( or common or possible ) network port on some remote machine, and to attempt
to make contact. Once it is determined that a particular computer is listening to a particular port,
then the next step is to determine what daemon is listening, and whether or not it is a version
containing a known security flaw that can be exploited.
 Because port scanning is easily detected and traced, it is usually launched from zombie systems,
i.e. previously hacked systems that are being used without the knowledge or permission of their
rightful owner. For this reason it is important to protect "innocuous" systems and accounts as well
as those that contain sensitive information or special privileges.
 Denial of Service - Denial of service attacks normally prevents user to make legitimate use of the
system. For example user may not be able to use internet if denial of service attacks browser's
content settings.
Some of the forms of viruses include:
 File - A file virus attaches itself to an executable file, causing it to run the virus code first and then
jump to the start of the original program. These viruses are termed parasitic, because they do not leave
any new files on the system, and the original program is still fully functional.
 Boot - A boot virus occupies the boot sector, and runs before the OS is loaded. These are also known
as memory viruses, because in operation they reside in memory, and do not appear in the file system.
 Macro - These viruses exist as a macro ( script ) that are run automatically by certain macro-capable
programs such as MS Word or Excel. These viruses can exist in word processing documents or
spreadsheet files.
 Source code viruses look for source code and infect it in order to spread.
 Polymorphic viruses change every time they spread - Not their underlying functionality, but just their
signature, by which virus checkers recognize them.
 Encrypted viruses travel in encrypted form to escape detection. In practice they are self-decrypting,
which then allows them to infect other files.
 Stealth viruses try to avoid detection by modifying parts of the system that could be used to detect it.
For example the read( ) system call could be modified so that if an infected file is read the infected part
gets skipped and the reader would see the original unadulterated file.
 Tunneling viruses attempt to avoid detection by inserting themselves into the interrupt handler chain,
or into device drivers.
 Multipartite viruses attack multiple parts of the system, such as files, boot sector, and memory.
 Armored viruses are coded to make them hard for anti-virus researchers to decode and understand. In
addition many files associated with viruses are hidden, protected, or given innocuous looking names
such as "...".
 In 2004 a virus exploited three bugs in Microsoft products to infect hundreds of Windows servers
( including many trusted sites ) running Microsoft Internet Information Server, which in turn
infected any Microsoft Internet Explorer web browser that visited any of the infected server sites.
One of the back-door programs it installed was a keystroke logger, which records users
keystrokes, including passwords and other sensitive information.
 There is some debate in the computing community as to whether a monoculture, in which nearly
all systems run the same hardware, operating system, and applications, increases the threat of
viruses and the potential for harm caused by them.

Password Vulnerabilities
 Passwords can be guessed.
 Intelligent guessing requires knowing something about the intended target in specific, or
about people and commonly used passwords in general.
 Brute-force guessing involves trying every word in the dictionary, or every valid
combination of characters. For this reason good passwords should not be in any
dictionary ( in any language ), should be reasonably lengthy, and should use the full range
of allowable characters by including upper and lower case characters, numbers, and
special symbols.
 "Shoulder surfing" involves looking over people's shoulders while they are typing in their
password.
 Even if the lurker does not get the entire password, they may get enough clues to narrow
it down, especially if they watch on repeated occasions.
 Common courtesy dictates that you look away from the keyboard while someone is
typing their password.
 Passwords echoed as stars or dots still give clues, because an observer can determine how
many characters are in the password. :-(
 "Packet sniffing" involves putting a monitor on a network connection and reading data contained
in those packets.
 SSH encrypts all packets, reducing the effectiveness of packet sniffing.
 However you should still never e-mail a password, particularly not with the word
"password" in the same message or worse yet the subject header.
 Beware of any system that transmits passwords in clear text. ( "Thank you for signing up
for XYZ. Your new account and password information are shown below". ) You probably
want to have a spare throw-away password to give these entities, instead of using the
same high-security password that you use for banking or other confidential uses.
Protected Objects
The rise of multiprogramming meant that several aspects of a computing system required protection.
 memory
 sharable I/O devices, such as disks
 serially reusable I/O devices, such as printers and tape drives
 sharable programs and subprocedures
 networks
 sharable data
As it assumed responsibility for controlled sharing, the operating system had to protect these objects.
Security Methods of Operating Systems
The basis of protection is separation: keeping one user's objects separate from other users. Separation in
an operating system can occur in several ways.
 Physical separation , in which different processes use different physical objects, such as separate
printers for output requiring different levels of security
 Temporal separation , in which processes having different security requirements are executed at
different times
 Logical separation , in which users operate under the illusion that no other processes exist, as
when an operating system constrains a program's accesses so that the program cannot access
objects outside its permitted domain
 Cryptographic separation , in which processes conceal their data and computations in such a way
that they are unintelligible to outside processes
Combinations of two or more of these forms of separation are also possible.
 The first two approaches are very stringent and can lead to poor resource utilization. Therefore, we
would like to shift the burden of protection to the operating system to allow concurrent execution of
processes having different security needs.

There are several ways an operating system can assist, offering protection at any of several levels.
 Do not protect . Operating systems with no protection are appropriate when sensitive procedures
are being run at separate times.
 Isolate . When an operating system provides isolation, different processes running concurrently
are unaware of the presence of each other. Each process has its own address space, files, and other
objects. The operating system must confine each process somehow, so that the objects of the
other processes are completely concealed.
 Share all or share nothing . With this form of protection, the owner of an object declares it to be
public or private. A public object is available to all users, whereas a private object is available
only to its owner.
 Share via access limitation . With protection by access limitation, the operating system checks the
allowability of each user's potential access to an object. That is, access control is implemented for
a specific user and a specific object.
 Share by capabilities . An extension of limited access sharing, this form of protection allows
dynamic creation of sharing rights for objects. The degree of sharing can depend on the owner or
the subject, on the context of the computation, or on the object itself.
 Limit use of an object . This form of protection limits not just the access to an object but the use
made of that object after it has been accessed. For example, a user may be allowed to view a
sensitive document, but not to print a copy of it. More powerfully, a user may be allowed access
to data in a database to derive statistical summaries (such as average salary at a particular grade
level), but not to determine specific data values (salaries of individuals).
These modes of sharing are arranged in increasing order of difficulty to implement, but also in increasing
order of fineness of protection they provide.
A given operating system may provide different levels of protection for different objects, users, or
situations.

Memory and Address Protection

 Preventing one program from affecting the memory of other programs.
 Protection can be built into the hardware mechanisms that control efficient use of memory, so that solid
protection can be provided at essentially no additional cost.
Fence
 The simplest form of memory protection was introduced in single-user operating systems, to prevent
a faulty user program from destroying part of the resident portion of the operating system. As its
name implies, a fence is a method to confine users to one side of a boundary.
 Another implementation used a hardware register, often called a fence register , containing the
address of the end of the operating system. In contrast to a fixed fence, in this scheme the location of
the fence could be changed. Each time a user program generated an address for data modification, the
address was automatically compared with the fence address. If the address was greater than the fence
address (that is, in the user area), the instruction was executed; if it was less than the fence address
(that is, in the operating system area), an error condition was raised.

Relocation
If the operating system can be assumed to be of a fixed size , programmers can write their code assuming
that the program begins at a constant address. This feature of the operating system makes it easy to
determine the address of any object in the program.
It also makes it essentially impossible to change the starting address if, for example, a new version of the
operating system is larger or smaller than the old. If the size of the operating system is allowed to change,
then programs must be written in a way that does not depend on placement at a specific location in
memory.

Base/Bounds Registers
 With two or more users, none can know in advance where a program will be loaded for execution.
 The relocation register solves the problem by providing a base or starting address.
 All addresses inside a program are offsets from that base address.
 A variable fence register is generally known as a base register .

Segmentation
Segmentation , involves the dividing a program into separate pieces. Each piece has a logical unity,
exhibiting a relationship among all of its code or data values.
Segmentation allows a program to be divided into many pieces having different access rights.
This hiding of addresses has three advantages for the operating system.
1. The operating system can place any segment at any location or move any segment to any location,
even after the program begins to execute. Because the operating system translates all address
references by a segment address table, the operating system needs only to update the address in that
one table when a segment is moved.
2. A segment can be removed from main memory (and stored on an auxiliary device) if it is not being
used currently.
3. Every address reference passes through the operating system, so there is an opportunity to check each
one for protection.
Segmentation offers these protective benefits.
 Each address reference is checked for protection.
 Many different classes of data items can be assigned different levels of protection.
 Two or more users can share access to a segment, with potentially different access rights.
 A user cannot generate an address or access to an unpermitted segment.

Control of Access to General Objects

Protecting memory is a specific case of the more general problem of protecting objects. As
multiprogramming has developed, the numbers and kinds of objects shared have also increased.
Examples of the kinds of objects for which protection is desirable:
1. memory
 2. a file or data set on an auxiliary storage device
3. an executing program in memory
4. a directory of files
5. a hardware device
6. a data structure, such as a stack
7. a table of the operating system
8. instructions, especially privileged instructions
9. passwords and the user authentication mechanism
10.the protection mechanism itself
The memory protection mechanism can be fairly simple because every memory access is guaranteed to go
through certain points in the hardware. With more general objects, the number of points of access may be
larger, a central authority through which all accesses pass may be lacking, and the kind of access may not
simply be limited to read, write, or execute.
There are several complementary goals in protecting objects.
 Check every access . We may want to revoke a user's privilege to access an object. If we have
previously authorized the user to access the object, we do not necessarily intend that the user
should retain indefinite access to the object.
 Enforce least privilege . The principle of least privilege states that a subject should have access to
the smallest number of objects necessary to perform some task. Even if extra information would
be useless or harmless if the subject were to have access, the subject should not have that
additional access.
 Verify acceptable usage . Ability to access is a yes-or-no decision. But it is equally important to
check that the activity to be performed on an object is appropriate.
Directory
 One simple way to protect an object is to use a mechanism that works like a file directory.
 Every file has a unique owner who possesses "control" access rights (including the rights to declare
who has what access) and to revoke access to any person at any time.
 Each user has a file directory, which lists all the files to which that user has access.
Several difficulties can arise.
 The list becomes too large if many shared objects, such as libraries of subprograms or a common table
of users, are accessible to all users.
 The directory of each user must have one entry for each such shared object, even if the user has no
intention of accessing the object.
 Deletion must be reflected in all directories.

4.