Java Feels Secure

Thang M. Le
[email protected]

ABSTRACT
Java has become a favorable language in software development. For the past ten years, Java has dominated the Internet
under the forms of Java applet and Java web server. Recently, Java has also proved its advantages in embedded world (e.g.
Android platform). Being a popular language, Java has become a target for numerous attacks. We still remember Sapphire
worm incident which caused flooding the Internet when exploiting the security flaw in Microsoft SQL server. It would be
more destructive if a similar exploit existed in Java. Java security is again evaluated in this paper. The focus is on existing
methods of protection provided by Java language, the bytecodes and the system security design. The paper then reviews
important known bugs and further discusses some concerns and improvements based on the current design.

1. Introduction
The question “So, how does Java feel?” arises in [5] becomes the topic of this paper. Java feels fun because it is creative.
Importantly, Java feels safe as it is secured. Security is always an important part of Java platform since the first release.
Secure by design has been built into Java since its inception. What makes Java feel secure is the subject of this paper. The
term “security” is vaguely used in this paper to refer to three important aspects in computer security: confidentiality,
integrity and availability. The term “bytecodes” makes reference to Java bytecodes instruction set. The paper also frequently
uses the word “Java” to denote anything relating to Java language, bytecodes and Java Virtual Machine (JVM).

2. Language Protection in Java 1.0

Java evolved from the project to build a portable environment for embedded systems back in 1991 [6]. Out of the box,
Java was set to achieve portability and security. Java 1.0 was first released in 1995. It is known for its secure by design. The
concept of secure by design has been broadly adopted in building trustworthy systems. Following this concept, a trusted
system meets its security requirements based on its architectural design rather than its implementation. Such a system would
direct all execution paths and data flow to its “gatekeeper” who is responsible to control resource accesses based on security
policy in effect. Such a system would have its design follows common principles recommended in [8]. It is only by design
security constraints of the system are enforced firmly without individual effort from each component. Achieving secure by
design helps the task of evaluating a system more effectively, hence, security flaws inherent from the design can be quickly
discovered and addressed.
From its design, Java supports codes written in Java and codes written in bytecodes. Since supporting two different
formats of code, the language validates source codes during both compile time and load time. These validations are
necessary to protect the three security aspects of the system.

2.1. Confidentiality
Java language introduces access qualifiers: public, protected, private and default to define different access controls to a
class property. The compiler and the bytecode verifier are responsible to detect any violations to the rules of these qualifiers
in codes written in Java and bytecodes respectively. Another important method of protection is the complete removal of
pointer arithmetic in the language [5]. Pointer arithmetic is a source of illegal memory access in languages such as C, C++.
Lack of supporting pointer arithmetic in the language contributes a large portion to a safe system.

2.2. Integrity
Java language uses keyword final to mark certain information cannot be modified or circumvented. This keyword is
checked by both the compiler and the verifier. The bytecodes imposes a set of important restrictions which the verifier needs
to check when a class is loaded into the system. Once the loading is complete, these properties are held: “no operand stack
overflow or underflow, type of parameters of all opcodes are always correct, no illegal data conversions are done, no illegal
type casting” [6]. Integrity of data resided in memory is protected by the use of garbage collection which allocates and frees
memory for running programs. Pointers are further restricted to protect from integrity problems. It is to say memory is well
protected in Java.

2.3. Availability
Availability aspect of Java lies in its philosophy of write once run everywhere. The introduction of platform neutral
machine code known as Java bytecodes is necessary to support this versatile requirement. While the bytecodes is
independent from underlying platform, JVM is the only platform dependent piece in Java. That means there must be a

1
separated JVM for each different platform. The bytecodes is designed to be type-safe, checkable, portable and executable,
which precludes malicious codes from causing any damage to JVM.

3. Runtime Protection in Java 1.0

Runtime protection in Java 1.0 is pretty straight forward following “all or nothing” level of protection. Java defines
programs loaded from local file system are trusted while programs loaded over network (such as applets) are untrusted. The
whole system security is designed around this definition. While trusted codes enjoy full privileges to access protected
resources, untrusted codes have no permission to use the underlying resources. This security enforcement is achieved
through the use of Java sandbox model. Java ensures an untrusted program to be executed inside its sandbox and cannot
interfere with other programs. The sandbox when created loads an appropriate security policy into it. This policy has a list of
permissions to either grant or deny execution of sensitive methods. ClassLoader and SecurityManager are implementations
of the sandbox. ClassLoader is responsible to load classes into JVM. SecurityManager is considered a gatekeeper of the
system. SecurityManager has a set of methods prefixed with “check” keyword. These methods are required to get call in
protected routines. The least privilege principal is used in each of these check methods which assume callers do not have
permission to perform the requested operation. Java 1.0 did not have a strong mechanism to differentiate trusted program
from untrusted program. At various points, the system inferred the call is coming from an applet if the current executing
stack has less than 2 or 3 stack frames. There were exceptions to this rule and the whole system was patched altogether
because of this unreliable observation [3].

4. Runtime Protection in Java 1.2

“All or nothing” level of protection in Java 1.0 is too rigid. This protection did not meet growing demands from Java
users. The security goal was set for Java 1.2 to achieve a fine-grained access control model and flexibility to adopt rapidly
emerging technologies [4]. Java 1.2 released in 1998 witnessed the most significant improvement in system security thanks
to the hard work of Li Gong and his team from JavaSoft. Many of security features introduced in Java 1.2 later called Java 2
are still being carried forward to recent Java releases.

4.1. Evolution of Java Sandbox Model

The weakness of Java 1.0 sandbox model is its lack of reliable mechanism to distinguish trusted codes from hostile
codes. This gap was filled in Java 1.2 through a set of new classes including SecureClassLoader, CodeSource,
ProtectionDomain and Permission. In the new design, SecureClassLoader is used to load untrusted codes over network. The
class loader then assigns each loaded class to associated ProtectionDomain based on its CodeSource. CodeSource is the
solution to the weakness in Java 1.0. CodeSource stores a set of certificates and the source URL of the loaded class. The
combination of data encapsulated in CodeSource object must be unique to identify the group of classes of a program from
others in JVM. If there were exception cases just as the lesson in Java 1.0, the whole security model would fall apart.
ProtectionDomain contains a CodeSource object and a list of Permission objects which can be programmatically
constructed or created when loading policy file. It is worth to reiterate Permission object represents an access control to a
sensitive resource such as file I/O, net I/O, abstract window toolkit (AWT)… The existence of Permission object does not
automatically grant access to the resource without being assigned. Once a Permission object is assigned to a CodeSource
object, classes under this CodeSource are granted permission described in the Permission object

4.2. Stack Inspection Algorithm

SecurityManager contains a list of checking methods. However, how to properly perform security checking is more
important than what should be checked. AccessController implements the stack inspection algorithm to facilitate access
controlling process. The algorithm walks down current thread stack. At each stack frame, the involved class is known. The
algorithm searches for ProtectionDomain corresponding to this involved class. Once the ProtectionDomain is retrieved, list
of permissions are known. The algorithm then verifies whether the class has enough permission to perform the operation. If
the class is authorized for the operation, the algorithm continues on next stack frame until reaching the end of the stack. If
the involved class is not permitted for the operation, the check will stop with the denial of access and SecurityException
might get thrown. AccessController also includes doPrivilege() method which is used to mark a class “privileged”. When
encounters a “privilege” class, the check for permission is still required for the “privileged” class and immediately returns
the result of the last check without continuing on next stack frame. Stack based inspection is a simple and effective
procedure. Given the type safety property of Java language and the nature of stack based computation, stack inspection
algorithm is proved to be accurate and useful in access control decision [10]. The algorithm might not achieve a desire level
of efficiency compared with other methods. But the simplicity and the effectiveness overtake its performance disadvantage.

4.3. Protection Object

The last missing piece introduced in Java 1.2 is the existence of SignedObject, SealedObject and GuardObject. Since the
beginning, Java has only anticipated to build its system security around access control on operation basic which is at the
class level. There is no effort put into protecting an object which contains confidential data. Transmitting an object storing
password information over the Internet in plain text was the concern. SignedObject, SealedObject and GuardObject stand to
support this type of protection. SignedObject uses digital signature scheme to protect the object integrity while SealedObject
uses symmetric key cryptography to perform encryption and decryption during serialization and deserialization.
GuardObject is used to put protections on object’s methods to either grant or deny execution.

2
5. Known Implementation Bugs
There is no existing trustworthy system. Java is not an exception. Despite inherent the good design, Java still has various
security bugs most are from bad implementations. Below provides a list of known vulnerabilities in chronological order.
In 1996, a group of researchers from Princeton released their work on Java 1.0 security [2]. Their paper provided a list of
serious security issues including denial of service attack, covert channel, DNS weakness, buffer overflow and so on. It was
the most notable research attacking Java in all aspects from the implementation to the design and the language.
In 2002, there was a flaw in Microsoft VM which could allow malicious applets to bypass class restrictions imposed by
StandardSecurityManager. An attacker could take advantage of this vulnerability to perform denial of service attack [11].
In 2005, buffer overflow and integer overflow exploited the vulnerabilities in native code shipped with JVM. [13]
Most recently, in 2008, the vulnerability in deserializing Calendar objects could result in elevation of privilege to
malicious applet. This vulnerability was nominated for best client side bug in 2009. [12]

6. Future Improvement
From the beginning, SecurityManager is designed to be a single point in the system for classes which require controlling
access to check with for a decision either to allow or deny access [3]. This design effectively forces various system routines
to include methods of SecurityManager in their logic. The consequence is the whole system security operates following
callback model. This puts SecurityManager who is a gatekeeper in passive mode waiting to be called. Security by callback
is fragile and is easily circumvented. Security gatekeeper must be active rather passive. The current security architecture
would need a revisit to change its security model to dispatcher model. Only with dispatcher in place, we can guarantee all
accesses are completely mediated by SecurityManager. Windows operating system kernel is known to use dispatcher model
when transitioning from user mode to kernel mode during system calls. Current Java security architecture can easily adopt
dispatcher model without a lot of required work. System libraries would have to be split into client system libraries and
kernel system libraries. All executing methods of client system libraries will be delegated to corresponding kernel system
libraries through a dispatcher where SecurityManager is installed to exercise access control for every sensitive method.
Another gap in Java security model is it only exercises one way checking. A complete mediation principle should require
bi-directional checking on incoming access and outgoing returned object. Java security is designed centric around checking
access control on incoming methods only. It completely lacks of mechanism to check whether a caller has permission to
receive a certain returned object. This makes the system vulnerable to implementation flaws exist in trusted codes. The
vulnerability in deserializing Calendar mentioned above is an excellent example. The correctness of checking object upon
returning relies on the relationship of construction and possession:

If a subject is not allowed to create a certain type of object, this subject cannot possess an object of this type.

For example, an applet by default does not have permissions to “createClassLoader” or “getClassLoader”. As a result, if
a system routine returns an instance of ClassLoader to this applet, it indirectly violates these two rules. By careful
examining whether callers have constructor permissions over returned object, the system can wisely detect errors made by
trusted codes. From case to case, the construction and possession relationship might not hold. In such cases, system should
return the returned object in the form of GuardObject to the caller.

7. Conclusion
Building a trustworthy general-purpose system is hard. Even a system is designed to immune from certain attacks, over
time this property might no longer hold due to the conflicts of new features introduced to the system. It is said that only
system built on solid security architecture sustains through such changes. A rule of thumb is protection scheme must be
rather centric than dispersible. It is unreasonable to ask every piece of codes in the system to check for certain attacks.
Methods of protection must make no or less assumptions about others. Such assumptions add uncertainty to the system
security. From the beginning, Java has a good start based on the protection from its intermediate bytecodes. But at runtime,
its protection methods are still insufficient. The current security design has been widely recognized for its simplicity while
being flexible. There are still concerns existing in the current design and they were briefly discussed in this paper. To its
end, Java is still a language of joys and yet feels secure enough to rely on for certain critical tasks.

References
[1] Eric Allman. A Conversation with James Gosling. Queue, Volume 2, Issue 5, 2004
[2] D. Dean, E. W. Felten, and D. S. Wallach. Java Security: From HotJava to Netscape and Beyond. Proceedings of 1996
IEEE Symposium on Security and Privacy (Oakland, California), May 1996.
[3] Li Gong. Java Security: A Ten Year Restrospective. Computer Security Applications Conference, 2009. ACSAC '09.
[4] Li Gong. Java Security: Present and Near Future. Micro, IEEE, Volume 17, Issue 3, 1997
[5] James Gosling. The Feel of Java. IEEE Computer, Volume 30, Issue6, pages 53-57, 1997.
[6] James Gosling. Java intermediate bytecode. Proceeding IR '95 Papers from the 1995 ACM SIGPLAN workshop on
Intermediate representations, 1995.
[7] L. Koved, A. J. Nadalin, D. Neal, T. Lawson. The Evolution of Java Security. IBM Systems Journal, Vol.37, No.3, 1998
[8] J. H. Saltzer, M. D. Schroeder. The Protection of Information in Computer Systems. Proceedings of the IEEE, Volume
63, Issue 9, pages 1278-1308, 1975.

3
[9] Andreas Sterbenz. An Evaluation of the Java Security Model. Computer Security Applications Conference, 1996., 12th
Annual
[10] D. S. Wallach, E. W. Fellen. Understanding Java Stack Inspection. Proceedings 1998 IEEE Symposium on Security and
Privacy, 1998.
[11] CERT security report #237777, http://www.kb.cert.org./vuls/id/237777.
[12] CVE-2008-5353. http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-5353
[13] http://scary.beasts.org/security/CESA-2005-008.txt
[14] http://download.oracle.com/javase/1.5.0/docs/guide/security/spec/security-specTOC.fm.html