java.net.

RTP
Waqar Ali
Columbia University
New York, NY 10027
USA
[email protected]

Akhil Nigam
Columbia University
New York, NY 10027
USA
[email protected]

Abstract
The Real-time Transport Protocol (RTP [1]) has become a widely implemented inter
net-standard protocol for transport of real-time data. It provides a mechanism f
or the participants to give feedback about the service using the RTP Control Pro
tocol (RTCP [1]). This document presents the platform independent RTP implementa
tion in Java. This implementation, which is available as a java.net.RTP package,
can be easily incorporated into an application, which then has access to all of
the transport level features RTP and RTCP provide. This package also provides o
ut of band initialization classes, which implement a late-initialization protoco
l. These classes could be used to implement a late initialization server which i
s capable of synchronizing late-joining clients.
Introduction
This report describes the Java implementation of RTP protocol. It covers the fol
lowing topics.
1) Related Work
2) Background
3) Architecture
3.1) Object Model
3.2) Event Model
3.3) Late Initialization Client/Server
4) Program Documentation
5) Java RTP API documentation
6) References
7) Downloads

Related Work
RTP Library:
C++ library which provides application independent implementation of the RTP pro
tocol. This library is available for DEC OSF/1 v3.2 and Solaris 2.3 platforms on
ly.
Java Media Framework:
The Java Media Framework API (JMF) specifies a simple, unified architecture to s
ynchronize and display time-based data, such as audio and video, within Java app
lications and applets. The JMF 1.0 API was developed by Sun Microsystems, Inc.,
Silicon Graphics Inc., and Intel Corporation.[2]
Elemedia H.323 stack
The elemedia PX3230S H.323 Protocol Stack software module is a software implemen
tation of the ITU-T H.323 standard. The RTP/RTCP module supports header validity
checks, statistic logging, sender reports, reception reports, and a system time
-to-media time utility. [3]
Webcanal
WebCanal contains a set of networking packages and applications mainly for infor
mation distribution using IP Multicast. WebCanal networking packages include a J
ava implementation of RTP (real time transport protocol), LRMP, SAP/SDP (session
announcement protocol) and more. [3]

Background
The project was motivated by the need to have a platform independent, multicast-
capable RTP implementation, which could be used in collaborative applications li
ke the Whiteboard. The whiteboard application has special initialization require
ments where all the instances of the collaborating whiteboards have to display t
he identical content, i.e. they need to have a common state. This becomes a prob
lem for late joiners which then have to be synchronized and brought to a common
state like the other instances, hence there must exist either a dedicated server
or such, which is capable of serving such clients and is responsible to provide
late joiners with historic data. A generic functionality to build such a client
and a server needs to be available for such an application. The package was des
igned and developed to be modular with a generic late-initialization client/serv
er classes implementing a late-initialization protocol.
Architecture
This section describes the architecture of the software. When appropriate, the d
esign decisions and alternatives are discussed.
Object Model
Overall architecture of the software was motivated by the need to have a modular
, simple and platform-independent RTP implementation which could be easily integ
rated into any application requiring RTP as its transport protocol. By choosing
Java as the development platform, the goal of having platform-independent implem
entation was immediately met. The object oriented features of Java and the langu
age's simplicity made it possible to make the software simple and modular.
Session:
The top most class the user of the package interacts with is Session. This class
encapsulates the RTP and RTCP related setup, startup and shutdown procedures. S
ession also serves as the interface using which object's states and their intera
ctions are controlled.
Source objects. Source class encapsulates the source-related attributes, which a
re updated by the sender and receiver modules. The participants include active s
enders as well as listeners. Source table is implemented as a Java hash-table, w
hich essentially is an associative array with search functionality built into it
. The key in the hash-table is the SSRC of the source and the value is the Sourc
e object itself. Java Map and Vector classes (in java.util) were considered as i
mplementation alternatives. Both of these are container classes just like the ha
sh-table but due to the reasons discussed below, it was agreed upon to use the h
ash-table.
Map (java.util.Map) is only available in JDK 1.2 (which at the time of writing t
his document is in beta release). Selecting this as a source container would hav
e limited the usability of the package to 1.2 or later versions of JDK. Hash-tab
le, on the other hand, is available in all JDK releases as early as the version
1.02.
Vector (java.util.Vector) was not chosen because it does not provide the key-val
ue associations and would have required manual search to find the desired source
object. The hash-table, on the other hand, has the search functionality and ass
ociation built-in making it trivial to retrieve the desired source object given
its SSRC as the key.
An implementation aspect of the source table needs to be pointed out; it is that
the sample implementation suggested rfc-1889 distinguishes between the senders
and members by maintaining two separate tables. In this package, the source tabl
e provided by the session holds the members and there is no separate table for s
enders. A source attribute ActiveSender is used to distinguish the senders from
listeners in the member table. Functions in Session are provided to get the send
ers or the members.
From the user's perspective, the Session interacts with the network and is respo
nsible for sending and receiving RTP and RTCP packets. What constitutes the netw
ork interaction can be classified into two distinct processes:
(i) Synchronous processes:
Sending RTP packets
(i) Asynchronous processes:
Receiving RTCP packets
Receiving RTP packets
Sending RTCP packets

The synchronous interaction with the network is straightforward, this is driven

by the application (the user of the package) and is invoked by calling the Sessi
on.SendRTPPacket() function.
The asynchronous interaction, on the other hand, requires the RTP and RTCP recei
vers to run on separate threads and wait for the packet arrival. Following the r
eception of a packet, several tasks are performed among which is posting of the
packet arrival event. Using events, the application has the option to ignore a p
articular event family or subscribe to it. The event model and the object intera
ction that takes place when asynchronous events occur are described in the Event
Model section of this document.
Figure 1: Higher level functional modules
RTP Packet Sending and Receiving:
Figure 1 depicts the object model where Session uses the RTPThreadHandler object
to send and receive RTP packets. RTPThreadHandler class is inherited from Threa
d and the function which receives the RTP packets, runs in an infinite loop. Thi
s loop is started directly by calling the or using Session.Start() method which
starts all the receivers. RTPThreadHandler class also doubles as a sender by pro
viding the SendPacket() function.
RTCP Packet Sending and Receiving
RTCP receiver RTCPReceiverThread is similar to RTPThreadHandler described above
as it receives RTCP packets in an endless loop, but it is dissimilar as it does
not provide any send functionality.
For sending RTCP packets, a separate threaded class RTCPSenderThread is provided
. The functionality of sending and receiving RTCP packets is not folded into one
class to keep the design modular and simple. RTCP sender and receiver both have
complicated algorithms and to keep the changes in the send RTCP module from eff
ecting the receiver module (or vice-versa), these functional modules are impleme
nted as separate encapsulations. This was decided upon to keep the overall desig
n in accordance with the design guideline of isolating the package modules from
the changes made in other modules, while allowing for extensibility.
RTCPThreadHandler class is the parent class, which serves as an intstantiator of
the sender and receiver thread classes. This class has the sole purpose of poli
cing the RTCP sender and receiver threads.
Although sending of the RTCP packets is an asynchronous process, the sender thre
ad does not run in an endless loop like the receiver -- the Session has some con
trol over it. For instance, after the RTCP sender thread is started, it does not
send any RTCP packets unless the application (user of the Session) sends a RTP
packet. Likewise, the RTCP sender stops sending packets after application reques
ts a shutdown by calling the Session.Stop() function. According to the RTP proto
col specifications, it is mandatory for the RTCP sender thread to be silent foll
owing the transmission of the BYE packet. Although the Session user has some con
trol over the RTCPSenderThread's packet sending behavior, it has no explicit con
trol over when the RTCP packet will be sent, this is due to the randomness intro
duced while calculating the RTCP sending interval.
Sending RTCP packets has to follow certain rules and guidelines as dictated by t
he rfc[1]. The flowchart shown in Figure 2 summarizes the algorithms used in the
implementation of the sender.

Figure 2: RTCP Send flowchart. The thread loop is entered at the top-left and ex
ited at the right.

This flow depicted in the flow-diagram above implements the BYE back-off algorit
hm (rfc-1889, section 7.3.7). Furthermore, SSRC of sources are timed out followi
ng the algorithm described in Option B of rfc-1889 (section 7.3.6).
RTCP reception involves parsing the incoming RTCP packets, which can be of type
described in RTCPConstants class. After parsing the packet, source table is upda
ted if required and an event is generated for the registered listener.
RTP and RTCP reception update the Session and source level statistics that are n
ecessary to sustain an RTP session with minimal interaction from the application
. This makes it easy for applications, which do not necessary need to concern th
emselves with the protocol intricacies, to use this software package and with on
ly a few lines of code, have a fully functional RTP session. For instance, notic
e the following sample code

// Construct a new Session object

rtpSession = new Session ( "234.5.6.7", // MulticastGroupIPAddress
8000, // MulticastGroupPort
8001, // RTCPGroupPort
8051, // RTPSendFromPort
8052, // RTCPSendFromPort
10000 // bandwidth
);
// Set the session parameters
rtpSession.setPayloadType ( 5 );
rtpSession.setCName ( "RTPUser" );
rtpSession.setEMail ( "[email protected]" );
// Start the session
rtpSession.Start();
// Send a test packet.
rtpSession.SendPacket ( String ( "TestString" ).getBytes() );
// Stop the session
rtpSession.Stop();

However, for the applications requiring finer control over the RTP session, it i
s necessary to expose some of the session internals. This allows for application
s, requiring distinct level of control, to use the same software package. Sessio
n exposes some of the internal functionality, such as, thread handling, source-t
able modification methods etc. for flexibility and extendibility.
It was mentioned earlier that when a RTP or RTCP packet arrives, session level s
tate and statistics are updated, furthermore, it is necessary to forward the RTP
packets over to the user of the session (application or any other class requiri
ng the received RTP packets). To avoid using hard-coded callback functions, an e
vent model was designed which is discussed in the next section.
Event Model
The event model in this package is based on the one implemented by the Abstract
Windowing Toolkit (AWT) 1.1. In the AWT event-model, asynchronous events, such a
s button-click etc. are handled by registered listeners. A listener is a class w
hich implements an interface specified by the event-model. For instance, there e
xists in AWT, the mouseListener interface. Any class interested in mouse events,
must implement this interface. Additionally, the interested class instance must
register itself with the event source by calling the method addMouseListener an
d passing its own instance reference. This step of registration will allow the e
vent dispatcher to call the appropriate event handling functions in the listener
 when an event is posted. The dispatcher is guaranteed that the listener, which
declares that it implements the interface, indeed does implement all the methods
. This guarantee is in effect by the compiler, which will produce compile-time e
rrors until an implementation, even if empty, is provided by the interface imple
mentor.
Following the same model, two interfaces, RTCP_actionListener are made available
. It is not required by the application classes to implement any of these interf
aces. The only requirement is that once an application class is declared that it
implements the interface, it must provide handlers for each and every method. T
he objects passed to the event handler are classes which encapsulate the RTP and
RTCP packets. Following object model diagram will help clarify how these packet
classes are interrelated.
RTP Packet:
Attributes of the RTP Packet are listed in the following model diagram.

When the Session.postAction() is called. This method is provided with the newly
constructed RTPPacket object. An alternative is to post the byte data stream ins
tead, but from the user's point of view, it is easier to work with objects and a
ttributes than to parse bits and bytes out of a byte datagram. To the user, an i
ncoming RTP packet is nothing more than the object instance of the RTPPacket cla
ss. Note that the user is not required to do anything with any of the attributes
, most often, it will only need the data attribute, which contains the payload.
But an application like RTP monitor needs to have access to the RTP packet heade
r and this encapsulation targets such users.

RTCPSenderReportPacket,
RTCPReceiverReportPacket:
RTCP receiver and sender report packets inherit from the RTCPPacket class, which
contains zero or one Report Block. It is possible that there are more than 31 m
embers in an RTP session, and the specifications allow for a RTCP packet to have
at most 31 report blocks, hence, the feedback about this source might be in the
next RTCP packet.

When a RTCP sender report or receiver report packet is encountered in the compou
nd packet, a corresponding packet object is instantiated and handleRTCPEvent() m
ethod and hands the packet object to it.
SDES Packets:
Following object diagram displays the SDES and BYE packets. Similar to the RTCP
sender/receiver report reception, when SDES and BYE packets are received, respec
tive packet objects are created and posted to Session.

Sequence Diagram:
Following sequence diagram shows the interaction between the Session, the sessio
n instantiator which implements the RTP_actionListener interface, the RTP receiv
er and the RTPPacket object.

Sequence 1 and 1.5: The registration takes place. Here instance of the class whi
ch is implementing the reference in order to post the RTP events to this listene
r.
Sequence 2: The asynchronous event, i.e. the RTP packet reception.
Sequence 3: The RTP receiver, after performing the house-keeping tasks, such as
updating the source table, upgrading the sender of this RTP packet to active sen
der etc., instantiates a RTPPacket object and populates its fields. It then post
s the packet object with the session in sequence 3.5.
Sequence 4: Session determines if any RTP_actionListener implementing object is
registered. In this case, there is a registered object.
Sequence 5: The packet object is handled by the RTP_actionListener.

Following rules and guidelines apply while working with the events model and act
ionListener interfaces:
It is recommended that a class implement the RTP_actionListener interface.
class MyClass implements RTP_actionListener { }

Session maintains registration and in the current implementation, there can only
be one listener per interface. Registration is performed using the methods Sess
ion.addRTCP_actionListener().
Only one registered listener will get the event notifications. For instance, the
re are two classes, A and B, that implement the RTP_actionListener interface, an
d if instance of A followed by B registers itself by calling the Session.addRTP_
actionListener() function, then the last registration will overwrite the one pre
ceding it. It is necessary to understand that there can only be one instance reg
istered for RTP_actionListener. Similarly, for RTCP events, there should be no m
ore than one class implementing the RTCP_actionListener interface and registerin
g itself.
If a class implements both interfaces, it must provide implementation for method
s in both the interfaces.

Following code examples may help clarify the event model.

// Example 1: One class that implements the RTP_actionListener interface and reg
isters itself
class MyClass implements RTP_actionListener {
MyClass {
Session rtpSession = new Session ( );
rtpSession.addRTP_actionListener ( this );

}
public void handleRTPEvent( RTPPacket rtppkt) {
// RTP event handler..
}
} // end

// Example 2: One class that implements both RTP_actionListener and RTCP_actionL

istener
class MyClass implements RTP_actionListener, RTCP_actionListener {
MyClass {
Session rtpSession = new Session ( );
rtpSession.addRTP_actionListener ( this );

}
public void handleRTPEvent(RTPPacket pkt) {
// RTP event handler
..
}
public void handleRTCPEvent(RTCPReceiverReportPacket RRpkt) {
// receiver report packet event handler
..
}
public void handleRTCPEvent(RTCPSenderReportPacket SRpkt) {
// sender report packet event handler
..
}
public void handleRTCPEvent(RTCPSDESPacket sdespkt) {
// Sdes packet event handler
..
}
public void handleRTCPEvent(RTCPBYEPacket byepkt) {
// bye packet event handler
..
}
One or more than one class can implement one or both the interfaces but only one
class should register.

Late Initialization Client/Server

Consider a whiteboard implementation which uses this package as its transport. A
ssume that a conference using these whiteboard instances has been active for som
e time. Now a new instance of the whiteboard is brought up which starts to recei
ve packets originated after its instantiation. The contents of the late-joiner w
hiteboard are not in sync with other instances that have been active for a while
. In order to synchronize the contents of this with others, there must exist a s
erver which is capable of replaying all the historic data. The Late Initializati
on Client/server tries to provide such functionality.
The repository of the historic data is maintained by the session as a Vector nam
ed Session.InitializationData. The applications is responsible for keeping this
vector up-to-date. The Vector is assumed to contain objects which could be conve
rted to String (proprietary classes must override the .toString() method). Conte
nts of this vector are given to the late-clients, which update their Session.Lat
eInitializationData vector. Hence, after a successful initialization, all the co
llaborating participants will have identical data in the LateInitializationData
vector.
A sample usage of this scheme would consist of a dedicated server, which is alwa
ys alive and listens to the RTP packets on the multicast groups. This server sto
res the data it receives (in String format) in the LateInitializationData vector
. The clients, upon instantiation, will connect to this server and populate thei
r LateInitializationData vector, thereby allowing all the clients to share a com
mon state irrespective of their startup times.

High Level Overview

Late Initialization Server
This encapsulates the server side functionality of the late initialization proce
dure. In the application it is implemented as separate thread, which monitors th
e activity on a specified port, waiting for client connections. All clients init
ially connect to this port when they want to initiate a new session with the ser
ver. As soon as the server gets a client request on this initial port, it assign
s the client a random port number (which is available) to connect to, so that th
e initial port is freed for other client requests.. The server spawns a new thre
ad connecting to the randomly generated port number and creates a Server Socket
on that port . The client and the server go through a series of "handshaking" si
gnals which maintain the state of the server and the client. The server transfer
s all the data to the client and sends a bye to the client which closes the conn
ection on that particular port and the port is freed up.
Late Initialization Client
This encapsulates the client side functionality of the late initialization proce
dure. It starts the client thread which connects to the server on a specified po
rt to initialize a new session with the server. The server assigns the client a
newly generated port number (which is available) to connect to, so that the init
ial port is freed for other client requests. The client closes that connection a
nd spawns a new thread connecting to the port number assigned by the server and
the client and the server go through a series of "handshaking" signals which mai
ntain the state of the server and the client. The server transfers all the data
to the client and sends a bye to the client which closes the connection on that
particular port and the port is freed up.
Program Documentation
Authors
Abstract
system requirements
installation instructions
Sample Usage (JRTPMon)
Limitations and Bugs
Enhancements
Acknowledgements

Measurements and Tests

The application used to test the sender and receiver functionality is available
to download from here .
References
1 Henning. Schulzrinne, et. al., RFC-1889, RTP, a Transport Protocol for Realtim
e Applications
2.JavaTM Media Framework FAQ
3.Henning. Schulzrinne, RTP Libraries and Tools.