Computer Science VIII

Chapter 2
Application Layer
Ing. Etson Guerrero

2: Application Layer 1
Chapter 2: Application layer
 2.1 Principles of  2.6 P2P applications
network applications  2.7 Socket programming
 2.2 Web and HTTP with TCP
 2.3 FTP  2.8 Socket programming
 2.4 Electronic Mail with UDP
 SMTP, POP3, IMAP
 2.5 DNS

2: Application Layer 2
Chapter 2: Application Layer
Our goals:  learn about protocols
 conceptual, by examining popular
implementation application-level
aspects of network protocols
application protocols  HTTP
 transport-layer  FTP
service models  SMTP / POP3 / IMAP
DNS
 client-server


paradigm  programming network

 peer-to-peer
applications
paradigm  socket API

2: Application Layer 3
Some network apps
 e-mail  voice over IP
 web  real-time video
 instant messaging conferencing
 remote login  grid computing
 P2P file sharing
 multi-user network
games
 streaming stored video
clips

2: Application Layer 4
Creating a network app application
transport
network
data link

write programs that physical

 run on (different) end

systems
 communicate over network
 e.g., web server software
communicates with browser
software application

No need to write software

transport
network
data link
for network-core devices
application
physical
transport
network
 Network-core devices do data link
physical
not run user applications
 applications on end systems
allows for rapid app
development, propagation
2: Application Layer 5
Chapter 2: Application layer
 2.1 Principles of  2.6 P2P applications
network applications  2.7 Socket programming
 2.2 Web and HTTP with TCP
 2.3 FTP  2.8 Socket programming
 2.4 Electronic Mail with UDP
 SMTP, POP3, IMAP  2.9 Building a Web
 2.5 DNS server

2: Application Layer 6
Application architectures
 Client-server
 Peer-to-peer (P2P)
 Hybrid of client-server and P2P

2: Application Layer 7
Processes communicating

Process: program running within a host.

 within same host, two processes communicate using

inter-process communication (defined by OS).

 processes in different hosts communicate by

exchanging messages

2: Application Layer 8
 Sockets
host or host or
server server

controlled by
app developer
process process

socket socket
TCP with TCP with
buffers, Internet buffers,
variables variables

controlled
by OS

2: Application Layer 9
Addressing processes

 to receive messages, process must have identifier

 host device has unique 32-bit IP address

 Q: does IP address of host suffice for

identifying the process?

2: Application Layer 10
Addressing processes

 identifier includes both IP address and port numbers

associated with process on host.

 Example port numbers:

 HTTP server: 80
 Mail server: 25

2: Application Layer 11
App-layer protocol defines
 Types of messages exchanged,
 e.g., request, response

 Message syntax:
 what fields in messages & how fields are delineated

 Message semantics
 meaning of information in fields

 Rules for when and how processes send & respond

to messages

2: Application Layer 12
What transport service does an app need?

2: Application Layer 13
 Transport service requirements of common apps

Application Data loss Throughput Time Sensitive

file transfer no loss elastic no

e-mail no loss elastic no
Web documents no loss elastic no
real-time audio/video loss-tolerant audio: 5kbps-1Mbps yes, 100’s msec
video:10kbps-5Mbps
stored audio/video loss-tolerant same as above yes, few secs
interactive games loss-tolerant few kbps up yes, 100’s msec
instant messaging no loss elastic yes and no

2: Application Layer 14
Internet transport protocols services

2: Application Layer 15
Internet apps: application, transport protocols

Application Underlying
Application layer protocol transport protocol

e-mail SMTP [RFC 2821] TCP

remote terminal access Telnet [RFC 854] TCP
Web HTTP [RFC 2616] TCP
file transfer FTP [RFC 959] TCP
streaming multimedia HTTP (eg Youtube), TCP or UDP
RTP [RFC 1889]
Internet telephony SIP, RTP, proprietary
(e.g., Skype) typically UDP

2: Application Layer 16
Chapter 2: Application layer
 2.1 Principles of  2.6 P2P applications
network applications  2.7 Socket programming
 app architectures with TCP
app requirements
 2.8 Socket programming


 2.2 Web and HTTP with UDP

 2.4 Electronic Mail
 SMTP, POP3, IMAP

 2.5 DNS

2: Application Layer 17
Web and HTTP
First some jargon
 Web page consists of objects
 Object can be HTML file, JPEG image, Java
applet, audio file,…
 Web page consists of base HTML-file which
includes several referenced objects
 Each object is addressable by a URL
 Example URL:
www.someschool.edu/someDept/pic.gif

host name path name

2: Application Layer 18
HTTP overview

HTTP: hypertext
transfer protocol
 Web’s application layer PC running
protocol Explorer

 client/server model
 client: browser that
requests, receives, Server
“displays” Web objects running
Apache Web
 server: Web server
server
sends objects in
response to requests
Mac running
Navigator

2: Application Layer 19
HTTP connections

2: Application Layer 20
Nonpersistent HTTP

2: Application Layer 21
Persistent HTTP

2: Application Layer 22
 HTTP request message

 two types of HTTP messages: request, response

 HTTP request message:
 ASCII (human-readable format)

request line
(GET, POST,
GET /somedir/page.html HTTP/1.1
HEAD commands)
Host: www.someschool.edu
header User-agent: Mozilla/4.0
lines Connection: close
Accept-language:fr
Carriage return,
line feed (extra carriage return, line feed)
indicates end
of message
2: Application Layer 23
HTTP request message: general format

2: Application Layer 24
Uploading form input
Post method:
 Web page often
includes form input URL method:
 Input is uploaded to  Uses GET method
server in entity body  Input is uploaded in
URL field of request
line:

www.somesite.com/animalsearch?monkeys&banana

2: Application Layer 25
Method types
HTTP/1.0 HTTP/1.1
 GET  GET, POST, HEAD
 POST  PUT
 HEAD  uploads file in entity
body to path specified
 asks server to leave
in URL field
requested object out of
response  DELETE
 deletes file specified in
the URL field

2: Application Layer 26
HTTP response message
status line
(protocol
status code
status phrase) HTTP/1.1 200 OK
Connection close
Date: Thu, 06 Aug 1998 12:00:15 GMT
header Server: Apache/1.3.0 (Unix)
lines Last-Modified: Mon, 22 Jun 1998 …...
Content-Length: 6821
Content-Type: text/html
data, e.g.,
requested data data data data data ...
HTML file

2: Application Layer 27
HTTP response status codes
In first line in server->client response message.
A few sample codes:
200 OK
 request succeeded, requested object later in this message
301 Moved Permanently
 requested object moved, new location specified later in
this message (Location:)
400 Bad Request
 request message not understood by server
404 Not Found
 requested document not found on this server
505 HTTP Version Not Supported
2: Application Layer 28
Trying out HTTP (client side) for yourself

2: Application Layer 29
User-server state: cookies
Example:
Many major Web sites
use cookies  Susan always access
Four components: Internet always from PC
1) cookie header line of  visits specific e-
HTTP response message commerce site for first
2) cookie header line in time
HTTP request message
3) cookie file kept on  when initial HTTP
user’s host, managed by requests arrives at site,
user’s browser
site creates:
4) back-end database at
Web site  unique ID
 entry in backend
database for ID
2: Application Layer 30
Cookies (continued)
aside
What cookies can bring: Cookies and privacy:
 authorization  cookies permit sites to
 shopping carts
learn a lot about you
 you may supply name
 recommendations
and e-mail to sites
 user session state
(Web e-mail)
How to keep “state”:
 protocol endpoints: maintain state
at sender/receiver over multiple
transactions
 cookies: http messages carry state

2: Application Layer 31
 Web caches (proxy server)
Goal: satisfy client request without involving origin server

origin
server

Proxy
server
client

client
origin
server

2: Application Layer 32
More about Web caching

 cache acts as both client and server

 typically cache is installed by ISP (university,

company, residential ISP)

Why Web caching?

2: Application Layer 33
 Conditional GET

 Goal: don’t send object if cache server

cache has up-to-date cached HTTP request msg
version If-modified-since:
object
 cache: specify date of
<date>
not
cached copy in HTTP request modified
HTTP response
If-modified-since:
HTTP/1.0
<date> 304 Not Modified
 server: response contains no
object if cached copy is up-
HTTP request msg
to-date: If-modified-since:
HTTP/1.0 304 Not <date> object
Modified modified
HTTP response
HTTP/1.0 200 OK
<data>
2: Application Layer 37
Chapter 2: Application layer
 2.1 Principles of  2.6 P2P applications
network applications  2.7 Socket programming
 2.2 Web and HTTP with TCP
 2.3 FTP  2.8 Socket programming
 2.4 Electronic Mail with UDP
 SMTP, POP3, IMAP  2.9 Building a Web
 2.5 DNS server

2: Application Layer 38
FTP: the file transfer protocol

FTP file transfer

FTP FTP
user client server
interface
user
at host remote file
local file system
system

 transfer file to/from remote host

 client/server model
 client: side that initiates transfer (either to/from
remote)
 server: remote host
 ftp: RFC 959
 ftp server: port 21

2: Application Layer 39
FTP: separate control, data connections

2: Application Layer 40
FTP commands, responses

Sample commands: Sample return codes

 sent as ASCII text over  status code and phrase (as
control channel in HTTP)
 USER username  331 Username OK,
 PASS password password required
 125 data connection
 LIST return list of file in
already open;
current directory
transfer starting
 RETR filename retrieves  425 Can’t open data
(gets) file connection
 STOR filename stores  452 Error writing
(puts) file onto remote file
host

2: Application Layer 41
Chapter 2: Application layer
 2.1 Principles of  2.6 P2P applications
network applications  2.7 Socket programming
 2.2 Web and HTTP with TCP
 2.3 FTP  2.8 Socket programming
 2.4 Electronic Mail with UDP
 SMTP, POP3, IMAP
 2.5 DNS

2: Application Layer 42
Electronic Mail
Three major components:
 user agents
 mail servers
 simple mail transfer protocol: SMTP

User Agent
 a.k.a. “mail reader”
 composing, editing, reading mail messages
 e.g., Eudora, Outlook, elm, Mozilla Thunderbird
 outgoing, incoming messages stored on server

2: Application Layer 43
Electronic Mail: SMTP [RFC 2821]

 uses TCP to reliably transfer email message from client

to server, port 25
 direct transfer: sending server to receiving server
 three phases of transfer
 handshaking (greeting)
 transfer of messages
 closure
 command/response interaction
 commands: ASCII text
 response: status code and phrase

 messages must be in 7-bit ASCII

2: Application Layer 44
 Scenario:
Anastasia sends message to Bartolo

2: Application Layer 45
Sample SMTP interaction
S: 220 hamburger.edu
C: HELO crepes.fr
S: 250 Hello crepes.fr, pleased to meet you
C: MAIL FROM: <[email protected]>
S: 250 [email protected]... Sender ok
C: RCPT TO: <[email protected]>
S: 250 [email protected] ... Recipient ok
C: DATA
S: 354 Enter mail, end with "." on a line by itself
C: Do you like ketchup?
C: How about pickles?
C: .
S: 250 Message accepted for delivery
C: QUIT
S: 221 hamburger.edu closing connection

2: Application Layer 46
Try SMTP interaction for yourself:

 telnet servername 25
 see 220 reply from server
 enter HELO, MAIL FROM, RCPT TO, DATA, QUIT
commands
above lets you send email without using email client
(reader)

2: Application Layer 47
SMTP: final words
 SMTP uses persistent Comparison with HTTP:
connections
 HTTP: pull
 SMTP requires message
(header & body) to be in 7-  SMTP: push
bit ASCII  both have ASCII
 SMTP server uses command/response
CRLF.CRLF to determine interaction, status codes
end of message
 HTTP: each object
encapsulated in its own
response msg
 SMTP: multiple objects
sent in multipart msg

2: Application Layer 48
Mail message format

SMTP: protocol for

exchanging email msgs header
blank
RFC 822: standard for text
line
message format:
 header lines, e.g.,
To:

body
 From:
 Subject:
different from SMTP
commands!
 body
 the “message”, ASCII
characters only

2: Application Layer 49
Mail access protocols
SMTP SMTP access user
user
agent protocol agent

sender’s mail receiver’s mail

server server

 SMTP: delivery/storage to receiver’s server

 Mail access protocol: retrieval from server
 POP: Post Office Protocol [RFC 1939]
• authorization (agent <-->server) and download
 IMAP: Internet Mail Access Protocol [RFC 1730]
• more features (more complex)
• manipulation of stored msgs on server
 HTTP: gmail, Hotmail, Yahoo! Mail, etc.

2: Application Layer 50
POP3 protocol S: +OK POP3 server ready
C: user bob
authorization phase S: +OK
C: pass hungry
 client commands: S: +OK user successfully logged on
 user: declare username
C: list
 pass: password S: 1 498
 server responses S: 2 912
S: .
 +OK
C: retr 1
 -ERR S: <message 1 contents>
transaction phase, client: S: .
C: dele 1
 list: list message numbers C: retr 2
 retr: retrieve message by S: <message 1 contents>
number S: .
C: dele 2
 dele: delete
C: quit
 quit S: +OK POP3 server signing off
2: Application Layer 51
POP3 (more) and IMAP
More about POP3 IMAP
 Previous example uses  Keep all messages in
“download and delete” one place: the server
mode.  Allows user to
 Bob cannot re-read e- organize messages in
mail if he changes folders
client  IMAP keeps user state
 “Download-and-keep”: across sessions:
copies of messages on  names of folders and
different clients mappings between
message IDs and folder
 POP3 is stateless
name
across sessions

2: Application Layer 52
Chapter 2: Application layer
 2.1 Principles of  2.6 P2P applications
network applications  2.7 Socket programming
 2.2 Web and HTTP with TCP
 2.3 FTP  2.8 Socket programming
 2.4 Electronic Mail with UDP
 SMTP, POP3, IMAP  2.9 Building a Web
 2.5 DNS server

2: Application Layer 53
DNS: Domain Name System

People: many identifiers: Domain Name System:

 SSN, name, passport #  distributed database
Internet hosts, routers: implemented in hierarchy of
many name servers
 IP address (32 bit) -
 application-layer protocol
used for addressing
host, routers, name servers to
datagrams
communicate to resolve names
 “name”, e.g., (address/name translation)
ww.yahoo.com - used by
 note: core Internet
humans
function, implemented as
Q: map between IP application-layer protocol
addresses and name ?  complexity at network’s
“edge”

2: Application Layer 54
 DNS
DNS services Why not centralize DNS?
 hostname to IP  single point of failure
address translation  traffic volume
 host aliasing  distant centralized
 Canonical, alias names database
 mail server aliasing  maintenance
 load distribution
 replicated Web doesn’t scale!
servers: set of IP
addresses for one
canonical name

2: Application Layer 55
Distributed, Hierarchical Database

Client wants IP for www.amazon.com; 1st approx:

 client queries a root server to find com DNS server
 client queries com DNS server to get amazon.com
DNS server
 client queries amazon.com DNS server to get IP
address for www.amazon.com

2: Application Layer 56
 DNS: Root name servers
 contacted by local name server that can not resolve name
 root name server:
 contacts authoritative name server if name mapping not known
 gets mapping
 returns mapping to local name server
a Verisign, Dulles, VA
c Cogent, Herndon, VA (also LA)
d U Maryland College Park, MD k RIPE London (also 16 other locations)
g US DoD Vienna, VA
h ARL Aberdeen, MD i Autonomica, Stockholm (plus
j Verisign, ( 21 locations) 28 other locations)
e NASA Mt View, CA m WIDE Tokyo (also Seoul,
f Internet Software C. Palo Alto, Paris, SF)
CA (and 36 other locations)

13 root name
servers worldwide
b USC-ISI Marina del Rey, CA
l ICANN Los Angeles, CA

2: Application Layer 57
TLD and Authoritative Servers
 Top-level domain (TLD) servers:
 responsible for com, org, net, edu, etc, and all
top-level country domains uk, fr, ca, jp.
 Network Solutions maintains servers for com TLD
 Educause for edu TLD
 Authoritative DNS servers:
 organization’s DNS servers, providing
authoritative hostname to IP mappings for
organization’s servers (e.g., Web, mail).
 can be maintained by organization or service
provider

2: Application Layer 58
Local Name Server
 does not strictly belong to hierarchy
 each ISP (residential ISP, company,
university) has one.
 also called “default name server”
 when host makes DNS query, query is sent
to its local DNS server
 acts as proxy, forwards query into hierarchy

2: Application Layer 59
DNS name
resolution example

iterated query:
 contacted server replies with name of server to
contact
 “I don’t know this name, but ask this server”

2: Application Layer 60
DNS name
resolution example
recursive query:
 puts burden of name resolution on contacted name server
 heavy load?

2: Application Layer 61
DNS: caching and updating records
 once (any) name server learns mapping, it caches
mapping
 cache entries timeout (disappear) after some
time
 TLD servers typically cached in local name
servers
• Thus root name servers not often visited
 update/notify mechanisms under design by IETF
 RFC 2136
 http://www.ietf.org/html.charters/dnsind-charter.html

2: Application Layer 62
DNS records

DNS: distributed db storing resource records (RR)

RR format: (name, value, type, ttl)

2: Application Layer 63
DNS protocol, messages
DNS protocol : query and reply messages, both with
same message format

msg header
 identification: 16 bit #
for query, reply to query
uses same #
 flags:
 query or reply
 recursion desired
 recursion available
 reply is authoritative

2: Application Layer 64
DNS protocol, messages

Name, type fields

for a query

RRs in response
to query

records for
authoritative servers

additional “helpful”
info that may be used

2: Application Layer 65
Inserting records into DNS

 example: new startup “Network Utopia”

 register name networkuptopia.com at DNS registrar
(e.g., Network Solutions)

 How do people get IP address of your Web site?

2: Application Layer 66
Chapter 2: Application layer
 2.1 Principles of  2.6 P2P applications
network applications  2.7 Socket programming
 app architectures with TCP
app requirements
 2.8 Socket programming


 2.2 Web and HTTP with UDP

 2.4 Electronic Mail
 SMTP, POP3, IMAP

 2.5 DNS

2: Application Layer 67
Pure P2P architecture
 no always-on server
 arbitrary end systems
directly communicate peer-peer
 peers are intermittently
connected and change IP
addresses

 Three topics:
 File distribution
 Searching for information
 Case Study: Skype

2: Application Layer 68
File Distribution: Server-Client vs P2P

Question : How much time to distribute file

from one server to N peers?

2: Application Layer 69
Server-client vs. P2P: example
Client upload rate = u, F/u = 1 hour, us = 10u, dmin ≥ us

2: Application Layer 70
File distribution: BitTorrent
 P2P file distribution

tracker: tracks peers torrent: group of

participating in torrent peers exchanging
chunks of a file

obtain list
of peers

trading
chunks

peer

2: Application Layer 71
Distributed Hash Table (DHT)
 DHT = distributed P2P database
 Database has (key, value) pairs;
 key: ss number; value: human name
 key: content type; value: IP address

 Peers query DB with key

 DB returns values that match the key
 Peers can also insert (key, value) peers
How to assign keys to peers?

 Ex: n=4; peers: 1,3,4,5,8,10,12,14;

 key = 13, then successor peer = 14
 key = 15, then successor peer = 1
Circular DHT (1)
1

3
15

4
12
5
10
8
 Each peer only aware of immediate successor
and predecessor.
 “Overlay network”
Circular DHT with Shortcuts
Peer Churn
1
•To handle peer churn, require
3 each peer to know the IP address
15
of its two successors.
• Each peer periodically pings its
4 two successors to see if they
12 are still alive.
5
10
8
 Peer 5 abruptly leaves
 Peer 4 detects; makes 8 its immediate successor;
asks 8 who its immediate successor is; makes 8’s
immediate successor its second successor.
 What if peer 13 wants to join?
P2P Case study: Skype
Skype clients (SC)
 inherently P2P: pairs
of users communicate.
 proprietary Skype
application-layer login server Supernode
protocol (inferred via (SN)
reverse engineering)
 hierarchical overlay
with SNs
 Index maps usernames
to IP addresses;
distributed over SNs

2: Application Layer 79
Peers as relays
 Problem when both
Alice and Bob are
behind “NATs”.
 NAT prevents an outside
peer from initiating a call
to insider peer
 Solution:
 Using Alice’s and Bob’s
SNs, Relay is chosen
 Each peer initiates
session with relay.
 Peers can now
communicate through
NATs via relay

2: Application Layer 80
Chapter 2: Summary
our study of network apps now complete!
 application architectures  specific protocols:
 client-server  HTTP

 P2P  FTP

 hybrid  SMTP, POP, IMAP

 DNS
 application service
 P2P: BitTorrent, Skype
requirements:
 reliability, bandwidth,  socket programming
delay
 Internet transport
service model
 connection-oriented,
reliable: TCP
 unreliable, datagrams: UDP
2: Application Layer 81
Chapter 2: Summary
Most importantly: learned about protocols

 typical request/reply Important themes:

message exchange:  control vs. data msgs
client requests info or
 in-band, out-of-band

service
 server responds with  centralized vs.
data, status code decentralized
 message formats:  stateless vs. stateful
 headers: fields giving  reliable vs. unreliable
info about data msg transfer
 data: info being
communicated  “complexity at network
edge”
2: Application Layer 82