Network Protocols

Overview
Prof. Osama Abdel Raof
Lecture-8
1. What Are Network
Protocols?
Network protocols are standardized rules that
dictate how data is transmitted, formatted, and
processed over a network. They enable
communication between devices, ensuring that
data is sent and received correctly.
 Why Are Protocols Important?
o Ensure seamless communication between
diverse systems and applications.
o Provide a common language for devices
across the internet and private networks.
o Define error handling, connection setup, and
termination procedures.
2. Key Protocols in Packet
Analysis

 TCP/IP (Transmission Control Protocol/Internet

Protocol):

o The backbone of the internet, TCP/IP provides the

foundational framework for most network
communications.

o TCP: Transmission Control Protocol

 Reliable, connection-oriented protocol.

 Ensures data packets are delivered in sequence

and without loss.

 Vulnerabilities: Susceptible to SYN floods, which

can lead to Denial of Service (DoS) attacks.

o IP: internet protocol

 Handles addressing and routing of packets

between devices.

 Vulnerabilities: IP spoofing can enable attackers

to impersonate a trusted source.
2. Key Protocols
in Packet Analysis
 HTTP (Hypertext Transfer Protocol):

o Used for web communications, such as accessing

websites.

o Operates over TCP (commonly on port 80).

o Vulnerabilities:

 Plaintext transmission (unless HTTPS is used),

making it prone to data interception.

 Man-in-the-Middle (MitM) attacks can

compromise HTTP sessions.

 HTTPS (HTTP Secure):

o Secure version of HTTP, using SSL/TLS for

encryption.

o Protects data integrity and confidentiality.

o Vulnerabilities: Outdated SSL/TLS versions and

misconfigured certificates can be exploited.
2. Key Protocols in Packet
Analysis

 HTTP (Hypertext Transfer Protocol):

o Used for web communications, such as accessing websites.
o Operates over TCP (commonly on port 80).
o Vulnerabilities:
 Plaintext transmission (unless HTTPS is used), making it prone to data interception.

 Man-in-the-Middle (MitM) attacks can compromise HTTP sessions.

 HTTPS (HTTP Secure):

o Secure version of HTTP, using SSL/TLS for encryption.
o Protects data integrity and confidentiality.
o Vulnerabilities: Outdated SSL/TLS versions and misconfigured certificates can be exploited.
2. Key Protocols in Packet
Analysis
Facilitates the transfer of files between devices.
FTP (File Operates on ports 20 and 21.
Transfer Vulnerabilities:
• Transmits data, including credentials, in plaintext, making it susceptible to eavesdropping.
Protocol): • Often replaced by SFTP or FTPS for enhanced security.

DNS Translates human-readable domain names (e.g., www.example.com) into IP

addresses.
(Domain Operates over UDP (port 53) for queries and TCP for zone transfers.
Name Vulnerabilities:
• DNS spoofing or poisoning can redirect users to malicious websites.
System): • DNS amplification attacks can be used to overwhelm a target server.
3. Role of Protocols in
Communication
Protocols ensure data is transmitted efficiently and reliably across
networks:
• Session Establishment and Management:
• TCP’s three-way handshake ensures a connection is established before data
transfer begins.
• Protocols manage connection states and gracefully terminate them when
communication is complete.
• Routing and Addressing:
• IP ensures data packets are routed through the network to the correct
destination.
• DNS simplifies routing by converting domain names to IP addresses.
• Error Detection and Recovery:
• TCP implements error-checking mechanisms to detect and retransmit lost
packets.
• Protocols like ARP (Address Resolution Protocol) resolve addressing conflicts.
4. Common Vulnerabilities and
Exploits in Protocols
Protocol-Specific Exploits:
• TCP Reset Attacks: Interrupt ongoing connections by sending
forged TCP reset packets.
• HTTP Injection: Manipulating HTTP requests to execute malicious
commands.
• DNS Cache Poisoning: Altering DNS records to redirect traffic to
General
maliciousExploits
IPs. Across Protocols:
• Packet sniffing to intercept unencrypted data.
Session hijacking by stealing authentication tokens or
cookies
5. Importance of Protocol
Knowledge in Packet Analysis
Understandi Identifying anomalies in traffic (e.g., unusual
ng protocols
is crucial for DNS queries or repeated TCP SYN packets).
effective
packet
analysis: Detecting malicious payloads embedded in
protocol data.

Troubleshooting network issues by analyzing

protocol-specific fields (e.g., TCP flags, HTTP
headers).
Analyzing TCP Handshakes

The TCP three-way

handshake is the
foundational process that
establishes a reliable
1. What is the TCP Three- connection between two
Way Handshake? devices on a network. It
ensures both parties are
ready for communication
and agree on initial
parameters.
Analyzing TCP Handshakes
Steps in the Three-Way Handshake:

1. SYN (Synchronize):
• The client sends a SYN packet to the server, indicating it wants to start a
connection.
• This packet contains an initial sequence number for tracking communication.
2. SYN-ACK (Synchronize-Acknowledge):
• The server responds with a SYN-ACK packet, acknowledging the client’s SYN and
providing its own sequence number.

3. ACK (Acknowledge):
• The client replies with an ACK packet, confirming the server's SYN-ACK.
• The connection is now established, and data transmission can begin.
 Synchronizes sequence
numbers to track packets.

Key Features Ensures both parties

of the agree to the connection
parameters.
Handshake:

Provides a reliable
foundation for data
exchange.
 The handshake plays a critical role
in maintaining a reliable and secure
network:
• Connection Reliability:

2. • Ensures both client and server are ready

before data transfer begins.
Importance • Prevents data loss by establishing
synchronization of sequence numbers.
of the • Flow Control:
• Sets the stage for managing the flow of
Three-Way data, preventing overwhelming either
party.
Handshake • Security and Authentication:
• Helps identify unauthorized or rogue
connection attempts.
• The sequence numbers and
acknowledgments make it harder for
attackers to spoof connections without
detection.
3. Detecting Anomalies in
Connection Setups
Anomalies during the handshake process often indicate
malicious activities or network issues. Analysts use packet
analysis to detect these irregularities.
• Common Anomalies and Their Implications:
1.Incomplete Handshakes:
• Symptoms: SYN packets sent without receiving corresponding SYN-
ACKs.
• Possible Cause: SYN flood attacks designed to exhaust server
resources.
2.Repeated SYN Packets:
• Symptoms: Multiple SYN packets from the same source IP without
completing the handshake.
• Possible Cause: Network misconfigurations or DDoS attacks..
3.
Detecting
Anomalies
3- Common Anomalies and 4- Indicators of Scanning
Their Implications:. Tools:
Unusual Sequence Numbers: Tools like Nmap often send SYN packets

in • Symptoms: Sequence numbers that

don’t match expected patterns.
• Possible Cause: Spoofed packets
to multiple ports to identify open ones.
These packets can be detected by
observing patterns of SYN packets

Connectio
without follow-up ACKs.
attempting to hijack a session.
Unexpected Resets (RST):
• Symptoms: Sudden RST packets

n Setups
during or after the handshake.
• Possible Cause: Scanning tools,
firewalls blocking connections, or
malicious attempts to disrupt
communication.
 Wireshark:
4. Tools • Filter to analyze handshakes:
for tcp.flags.syn == 1 && tcp.flags.ack ==
0
Analyzing • Detect SYN floods: Monitor high
volumes of SYN packets with no
TCP corresponding
Intrusion ACKs.
Detection
Systems (IDS):
Handshak • IDS like Snort can alert on anomalies
during handshake processes.
es • Custom rules can identify irregular
handshake patterns indicative of
attacks.
 For SYN Floods:

5. • Implement SYN cookies: Ensures

server resources are not over-
Mitigating utilized during incomplete
handshakes.
TCP • Use rate limiting: Limits the
number of SYN packets accepted
Handshak from a single IP.
For Scans and Probes:
e
• Configure firewalls to block
Anomalies repeated connection attempts.
• Use network monitoring tools to
detect suspicious patterns.
 • 1. What Are Packet Headers?
A packet header is the metadata
at the beginning of a network
Interpretin packet. It provides essential
information for routing,
g Packet managing, and delivering data
across a network. Each protocol
Headers layer adds its own header during
encapsulation, and these headers
are critical for understanding
network traffic during analysis.
 • Source and Destination IP Addresses:
• Source IP: Identifies the device that sent the
packet. Useful for tracing the origin of traffic,
2. Key especially during security investigations.

Details in
• Destination IP: Indicates the intended
recipient of the packet. Helps verify whether
traffic is directed to legitimate or suspicious
Packet targets.
• Port Numbers:
Headers • Source and destination ports define the
applications or services involved in the
communication (e.g., HTTP uses port 80,
HTTPS uses port 443).
• Anomalous or uncommon ports may
indicate malicious activity.
  Protocol Information:
o Specifies the protocol used (e.g., TCP, UDP, ICMP).
o Knowing the protocol helps interpret the data structure
and identify potential exploits.

2. Key
 Flags and Control Bits:
o TCP flags (SYN, ACK, RST, FIN, etc.) indicate the state of

Details in
the connection and any special actions required.
o Example: Repeated SYN packets without ACKs may
signify a SYN flood attack.

Packet  Sequence and Acknowledgment Numbers:

Headers o Used in TCP communications to ensure reliable data

transfer.
o Unexpected jumps in sequence numbers might indicate
malicious tampering or spoofing.

 Time-to-Live (TTL):
o Indicates how many hops a packet can take before being
discarded.
o Abnormal TTL values can signal spoofed packets.
  Tracing Source IPs:
o The source IP in the packet header helps identify the
origin of traffic. Analysts use it to:

3. Using  Trace malicious traffic to its point of origin.

Headers
 Correlate with logs from firewalls and intrusion
detection systems (IDS).

to Track
o Challenges:
 Attackers often use spoofed IPs or proxy services to
hide their true location.

Attackers  Analyzing Geolocation:

o Tools like MaxMind or IP Geolocation APIs can determine
the geographical origin of an IP address.
o Geolocation data can provide clues about the attacker's
location and potential intent.
3. Using Headers to Track Attackers

Detecting Inspecting Ports:

Anomalies:
Unusual source or destination IPs (e.g., Repeated connections to uncommon
private IPs on public networks) may ports (e.g., port 3389 for RDP) could
indicate misconfigurations or malicious indicate reconnaissance or an ongoing
activities. attack.
Example: Internal traffic with external Matching header port information to
destination IPs might point to data known attack patterns can uncover
exfiltration attempts. specific threats (e.g., SSH brute force
attacks on port 22).
 DDoS Attack:
4. Case o A Distributed Denial of Service
Studies (DDoS) attack generates a flood
of packets with varied source
in IPs.

Header- o Analyzing headers can reveal:

 Common patterns in spoofed
Based source IPs.
 The true attacker, if packets from
Tracking certain IPs appear
disproportionately.
4. Case Studies in Header-Based
Tracking
Phishing Campaign: Botnet Communication:
• By analyzing the headers of • Headers reveal
packets carrying phishing communication between
emails or malicious web infected devices and
requests, investigators can: Command-and-Control (C2)
• Identify the source IP of servers.
the attacker’s server. • Key indicators include:
• Correlate with domain • Repeated outbound traffic
lookups to uncover to the same destination IP.
additional infrastructure. • Use of unusual protocols
or payloads.
5. Tools for Header Analysis

Intrusion Detection
Wireshark: tcpdump:
Systems (IDS):
• Use filters to isolate • Command-line tool to • Tools like Snort or
traffic by capture and analyze Suricata flag
source/destination IP packet headers. suspicious header
or port. • Example: tcpdump -n patterns (e.g.,
• Inspect header fields src 192.168.1.1 malformed packets,
in detail for anomalies (captures traffic unusual ports).
or key indicators. originating from a
• Example filter: ip.addr specific source IP)
== 192.168.1.1
(analyzes all traffic
involving a specific IP).
6. Challenges in Using Headers to
Track Attackers

Spoofed IPs:

• Attackers can forge source IP addresses, complicating attribution.

• Techniques like ingress filtering can reduce the impact of spoofed packets.

Proxies and VPNs:

• Attackers often use proxies, VPNs, or The Onion Router (TOR) to anonymize
their traffic.
Encrypted Traffic:

• Headers remain accessible, but payload data is obscured, limiting full

analysis.
Malicious Payloads

• 1. What Are Malicious

Payloads?
In the context of network traffic,
the payload is the actual data
transmitted within a packet. A
malicious payload contains
harmful code or data intended to
exploit vulnerabilities, compromise
systems, or extract sensitive
information. Malicious payloads
often target vulnerabilities in
software, systems, or protocols to
achieve the attacker’s goals.
2. Examples of Malware in
Payloads
 Ransomware:
o Ransomware encrypts files on the victim’s system and demands
payment for decryption.
o How It Spreads:
 Via email attachments or links containing malicious payloads.
 Through network shares or remote code execution exploits.
o Indicators in Payloads:
 Encoded or compressed executable files.
 Suspicious commands targeting encryption libraries.
2. Examples of Malware in
Payloads
 Trojan Horses:
o Malicious payloads disguised as legitimate software.
o How It Spreads:
 Distributed through seemingly harmless applications or software
updates.
o Indicators in Payloads:
 Embedded commands to download additional malicious
components.
 Calls to external Command-and-Control (C2) servers.
2. Examples of Malware in
Payloads
 Worms:
o Self-replicating payloads that spread across networks without user
intervention.
o How It Spreads:
 Exploiting vulnerabilities in network services or software.
o Indicators in Payloads:
 Scripts or commands targeting common exploits.
 Repeated requests to multiple hosts.
2. Examples of Malware in Payloads
 Botnets:
o Payloads that turn infected devices into bots, controlled remotely by
attackers.
o How It Spreads:
 Via malicious downloads or phishing campaigns.
o Indicators in Payloads:
 Encrypted or obfuscated instructions for remote control.
 Repeated connections to specific C2 servers.
2. Examples of Malware in Payloads
 Data Exfiltration:
o Payloads designed to extract sensitive information, such as
passwords, financial data, or intellectual property.
o How It Spreads:
 Through malware like keyloggers or spyware.
o Indicators in Payloads:
 Large amounts of outbound traffic.
 Encoded data that appears unrelated to normal communication.
3. How to
Recognize Encoded
Data

 Why Attackers Encode

Data:
o Encoding hides malicious
content to evade detection
by firewalls, intrusion
detection systems (IDS), and
antivirus software.
o Examples include Base64
encoding, hex encoding, and
custom encoding schemes.
Techniques to
Recognize Encoded
Data:

1- Analyzing Packet Payloads:

 Use packet analysis tools (e.g.,
Wireshark) to inspect the payload.
 Look for patterns that deviate from
typical plaintext or expected data
formats.

2- Common Indicators of Encoding:

 Base64: A series of alphanumeric
characters ending with = or ==.
 Hexadecimal: Strings of numbers and
letters (0-9, A-F) in pairs.
 Custom Encoding: Random-seeming
byte patterns, often repetitive.
 3. Analyzing Repeated
Patterns:
• Encoded data often has repeating
structures or delimiters, indicating
Techniques compressed or encrypted content.

to 4. Entropy Analysis:

Recognize • Encoded data has higher entropy than

plaintext. Tools like binwalk or file can
Encoded estimate entropy levels.

Data: 5.
Traffic:
Inspecting HTTP/HTTPS

• Malicious payloads are often hidden in

parameters or POST requests.
• Analyze request bodies for irregular
data.
 Example 1: Encoded Malware in
HTTP Payload
• Payload: GET /update?
data=SGVsbG8gd29ybGQ= (Base64
encoding).

4. Real- • Decoded result: Contains the command to

download malicious software.
World • Detection: Use tools like Wireshark or an IDS
to decode the Base64 content.
Examples Example 2: Malicious Shellcode
in a Packet
• Payload contains a hexadecimal string (\x68\
x65\x6c\x6c\x6f) representing executable
instructions.
• Detection: Tools like objdump or Ghidra can
analyze the code.
 Wireshark:
• Inspect packet payloads in the
"Packet Bytes" pane.
5. Tools for • Use filters to isolate suspicious
traffic, e.g., http.request.method
Detecting == POST.
Malicious Snort/Suricata:
Payloads • Intrusion detection/prevention
systems that detect known
payload patterns.
• Example: Rules to flag Base64-
encoded data in HTTP headers.
 Hex Editors:
• Tools like HxD allow
5. Tools for manual inspection of
Detecting packet payloads for
Malicious unusual content.
Entropy Analyzers:
Payloads
• Tools like Entropy Explorer
help detect compressed,
encrypted, or encoded
payloads.
 Encryption:
• Encrypted traffic (e.g., HTTPS)
6. hides payload data from packet
Challenges inspection.
• Solution: Use TLS decryption
in tools or analyze metadata like
Detecting DNS queries and packet sizes.
Malicious Polymorphism:
• Malware that changes its
Payloads payload structure to evade
detection.
• Solution: Focus on behavioral
patterns rather than static
signatures.
Anomalies in Traffic
Flow

• 1. Understanding Anomalies in
Traffic Flow
Anomalies in traffic flow refer to
patterns or behaviors that deviate
from the expected network
activity. These irregularities are
often indicators of malicious
activity or underlying network
issues. Detecting and analyzing
these anomalies is critical for
identifying security threats, such
as attacks, data breaches, or
malware infections.
2. Indicators of Malicious Traffic
 Sudden Spikes in Traffic:
o Description:
 A significant and unexplained increase in traffic volume over a short period.
 May occur on specific ports, IP addresses, or entire subnets.
o Potential Causes:
 Distributed Denial of Service (DDoS) attacks overwhelming a server.
 Malware spreading across the network, generating excessive outbound traffic.
 Data exfiltration attempts involving large file transfers to an external server.
o Detection:
 Use network monitoring tools to identify sudden changes in traffic volume.
 Analyze flow data to isolate the source and destination of the spikes.
Repeated Connection Attempts:
o Description:
 Multiple attempts to establish connections to the same or different destinations,
often without completing them.
o Potential Causes:
 Port scanning by attackers searching for open or vulnerable services.
 Brute force login attempts targeting authentication systems.
 Malware beaconing to Command-and-Control (C2) servers.
o Detection:
 Monitor failed connection attempts using Intrusion Detection Systems (IDS) or
network logs.
 Look for patterns, such as SYN packets without corresponding ACKs.
3. Tools
 Wireshark:
o Apply filters to focus on specific traffic types (e.g., TCP SYN
packets).

for
o Example filters:
 tcp.flags.syn == 1 && tcp.flags.ack == 0: Detects SYN
packets without ACKs (potential port scan).

Detectin  ip.src == <specific_IP>: Analyzes traffic from a particular

source.

 Network Monitoring Tools:

g Traffic o Tools like SolarWinds, PRTG, or Zabbix visualize traffic patterns

and help identify anomalies.

Anomali
o Look for sudden spikes in traffic graphs or alerts for abnormal
behavior.

 Flow-Based Tools:

es o NetFlow or sFlow tools provide aggregated data to identify

bandwidth anomalies or unusual flow patterns.

 IDS/IPS Systems:
o Systems like Snort or Suricata can flag suspicious behaviors
such as repeated connection attempts or high volumes of traffic
on uncommon ports.
4. Case Studies of Anomalies in Traffic Flow

 Case Study 1: DDoS Attack

o Indicators:
 Sudden surge in inbound traffic to a web server.
 Large volumes of SYN packets without follow-up ACKs (SYN
flood).
o Response:
 Analyze the source IPs to determine if they are spoofed.
 Use rate-limiting or IP blacklisting to mitigate the attack.
4. Case Studies of Anomalies in Traffic Flow

 Case Study 2: Port Scanning

o Indicators:
 Multiple SYN packets to different ports on a server from the same
source IP.
 Lack of completed TCP handshakes.
o Response:
 Use firewalls to block scanning attempts.
 Analyze the scanning pattern to identify the tool used (e.g.,
Nmap).
4. Case Studies of Anomalies in Traffic Flow

 Case Study 3: Data Exfiltration

o Indicators:
 Unusual outbound traffic to an unfamiliar external IP.
 Large volumes of data being transferred outside normal business
hours.
o Response:
 Isolate the device generating the traffic.
 Inspect payloads to determine if sensitive data is being
transmitted.
5. Challenges in Detecting
Anomalies
Encrypted Traffic:
• Encrypted traffic (e.g., HTTPS) makes it harder to inspect
payloads directly.
• Solution: Focus on metadata such as traffic volume,
destination, and timing patterns.
False Positives:
• Sudden spikes or repeated attempts may occur during
legitimate activities, such as software updates or backup
operations.
• Solution: Correlate anomalies with expected network
behavior or logs.
6. Mitigation Strategies for
Anomalous Traffic
• Establish normal traffic patterns to identify deviations
quickly.
Set Baselines: • Use machine learning-based tools to dynamically adjust
baselines.

• Prevent abuse by limiting the number of connections or

Rate Limiting: requests a client can make within a specific timeframe.

Real-Time • Configure alerts for specific thresholds of traffic volume or

Alerts: repeated failed connection attempts.

Incident • Have predefined response strategies for specific

anomalies, such as isolating affected systems or blocking
Response Plans: malicious IPs.